Miracle-dz/UniFAR-FTUs huggingface dataset:
Roman: Making Everyday Objects Robotically Manipulable with 3D-Printable Add-on Mechanisms

One important vision of robotics is to provide physical assistance by manipulating different everyday objects, e.g., hand tools, kitchen utensils. However, many objects designed for dexterous hand-control are not easily manipulable by a single robotic arm with a generic parallel gripper. Complementary to existing research on developing grippers and control algorithms, we present Roman, a suite of hardware design and software tool support for robotic engineers to create 3D printable mechanisms attached to everyday handheld objects, making them easier to be manipulated by conventional robotic arms. The Roman hardware comes with a versatile magnetic gripper that can snap on/off handheld objects and drive add-on mechanisms to perform tasks. Roman also provides software support to register and author control programs. To validate our approach, we designed and fabricated Roman mechanisms for 14 everyday objects/tasks presented within a design space and conducted expert interviews with robotic engineers indicating that Roman serves as a practical alternative for enabling robotic manipulation of everyday objects.
“I Want It That Way”: Exploring Users’ Customization and Personalization Preferences for Home Assistants

Home assistants are becoming a widespread product, but they mostly come as a compact device and offer very few customization and personalization features, which often leads to dissatisfaction. With the technological advancements, these systems are becoming more adaptable to the users’ needs and can better imitate a human personality. To achieve that efficiently, understanding how different users envision their desired assistant is crucial. To identify people’s customization and personalization preferences and their desired personality for a home assistant, we designed a set of storyboards depicting a variety of possible features in a domestic setting and conducted a user study (), including a series of semi-structured interviews. Our quantitative results suggest that users prefer an agent which is highly agreeable and has higher conscientiousness and emotional stability. Furthermore, we discuss users’ customization and personalization preferences for a home assistant, which could be considered when designing the future generation of home assistants.
Investigating Potentials of Shape-Changing Displays for Sound Zones

In this paper, we investigate the use of shape-change for interaction with sound zones. A core challenge to designing interaction with sound zone systems is to support users’ understanding of the unique spatial properties of sound zones. Shape-changing interfaces present new opportunities for addressing this. We present a structured investigation into this. We leveraged the knowledge of 12 sound experts to define a set of basic shapes and movements. Then, we constructed a prototype and conducted an elicitation study with 17 novice users, investigating the experience of these shapes and movements. Our findings show that physical visualizations of sound zones can be useful in supporting users’ experience of sound zones. We present a framework of 4 basic pattern categories that prompt different sound zone experiences and outline further research directions for shape-change in supporting sound zone interaction.
Digital Emotion Regulation in Everyday Life

Two decades of focus on User Experience has yielded an array of digital technologies that help people experience, understand and share emotions. Although the effects of specific technologies upon emotion have been well studied, less is known about how people actively appropriate and combine the full range of devices, apps and services at their disposal to deliberately manage emotions in everyday life. We conducted a one-week diary study in which 23 adults recorded interactions between their emotions and technology use. They reported using a diverse range of emotion-shaping tools and strategies as part of coping with daily challenges, managing routines, and pursuing work and social goals. We analyse these data in the light of psychological theories of emotion. Our findings point to the significance of digital emotion regulation as a powerful perspective to inform wider debates about the impacts of technology on social and emotional well-being.
Mobilizing Crowdwork:A Systematic Assessment of the Mobile Usability of HITs

There is a growing interest in extending crowdwork beyond traditional desktop-centric design to include mobile devices (e.g., smartphones). However, mobilizing crowdwork remains significantly tedious due to a lack of understanding about the mobile usability requirements of human intelligence tasks (HITs). We present a taxonomy of characteristics that defines the mobile usability of HITs for smartphone devices. The taxonomy is developed based on findings from a study of three consecutive steps. In Step 1, we establish an initial design of our taxonomy through a targeted literature analysis. In Step 2, we verify and extend the taxonomy through an online survey with Amazon Mechanical Turk crowdworkers. Finally, in Step 3 we demonstrate the taxonomy’s utility by applying it to analyze the mobile usability of a dataset of scraped HITs. In this paper, we present the iterative development of the taxonomy, highlighting the observed practices and preferences around mobile crowdwork. We conclude with the implications of our taxonomy for accessibly and ethically mobilizing crowdwork not only within the context of smartphone devices, but beyond them.
Computational Rationality as a Theory of Interaction

How do people interact with computers? This fundamental question was asked by Card, Moran, and Newell in 1983 with a proposition to frame it as a question about human cognition – in other words, as a matter of how information is processed in the mind. Recently, the question has been reframed as one of adaptation: how do people adapt their interaction to the limits imposed by cognition, device design, and environment? The paper synthesizes advances toward an answer within the theoretical framework of computational rationality. The core assumption is that users act in accordance with what is best for them, given the limits imposed by their cognitive architecture and their experience of the task environment. This theory can be expressed in computational models that explain and predict interaction. The paper reviews the theoretical commitments and emerging applications in HCI, and it concludes by outlining a research agenda for future work.
CareMouse: An Interactive Mouse System that Supports Wrist Stretching Exercises in the Workplace

Knowledge workers suffer from wrist pain due to their long-term mouse and keyboard use. In this study, we present CareMouse, an interactive mouse system that supports wrist stretching exercises in the workplace. When the stretch alarm is given, users hold CareMouse and do exercises, and the system collects the wrist movement data and determines whether they follow the accurate stretching motions based on a machine learning algorithm, enabling real-time guidance. We conducted a preliminary user study to understand the users’ perception and user experience of the system. Our results showed the feasibility of CareMouse in guiding stretching exercises interactively. We provided design implications for the augmentation of existing tools when offering auxiliary functions.
Exploring the Role of Paradata in Digitally Supported Qualitative Co-Research

Academics and community organisations are increasingly adopting co-research practices where participants contribute to qualitative data collection, analysis, and dissemination. These qualitative practices can often lack transparency that can present a problem for stakeholders (such as funding agencies) who seek evidence of the rigour and accountability in these decision-making processes. When qualitative research is done digitally, paradata is available as interaction logs that reveal the underlying processes, such as the time spent engaging with different segments of an interview. In practice, paradata is seldom used to examine the decisions associated with undertaking qualitative research. This paper explores the role of paradata arising from a four-month engagement with a community-led charity that used a digital platform to support their qualitative co-research project. Through observations of platform use and reflective post-deployment interviews, our findings highlight examples of paradata generated through digital tools in qualitative research, e.g., listening coverage, engagement rate, thematic maps and data discards. From this, we contribute a conceptualisation of paradata and discuss its role in qualitative research to improve process transparency, enhance data sharing, and to create feedback loops with research participants.
An Agile New Research Framework for Hybrid Human-AI Teaming: Trust, Transparency, and Transferability

We propose a new research framework by which the nascent discipline of human-AI teaming can be explored within experimental environments in preparation for transferal to real-world contexts. We examine the existing literature and unanswered research questions through the lens of an Agile approach to construct our proposed framework. Our framework aims to provide a structure for understanding the macro features of this research landscape, supporting holistic research into the acceptability of human-AI teaming to human team members and the affordances of AI team members. The framework has the potential to enhance decision-making and performance of hybrid human-AI teams. Further, our framework proposes the application of Agile methodology for research management and knowledge discovery. We propose a transferability pathway for hybrid teaming to be initially tested in a safe environment, such as a real-time strategy video game, with elements of lessons learned that can be transferred to real-world situations.
SpinOcchio: Understanding Haptic-Visual Congruency of Skin-Slip in VR with a Dynamic Grip Controller

This paper’s goal is to understand the haptic-visual congruency perception of skin-slip on the fingertips given visual cues in Virtual Reality (VR). We developed SpinOcchio (Spin for the spinning mechanism used, Occhio for the Italian word “eye”), a handheld haptic controller capable of rendering the thickness and slipping of a virtual object pinched between two fingers. This is achieved using a mechanism with spinning and pivoting disks that apply a tangential skin-slip movement to the fingertips. With SpinOcchio, we determined the baseline haptic discrimination threshold for skin-slip, and, using these results, we tested how haptic realism of motion and thickness is perceived with varying visual cues in VR. Surprisingly, the results show that in all cases, visual cues dominate over haptic perception. Based on these results, we suggest applications that leverage skin-slip and grip interaction, contributing further to realistic experiences in VR.
A Personalized Visual Aid for Selections of Appearance Building Products with Long-term Effects

It is challenging for customers to select appearance building products (e.g., skincare products, weight loss programs) that suit them personally as such products usually demonstrate efficacy only after long-term usage. Although e-retailers generally provide product descriptions or other customers’ reviews, users often find it hard to relate to their own situations. In this work, we proposed a pipeline to display envisioned users’ appearance after long-term use of appearance building products to deliver their efficacy on each individual visually. We selected skincare as a case and developed SkincareMirror which predicts skincare effects on users’ facial images by analyzing product function labels, efficacy ratings, and skin models’ images. The results of a between-subjects study (N=48) show that (1) SkincareMirror outperforms the baseline shopping site in terms of perceived usability, usefulness, user satisfaction and helps users select products faster; (2) SkincareMirror is especially effective to males and users with limited product domain knowledge.
Recentering Reframing as an RtD Contribution: The Case of Pivoting from Accessible Web Tables to a Conversational Internet

Design produces valuable knowledge by offering new perspectives that reframe problematic situations. Research through Design (RtD) contributes new frames along with design work demonstrating a frame's value. Interestingly, RtD papers rarely describe how reframing happens. This gap in documentation unintentionally implies a romantic account of design, it implies that the first step of an RtD project is to have a brilliant idea. This is especially problematic in cases where the reframing causes a pivot that leads to a new research program. To help address this gap, we describe a case where through a series of three design experiments we experienced a research pivot. We describe how our work to improve web-table navigation for screen-reader users broke our frame. The break led to a new research program focused on constructing a conversational internet. This paper offers our case along with reflection on reporting design work that drives reframing.
Wizard of Errors: Introducing and Evaluating Machine Learning Errors in Wizard of Oz Studies

When designing Machine Learning (ML) enabled solutions, designers often need to simulate ML behavior through the Wizard of Oz (WoZ) approach to test the user experience before the ML model is available. Although reproducing ML errors is essential for having a good representation, they are rarely considered. We introduce Wizard of Errors (WoE), a tool for conducting WoZ studies on ML-enabled solutions that allows simulating ML errors during user experience assessment. We explored how this system can be used to simulate the behavior of a computer vision model. We tested WoE with design students to determine the importance of considering ML errors in design, the relevance of using descriptive error types instead of confusion matrix, and the suitability of manual error control in WoZ studies. Our work identifies several challenges, which prevent realistic error representation by designers in such studies. We discuss the implications of these findings for design.
TypeOut: Leveraging Just-in-Time Self-Affirmation for Smartphone Overuse Reduction

Smartphone overuse is related to a variety of issues such as lack of sleep and anxiety. We explore the application of Self-Affirmation Theory on smartphone overuse intervention in a just-in-time manner. We present TypeOut, a just-in-time intervention technique that integrates two components: an in-situ typing-based unlock process to improve user engagement, and self-affirmation-based typing content to enhance effectiveness. We hypothesize that the integration of typing and self-affirmation content can better reduce smartphone overuse. We conducted a 10-week within-subject field experiment (N=54) and compared TypeOut against two baselines: one only showing the self-affirmation content (a common notification-based intervention), and one only requiring typing non-semantic content (a state-of-the-art method). TypeOut reduces app usage by over 50%, and both app opening frequency and usage duration by over 25%, all significantly outperforming baselines. TypeOut can potentially be used in other domains where an intervention may benefit from integrating self-affirmation exercises with an engaging just-in-time mechanism.
Play With Your Emotions: Exploring Possibilities of Emotions as Game Input in NERO

This work presents NERO, a game concept using the player’s active emotional input to map the emotional state of the player to representative in-game characters. Emotional input in games has been mainly used as a passive measure to adjust for game difficulty or other variables. However the player has not been given the possibility to explore and play with one’s emotions as an active feature. Given the high subjectivity of felt emotions we wanted to focus on the player’s experience of emotional input rather than the objective accuracy of the input sensor. We therefore implemented a proof-of-concept game using heart-rate as a proxy for emotion measurement and through repeated player tests the game mechanics were revised and evaluated. Valuable insight for the design of entertainment-focused emotional input games were gained, including emotional connection despite limited accuracy, influence of the environment and the importance of calibration. The players overall enjoyed the novel game experience and their feedback carries useful implications for future games including active emotional input.
Community Resilience: Quantifying the Disruptive Effects of Sudden Spikes in Activity within Online Communities

Online communities often experience some form of crisis. Regardless of whether these communities are small groups or entire platforms, the ability to handle disruptions during volatile periods signifies the resilience of a community. In this paper, we analyze the effects of a specific type of crisis event on Reddit: sudden spikes in attention received by a community due to a post from the subreddit hitting r/popular—the default feed that aggregates the most popular content on Reddit. Our results show that r/popular is a source of potential disruptions due to the higher number of comments, authors, and especially removed comments found compared to threads that do not reach r/popular. When looking at r/popular’s effects across subreddits, large communities’ commenting and posting behaviors were less impacted by r/popular threads compared to smaller communities. However, similar-sized subreddits had substantial variations in how their commenting and posting activities were influenced by an r/popular thread. Understanding the causal factors behind these differences has implications for online governance, as well as fostering resilient and healthy communities.
VisGuide: User-Oriented Recommendations for Data Event Extraction

Data exploration systems have become popular tools with which data analysts and others can explore raw data and organize their observations. However, users of such systems who are unfamiliar with their datasets face several challenges when trying to extract data events of interest to them. Those challenges include progressively discovering informative charts, organizing them into a logical order to depict a meaningful fact, and arranging one or more facts to illustrate a data event. To alleviate them, we propose VisGuide—a data exploration system that generates personalized recommendations to aid users’ discovery of data events in breadth and depth by incrementally learning their data exploration preferences and recommending meaningful charts tailored to them. As well as user preferences, VisGuide’s recommendations simultaneously consider sequence organization and chart presentation. We conducted two user studies to evaluate 1) the usability of VisGuide and 2) user satisfaction with its recommendation system. The results of those studies indicate that VisGuide can effectively help users create coherent and user-oriented visualization trees that represent meaningful data events.
Generating Embodied Storytelling and Interactive Experience of China Intangible Cultural Heritage “Hua'er” in Virtual Reality

Hua'er, a type of traditional oral performance, is one of the national intangible cultural heritages (ICH) in China. Experts have been trying to enhance public's awareness of Hua'er protection through digital documentation technology, but there's no efficacious means to attract interest and popularize knowledge yet. In this paper, we propose an interactive VR system engaging audiences to experience and understand the connotation of Hua'er performance. Based on an online survey with the public and in-depth interviews with Hua'er experts, we derive a set of design requirements for generating embodied storytelling and interactive experience in VR. Accordingly, we design “Hua'er and the Youth” (HY) that integrates three methods (virtual avatar, participatory performance, and game-based knowledge acquisition) and conduct a between-subjects user study to compare HY with the Comparison system. The results suggest that our methods significantly improved audience's interactive experience, knowledge level and the awareness of ICH safeguarding.
Demonstrating InfraredTags: Decoding Invisible 3D Printed Tags with Convolutional Neural Networks

We demonstrate InfraredTags, which are invisible 2D markers and barcodes that are 3D printed as part of objects. We show how InfraredTags can be decoded using a convolutional neural network (CNN) after being captured by low-cost near-infrared cameras. InfraredTags are formed by printing objects from an infrared-transmitting filament, which infrared cameras can see through, and by having air gaps inside for the tag’s bits, which appear at a different intensity in the infrared image. We built a user interface that facilitates the integration of common tags (QR codes, ArUco markers) with the object geometry to make them 3D printable as InfraredTags. Once printed, the tags can be robustly decoded with a U-Net model that we trained using a custom dataset for optimal binarization and detection. We show how our method enables different applications, such as object tracking and embedding metadata for tangible interactions and augmented reality.
Barriers to Expertise in Citizen Science Games

Expertise-centric citizen science games (ECCSGs) can be powerful tools for crowdsourcing scientific knowledge production. However, to be effective these games must train their players on how to become experts, which is difficult in practice. In this study, we investigated the path to expertise and the barriers involved by interviewing players of three ECCSGs: Foldit, Eterna, and Eyewire. We then applied reflexive thematic analysis to generate themes of their experiences and produce a model of expertise and its barriers. We found expertise is constructed through a cycle of exploratory and social learning but prevented by instructional design issues. Moreover, exploration is slowed by a lack of polish to the game artifact, and social learning is disrupted by a lack of clear communication. Based on our analysis we make several recommendations for CSG developers, including: collaborating with professionals of required skill sets; providing social features and feedback systems; and improving scientific communication.
Improving Colour Patterns to Assist People with Colour Vision Deficiency

Many daily tasks rely on accurately identifying and distinguishing between different colours. However, these tasks can be frustrating and potentially dangerous for people with Colour Vision Deficiency (CVD). Despite prior work exploring how pattern overlays on top of colours can support people with CVD, the solutions were often unintuitive or required significant training to become proficient. We address this problem by creating two new colour patterns (ColourIconizer, ColourMix). We evaluated these patterns against a previously published colour pattern (ColourMeters) using an online evaluation with three new colour identification tasks (Selection Task, Transition Task, Sorting Task). ColourMeters helped with the Transition Task, but struggled with the Selection and Sorting Tasks. Conversely, ColourIconizer helped with the Selection and Sorting Tasks but struggled to help on the Transition Task. ColourMix provided general assistance on all tasks. Our combined results help inform and improve the design of future colour patterns.