Reviewerly/exHarmony huggingface dataset:
Classifying Head Movements to Separate Head-Gaze and Head Gestures as Distinct Modes of Input

Head movement is widely used as a uniform type of input for human-computer interaction. However, there are fundamental differences between head movements coupled with gaze in support of our visual system, and head movements performed as gestural expression. Both Head-Gaze and Head Gestures are of utility for interaction but differ in their affordances. To facilitate the treatment of Head-Gaze and Head Gestures as separate types of input, we developed HeadBoost as a novel classifier, achieving high accuracy in classifying gaze-driven versus gestural head movement (F1-Score: 0.89). We demonstrate the utility of the classifier with three applications: gestural input while avoiding unintentional input by Head-Gaze; target selection with Head-Gaze while avoiding Midas Touch by head gestures; and switching of cursor control between Head-Gaze for fast positioning and Head Gesture for refinement. The classification of Head-Gaze and Head Gesture allows for seamless head-based interaction while avoiding false activation.
Comparison of algorithms for contrast enhancement based on triangle orientation discrimination assessments by convolutional neural networks

Within the last decades, a large number of techniques for contrast enhancement has been proposed. There are some comparisons of such algorithms for few images and figures of merit. However, many of these figures of merit cannot assess usability of altered image content for specific tasks, such as object recognition. In this work, the effect of contrast enhancement algorithms is evaluated by means of the triangle orientation discrimination (TOD), which is a current method for imager performance assessment. The conventional TOD approach requires observers to recognize equilateral triangles pointing in four different directions, whereas here convolutional neural network models are used for the classification task. These models are trained by artificial images with single triangles. Many methods for contrast enhancement highly depend on the content of the entire image. Therefore, the images are superimposed over natural backgrounds with varying standard deviations to provide different signal-to-background ratios. Then, these images are degraded by Gaussian blur and noise representing degradational camera effects and sensor noise. Different algorithms, such as the contrast-limited adaptive histogram equalization or local range modification, are applied. Then accuracies of the trained models on these images are compared for different contrast enhancement algorithms. Accuracy gains for low signal-to-background ratios and sufficiently large triangles are found, whereas impairments are found for high signal-to-background ratios and small triangles. A high generalization ability of our TOD model is found from the similar accuracies for several image databases used for backgrounds. Finally, implications of replacing triangles with real target signatures when using such advanced digital signal processing algorithms are discussed. The results are a step toward the assessment of those algorithms for generic target recognition.
Studying Children’s Object Interaction in Virtual Reality: A Manipulative Gesture Taxonomy for VR Hand Tracking

In this paper, we propose a taxonomy for the classification of children's gestural input elicited from spatial puzzle play in VR hand tracking. The taxonomy builds on the existing manipulative gesture taxonomy in human-computer interaction, and offers two main analytical categories; Goal-directed actions and Hand kinematics as complementary dimensions for analysing gestural input. Based on our study with eight children (aged between 7-14), we report the qualitative results for describing the categories for analysis and quantitative results for their frequency in occurring in children's interaction with the objects during the spatial task. This taxonomy is an initial step towards capturing the complexity of manipulative gestures in relation to mental rotation actions, and helps designers and developers to understand and study children's gestures as an input for object interaction as well as an indicator for spatial thinking strategies in VR hand tracking systems.
Co-designing Community-based Sharing of Smarthome Devices for the Purpose of Co-monitoring In-home Emergencies

We conducted 26 co-design interviews with 50 smarthome device owners to understand the perceived benefits, drawbacks, and design considerations for developing a smarthome system that facilitates co-monitoring with emergency contacts who live outside of one's home. Participants felt that such a system would help ensure their personal safety, safeguard from material loss, and give them peace of mind by ensuring quick response and verifying potential threats. However, they also expressed concerns regarding privacy, overburdening others, and other potential threats, such as unauthorized access and security breaches. To alleviate these concerns, participants designed flexible and granular access control and fail-safe back-up features. Our study reveals why peer-based co-monitoring of smarthomes for emergencies may be beneficial but also difficult to implement. Based on the insights gained from our study, we provide recommendations for designing technologies that facilitate such co-monitoring while mitigating its risks.
Blended Collaboration: Communication and Cooperation Between Two Users Across the Reality-Virtuality Continuum

Mixed reality (MR) technologies provide enormous potential for collaboration between multiple users across the Reality–Virtuality continuum. We evaluate communication in a MR-based two-user collaboration task, in which the users have to move an object through an obstacle without collision. We used a blended reality environment, in which one user is immersed in virtual reality, whereas the other uses mobile augmented reality. Both users have different abilities and information and mutually depend on each other for successful completion of the task. Communication consensus can either be achieved by using speech, visual widgets, or a combination of both. The results indicate that speech plays a fundamental role. The usage of widgets served as an extension rather than a replacement of language. However, the combination of speech and widgets improved the clearness of communication with less miscommunication. These results provide important indications about how to design blended collaboration across the Reality–Virtuality continuum.
Making Smart Cities Explainable: What XAI Can Learn from the “Ghost Map”

How can we visualize civic algorithms in ways that illuminate both their positive and negative spatial impacts? Civic algorithms guide everyday decisions that cumulatively create city life. Yet, their broader effects remain invisible to their creators and city inhabitants. Recent scholarship on “algorithmic harms” presents an urgent need to make smart cities explainable. We argue that existing Explainable AI (XAI) approaches are limited across four important dimensions: accessibility, cultural reflexivity, situatedness, and visibility into internal representations. Our research explores the potential of conventional maps in addressing these limits and providing what we call “grounded explanations”. As a salient example, we harness the historical case of the “Ghost Map”, designed by John Snow to visualize and resolve the 1854 London Cholera epidemic. We believe that such examples can help the XAI community learn from the cultural history of city representations, as they seek to establish public processes for explaining and evaluating “smart cities”.
Autospeculation: Reflecting on the Intimate and Imaginative Capacities of Data Analysis

Given decades of Human computer interaction (HCI) research focused on scientific empiricism, it can be hard for the field to acknowledge that data analysis is both an emotional and speculative process. But what does it mean for this process of data analysis to embrace its situated and speculative nature? In this paper, we explore this possibility by building on decades of HCI mixed methods that root data analysis in design. Drawing on an autoethnographic design inquiry, we examine how data analysis can work as an implicating process, one that is not only critically grounded in a designer’s own situation but also offers modes of imagining the world otherwise. In this analysis, we find that autobiographical design can help HCI scholars to respond to current critiques of speculative design by grounding and rendering more personal certain kinds of speculation, opening a space for diverse voices to emerge.
ACTIVE4: A Conceptual Framework for Gathering Empathetic Insights toward Office Workers’ Vitality Ecosystem Design

Sedentary behavior (SB) is prevalent in workplaces, putting office workers at an increased risk of severe health problems. To help designers and researchers gain a better understanding of office workers' contextual concerns for physical inactivity (reducing SB and enhancing physical activity (PA)), we have proposed a conceptual framework ACTIVE4. This framework advises designers and researchers to consider four key factors that influence office workers' physical inactivity: active mind, active behavior, active support, and active environment. We conducted three workshops (N=28 design students) to evaluate the framework. The participants found ACTIVE4 helpful in guiding them towards a more systematic understanding of the environmental influences and office workers' personal needs for reducing physical inactivity. In future work, we will optimize the ACTIVE4 framework's learning curve as suggested by participants and conduct an expert study to further discuss design opportunities and requirements for the ACTIVE4-related vitality toolkit.
Defining Embodied Provenance for Immersive Sensemaking

Immersive analytics research has explored how embodied data representations and interactions can be used to engage users in sensemaking. Prior research has broadly overlooked the potential of immersive space for supporting analytic provenance, the understanding of sensemaking processes through users' interaction histories. We propose a concept of embodied provenance, the use of three-dimensional space and embodiment in supporting recalling, reproducing, annotating and sharing analysis history in immersive environments. We highlight a set of design criteria for analytic provenance drawn from prior work and propose a conceptual framework for embodied provenance. We develop a prototype system in virtual reality to demonstrate the concept and support the conceptual framework by providing multiple data views and embodied interaction metaphors in a large space. We present a use case scenario of energy consumption analysis, which shows the system's potential for assisting analytic provenance using embodiment.
Trust and Perceived Control in Burnout Support Chatbots

Increased levels of user control in learning systems is commonly cited as good AI development practice. However, the evidence as to the effect of perceived control over trust in these systems is mixed. This study investigated the relationship between different trust dimensions and perceived control in postgraduate student burnout support chatbots, and modelled the moderating factors therein. We present an in-between subject controlled experiment using simulated therapy-goal learning to study the effect of perceived control (as manipulated by feedback incorporation) on perceived agent benevolence, competence, and trust. Our results showed that perceived control was moderately correlated with benevolence (r = 0.448, BF10 = 7.150), and weakly correlated with competence and trust.
HCI Research in Sensitive Settings: Lessons Learned from Technology Design and Ethical Challenges in Dementia

In this course, we will share concrete guidelines for HCI research in sensitive settings based on our research experience in dementia. We share how these lessons learned apply in sensitive settings regarding how designing for and with people with dementia has general implications for design and computing. First, we will provide real-world insights on designing technologies in sensitive settings by analyzing example cases from academia and industry. Next, we instruct designers and researchers in practical ethical conduct with stakeholders through hands-on exercises, such as facilitating informed consent and modifying common co-design activities for sensitive contexts.
A Study on User Perception and Experience Differences in Recommendation Results by Domain Expertise: The Case of Fashion Domains

To improve user satisfaction to the recommender systems (RS), it is essential to identify how users would perceive the system and what recommendation results they would prefer. Domain experts also use RS that relate to their work and decision-making, but existing studies have primarily focused on user experience from the general public and somewhat neglected the degree of user perception, understanding, and preference to the RS according to domain knowledge and interest and recommendation algorithm types. In this paper, we present My Own Style (MOS), a dashboard tool designed to analyze a given input fashion image and recommend outfit based on three recommendation algorithms that have different degrees of similarity and diversity. Based on the results from a large-scale user study with 166 participants, our results showed that the participants who have high fashion knowledge and interest (i.e., domain experts) well understood the results of RS and preferred the recommendation algorithm that provides similar outfit, while those who have low fashion knowledge and interest did not well understand the recommendation results as much as the expert group and preferred the algorithm that suggests diverse outfit. Our work is meaningful in providing empirical evidence on how to develop, select, and utilize recommendation algorithms according to domain expertise.
H4VR: One-handed Gesture-based Text Entry in Virtual Reality Using a Four-key Keyboard

H4VR is a text entry method offering two layouts – flat and cross – of a four-key keyboard in VR. Hand gestures select the keys with a different selection method for each layout. The entry speed and error rate were measured and analyzed in a five-session longitudinal user study. Five participants entered ten phrases on each keyboard in each session. They reached an average entry speed of 4.57 and 3.63 words per minute on the flat and cross keyboards, respectively. The average error rate was less than 2% on both keyboards.
Accuracy of AI-generated Captions With Collaborative Manual Corrections in Real-Time

Automatic Speech Recognition (ASR) is a cost-efficient and scalable tool to automate real-time captioning. Even though its overall quality has improved rapidly, generated transcripts can be inaccurate. While manual correction helps to increase transcription accuracy, this causes new real-time challenges, especially for live-streaming. Crowd-sourcing can make the high workload more manageable by distributing the work across multiple individuals. In this paper, we developed a prototype that enables humans to collaboratively correct AI-generated captions in real-time. We conducted an experiment with 40 participants to measure the accuracy of the created and manually corrected captions. The results show that manual corrections improved the overall text accuracy according to multiple metrics as well as overall qualitative analysis.
Spot The Difference: AI, Please Make This for Me!

Play and games are an inseparable part of our lives, and we have been playing various games from our childhood to adulthood. One of the widely recognized games is "Spot-the-Difference", which has been played and employed in a variety of domains, such as education, training, and entertainment. However, designing a custom "Spot-the-Difference" game for a certain domain requires consideration in terms of the level of game difficulty, game interest, and human intervention, therefore, is non-trivial. In this paper, we propose a novel framework based on Human-AI collaboration to automatically generate images for the "Spot-the-Difference" game. We employ Maskformer and Inpainting Stable Diffusion to automatically identify and re-draw a set of regions in the image. Finally, we conducted a user study with 19 participants to evaluate our framework. From the experimental result, we found that AI-generated game images were enjoyable, but there is still room for improvement in the quality of the overall game playing.
An Approach to Solve the Identification and Authentication Challenges in Metaverse

Technology is getting updated day by day, and the current pandemic has added to the need for the digital availability of various components of life. With this advancement came the concept of the metaverse, which has been predicted to give rise to businesses in the world as people will be able to access everything by sitting in their homes. However, along with the boon comes a bane for any new technology. With adoption of the various advance technologies, privacy and security concerns also appear which needs to be catered. Here, the bane we talk about is the security issues faced by metaverse in terms of authentication and identification. Enhancing the security and privacy features will allow metaverse to attract more and more users. This paper highlights the reasons for the traditional methods being less effective. We have also focused on the other security concerns in metaverse and how we can mitigate them. Along with this, the paper also provides a new model that will be more effective in terms of security.
Negotiating Experience and Communicating Information Through Abstract Metaphor

An implicit assumption in metaphor use is that it requires grounding in a familiar concept, prominently seen in the popular Desktop Metaphor. In human-to-human communication, however, abstract metaphors, without such grounding, are often used with great success. To understand when and why metaphors work, we present a case study of metaphor use in voice teaching. Voice educators must teach about subjective, sensory experiences and rely on abstract metaphor to express information about unseen and intangible processes inside the body. We present a thematic analysis of metaphor use by 12 voice teachers. We found that metaphor works not because of strong grounding in the familiar, but because of its ambiguity and flexibility, allowing shared understanding between individual lived experiences. We summarise our findings in a model of metaphor-based communication. This model can be used as an analysis tool within the existing taxonomies of metaphor in user interaction for better understanding why metaphor works in HCI. It can also be used as a design resource for thinking about metaphor use and abstracting metaphor strategies from both novel and existing designs.
THERIF: Themes for Readability from Iterative Feedback

Digital reading applications give readers the ability to customize fonts, sizes, and spacings, all of which have been shown to improve the reading experience for readers from different demographics. However, tweaking these text features can be challenging, especially given their interactions on the final look and feel of the text. Our solution is to offer readers preset combinations of font, character, word and line spacing, which we bundle together into reading themes. To arrive at a recommended set of reading themes, we combine crowdsourced text adjustments, ML-driven clustering of the resulting text formats, and design sessions. After four iterations of these steps, we converge on a set of three COR (Compact, Open, and Relaxed) themes that are designed for different readers.
PointShopAR: Supporting Environmental Design Prototyping Using Point Cloud in Augmented Reality

We present PointShopAR, a novel tablet-based system for AR environmental design using point clouds as the underlying representation. It integrates point cloud capture and editing in a single AR workflow to help users quickly prototype design ideas in their spatial context. We hypothesize that point clouds are well suited for prototyping, as they can be captured more rapidly than textured meshes and then edited immediately in situ on the capturing device. We based the design of PointShopAR on the practical needs of six architects in a formative study. Our system supports a variety of point cloud editing operations in AR, including selection, transformation, hole filling, drawing, morphing, and animation. We evaluate PointShopAR through a remote study on usability and an in-person study on environmental design support. Participants were able to iterate design rapidly, showing the merits of an integrated capture and editing workflow with point clouds in AR environmental design.
An Evaluation Study of 2D and 3D Teleconferencing for Remote Physical Therapy

Abstract The present research investigates the effectiveness of using a telepresence system compared to a video conferencing system and the effectiveness of using two cameras compared to one camera for remote physical therapy. We used Telegie as our telepresence system, which allowed users to see an environment captured with RGBD cameras in 3D through a VR headset. Since both telepresence and the inclusion of a second camera provide users with additional spatial information, we examined this affordance within the relevant context of remote physical therapy. Our dyadic study across different time zones paired 11 physical therapists with 76 participants who took on the role of patients for a remote session. Our quantitative questionnaire data and qualitative interviews with therapists revealed several important findings. First, after controlling for individual differences among participants, using two cameras had a marginally significant positive effect on physical therapy assessment scores from therapists. Second, the spatial ability of patients was a strong predictor of therapist assessment. And third, the video clarity of remote communication systems mattered. Based on our findings, we offer several suggestions and insights towards the future use of telepresence systems for remote communication.
MOFA: Exploring Asymmetric Mixed Reality Design Strategy for Co-located Multiplayer Between Handheld and Head-mounted Augmented Reality

For co-located multiplayer asymmetric Mixed Reality (MR) scenarios involving both Handheld Augmented Reality (HAR) and head-mounted Stereoscopic Augmented Reality (StAR) devices, we propose a design strategy to distinguish the roles of different players based on the affordances, limitations, and capabilities of these two kinds of AR devices. Specifically, the roles include: a HAR player as a third-person passive participant (termed spectator), a third-person active participant (termed puppeteer), a first-person participant, and a first-person partial-information participant; and a StAR player as a first-person participant and a first-person partial-information participant. In order to explore this concept, we designed four multiplayer MR game prototypes: The Duck, The Ghost, The Dragon, and The Duel to demonstrate strategies for creating interesting, engaging, and strategic cooperative experiences between players.