Reviewerly/exHarmony huggingface dataset:
Social cognition of humanoid robots on customer tolerance of service failure

Purpose This study aims to examine the effects of the perceived warmth and competence of humanoid robots on customer tolerance of service failure through the affective response and the boundary condition of relationship norms. Design/methodology/approach Two experimental studies were conducted to investigate the effects of perceived warmth and competence of humanoid robots’ physical appearances on tolerance of service failure and the mediating role of anger. The boundary influence of relationship norms is also explored. Findings The results reveal that the perception of warmth (vs. competence) robot leads to less (more) anger, which significantly results in tolerance of service failure. However, customer tolerance is insignificant under exchange norms, as the undelivered service violates the expectations of both warm and competent robots. Practical implications This study provides practical guidance for hospitality managers to implement humanoid robots in a way that minimizes the negative outcomes of service failure. Managers should also think about the appropriate match of different types of humanoid robots and relationship norms in which robots will be deployed. Originality/value This study contributes to the tolerance literature by taking a social cognition perspective to investigate the effect of humanoid robots’ physical appearances on customers’ reactions to service failure. The findings also reveal that its affective mechanism lies in the effect of expectancy violations of service failure on tolerance. Furthermore, this study extends the literature on relationship norms to the influence of company factors on effective humanoid robot implementation.
VQAPT: A New visual question answering model for personality traits in social media images

Visual Question Answering (VQA) for personality trait images on social media is challenging because of multiple emotions and actions with complex backgrounds in social media images. This work aims at developing a new VQA model for different personality traits (VQAPT) identification in a single image. This work considers the Big Five Factors (BFF) for personality traits namely, Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. VQA is proposed based on the observation that multiple personality traits can be seen in a single image. We propose a model integrating text recognition and person/face recognition to derive the unique relationship between the text and the person's action in the image. Furthermore, a dynamic text-object graph for personality traits identification is constructed according to the query. For understanding a query, we explore the Contrastive Language-Image Pre-trained (CLIP) transformer encoder in this work. Since it is the first work of its kind, we have created a new dataset under this work for evaluation and the dataset is available publicly as mentioned in Section 4. The effectiveness of the proposed method is also evaluated on two benchmark datasets, namely TextVQA for VQA and PTI for personality traits identification.
Time-series visual explainability for Alzheimer’s disease progression detection for smart healthcare

Artificial intelligence (AI)-based diagnostic systems provide less error-prone and safer support to clinicians, enhancing the medical decision-making process. This study presents a smart and reliable healthcare framework for detecting Alzheimer's disease (AD) progression. Early detection of AD before the onset of clinical symptoms is the most crucial step in starting timely treatment. To predict the conversion of cognitively normal patients to those with AD, three-dimensional 3D magnetic resonance imaging (MRI) whole-brain neuroimaging methods have been extensively studied. However, depending on the 3D volume, this method is computationally expensive. To solve this problem, we used an approximate rank pooling method originally designed for video action recognition with a 3D MRI volume to obtain a compressed representation of multiple two-dimensional (2D) MRI slices. This study proposes a hybrid multimodal CNN-BiLSTM deep model for AD progression detection, in which the resulting dynamic 2D images are fused with cognitive features. Moreover, a novel explainable AI approach is proposed to provide visual explanations using the resulting longitudinal 2D dynamic images. Temporal explanations were provided by visualizing the affected brain regions captured using longitudinal 2D MRIs. By utilizing a sample of 1,692 subjects with multimodal data from the Alzheimer’s Disease Neuroimaging Initiative dataset, our method was assessed using a 10-fold cross-validation process. The model achieved an area under the receiver operating characteristics curve (AUC) of 94% using longitudinal 2D three-time-step dynamic image data. The fusion of 2D dynamic images with cognitive features enhanced the performance by 2% in terms of the AUC. Patients who gradually develop AD, show changes in various brain regions. For such patients, our system highlights the critical role of the hippocampus, medial amygdala, caudal hippocampus, and lateral amygdala at the initial time steps. In the late stages of AD, the system detects abnormalities in extra brain regions such as the medial temporal gyrus, superior temporal gyrus, fusiform gyrus, and caudal hippocampus; indicating that patients have completely progressed to AD.
Reduced order modeling of hydrodynamic interactions between a submarine and unmanned underwater vehicle using non-myopic multi-fidelity active learning

Several efforts have been dedicated to developing computational tools capable of predicting the hydrodynamic forces and moments of Unmanned Underwater Vehicles (UUVs). However, there is no method at the moment that allows for real-time computational modeling of all the complex hydrodynamic interaction forces and moments that a UUV experiences when operating in close proximity to a moving submarine. Real-time modeling of these hydrodynamic interactions is essential to simulate the motion required to launch and recover UUVs from submarines. Potential flow models are often fast enough to be used in real time, but lack the accuracy of Computational Fluid Dynamics (CFD) simulations, which often take hours or days to solve. Here, we formulate the problem in the context of machine learning, specifically active learning. The goal is to develop a surrogate model capable of predicting the UUV and submarine hydrodynamic interactions in real time using a very small number of carefully selected CFD simulations. We introduce a new active learning framework called Non-Myopic Multi-Fidelity Active Learning for Gaussian Process (GP) regression that accelerates the convergence of the surrogate model by utilizing the low cost of the low fidelity simulations to explore the domain, as well as optimally selected high fidelity simulations to improve the model accuracy. The resulting surrogate model can be integrated into UUV control and autonomy systems and motion simulators to further enable UUV launch and recovery from submarines. This new active learning method may also be used to create higher accuracy and lower cost surrogate models in other real world applications.
VRoxy: Wide-Area Collaboration From an Office Using a VR-Driven Robotic Proxy

Recent research in robotic proxies has demonstrated that one can automatically reproduce many non-verbal cues important in co-located collaboration. However, they often require a symmetrical hardware setup in each location. We present the VRoxy system, designed to enable access to remote spaces through a robotic embodiment, using a VR headset in a much smaller space, such as a personal office. VRoxy maps small movements in VR space to larger movements in the physical space of the robot, allowing the user to navigate large physical spaces easily. Using VRoxy, the VR user can quickly explore and navigate in a low-fidelity rendering of the remote space. Upon the robot's arrival, the system uses the feed of a 360 camera to support real-time interactions. The system also facilitates various interaction modalities by rendering the micro-mobility around shared spaces, head and facial animations, and pointing gestures on the proxy. We demonstrate how our system can accommodate mapping multiple physical locations onto a unified virtual space. In a formative study, users could complete a design decision task where they navigated and collaborated in a complex 7.5m x 5m layout using a 3m x 2m VR space.
Dynamic Toolchains: Software Infrastructure for Digital Fabrication Workflows

New digital fabrication workflows require both software development and digital/physical material exploration. To support digital fabrication workflow development, we contribute infrastructure that prioritizes extensibility and iteration. Dynamic Toolchains are dataflow programs with event-driven feedback between interactive, stateful modules. We contribute a browser-based dataflow environment for running Dynamic Toolchains, a library of fabrication-oriented front- and back-end modules for design and machine control, and a development framework for building custom modules. Furthermore, we show how our infrastructure supports unconventional fabrication workflows with demonstrations that include interactive watercolor painting, map plotting, machine knitting, audio embroidery, textured 3d printing, and computer-controlled milling. These demonstrations show how our infrastructure supports multiple kinds of engagement including reuse, remix, and extension. Finally, we discuss how this work contributes to broader conversations in HCI on creativity across the digital/physical divide.
Interactive Flexible Style Transfer for Vector Graphics

Vector graphics are an industry-standard way to represent and share visual designs. Designers frequently source and incorporate styles from existing designs into their work. Unfortunately, popular design tools are not well suited for this task. We present VST, Vector Style Transfer, a novel design tool for flexibly transferring visual styles between vector graphics. The core of VST lies in leveraging automation while respecting designers' tastes and the subjectivity inherent to style transfer. In VST, designers tune a cross-design element correspondence and customize which style attributes to change. We report results from a user study in which designers used VST to control style transfer between several designs, including designs participants created with external tools beforehand. VST shows that enabling design correspondence tuning and customization is one way to support interactive, flexible style transfer.
Pantœnna: Mouth pose estimation for ar/vr headsets using low-profile antenna and impedance characteristic sensing

Methods for faithfully capturing a user's holistic pose have immediate uses in AR/VR, ranging from multimodal input to expressive avatars. Although body-tracking has received the most attention, the mouth is also of particular importance, given that it is the channel for both speech and facial expression. In this work, we describe a new RF-based approach for capturing mouth pose using an antenna integrated into the underside of a VR/AR headset. Our approach side-steps privacy issues inherent in camera-based methods, while simultaneously supporting silent facial expressions that audio-based methods cannot. Further, compared to bio-sensing methods such as EMG and EIT, our method requires no contact with the wearer's body and can be fully self-contained in the headset, offering a high degree of physical robustness and user practicality. We detail our implementation along with results from two user studies, which show a mean 3D error of 2.6 mm for 11 mouth keypoints across worn sessions without re-calibration.
Statslator: Interactive Translation of NHST and Estimation Statistics Reporting Styles in Scientific Documents

Inferential statistics are typically reported using p-values (NHST) or confidence intervals on effect sizes (estimation). This is done using a range of styles, but some readers have preferences about how statistics should be presented and others have limited familiarity with alternatives. We propose a system to interactively translate statistical reporting styles in existing documents, allowing readers to switch between interval estimates, p-values, and standardized effect sizes, all using textual and graphical reports that are dynamic and user customizable. Forty years of CHI papers are examined. Using only the information reported in scientific documents, equations are derived and validated on simulated datasets to show that conversions between p-values and confidence intervals are accurate. The system helps readers interpret statistics in a familiar style, compare reports that use different styles, and even validate the correctness of reports. Code and data: https://osf.io/x4ue7
Beyond the Artifact: Power as a Lens for Creativity Support Tools

Researchers who build creativity support tools (CSTs) define abstractions and software representations that align with user needs to give users the power to accomplish tasks. However, these specifications also structure and limit how users can and should think, act, and express themselves. Thus, tool designers unavoidably exert power over their users by enacting a "normative ground" through their tools. Drawing on interviews with 11 creative practitioners, tool designers, and CST researchers, we offer a definition of empowerment in the context of creative practice, build a preliminary theory of how power relationships manifest in CSTs, and explain why researchers have had trouble addressing these concepts in the past. We re-examine CST literature through a lens of power and argue that mitigating power imbalances at the level of technical design requires enabling users in both vertical movement along levels of abstraction as well as horizontal movement between tools through interoperable representations. A lens of power is one possible orientation that lets us recognize the methodological shifts required towards building "artistic support tools."
From Gap to Synergy: Enhancing Contextual Understanding through Human-Machine Collaboration in Personalized Systems

This paper presents LangAware, a collaborative approach for constructing personalized context for context-aware applications. The need for personalization arises due to significant variations in context between individuals based on scenarios, devices, and preferences. However, there is often a notable gap between humans and machines in the understanding of how contexts are constructed, as observed in trigger-action programming studies such as IFTTT. LangAware enables end-users to participate in establishing contextual rules in-situ using natural language. The system leverages large language models (LLMs) to semantically connect low-level sensor detectors to high-level contexts and provide understandable natural language feedback for effective user involvement. We conducted a user study with 16 participants in real-life settings, which revealed an average success rate of 87.50% for defining contextual rules in a variety of 12 campus scenarios, typically accomplished within just two modifications. Furthermore, users reported a better understanding of the machine's capabilities by interacting with LangAware.
TouchType-GAN: Modeling Touch Typing with Generative Adversarial Network

Models that can generate touch typing tasks are important to the development of touch typing keyboards. We propose TouchType-GAN, a Conditional Generative Adversarial Network that can simulate locations and time stamps of touch points in touch typing. TouchType-GAN takes arbitrary text as input to generate realistic touch typing both spatially (i.e., (x, y) coordinates of touch points) and temporally (i.e., timestamps of touch points). TouchType-GAN introduces a variational generator that estimates Gaussian Distributions for every target letter to prevent mode collapse. Our experiments on a dataset with 3k typed sentences show that TouchType-GAN outperforms existing touch typing models, including the Rotational Dual Gaussian model [36] for simulating the distribution of touch points, and the Finger-Fitts Euclidean Model [30] for simulating typing time. Overall, our research demonstrates that the proposed GAN structure can learn the distribution of user typed touch points, and the resulting TouchType-GAN can also estimate typing movements. TouchType-GAN can serve as a valuable tool for designing and evaluating touch typing input systems.
PressurePick: Muscle Tension Estimation for Guitar Players Using Unobtrusive Pressure Sensing

When learning to play an instrument, it is crucial for the learner’s muscles to be in a relaxed state when practicing. Identifying, which parts of a song lead to increased muscle tension requires self-awareness during an already cognitively demanding task. In this work, we investigate unobtrusive pressure sensing for estimating muscle tension while practicing songs with the guitar. First, we collected data from twelve guitarists. Our apparatus consisted of three pressure sensors (one on each side of the guitar pick and one on the guitar neck) to determine the sensor that is most suitable for automatically estimating muscle tension. Second, we extracted features from the pressure time series that are indicative of muscle tension. Third, we present the hardware and software design of our PressurePick prototype, which is directly informed by the data collection and subsequent analysis.
WorldSmith: Iterative and Expressive Prompting for World Building with a Generative AI

Crafting a rich and unique environment is crucial for fictional world-building, but can be difficult to achieve since illustrating a world from scratch requires time and significant skill. We investigate the use of recent multi-modal image generation systems to enable users iteratively visualize and modify elements of their fictional world using a combination of text input, sketching, and region-based filling. WorldSmith enables novice world builders to quickly visualize a fictional world with layered edits and hierarchical compositions. Through a formative study (4 participants) and first-use study (13 participants) we demonstrate that WorldSmith offers more expressive interactions with prompt-based models. With this work, we explore how creatives can be empowered to leverage prompt-based generative AI as a tool in their creative process, beyond current "click-once" prompting UI paradigms.
STAR: Smartphone-analogous Typing in Augmented Reality

While text entry is an essential and frequent task in Augmented Reality (AR) applications, devising an efficient and easy-to-use text entry method for AR remains an open challenge. This research presents STAR, a smartphone-analogous AR text entry technique that leverages a user's familiarity with smartphone two-thumb typing. With STAR, a user performs thumb typing on a virtual QWERTY keyboard that is overlain on the skin of their hands. During an evaluation study of STAR, participants achieved a mean typing speed of 21.9 WPM (i.e., 56% of their smartphone typing speed), and a mean error rate of 0.3% after 30 minutes of practice. We further analyze the major factors implicated in the performance gap between STAR and smartphone typing, and discuss ways this gap could be narrowed.
Vision-based approach to assess performance levels while eating

Abstract The elderly population is increasing at a rapid rate, and the need for effectively supporting independent living has become crucial. Wearable sensors can be helpful, but these are intrusive as they require adherence by the elderly. Thus, a semi-anonymous (no image records) vision-based non-intrusive monitoring system might potentially be the answer. As everyone has to eat, we introduce a first investigation into how eating behavior might be used as an indicator of performance changes. This study aims to provide a comprehensive model of the eating behavior of individuals. This includes creating a visual representation of the different actions involved in the eating process, in the form of a state diagram, as well as measuring the level of performance or decay over time during eating. Also, in studies that involve humans, getting a generalized model across numerous human subjects is challenging, as indicative features that parametrize decay/performance changes vary significantly from person to person. We present a two-step approach to get a generalized model using distinctive micro-movements, i.e., (1) get the best features across all subjects (all features are extracted from 3D poses of subjects) and (2) use an uncertainty-aware regression model to tackle the problem. Moreover, we also present an extended version of EatSense, a dataset that explores eating behavior and quality of motion assessment while eating.
Amharic political sentiment analysis using deep learning approaches

Abstract This study delves into the realm of sentiment analysis in the Amharic language, focusing on political sentences extracted from social media platforms in Ethiopia. The research employs deep learning techniques, including Convolutional Neural Networks (CNN), Bidirectional Long Short-Term Memory (Bi-LSTM), and a hybrid model combining CNN with Bi-LSTM to analyze and classify sentiments. The hybrid CNN-Bi-LSTM model emerges as the top performer, achieving an impressive accuracy of 91.60%. While these results mark a significant milestone, challenges persist, such as the need for a more extensive and diverse dataset and the identification of nuanced sentiments like sarcasm and figurative speech. The study underscores the importance of transitioning from binary sentiment analysis to a multi-class classification approach, enabling a finer-grained understanding of sentiments. Moreover, the establishment of a standardized corpus for Amharic sentiment analysis emerges as a critical endeavor with broad applicability beyond politics, spanning domains like agriculture, industry, tourism, sports, entertainment, and satisfaction analysis. The exploration of sarcastic comments in the Amharic language stands out as a promising avenue for future research.