Miracle-dz/UniFAR-FTUs huggingface dataset:
Preliminary Study on Effect of Secretly Increasing or Decreasing Predicted Number of Steps to Promote Walking

With the widespread use of wearable devices, it is becoming easier to predict the future of an individual’s behavior regarding health. However, little is known about how presenting future predictions about an individual’s behavior affects awareness and behavior. The “StepUp Forecast” project is investigating walking behavior to clarify the impact of presenting predictions of the number of steps an individual will take on human behavior. In a previous study, it was found that self-efficacy and the number of steps taken increased significantly when the predicted number of steps was presented, compared with when only a record of the number of steps taken was shown. To advance this research, we investigated the effects of intentionally and secretly increasing or decreasing the predicted value rather than simply presenting it. Specifically, we observed changes in awareness and behavior when users were presented with a predicted value with 1,000 steps added or subtracted. In our 5-week experiment, 40 participants used StepUp Forecast, an application that presents the predicted number of steps on the basis of past life logs. The results indicate a non-significant but increasing trend in self-efficacy and number of steps added or subtracted compared with when only the step-count record was presented.
µGeT: Multimodal eyes-free text selection technique combining touch interaction and microgestures

We present µGeT, a novel multimodal eyes-free text selection technique. µGeT combines touch interaction with microgestures. µGeT is especially suited for People with Visual Impairments (PVI) by expanding the input bandwidth of touchscreen devices, thus shortening the interaction paths for routine tasks. To do so, µGeT extends touch interaction (left/right and up/down flicks) using two simple microgestures: thumb touching either the index or the middle finger. For text selection, the multimodal technique allows us to directly modify the positioning of the two selection handles and the granularity of text selection. Two user studies, one with 9 PVI and one with 8 blindfolded sighted people, compared µGeT with a baseline common technique (VoiceOver like on iPhone). Despite a large variability in performance, the two user studies showed that µGeT is globally faster and yields fewer errors than VoiceOver. A detailed analysis of the interaction trajectories highlights the different strategies adopted by the participants. Beyond text selection, this research shows the potential of combining touch interaction and microgestures for improving the accessibility of touchscreen devices for PVI.
Assessment and Analysis of Wearables and Companion Mobile (Health) Applications: A Usability Evaluation Framework

Wearable technologies have gained significant traction in various domains with healthcare being a particularly prominent area for their use, often in conjunction with mobile (health) applications. The increasing demand and versatility of these technologies pose challenges in meeting the specific needs of patients, healthy subjects, as well as healthcare professionals. Despite their growing adoption, there is a notable lack of empirical research focusing on user behavior, adaptation, acceptance, and usability evaluation for wearable technologies and companion apps. Research focusing on human factors and usability is currently inadequate in this specific area. Therefore, the objective of my research is to develop a structured guideline to address these limitations. The primary goal is to create a practical and flexible framework for assessing the usability of wearables and their associated mobile apps. This framework will be designed to be accessible and adaptable, allowing both usability experts and non-experts such as clinical researchers to evaluate wearable technology together with its companion mobile app. Through the development, validation, and implementation of this framework, this investigation aims to support the evaluation in a structured manner, and thus enhance the long-term adoption and improve the acceptance of wearable technologies and mobile (health) apps.
Recognition of Engagement from Electrodermal Activity Data Across Different Contexts

Engagement is a human experience relevant in multiple contexts, including classrooms, presentations and workplaces. Stemming from flow theory, engagement in these contexts has been studied using wearable devices, which can unobtrusively measure physiological changes, specifically Electrodermal Activity (EDA). However, researchers have not explored how EDA markers might be similar or different between various engagement scenarios, namely student, audience and workplace engagement. In this study, we investigated possible similarities through the use of three datasets containing EDA data and engagement self-report labels, collected in the wild in different settings using research-grade wrist-worn wearable devices. We analysed the correlation between hand-crafted EDA features and the engagement level and we leveraged a machine learning framework for engagement prediction. We found that similar features are correlated with the engagement level across the various settings. We also found that our machine learning model identified related markers as important across the three engagement contexts. Our results highlight that similarities are present in the EDA features between different engagement contexts, while also identifying possible dataset specific differences.
Explainable Depression Detection using Multimodal Behavioural Cues

Depression is a severe mental illness that not only affects the patient but also has major social and economical implications. Recent studies have employed artificial intelligence using multimodal behavioural cues to objectively investigate depression and alleviate the subjectivity involved in current depression diagnostic process. However, head motion has received a fairly limited attention as a behavioural marker for detecting depression and the lack of explainability of the "black box" approaches have restricted their widespread adoption. Consequently, the objective of this research is to examine the utility of fundamental head-motion units termed kinemes and explore the explainability of multimodal behavioural cues for depression detection. To this end, the research to date evaluated depression classification performance on the BlackDog and AVEC2013 datasets using multiple machine learning methods. Our findings indicate that: (a) head motion patterns are effective cues for depression assessment, and (b) explanatory kineme patterns can be observed for the two classes, consistent with prior research.
Breezy the Calm Monster: Soft Toy Design Combined with Pervasive Technology to Teach Deep Breathing

Anger dysregulation can lead to aggression and peer rejection in early childhood. Anger episodes and temper tantrums frequently escalate, leaving parents and young children needing effective emotion regulation interventions. This research investigates how designing a playful interaction with ubiquitous technology can help parents teach young children emotion regulation strategies such as deep breathing. Breezy is a soft toy with an app and a storybook that helps parents and engages them in teaching deep breathing as an anger regulation strategy. The soft toy has sensory and physical features and can be held as a puppet, which helps young children to connect with the toy. The digital app addresses facial expressions and physiological components of emotions through sensory and interactive features that encourage identifying emotions. The storybook facilitates a familiar context for parents' engagement through playful learning. The interactive system provides parents with opportunities to teach their young children to identify and label anger and practice and model age-appropriate breathing exercises. This research aims to inform the design of future educational and playful interactions for emotional competence skills, as well as to broaden their application and promote their use in homes.
GCFormer: A Graph Convolutional Transformer for Speech Emotion Recognition

Graph convolutional networks (GCNs) have achieved excellent results in image classification and natural language processing. However, at present, the application of GCNs in speech emotion recognition (SER) is not widely studied. Meanwhile, recent studies have shown that GCNs may not be able to adaptively capture the long-range context emotional information over the whole audio. To alleviate this problem, this paper proposes a Graph Convolutional Transformer (GCFormer) model which empowers the model to extract local and global emotional information. Specifically, we construct a cyclic graph and perform concise graph convolution operations to obtain spatial local features. Then, a consecutive transformer network further strives to learn more high-level representations and their global temporal correlation. Finally and sequentially, the learned serialized representations from the transformer are mapped into a vector through a gated recurrent unit (GRU) pooling layer for emotion classification. The experiment results obtained on two public emotional datasets demonstrate that the proposed GCFormer performs significantly better than other GCN-based models in terms of prediction accuracy, and surpasses the other state-of-the-art deep learning models in terms of prediction accuracy and model efficiency.
Detecting When the Mind Wanders Off Task in Real-time: An Overview and Systematic Review

Research on the ubiquity and consequences of task-unrelated thought (TUT; often used to operationalize mind wandering) in several domains recently sparked a surge in efforts to create “stealth measurements” of TUT using machine learning. Although these attempts have been successful, they have used widely varied algorithms, modalities, and performance metrics — making them difficult to compare and inform future work on best practices. We aim to synthesize these findings through a systematic review of 42 studies identified following PRISMA guidelines to answer two research questions: 1) are there any modalities that are better indicators of TUT than the rest; and 2) do multimodal models provide better results than unimodal models? We found that models built on gaze typically outperform other modalities and that multimodal models do not present a clear edge over their unimodal counterparts. Our review highlights the typical steps involved in model creation and the choices available in each step to guide future research, while also discussing the limitations of the current “state of the art” — namely the barriers to generalizability.
Empowering Autonomy and Agency: Exploring and Augmenting Accessible Cyber-Physical Systems

People with Vision Impairments face various barriers in physical activities that are highly visual (e.g., cooking, visiting art museums), which strongly impact their agency, autonomy, and quality of life. Cyber-Physical Systems (CPS) provide opportunities for people with vision impairments to better interact with physical space. In this research, we first explore current practices and challenges of the adaptation of CPS by people with vision impairments in physical activities. We then establish a comprehensive design space and design guidelines for CPS to satisfy the needs of blind individuals. Finally, we integrate the user-centered design guidelines, implement CPS for people with vision impairments to reduce the barrier of interacting with physical space, and conduct deployment studies to validate our design. Overall, our research provide a roadmap of augmenting accessible CPS for people with disabilities in the physical space.
Can Data Augmentation Improve Daily Mood Prediction from Wearable Data? An Empirical Study

Mobile sensing data, approximating human behavior and physiology, can be processed by machine learning models to predict mental health symptoms. While these models are accurate in smaller samples, their generalization accuracy decreases in larger samples, potentially because it is difficult to collect enough mobile sensing and mental health outcomes data at scale to enable generalization. In this study, we hypothesized that augmenting training data with synthetic data samples could improve the generalizability of these machine learning models. We created a data augmentation system that generated synthetic mobile sensing and mental health outcomes data, and evaluated the utility of this system via the downstream machine learning task of predicting daily mood from wearable sensing data. We experimented with both simple (e.g. noise addition) and novel generative data augmentation methods, based upon conditional generative adversarial networks and multi-task learning. Our initial findings suggest that the data augmentation system generated realistic synthetic data, but did not improve mood prediction. We propose future work to validate our findings and test other methods to improve the generalizability of mental health symptom prediction models.
MoCa'Collection: Normalizing Dynamic Textile Geometry with Capacitive Sensing in Design Centric Wearables

In this work, we promote capacitive sensing as a versatile smart textile modality through a collection of functional wearable designs. Considering the large variety of possible garment design concepts, we outline an approach to implement smart sensing technology into garments while maintaining these diverse design possibilities. After introducing the basic functionalities of capacitive sensing and the process of designing and building a smart garment, we present an assortment of garments enabled by this technology within the MoCa’Collection. Each of the projects serves a different purpose, built by people representing different backgrounds from electrical engineers, computer scientists, digital artists to smart fashion designers, starting from technical design over digital art to our latest design of a strongly design-oriented full-body capturing suit implementing the proposed technology.
Frame-Level Event Representation Learning for Semantic-Level Generation and Editing of Avatar Motion

Understanding an avatar’s motion and controlling its content is important for content creation and has been actively studied in computer vision and graphics. An avatar’s motion consists of frames representing poses each time, and a subsequence of frames can be grouped into a segment based on semantic meaning. To enable semantic-level control of motion, it is important to understand the semantic division of the avatar’s motion. We define a semantic division of avatar’s motion as an “event”, which switches only when the frame in the motion cannot be predicted from the previous frames and information of the last event, and tackled editing motion and inferring motion from text based on events. However, it is challenging because we need to obtain the event information, and control the content of motion based on the obtained event information. To overcome this challenge, we propose obtaining frame-level event representation from the pair of motion and text and using it to edit events in motion and predict motion from the text. Specifically, we learn a frame-level event representation by reconstructing the avatar’s motion from the corresponding frame-level event representation sequence while inferring the sequence from the text. By doing so, we can predict motion from the text. Also, since the event at each motion frame is represented with the corresponding event representation, we can edit events in motion by editing the corresponding event representation sequence. We evaluated our method on the HumanML3D dataset and demonstrated that our model can generate motion from the text while editing motion flexibly (e.g., allowing the change of the event duration, modification of the event characteristics, and the addition of new events).
Out of Sight,... How Asymmetry in Video-Conference Affects Social Interaction

As social-mediated interaction is becoming increasingly important and multi-modal, even expanding into virtual reality and physical telepresence with robotic avatars, new challenges emerge. For instance, video calls have become the norm and it is increasingly common that people experience a form of asymmetry, such as not being heard or seen by their communication partners online due to connection issues. Previous research has not yet extensively explored the effect on social interaction. In this study, 61 Dyads, i.e. 122 adults, played a quiz-like game using a video-conferencing platform and evaluated the quality of their social interaction by measuring five sub-scales of social presence. The Dyads had either symmetrical access to social cues (both only audio, or both audio and video) or asymmetrical access (one partner receiving only audio, the other audio and video). Our results showed that in the case of asymmetrical access, the party receiving more modalities, i.e. audio and video from the other, felt significantly less connected than their partner. We discuss these results in relation to the Media Richness Theory (MRT) and the Hyperpersonal Model: in asymmetry, more modalities or cues will not necessarily increase feeling socially connected, in opposition to what was predicted by MRT. We hypothesize that participants sending fewer cues compensate by increasing the richness of their expressions and that the interaction shifts towards an equivalent richness for both participants.
Serial correlation in the Italian futures market

We study the serial correlation of high-frequency intraday returns on the Italian stock index futures (FIB30) in the period 2000-2002. We adopt three different methods of analysis: the spectral density via Fast Fourier Transform, Detrended Fluctuation Analysis (DFA) and the Variance Ratio test. We find that intraday autocorrelation is mostly negative for time scales lower than 20 minutes, but we support the efficiency of the Italian futures market.
A Tailored Textile Sensor-based Wrap for Shoulder Complex Angles Monitoring

The shoulder joint plays a crucial role in the recovery of upper limb function. However, conventional wearable technologies employed for monitoring shoulder joint movements predominantly rely on inertial sensing units (IMUs), which may suffer alignment errors and compromise the freedom and wearability experienced by patients during their daily activities. This paper contributes in two facets, first, it presents the design, implementation, and technical evaluation of a new wearable system, a customized unilateral shoulder wrap that utilizes flexible and breathable textile sensors. Diverging from earlier studies, our system not only facilitates the monitoring of glenohumeral joint angles but also concurrently tracks the movement angles of the scapula. Secondly, to estimate joint angles, we propose a specific model called the Channel-Temporal Encoding Network (CTEN), which leverages Transformer and Long Short-Term Memory (LSTM) architectures. In a preliminary technical evaluation, the results demonstrate root mean square errors (RMSEs) of 2.24°and 1.13°for the glenohumeral joint and scapula, respectively. This study is intended to contribute to the development of more advanced wearables tailored for shoulder joint rehabilitation training.
BioSparks: Jewelry as Electrochemical Sweat Biosensors with Modular, Repurposing and Interchangeable Approaches

This paper presents BioSparks, a wearable device that detects glucose levels in sweat through electrochemical biosensors crafted with traditional jewelry techniques. Unlike conventional biosensors that are disposed of after use, BioSparks employs a repurposing method, allowing for the reuse of discarded electrodes within the jewelry’s chain, as pendants or earrings. It incorporates interchangeable electrodes that facilitates their replacement after timelife. The modular design enables the wearable to be placed on various body parts, including the neck, wrist and waist. The paper outlines our design considerations for Wearability Factors for Jewelry Biosensors, and the fabrication process combining traditional jewelry techniques and electromistry. Our technical evaluation shows the performance of our biosensor under ten different glucose concentrations.
Towards using Digital Phenotypes in Mobile Tactile Stimulation for Children with ASD

86% of children with Autism Spectrum Disorders (ASD) exhibit tactile impairments. Recent research has shown Mobile Tactile Stimulation is effective as a sensory processing therapy for children with ASD. The amount of time, duration, and intensity of such tactile stimulation depends on the appreciation of therapists and their interpretability based on experience and judgment. So, there is an untapped potential to propose an approach that can objectively quantify progress by leveraging the capabilities of Mobile Tactile Stimulation to gather continuous streams of data from touch interactions. In this position paper, we present a proposal to use Digital Phenotypes as an approach to adjust the “digital active ingredients” of tactile stimulation. We discuss our vision towards using Digital Phenotypes as a tool to follow up patient trajectories and create adaptive digital therapies. We close, discussing directions for future work.
Increasing Heart Rate and Anxiety Level with Vibrotactile and Audio Presentation of Fast Heartbeat

Heartbeat is not only one of our physical health indicators, but also plays an important role in our emotional changes. Previous investigations have been repeatedly investigated to the soothing effects of low frequency vibrotactile cues which evoke a slow heartbeat in stressful situations. The impact of stimuli which evoke faster heartbeats on users’ anxiety or heart rate is, however, poorly understood. We conducted two studies to evaluate the influence of the presentation of a fast heartbeat via vibration and/or sound, both in calm and stressed states. Results showed that the presentation of fast heartbeat stimuli can induce increased anxiety levels and heart rate. We use these results to inform how future designers could carefully present fast heartbeat stimuli in multimedia application to enhance feelings of immersion, effort and engagement.
EarBender: Enabling Rich IMU-based Natural Hand-to-Ear Interaction in Commodity Earables

Earables have been gaining popularity over the past few years for their ease of use and convenience over wired earphones. However, modern-day earables usually have a limited interface, inhibiting their potential as an accessible medium of input. To this end, we present EarBender: an ear-based real-time system that bridges the gap between earables and on-body interaction, providing a more diverse and natural form of interaction with devices. EarBender enables touch-based hand-to-ear gestures on mobile devices by leveraging inertial sensors in commercially available earable devices. Our proposed system detects the slight deformation in a user’s ear resulting from different ear-based actions including swiping and tapping and classifies the action performed. EarBender is designed to be energy-efficient, easy to deploy and robust to different users, requiring little to no calibration. We implement a prototype of EarBender using eSense, a multi-sensory earable platform, and evaluate it in different scenarios and parameter settings. Results show that the system can detect the occurrence of gestures with a 96.8% accuracy and classify seven different hand-to-ear gestures with an accuracy up to 97.4% maintained across four subjects.