Miracle-dz/UniFAR-FTUs huggingface dataset:
Automatic, Manual, or Hybrid? A Preliminary Investigation of Users' Perception of Features for Supporting Notification Management

Mobile notifications, crucial to our daily activities and informational needs, are often undermined by insufficient management features. Previous research has suggested enhancements like automatic sorting, filtering, and categorization, but empirical evidence supporting these strategies is yet to be seen. This study bridges the gap, developing an Android application to assess these proposed features’ efficacy in improving notification management efficiency and user experience. We utilized the Experience Sampling Method (ESM) for in-depth user insights, and our preliminary findings indicate a perceived superiority for a hybrid system combining automatic and manual functionalities, over systems solely dependent on either approach. This research paves the way for an optimized notification system, better equipped to assist users in managing mobile notifications effectively.
Annotations from speech and heart rate: impact on multimodal emotion recognition

The focus of multimodal emotion recognition has often been on the analysis of several fusion strategies. However, little attention has been paid to the effect of emotional cues, such as physiological and audio cues, on external annotations used to generate the Ground Truths (GTs). In our study, we analyze this effect by collecting six continuous arousal annotations for three groups of emotional cues: speech only, heartbeat sound only and their combination. Our results indicate significant differences between the three groups of annotations, thus giving three distinct cue-specific GTs. The relevance of these GTs is estimated by training multimodal machine learning models to regress speech, heart rate and their multimodal fusion on arousal. Our analysis shows that a cue(s)-specific GT is better predicted by the corresponding modality(s). In addition, the fusion of several emotional cues for the definition of GTs allows to reach a similar performance for both unimodal models and multimodal fusion. In conclusion, our results indicates that heart rate is an efficient cue for the generation of a physiological GT; and that combining several emotional cues for GTs generation is as important as performing input multimodal fusion for emotion prediction.
Mazi Umntanakho “Know Your Child”: An Accessible Social-Emotional Assessment Tool for Children in Low-Income South African Communities

Ensuring proper developmental and behavioral assessments for children is crucial, particularly in contexts where resources are scarce, developmental delays carry a stigma, and specialists are lacking. However, adequate and culturally appropriate methods for screening and assessing children in the Majority World still need to be improved. In this context, mobile technology can be crucial in providing accessible assessments and screenings that better fulfill the needs. In this work, we present the design and development of Mazi Umntanakho "Know Your Child," a conversational agent on WhatsApp that supports South African home visitors in assessing and tracking children’s socio-emotional skills. The agent is based on the Strengths and Difficulties Questionnaire (SDQ) and International Development and Early Learning Assessment (IDELA) to provide culturally appropriate assessments. The agent is personalized to user preferences and language and provides feedback and advice in various multimedia formats, including infographics, video, audio, and photos.
Machine Learning-based Multi-Class Traffic Management for Smart Grid Communication Network

Smart grid communication networks are facing the increasing challenge of heterogeneous facilities and diverse communication requirements. Traditional communication technologies without the ability to adapt cannot meet these requirements. A data-driven and machine learning-based multi-class traffic management scheme is proposed in this study, which classifies network traffic into different service levels for better transmission provision. Numerical experiments with a real-world dataset are conducted to validate the effectiveness of the multi-class traffic management scheme, in which XGBoost achieves an accuracy of 0.9842 and an F1 score of 0.9914.
Departure Time Prediction Using Smartphone Data for Delayed-Full Charging BMS Algorithm

Battery degradation, a gradual loss of capacity and performance due to frequent charging and discharging cycles, is a significant challenge to the widespread adoption of electric vehicles (EVs). This study proposes a BMS algorithm that delays full charging under selective conditions and completes charging immediately just before use to reduce battery degradation rate caused by fully charged state time. Our goal is to predict the charging end time based on an individual’s departure time by capturing digital behavioral markers extracted from smartphone data, while minimizing reduction in driving range due to undesired predictions. Preliminary experiment was conducted with 41 subjects to assess the feasibility of the proposed approach. Our results demonstrate that the mobile passive features are capable of learning the departure behavior pattern, achieving an average mean absolute error (MAE) of 0.2336.
On Head Motion for Recognizing Aggression and Negative Affect during Speaking and Listening

Affective aggression is a form of aggression characterized by impulsive reactions driven by strong negative emotions. Despite the extensive research in the area of automatic emotion recognition, affective aggression is a phenomenon that has received less attention. This study investigates the use of head motion as a potential indicator of affective aggression and negative affect. It provides an analysis of head movement patterns associated with various levels of aggression, valence, arousal and dominance, and compares behaviors and recognition performance under speaking and listening conditions. The study was conducted on the Negative Affect and Aggression database - a multimodal corpus of dyadic interactions between aggression regulation training actors and non-actors, annotated for levels of aggression, valence, arousal, and dominance. Results demonstrate that head motion features can serve as promising indicators of affect during both speaking and listening. Valence and arousal prediction achieved better performance during speaking, while aggression and dominance were better predicted during listening. Significant increases in the magnitude of pitch angular acceleration were associated with escalation along all four annotated dimensions. Interestingly, higher escalation was accompanied by a significant increase in the total number of movements during speaking, but a significant decrease of the number of movements was observed as escalation increased along listening intervals. These findings are particularly relevant as head motion can be used solely or potentially as a supplementary modality when other modalities such as speech or facial expressions are unavailable or altered.
Smartwatch-Based Sensing Framework for Continuous Data Collection: Design and Implementation

Smartwatches are an increasingly popular technology that employs advanced sensors (e.g., location, motion, and microphone) comparable to those used by smartphones. Passive mobile sensing, a method of acquiring human behavior data from mobile and wearable devices inconspicuously, is widely used in research fields related to behavior analysis. In combination with machine learning, passive mobile sensing can be used to interpret various human and environmental contexts without requiring user intervention. Because smartwatches are always worn on the wrist, they have the potential to collect data that cannot be collected by smartphones. However, the effective use of smartwatches as platforms for passive mobile sensing poses challenges in terms of battery life, storage, and communication. To address these challenges, we designed and implemented a tailored framework for off-the-shelf smartwatches. We evaluated power consumption under eight different sensing conditions using three smartwatches. The results demonstrate that the framework can collect sensor data with a battery life of 16-31 h depending on the settings. Finally, we considered potential future solutions for optimizing power consumption in passive sensing with off-the-shelf smartwatches.
User-Independent Motion and Location Analysis for Sussex-Huawei Locomotion Data

Transportation mode detection (TMD) is a context-aware computing technology with significant potential in several applications. However, the development of TMD technologies for real-world scenarios remains challenging, including user-independent evaluations and multimodal analyses. In this study, our team (HYU-CSE) suggested a TMD model as part of the Sussex-Huawei Locomotion (SHL) recognition challenge, and we used the SHL motion and location data. The proposed TMD model was based on the DenseNet architecture, and post-processing using voting schemes was applied to refine the detection performance. The results suggested that the proposed method achieved 94.13% of an F1 score with user-independent analysis. We hope that our study will ultimately help in the design of better TMD applications.
Unveiling Privacy Measures in Mental Health Applications

Mental health conditions have become a global public health issue, especially in the context of the COVID-19 pandemic. To cope with the increasing demand for mental health services, many people have turned to smartphone applications that offer various mental health solutions, such as therapy, counseling, and self-help. However, these applications also pose significant privacy risks for their users, as they collect and share sensitive personal and health information with third parties, often without adequate consent or transparency. In this study, we examine the privacy policies of popular mental health smartphone applications using the Fair Information Practice Principles (FIPPs), a widely recognized privacy framework. Our objective is to assess the extent to which these applications adhere to the FIPPs guidelines and to identify the gaps and challenges in their privacy practices. We hope that our findings can inform and guide policy makers and application developers to design more user-centric and robust privacy policies that ensure the safety and security of users’ information.
Computational analyses of linguistic features with schizophrenic and autistic traits along with formal thought disorders

Formal Thought Disorder (FTD), which is a group of symptoms in cognition that affects language and thought, can be observed through language. FTD is seen across such developmental or psychiatric disorders as Autism Spectrum Disorder (ASD) or Schizophrenia, and its related Schizotypal Personality Disorder (SPD). Researchers have worked on computational analyses for the early detection of such symptoms and to develop better treatments more than 40 years. This paper collected a Japanese audio-report dataset with score labels related to ASD and SPD through a crowd-sourcing service from the general population. We measured language characteristics with the 2nd edition of the Social Responsiveness Scale (SRS2) and the Schizotypal Personality Questionnaire (SPQ), including an odd speech subscale from SPQ to quantize the FTD symptoms. We investigated the following four research questions through machine-learning-based score predictions: (RQ1) How are schizotypal and autistic measures correlated? (RQ2) What is the most suitable task to elicit FTD symptoms? (RQ3) Does the length of speech affect the elicitation of FTD symptoms? (RQ4) Which features are critical for capturing FTD symptoms? We confirmed that an FTD-related subscale, odd speech, was significantly correlated with both the total SPQ and SRS scores, although they themselves were not correlated significantly. In terms of the tasks, our result identified the effectiveness of FTD elicitation by the most negative memory. Furthermore, we confirmed that longer speech elicited more FTD symptoms as the increased score prediction performance of an FTD-related subscale odd speech from SPQ. Our ablation study confirmed the importance of function words and both the abstract and temporal features for FTD-related odd speech estimation. In contrast, embedding-based features were effective only in the SRS predictions, and content words were effective only in the SPQ predictions, a result that implies the differences of SPD-like and ASD-like symptoms. Data and programs used in this paper can be found here: https://sites.google. com/view/sagatake/resource.
Video-based Respiratory Waveform Estimation in Dialogue: A Novel Task and Dataset for Human-Machine Interaction

Respiration is closely related to speech, so respiratory information is useful for improving human-machine multimodal spoken interaction from various perspectives. A machine-learning task is presented for multimodal interactive systems to improve the compatibility of the systems and promote smooth interaction with them. This “video-based respiration waveform estimation (VRWE)” task consists of two subtasks: waveform amplitude estimation and waveform gradient estimation. A dataset consisting of respiratory data for 30 participants was created for this task, and a strong baseline method based on 3DCNN-ConvLSTM was evaluated on the dataset. Finally, VRWE, especially gradient estimation, was shown to be effective in predicting user voice activity after 200 ms. These results suggest that VRWE is effective for improving human-machine multimodal interaction.
EmotiW 2023: Emotion Recognition in the Wild Challenge

This paper describes the 9th Emotion Recognition in the Wild (EmotiW) challenge, which is being run as a grand challenge at the 25th ACM International Conference on Multimodal Interaction 2023. EmotiW challenge focuses on affect related benchmarking tasks and comprises of two sub-challenges: a) User Engagement Prediction in the Wild, and b) Audio-Visual Group-based Emotion Recognition. The purpose of this challenge is to provide a common platform for researchers from diverse domains. The objective is to promote the development and assessment of methods, which can predict engagement levels and/or identify perceived emotional well-being of a group of individuals in real-world circumstances. We describe the datasets, the challenge protocols and the accompanying sub-challenge.
Reconfigurable, Adhesive-Free, Wearable Skin Strain Device

Skin-strain – the act of stretching the skin – is an interesting (but generally understudied) mode of haptic stimulation. Producing artificial skin strain on the body using a wearable device requires a clear understanding of the body product relationship between the device and the user. We present a wearable device capable of creating dozens of unique skin strain experiences on the wearer through a novel shape memory alloy (SMA) actuator + reconfigurable hook-and-eye attachment architecture. Not only can this architecture create spatially- and temporally-customizable skin strain experiences, it does so without the use of temporary / permanent adhesives (a typical limitation of other skin strain device designs). We present the iterative design process from early working prototypes to the most recently developed devices, including the underlying design criteria and decisions
A weak coupling mechanism mediates the recovery stroke of myosin VI: a free energy simulation and string method analysis

Myosin motors use the energy of ATP to produce force and directed movement on actin by a swing of the lever arm. ATP is hydrolysed during the off-actin re-priming transition termed recovery stroke. To provide an understanding of chemo-mechanical transduction by myosin, it is critical to determine how the reverse swing of the lever arm and ATP hydrolysis are coupled. Previous studies concluded that the recovery stroke of myosin II is initiated by closure of the Switch II loop in the nucleotide-binding site. Recently, we proposed that the recovery stroke of myosin VI starts with the spontaneous re-priming of the converter domain to a putative pre-transition state (PTS) intermediate that precedes Switch II closing and ATPase activation. Here, we investigate the transition from the pre-recovery, post-rigor (PR) state to PTS in myosin VI using geometric free energy simulations and the string method. First, our calculations rediscover the PTS state agnostically and show that it is accessible from PR via a low free energy transition path. Second, separate path calculations using the string method illuminate the mechanism of the PR to PTS transition with atomic resolution. In this mechanism, the initiating event is a large movement of the converter/lever-arm region that triggers rearrangements in the Relay-SH1 region and the formation of the kink in the Relay helix with no coupling to the active site. Analysis of the free-energy barriers along the path suggests that the converter-initiated mechanism is much faster that the one initiated by Switch II closure, which supports the biological relevance of PTS as a major on-pathway intermediate of the recovery stroke in myosin VI. Our analysis suggests that lever-arm re-priming and ATP hydrolysis are only weakly coupled, so that the myosin recovery stroke is mostly driven by thermal fluctuations and stabilised by nucleotide consumption via a ratchet-like mechanism. Author summary Myosin is an ATP-powered motor protein that is crucial for cellular functions like muscle contraction, cell division, and the transport of molecular cargos. Understanding how myosin transforms chemical energy from ATP hydrolysis into mechanical work is a central open question in structural bioenergetics, which could guide the design of next-generation synthetic nanomachines. In myosin, ATP hydrolysis is coupled to the re-priming of the lever-arm during the recovery stroke transition. Using advanced molecular dynamics simulations, we described the sequence of events and the energetics of the recovery stroke of myosin VI with an unprecedented level of detail. The results support a mechanism in which the re-priming of the mechanical amplifier region of the protein is almost entirely driven by thermal fluctuations and stabilized by ATP hydrolysis. This weakly-coupled mechanism suggests that myosin can “rectify” thermal fluctuations to work efficiently in an isothermal environment dominated by stochastic fluctuations like the cell.
A Retina-inspired Optoelectronic Synapse Using Quantum Dots for Neuromorphic Photostimulation of Neurons

Neuromorphic electronics, inspired by the functions of neurons, have the potential to enable biomimetic communication with cells. Such systems require operation in aqueous environments, generation of sufficient levels of ionic currents for neurostimulation, and plasticity. However, their implementation requires a combination of separate devices, such as sensors, organic synaptic transistors, and stimulation electrodes. Here, we present a compact neuromorphic synapse that combines photodetection, memory, and neurostimulation functionalities all-in-one. The artificial photoreception is facilitated by a photovoltaic device based on cell-interfacing InP/ZnS quantum dots, which induces photo-faradaic charge-transfer mediated plasticity. The device sends excitatory post-synaptic currents exhibiting paired-pulse facilitation and post-tetanic potentiation to the hippocampal neurons via the biohybrid synapse. The electrophysiological recordings indicate modulation of the probability of action potential firing due to biomimetic temporal summation of excitatory post-synaptic currents. Our results pave the way for the development of novel bioinspired neuroprosthetics and soft robotics and highlight the potential of quantum dots for achieving versatile neuromorphic functionality in aqueous environments.
Technical note: Relief effects for passive microwave remote sensing

The signal of a microwave radiometer observing a land surface from space is composed of surface and atmospheric contributions, both of which depend on the relief. For proper interpretation of the data these effects should be quantified and, if necessary, taken into account. Relief effects are twofold. First, the path through the atmosphere between the surface and the sensor depends on the altitude of the emitting surface, thus leading to a height-dependent atmospheric influence. The effect can be taken into account by standard atmospheric radiative transfer models if the elevation of the surface and the atmospheric state are known. Second, more relevant for the present discussion is the variable topography of land surfaces, consisting of slopes, ridges and valleys, sometimes with characteristic alignments, and surfaces surrounded by elevated terrain. These surfaces interact radiatively, not only with the atmosphere, but also with each other, leading to the tendency to enhance the effective emission. Under such circumstances, deviations occur from the standard hemispheric emission of a horizontal surface. The interactions do not only depend on topography and emissivity, but also on the bistatic scattering behaviour. Special attention will be paid to the radiation enhancement in a landscape of Lambertian surfaces with elevated horizons. As an example, simulated data for southern Norway are shown.
To Broadcast or Not to Broadcast: Decision-Making Strategies for Mining Empty Blocks

Resource optimization in blockchain systems is a critical aspect of their architectural design. Despite frequent network congestion in Ethereum, a notable proportion of block space is underutilized, with occurrences of completely unused blocks exacerbating resource inefficiency in the network. This study investigates the motivations behind miners’ production of empty blocks. It is found that the immediate gains from mining empty blocks often outweigh the potential benefits derived from including transactions. Furthermore, our analysis indicates a substantial decrease in the frequency of empty blocks following Ethereum’s transition at the Merge, underscoring the effectiveness of the Proof-of-Stake (PoS) consensus mechanism in improving block space utilization in blockchain environments.