Reviewerly/exHarmony huggingface dataset:
Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

Observing a child’s interaction with their parents can provide us with important information about the child’s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child’s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.
Fusion of XLNet and BiLSTM-TextCNN for Weibo Sentiment Analysis in Spark Big Data Environment

This article proposes a Weibo sentiment analysis method to improve traditional algorithms' analysis efficiency and accuracy. The proposed algorithm uses deep learning in the Spark big data environment. First, the input data are converted into dynamic word vector representations using the Chinese version of the XLNet model. Then, dual-channel feature extraction is performed on the data using TextCNN and BiLSTM. The proposed algorithm uses an attention mechanism to allocate computing resources efficiently and realizes feature fusion and data classification. Comparative experiments are conducted on two public datasets under identical experimental conditions. In the NLPCC2014 and NLPCC2015 datasets, the proposed model improves the precision and F1 metrics by at least 4.26% and 2.64%, respectively. In the weibo_senti_100k dataset, the proposed model improves the precision and F1 metrics by at least 4.66% and 2.69%, respectively. The results indicate that the proposed method has better sentiment analysis and prediction abilities than existing methods.
The Limitations of Current Similarity-Based Objective Metrics in the Context of Human-Agent Interaction Applications

There are two main ways of evaluating a model generating an interactive virtual agent’s expressions. The first is through subjective perception tests, and the second is through objective metrics, which usually compare the model’s generated expressions to a test set of expressions considered the ground truth. In this work, we argue that using such objective metrics comparing generated expressions, to expressions contained in a test set limits the accuracy of the evaluation by failing to consider expressions that are different from the test set, but are still valid and well-perceived by users. We support this argument through experiments showing that different expression sequences are well perceived as listening responses to the same speaker’s utterance.
Instant Hand Redirection in Virtual Reality Through Electrical Muscle Stimulation-Triggered Eye Blinks

In this paper we investigate the use of electrical muscle stimulation (EMS) to trigger eye blinks for instant hand redirection in virtual reality (VR). With the rapid development of VR technology and increasing user expectations for realistic experiences, maintaining a seamless match between real and virtual objects becomes crucial for immersive interactions. However, hand movements are fast and sometimes unpredictable, increasing the need for instantaneous redirection. We introduce EMS to the field of hand redirection in VR through precise stimulation of the eyelid muscles. By exploiting the phenomenon of change blindness through natural eye blinks, our novel stimulation model achieves instantaneous, imperceptible hand redirection without the need for eye tracking. We first empirically validate the efficiency of our EMS model in eliciting full eye closure. In a second experiment, we demonstrate the feasibility of using such a technique for seamless instantaneous displacement in VR and its particular impact for hand redirection. Among other factors, our analysis also delves into the under-explored domain of gender influence on hand redirection techniques, revealing significant gender-based performance disparities.
Effects of symmetrical avatar arm movements on the sense of ownership of both hands inverted in a mirror

In this study, we investigated whether visual and tactile symmetrical stimuli in the arm affect the mirrored avatar’s sense of ownership of the hand. In the experiment, we tested the user, avatar, and mirrored avatar’s sense of ownership of their hand by catching a ball multiple times in a virtual space. The results suggest that non-ambidextrous individuals were significantly more likely to recognize the mirror-image avatar’s hand as their own in scenarios that included symmetrical visual and tactile stimuli than in scenarios without such stimuli. This experiment may contribute to a better understanding of the relationship between left-right sensation and body ownership sensation. It may contribute to the pursuit of immersive experiences in virtual environments.
A Pilot Study on the Impact of Discomfort Relief Measures on Virtual Reality Sickness and Immersion

While there are several theories of virtual reality (VR) sickness causes and pertinent methods suggested for mitigation, it remains an important problem. One possible solution might be to prescribe measures for just relieving the immediate symptoms (vs. addressing the very root causes). Understanding that the severity of the sickness may affect individuals differently, we examined three methods: (1) reducing the weight of the headset (using a suspension mechanism); (2) refreshing the user with a gentle breeze of wind (using a fan); (3) accompanying the VR viewing experience with mindful breathing. We assess the relative sickness reduction effect, if any, of these three measures through a comparative pilot experiment and individual case analysis. The preliminary results point to rather the importance of system usability and how it affects the relationship between the perceived immersion and the extent of sickness. The initial proposition to enhance the user’s physical condition as a way to better withstand VR sickness symptoms could not be established.
Comparing Mixed Reality Agent Representations: Studies in the Lab and in the Wild

Mixed-reality systems provide a number of different ways of representing users to each other in collaborative scenarios. There is an obvious tension between using media such as video for remote users compared to representations as avatars. This paper includes two experiments (total n = 80) on user trust when exposed to two of three different user representations in an immersive virtual reality environment that also acts as a simulation of typical augmented reality simulations: full body video, head and shoulder video and an animated 3D model. These representations acted as advisors in a trivia quiz. By evaluating trust through advisor selection and self-report, we found only minor differences between representations, but a strong effect of perceived advisor expertise. Unlike prior work, we did not find the 3D model scored poorly on trust, perhaps as a result of greater congruence within an immersive context.
DiffuGesture: Generating Human Gesture From Two-person Dialogue With Diffusion Models

This paper describes the DiffuGesture entry to the GENEA Challenge 2023. In this paper, we utilize conditional diffusion models to formulate the gesture generation problem. The DiffuGesture system generates human-like gestures from the two-person dialogue scenario, which are responsive to the interlocutor motions and accompany with the input speech. DiffuGesture system is built upon the recent DiffGesture [39]. Specifically, we introduce a lightweight transformer encoder to fuse the temporal relationships between human gestures and multi-modal conditions. Moreover, we adopt implicit classifier-free guidance to trade off between diversity and gesture quality. According to the collective evaluation released by GENEA Challenge 2023, our system demonstrates strong competitiveness in the appropriateness evaluation.
Guidelines for designing and building an automated multimodal textual annotation system

Automatic multimodal textual annotation systems have the potential to advance research in conversational analysis and computational linguistics; however, there is little research in this area. In this short paper, we outline guidelines for designing an automated multimodal textual annotation system. We discussed the guidelines using four guiding questions. First, we share the four possible use-cases for building an automated multimodal annotation system. Second, we share three guidelines for collecting multimodal data. Third, we share two guidelines on extracting multimodal features. Lastly, we share another four guidelines regarding what to present to the users.
GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos

Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.
Teaching Reform of Cultural and Creative Product Design Based on Virtual Reality (VR) Technology

Under the influence of visual culture environment, today's product design teaching will gradually change from the traditional theoretical method dominated by traditional classroom teachers to a more diversified and digital student practice mode. This study applies virtual reality technology to the teaching course of cultural and creative product design, and is committed to developing new possibilities for product design courses. Creating virtual design course environment with virtual reality equipment. Create an immersive design experience. Make up for the deficiency of traditional curriculum methods, and let students enjoy cultural and creative works from a richer and more diverse perspective in the virtual environment, so as to effectively cultivate the core literacy of the subject. It has become an inevitable trend for virtual reality technology to enter the traditional classroom. It is hoped that through this research, problems can be found, rules can be summarized, and practical experience of curriculum teaching can be provided for future “internet plus” and “VR+” education.
Comparative Performance and Energy Efficiency Analysis of JVM Variants and GraalVM in Java Applications

Java has dominated the ICT market for almost thirty years with various applications in nearly every sector all over the world. One of Java's main drawbacks comes from its heavyweight core - the Java Virtual Machine (JVM). Therefore, several JVM distributions have been developed to address this issue. GraalVM is the most promising amongst the recent distributions, providing better performance, low power consumption, and reduced carbon footprint emissions. In this research, a comparative analysis based on performance and energy efficiency metrics was conducted to assess this JVM distribution in light of three other classic JVM distributions: Amazon Corretto, Adopt OpenJDK, and Zulu. Findings showed that, although there was no significant difference between the test candidates, GraalVM seemed to be the leading JVM distribution. It is recommended that programmers and technology businesses consider adopting GraalVM in their future Java applications because of its energy efficiency.
A Methodology for Evaluating Multimodal Referring Expression Generation for Embodied Virtual Agents

Robust use of definite descriptions in a situated space often involves recourse to both verbal and non-verbal modalities. For IVAs, virtual agents designed to interact with humans, the ability to both recognize and generate non-verbal and verbal behavior is a critical capability. To assess how well an IVA is able to deploy multimodal behaviors, including language, gesture, and facial expressions, we propose a methodology to evaluate the agent’s capacity to generate object references in a situational context, using the domain of multimodal referring expressions as a use case. Our contributions include: 1) developing an embodied platform to collect human referring expressions while communicating with the IVA. 2) comparing human and machine-generated references in terms of evaluable properties using subjective and objective metrics. 3) reporting preliminary results from trials that aimed to check whether the agent can retrieve and disambiguate the object the human referred to, if the human has the ability to correct misunderstanding using language, deictic gesture, or both; and human ease of use while interacting with the agent.
Assessing Infant and Toddler Behaviors through Wearable Inertial Sensors: A Preliminary Investigation

Accurately assessing a child’s health status and taking appropriate action in case of illness or other emergencies is crucial to recording and sharing a childcare situation, such as sleeping hours, amount of exercise, and timing of meals. Although numerous applications and systems have been proposed to assist in recording and sharing these records, the process is still performed manually, representing a significant burden for parents. Therefore, automatic recording of infants’ and toddlers’ daily activities is required. Moreover, existing automatic infant behavior recognition methods have significant limitations on the space and target of application. In this study, we implement a machine learning model to investigate and propose a method to classify typical daily behaviors of infants and toddlers using a chest-mounted low-sampling rate accelerometer. In particular, the proposed method classifies eight activities: sleeping, crawling, walking, standing, sitting, drinking milk, eating baby food, and holding by a caregiver. As a dataset, we collected accelerometer data for nearly 18 hours with reference videos from ten infants and toddlers between 6 and 24 months. Based on the data, we extracted 60 time- and frequency-domain features calculated from the single accelerometer and a user feature to recognize the target daily activities. The performance evaluation considering different window sizes and machine learning models showed that our classification model reaches nearly 80% accuracy in the best-case scenario.
Duplicate Detection of Technical Documents in Petroleum Context Based on Near-Duplicate Image Search

Abstract Identifying technical document duplications is a crucial task to optimize storage resources and improve document management and retrieval processes. This helps to ensure that the most up-to-date version of documents is going to be available, which is important in all industrial domains including the petroleum sector to avoid errors and inconsistencies. In addition, deduplication is a key step to consider in data pre-processing for training large machine learning models. This paper tackles the task of duplicate detection in technical reports by proposing a hybrid solution based on open-source libraries on computer vision. The proposed solution has been evaluated on a large set of data and provided promising results which can help us to optimize our storage. We observed the solution is flexible enough to adapt to different scenarios and different types of reports. Having such a solution will potentially lead to reducing the carbon footprint in IT infrastructures.
The linkage between different learning contexts

The article analyzes the importance of the linkage in the pedagogical process of the different learning contexts. The role of the teacher is highlighted when conceiving and designing a classroom environment. Among the aspects that are addressed are the different contexts from a comprehensive perspective, which allow understanding the whole environment that surrounds and influences the student learning.&#x0D; Keywords:&#x0D; Education, skills, learning contexts.&#x0D; &#x0D; RESUMEN &#x0D; En el artículo se analiza la importancia de la vinculación en el proceso pedagógico de los distintos contextos de aprendizaje. Se destaca el papel del docente en el momento de concebir y diseñar un ambiente en el aula. Entre los aspectos que se abordan se encuentran los diferentes contextos desde una mirada integral, que permitan conocer todo el ambiente que rodea e influye en el aprendizaje de los estudiantes.&#x0D; Palabras clave: &#x0D; Educación, habilidades, contextos de aprendizaje.
Visual Intrusion as Nuisance: The View from the Top

In Fearn v Board of Trustees of the Tate Gallery (Fearn), the UK Supreme Court unanimously held that visual intrusions are in principle actionable under the tort of private nuisance. On the facts, a narrow 3:2 majority found that the Tate Modern was liable for the operation of its viewing gallery where the public could see into the claimants’ flats. This note argues that the court’s landmark determination on the scope of nuisance is a welcome one insofar as it decentres physical interference and aligns the operation of the tort with its normative underpinnings. More dubious is the majority’s emphasis on the ‘common and ordinary use’ of land, an uncertain concept which is likely to generate difficulties in future cases. Finally, the note considers some of the broader implications of Fearn, reflecting on the public reaction to the decision and considering what the case means for privacy protection in particular.
Cyclic Transfer Learning for Mandarin-English Code-Switching Speech Recognition

Transfer learning is a common method to improve the performance of the model on a target task via pre-training the model on pretext tasks. Different from the methods using monolingual corpora for pre-training, in this study, we propose a Cyclic Transfer Learning method (CTL) that utilizes both code-switching (CS) and monolingual speech resources as the pretext tasks. Moreover, the model in our approach is always alternately learned among these tasks. This helps our model can improve its performance via maintaining CS features during transferring knowledge. The experiment results on the standard SEAME Mandarin-English CS corpus have shown that our proposed CTL approach achieves the best performance with Mixed Error Rate (MER) of 16.3% on test <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{man}$</tex-math></inline-formula> , 24.1% on test <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{sge}$</tex-math></inline-formula> . In comparison to the baseline model that was pre-trained with monolingual data, our CTL method achieves 11.4% and 8.7% relative MER reduction on the test <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{man}$</tex-math></inline-formula> and test <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{sge}$</tex-math></inline-formula> sets, respectively. Besides, the CTL approach also outperforms compared to other state-of-the-art methods. The source code of the CTL method can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/caohongnga/CTL-CSSR</uri> .
The effect of model uncertainties in the reinforcement learning based regulation problem: An experimental case study with inverted pendulum

Abstract This work aims to improve the classical white‐box model of an inverted pendulum in order to reach a more accurate representation of an actual pendulum on a cart system. The purpose of the model is to train different controllers based on machine learning algorithms. In the context of this paper, the inverted pendulum system is driven by a belt drive that is controlled by a stepper motor. Due to the nature of the controller, the input to the stepper motor is in the form of a non‐smooth bang‐bang‐like signal that moves the cart to the left, right, or terminates its movement. One of the main challenges, in this case, is to find a proper function to model the stepper motor as its dynamics cannot be captured with a constant gain. It has been shown that the transient behavior of the stepper motor when changing direction or stopping is not negligible in the closed‐loop control performance. Accordingly, a grey‐box scheme, which accounts for the uncertainties that are not included in the vanilla white‐box model, is utilized to achieve a lower model mismatch compared to the actual pendulum. Initially, the equation of motion was derived using the Euler‐Lagrange equation with force on the cart as the control input. But in the real‐time experiment, the interface is realized by the stepper motor's frequency modulator, hence a transfer function representing the relationship between the frequency and the force applied on the cart (in the model) is calculated as a black‐box model. To improve the accuracy of the transfer function, an experimental data‐driven design of this function is performed based on modern schemes in system identification. For this purpose, the applied frequency to the stepper motor and the states from the actual system are recorded. Then, the applied force on the cart is calculated using the equation of motion and the recorded states. It is also shown that the frequency‐force transfer function uncertainty due to exogenous disturbances is non‐negligible and with the aim of having a more accurate model, an artificial neural network is introduced. Finally, the effectiveness of this grey‐box model is shown by training and implementing a deep Q‐network based controller to swing up and balance the inverted pendulum.
Efficient Blockchain-Based Data Integrity Auditing for Multi-Copy in Decentralized Storage

As the disruptor of cloud storage, decentralized storage could lead to a major shift in how organizations store data in the future. To ensure data availability, users generally encrypt the data and distribute it to multiple storage service providers. It is necessary to study data integrity verification in decentralized storage. Although some recent studies have proposed the using blockchain technology to assist auditing work in decentralized storage networks, the on-chain overhead still increases linearly with an increase in audit requests. Blockchain networks will inevitably be overloaded. In this study, we propose an efficient data integrity auditing scheme for multiple copies in decentralized storage. Particularly, using different polynomial commitment schemes, we first propose a basic scheme for verifying multiple copies of a single file, and then we propose an efficient batch auditing scheme for multiple copies of multiple files. Our scheme can significantly reduce the computation overhead of storage service providers while keeping the on-chain storage overhead constant. Security analysis and performance analysis show that our scheme is efficient and practical.