Reviewerly/exHarmony huggingface dataset:
ROAD-R: the autonomous driving dataset with logical requirements

Neural networks have proven to be very powerful at computer vision tasks. However, they often exhibit unexpected behaviours, violating known requirements expressing background knowledge. This calls for models (i) able to learn from the requirements, and (ii) guaranteed to be compliant with the requirements themselves. Unfortunately, the development of such models is hampered by the lack of datasets equipped with formally specified requirements. In this paper, we introduce the ROad event Awareness Dataset with logical Requirements (ROAD-R), the first publicly available dataset for autonomous driving with requirements expressed as logical constraints. Given ROAD-R, we show that current state-of-the-art models often violate its logical constraints, and that it is possible to exploit them to create models that (i) have a better performance, and (ii) are guaranteed to be compliant with the requirements themselves.
Multi-Class Confidence Detection Using Deep Learning Approach

The advancement of both the fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has enabled the development of effective automatic systems for analyzing human behavior. It is possible to recognize gestures, which are frequently used by people to communicate information non-verbally, by studying hand movements. So, the main contribution of this research is the collected dataset, which is taken from open-source videos of the relevant subjects that contain actions that depict confidence levels. The dataset contains high-quality frames with minimal bias and less noise. Secondly, we have chosen the domain of confidence determination during social issues such as interviews, discussions, or criminal investigations. Thirdly, the proposed model is a combination of two high-performing models, i.e., CNN (GoogLeNet) and LSTM. GoogLeNet is the state-of-the-art architecture for hand detection and gesture recognition. LSTM prevents the loss of information by keeping temporal data. So the combination of these two outperformed during the training and testing process. This study presents a method to recognize different categories of Self-Efficacy by performing multi-class classification based on the current situation of hand movements using visual data processing and feature extraction. The proposed architecture pre-processes the sequence of images collected from different scenarios, including humans, and their quality frames are extracted. These frames are then processed to extract and analyze the features regarding their body joints and hand position and classify them into four different classes related to efficacy, i.e., confidence, cooperation, confusion, and uncomfortable. The features are extracted using a combination framework of customized Convolutional Neural Network (CNN) layers with Long Short-Term Memory (LSTM) for feature extraction and classification. Remarkable results have been achieved from this study representing 90.48% accuracy with effective recognition of human body gestures through deep learning approaches.
Data Acquisition for Estimating Energy-Efficient Solar-Powered Sensor Node Performance for Usage in Industrial IoT

In the era of rapid technological growth, we are facing increased energy consumption. The question of using renewable energy sources is also essential for the sustainability of wireless sensor networks and the Industrial Internet of Things, especially in scenarios where there is a need to deploy an extensive number of sensor nodes and smart devices in industrial environments. Because of that, this paper targets the problem of monitoring the operations of solar-powered wireless sensor nodes applicable for a variety of Industrial IoT environments, considering their required locations in outdoor scenarios and the efficient solar power harvesting effects. This paper proposes a distributed wireless sensor network system architecture based on open-source hardware and open-source software technologies to achieve that. The proposed architecture is designed for acquiring solar radiation data and other ambient parameters (solar panel and ambient temperature, light intensity, etc.). These data are collected primarily to define estimation techniques using nonlinear regression for predicting solar panel voltage outputs that can be used to achieve energy-efficient operations of solar-powered sensor nodes in outdoor Industrial IoT systems. Additionally, data can be used to analyze and monitor the influence of multiple ambient data on the efficiency of solar panels and, thus, powering sensor nodes. The architecture proposal considers the variety of required data and the transmission and storage of harvested data for further processing. The proposed architecture is implemented in the small-scale variants for evaluation and testing. The platform is further evaluated with the prototype sensor node for collecting solar panel voltage generation data with open-source hardware and low-cost components for designing such data acquisition nodes. The sensor node is evaluated in different scenarios with solar and artificial light conditions for the feasibility of the proposed architecture and justification of its usage. As a result of this research, the platform and the method for implementing estimation techniques for sensor nodes in various sensor and IoT networks, which helps to achieve edge intelligence, is established.
Biased-Infinity Laplacian Applied to Depth Completion Using a Balanced Anisotropic Metric

This paper studies an anisotropic interpolation model that can fill in-depth data in a largely empty region of a depth map. We suppose the image is endowed with an anisotropic metric $$g_{ij}$$ considering spatial and photometric information. The interpolation model is the biased Infinity Laplacian or biased Absolutely Minimizing Lipschitz Extension (bAMLE). Given an implementation based on the “eikonal” operator, we investigate different metrics to improve the performance of bAMLE in the depth completion task. To perform this completion, we suppose we have a reference color image and a depth map at our disposal. We used first a standard metric that considers spatial and photometric differences, second, the same metric but using different exponents in each term, and third a new metric that considers a balance term between intensities and gradients of the image. Using this new metric, we outperform our version of the model and other contemporary models in KITTI dataset.
TeleSteer: Combining Discrete and Continuous Locomotion Techniques in Virtual Reality

Steering and teleporting are two common locomotion techniques in virtual reality (VR). Steering generates a great sense of spatial awareness and immersion but tends to lead to cybersickness; teleporting performs better in mitigating cybersickness but may lead to the loss of spatial awareness. Hence, we combined these two techniques and designed TeleSteer. This technique allows users to perform both steering and teleporting and customize the control. We discuss that a combined use of discrete (e.g. teleporting) and continuous (e.g. steering) locomotion techniques is necessary for scenarios that require both free explorations and close-range interaction tasks, making TeleSteer a suitable alternative.
Towards Augmented Reality Guiding Systems: An Engineering Design of an Immersive System for Complex 3D Printing Repair Process

Over the past decade, additive manufacturing (AM) has become widely adopted for both prototyping and, more recently, end-use products. In particular, fused deposition modeling (FDM) is the most widespread form of additive manufacturing due to its low cost, ease of use, and versatility. While additive processes are relatively automated, many steps in their operation and repair require trained human operators. Finding such operators can be difficult, as highlighted during the recent COVID-19 pandemic. Augmented reality (AR) systems could significantly help address this challenge by automating the training for 3D printer operators. Given multidimensional design choices, however, a research gap exists in the system requirements for such immersive guidance. To address this need, we explore the applicability of AR to guide users through a repair process. In that context, we report on the system design as well as the results of the AR system assessment in a qualitative study with experts.
Redirected Walking Based on Historical User Walking Data

With redirected walking (RDW) technology, people can explore large virtual worlds in smaller physical spaces. RDW controls the trajectory of the user's walking in the physical space through subtle adjustments, so as to minimize the collision between the user and the physical space. Previous predictive algorithms place constraints on the user's path according to the spatial layouts of the virtual environment and work well when applicable, while reactive algorithms are more general for scenarios involving free exploration or uncon-strained movements. However, even in relatively free environments, we can predict the user's walking to a certain extent by analyzing the user's historical walking data, which can help the decision-making of reactive algorithms. This paper proposes a novel RDW method that improves the effect of real-time unrestricted RDW by analyzing and utilizing the user's historical walking data. In this method, the physical space is discretized by considering the user's location and orientation in the physical space. Using the weighted directed graph obtained from the user's historical walking data, we dynamically update the scores of different reachable poses in the physical space during the user's walking. We rank the scores and choose the optimal target position and orientation to guide the user to the best pose. Since simulation experiments have been shown to be effective in many previous RDW studies, we also provide a method to simulate user walking trajectories and generate a dataset. Experiments show that our method outperforms multiple state-of-the-art methods in various environments of different sizes and spatial layouts.
Towards Cross-Reality Interaction and Collaboration: A Comparative Study of Object Selection and Manipulation in Reality and Virtuality

Cross-Reality (CR) is an important topic for the research of multiuser collaborative systems. It allows users to participate in the reality-virtuality continuum and select appropriate interactive systems to work with, such as Virtual Reality Head-Mounted Displays (VR HMDs). However, there is limited work showing how interaction in VR differs from the more commonly used Personal Computers (PCs) and tablet devices in terms of object selection and manipulation. In this paper, we present a comparative study that investigated how users perform and perceive workload on 3D object selection and manipulation tasks using different devices (e.g. PC, tablet, and VR). We recorded the time and accuracy as objective task performance measures, and users' self-reported workload as a subjective measure. Our results revealed that unlike the biased performances of PC and tablet, VR has a balanced performance and great potentials in complex tasks.
Masked FER-2013: Augmented Dataset for Facial Expression Recognition

This poster introduces the Masked FER-2013 dataset that can be used to analyze facial emotions in people wearing masks and train a recognizer. The FER-2013 dataset containing face images annotated with seven emotions was modified by synthesizing mask images into the lower portion of the face. Based on the quantitative evaluation, our dataset can improve the accuracy of CNN, VGG, and ResNet models' emotion recognition in masked images by a maximum of 46%. Additionally, as a use case, we present an application that recognizes the emotions of a masked person, generates emotional face animation, and augments it on a real mask during video conferencing.
Does interpupillary distance (IPD) relate to immediate cybersickness?

Widespread adoption of virtual reality (VR) will likely be limited by the common occurrence of cybersickness. Cybersickness sus- ceptibility varies across individuals, and previous research reported that interpupillary distance (IPD) may be a factor. However, that work emphasized cybersickness recovery rather than cybersickness immediately after exposure. The current study (N=178) examined if the mismatch between the user's IPD and the VR headset's IPD setting contributes to immediate cybersickness. Multiple linear re-gression indicated that gender and prior sickness due to screens were significant predictors of immediate cybersickness. However, no significant relationship between IPD mismatch and immediate cybersickness was observed.
Investigation of Thermal Perception and Emotional Response in Augmented Reality using Digital Biomarkers: A Pilot Study

Dialectical behavior therapy (DBT) is an evidence-based psychotherapy that helps patients learn skills to regulate emotions as a central strategy to improve life functioning. However, DBT skills require a long-term and consistent commitment, typically to group therapy over the course of months. Patients who might benefit may find this approach undesirable; it can be challenging to transfer learning from therapy sessions to daily life, and there is no way to personalize skills learning based on individualized needs. In this paper we propose the use of Augmented Reality (AR) and digital biomarkers to enhance DBT skill exercises to be more immersive and personalized by using physiological data as real-time feedback. To explore the feasibility of AR-based DBT skill implementation, we developed AR-based DBT skill exercises that manipulate the user's thermal perception by visualizing different thermal information in holograms. We conducted a user study to evaluate the impact of AR in changing the thermal perception and emotional states of the user with an analysis of physiological data collected from wearable devices.
LiteVR: Interpretable and Lightweight Cybersickness Detection using Explainable AI

Cybersickness is a common ailment associated with virtual reality (VR) user experiences. Several automated methods exist based on machine learning (ML) and deep learning (DL) to detect cyber-sickness. However, most of these cybersickness detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone energy-constrained VR head-mounted devices (HMDs). In this work, we present an explainable artificial intelligence (XAI)-based framework Lite VR for cybersickness detection, explaining the model's outcome, reducing the feature dimensions, and overall computational costs. First, we develop three cybersick-ness DL models based on long-term short-term memory (LSTM), gated recurrent unit (GRU), and multilayer perceptron (MLP). Then, we employed a post-hoc explanation, such as SHapley Additive Explanations (SHAP), to explain the results and extract the most dominant features of cybersickness. Finally, we retrain the DL models with the reduced number of features. Our results show that eye-tracking features are the most dominant for cybersickness detection. Furthermore, based on the XAI-based feature ranking and dimensionality reduction, we significantly reduce the model's size by up to 4.3×, training time by up to 5.6×, and its inference time by up to 3.8×, with higher cybersickness detection accuracy and low regression error (i.e., on Fast Motion Scale (FMS)). Our proposed lite LSTM model obtained an accuracy of 94% in classifying cyber-sickness and regressing (i.e., FMS 1–10) with a Root Mean Square Error (RMSE) of 0.30, which outperforms the state-of-the-art. Our proposed Lite VR framework can help researchers and practitioners analyze, detect, and deploy their DL-based cybersickness detection models in standalone VR HMDs.
OMG: Towards Effective Graph Classification Against Label Noise

Graph classification is a fundamental problem with diverse applications in bioinformatics and chemistry. Due to the intricate procedures of manual annotations in graphical domains, there may be abundant noisy labels of graphs in practice, resulting in poor performance for existing supervised methods. Thus, it is necessary and urgent to study the problem of graph classification with label noise. However, this problem is challenging due to the overfitting of noisy data as well as complicated relational structures of graphs. To handle this problem, we present a simple but effective approach called c <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</u> upled <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> ix for <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> raph Contrast (OMG), which combines coupled Mixup with graph contrastive learning in the feature space. On the one hand, to improve the model generalization, we take convex combination of sample pairs in the feature space for positive pair construction. On the other hand, to accomplish effective optimization, we offer challenging negatives by multiple sample Mixup with different emphasis. To further reduce the impact of noisy data, we develop a neighbour-aware noise removal strategy, which promotes the smoothness in the neighbourhood of samples following the principle of curriculum learning. Extensive experiments on a range of benchmark datasets demonstrate the superiority of our proposed OMG.
A Large-Scale Photonic CNN Based on Spike Coding and Temporal Integration

The real-time massive-data intelligent information processing tasks highlight a vital requirement for the novel, intelligent optimization hardware. Convolutional neural networks are highly capable of extracting the hierarchical feature map and enhancing recognition accuracy, with photonics-enabled implementations drawing considerable interest. Here, we propose a large-scale and reconfigurable photonic convolutional neural network (PCNN), based on a hardware-friendly distributed feedback laser diode (DFB-LD). Our approach applies biological time-to-first-spike coding to a DFB-LD neuron to perform a temporal convolutional operation (TCO) for image processing tasks. In PCNN, experimental results demonstrate that we successfully implement the TCO to extract the image features with convolutional kernels of size 11 × 11. Furthermore, we investigate the temporal pulse shaping of a DFB-LD neuron to build a densely-connected fully connected layer, which synaptic weights can be rapidly adjusted at a rate of 5 GHz, achieving full MNIST and Fashion-MNIST benchmark image classification tasks, with classification accuracies of 98.56% and 87.48%, respectively. This work highlights the potential of neuron-like optical analog computing platforms for real-time and more complex intelligent processing networks.
Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization

Domain generalization (DG) aims to learn from multiple known source domains a model that can generalize well to unknown target domains. The existing DG methods usually exploit the fusion of shared multi-source data to train a generalizable model. However, tremendous data is distributed across lots of places nowadays that can not be shared due to privacy policies. In this paper, we tackle the problem of federated domain generalization where the source datasets can only be accessed and learned locally for privacy protection. We propose a novel framework called Collaborative Semantic Aggregation and Calibration (CSAC) to enable this challenging problem. To fully absorb multi-source semantic information while avoiding unsafe data fusion, we conduct data-free semantic aggregation by fusing the models trained on the separated domains layer-by-layer. To address the semantic dislocation problem caused by domain shift, we further design cross-layer semantic calibration with an attention mechanism to align each semantic level and enhance domain invariance. We unify multi-source semantic learning and alignment in a collaborative way by repeating the semantic aggregation and calibration alternately, keeping each dataset localized, and the data privacy is carefully protected. Extensive experiments show the significant performance of our method in addressing this challenging problem.
Robust Subspace Clustering by Logarithmic Hyperbolic Cosine Function

As an important category of clustering methods, subspace clustering algorithms have arisen particular attention during the last decade. Most subspace clustering algorithms are designed by first constructing a similarity matrix and then using spectral clustering algorithms to perform clustering. How to learn a suitable representation matrix to construct the similarity matrix is essential to the clustering performance. In most existing algorithms, the representation matrix is solved by norm-minimization, which commonly enforces the error matrix with nuclear norm or sparsity norm. However, these methods may fail to achieve satisfactory performance for real data contaminated by complex noise. To this end, we propose a novel robust subspace clustering method based on the Logarithmic Hyperbolic Cosine Function (LHCF). We theoretically analyze the grouping effect, as well as the convergence behavior, which illustrates that highly correlated samples can be grouped into the same cluster. Experimental results conducted on the Extended Yale B dataset show that the newly proposed algorithm yields better clustering performance compared with some advanced methods.
Treating Noise and Anomalies in Vehicle Trajectories From an Experiment With a Swarm of Drones

Unmanned aerial systems, known as "drones," are relatively new in collecting traffic data.Data from drone videography can have potential applications for traffic research.Drones can record the vehicles from their aerial point-of-view and provide their naturalistic driving behavior.Processing raw data from drones to remove noise and anomalies is crucial to ensure that the data are fit for subsequent applications, e.g., the development of traffic flow or crash risk models.This study uses a part of the pNEUMA dataset, a large dataset with almost half a million trajectories captured by a swarm of drones over Athens, Greece.This novel dataset offers an opportunity to analyze the data attributes and treat the noise and outliers in the data.We use a combination of smoothing filters and Extreme Gradient Boosting with adaptive regularization to process the speed and acceleration profiles of the vehicle trajectories in the dataset.Our approach can help prospective data users treat this or similar trajectory datasets alternatively to applying manual thresholds and assist in accelerating research in microscopic traffic analysis.
Fully Ferroelectric-FETs Reservoir Computing Network for Temporal and Random Signal Processing

Reservoir computing (RC), a derivation of recurrent neural networks (RNNs), is an energy-efficient computational framework suitable for temporal signal processing. Owing to the short-term and long-term memory capability, the ferroelectric field-effect transistor (FeFET) is regarded as a promising hardware component for implementing RC networks. This article aims to optimize the fully FeFETs RC network by evaluating the recognition accuracy in various classification tasks, which includes the operating voltage sequence, device numbers as well as connection methods. The physical random telegraph noise (RTN), working as an ideal temporal and random signal, is investigated and extended by using the optimized fully FeFET RC network, resulting in a rapid time constant extraction method. Our findings may provide the broad potential for hardware security and cyber security based on the fully FeFET RC network.