Reviewerly/exHarmony huggingface dataset:
Enhancing Physical Rehabilitation Monitoring through Multi-View Fusion of Human Body Skeleton

The Kinect has gained popularity in physical therapy and rehabilitation systems as a skeleton body tracking device. However, limitations such as occlusion and noise in skeleton tracking have reduced its reliability. To address these issues, some studies have proposed multi-kinect systems to expand the field of vision and enhance data precision. In this paper, we introduce the design of a 360° multi-view fused human body skeleton system using Kinect devices. Our system harnesses the strengths of the Kinect's Software Development Kit (SDK) and the spatial arrangement of the Kinect devices to estimate the 3D position coordinates of human skeleton joints. Importantly, the system allows for easy customization to accommodate modifications. Our system enables the extraction of indicators related to human walking patterns within the context of physical rehabilitation. These indicators serve to motivate patients, provide valuable feedback, and track their rehabilitation progress. The system demonstrated its utility in initial laboratory tests for physical therapy and rehabilitation monitoring. Our research presents a novel approach to address the limitations of the Kinect in physical therapy and rehabilitation. The preliminary findings from laboratory tests indicate the system's potential to enhance physical therapy and rehabilitation monitoring.
Empirical Evaluation of Maintainability Index Accuracy for Measuring Maintainability of Python Programs*

The Maintainability Index is a well-known software metric that is commonly used to measure the maintainability of software. However, it has some potential issues due to its origin and components. In this paper, we empirically evaluate the accuracy of the maintainability index as a measure of maintainability for Python programs. We evaluated whether there is a correlation between the maintainability index and the cohesion metric for the classes in the subject programs. We also evaluated the effect of the module size on the accuracy of the maintainability index.
FPGA Implementation of Real-Time Video Watermarking Prototype Using PL-PS Embedded Processor Over Network Communication

This paper implements on FPGA a point-to-point video transmission prototype between an FPGA board and a PC as first step using Petalinux based system over an embedded processor in a hardware/software codesign approach, generalized later to the transmission between two ZedBoard FPGA plat-forms using TCP/IP ethernet protocol. The prototype hardware implementation is exploited to implement spatial watermarking for both visible and invisible techniques using additive and Least Significant Bit (LSB) methods respectively. The Prototype performances are evaluated in terms of video transmission, watermarking and hardware implementation complexity.
Exploring the Potential of Blockchain in Internet of Robotic Things: Advancements, Challenges, and Future Directions

In recent years, the concept of Internet of Robotic Things (IoRT) has gained significant attention, driven by the convergence of robotics, cloud computing, and diverse IoT devices and services. IoRT systems enable the autonomous exchange of data and decision-making within virtual infrastructures. However, managing and controlling diverse robotics applications that interact with IoT devices poses a major challenge in IoRT systems. The conventional centralized approach falls short in providing adequate security and privacy for the IoRT network, particularly due to the mobility of robots and the limited computational and energy resources of IoT devices. To address these limitations, this paper explores the integration of blockchain technology into IoRT networks. Blockchain, known for its distributed ledger based on peer-to-peer communication and trustworthy operations, offers a promising solution. This study provides a comprehensive overview of this integration, starting with an examination of the fundamental characteristics and challenges faced by IoRT networks. It then presents a survey of blockchain technology, highlighting its diverse applications in IoRT systems. Based on our findings, we will propose a new architecture for IoRT that utilizes Blockchain technology to address security and privacy concerns and enhance the overall performance of the network.
Thermal Sensation with Microheater for Virtual Reality Applications: Preliminary Results

In this work, we followed an energy-saving approach to design a micro-heater device for virtual reality (VR) applications, using a simple, low-cost step-lithography technique. First, spiral shapes of the Al-0.5Cu-based micro-heaters are designed and fabricated with a surface area of 5 mm x 5 mm. We, then, tested and evaluated the Joule heating characteristics of the proposed micro-heater using the proposed spiral design patterns. The fabricated micro-heaters are, then, used to realize a hand-worn VR device with thermal feedback. We present, in this paper, preliminary results of a wearable device with one micro-heater mounted on the user's index fingertip. The performance of the proposed system is verified under a VR application including touching three glasses with different water temperatures. The results denote that the geometry of the micro-heater enabled the sensation of temperature to be reached with low energy consumption. The proposed micro-structure provides researchers with the possibility to design more compact and lightweight devices for thermal sensation generation in VR applications.
Real-Time Human Action Representation Based on 2D Skeleton Joints

We present a new approach to capture human actions by using 2D skeletal joints as the foundation for the real-time representation. Our approach combines three distinct types of information, namely: (1) motion detection to identify salient regions within the action. This allows us to compute joint contribution ratios and save processing time by excluding still joints and focusing on the main joints involved in the action; (2) utilization of a predefined map for joints trajectory shapes, which encodes the temporal information and avoids noisy data using the map; and (3) incorporation of a direction map that captures the movement of the joints, in the spatial space. By integrating these elements, we have devised a comprehensive representation capable of discerning even highly similar actions. To evaluate the effectiveness of our proposed representation, we conducted experiments on the UTD-MHAD dataset [25], which encompasses a diverse range of 27 actions performed by 8 subjects (4 females and 4 males), each with 4 repetitions. The evaluation results exhibit a notable dissimilarity within the same action category (intra-class) and a significant similarity across different action categories (inter-class). Specifically, our approach achieved a commendable score of 94.81 on the UTD-MHAD dataset, thus demonstrating its efficacy and robustness.
YOLOv7-Face: A Real-Time Face Detector

Face detection plays a crucial role and serves as a fundamental requirement in several research areas, such as face recognition, face identification, and facial expression analysis, among others. Over the past few years, there has been a rapid growth in deep learning-based algorithms for object detection. These algorithms can be broadly classified into two categories: two-stage detectors, such as Faster R-CNN, and one-stage detectors, such as YOLO. Existing object detection methods aim to strike a balance between speed and accuracy. However, the YOLOv7 algorithm, known for its popularity in object detection, offers clear advantages in terms of both speed and accuracy. In this study, we introduced YOLOv7 as a novel proposed solution for face detection. The proposed method was trained and evaluated on the LFW dataset. The experimental results demonstrate that the face detection method based on YOLOv7 exhibits exceptional performance in terms of both speed and accuracy, achieving comparable results to existing state-of-the-art methods.
“If It’s Not Me It Doesn’t Make a Difference” - The Impact of Avatar Personalization on user Experience and Body Awareness in Virtual Reality

Body awareness is relevant for the efficacy of psychotherapy. However, previous work on virtual reality (VR) and avatar-assisted therapy has often overlooked it. We investigated the effect of avatar individualization on body awareness in the context of VR-specific user experience, including sense of embodiment (SoE), plausibility, and sense of presence (SoP). In a between-subject design, 86 participants embodied three avatar types and engaged in VR movement exercises. The avatars were (1) generic and gender-matched, (2) customized from a set of pre-existing options, or (3) personalized photorealistic scans. Compared to the other conditions, participants with personalized avatars reported increased SoE, yet higher eeriness and reduced body awareness. Further, SoE and SoP positively correlated with body awareness across conditions. Our results indicate that VR user experience and body awareness do not always dovetail and do not necessarily predict each other. Future research should work towards a balance between body awareness and SoE.
Point &amp; Portal: A New Action at a Distance Technique For Virtual Reality

This paper introduces Point & Portal, a novel Virtual Reality (VR) interaction technique, inspired by Point & Teleport. This new technique enables users to configure portals using pointing actions, and supports seamless action at a distance and navigation without requiring line of sight. By supporting multiple portals, Point & Portalenables users to create dynamic portal configurations to manage multiple remote tasks. Additionally, this paper introduces Relative Portal Positioning for reliable portal interactions, and the concept of maintaining Level Portals. In a comparative user study, Point & Portal demonstrated significant advantages over the traditional Point & Teleport technique to bring interaction devices within-arm’s reach. In the presence of obstacles, Point & Portal exhibited faster speed, lower cognitive load and was preferred by participants. Overall, participants required less physical movement, pointing actions, and reported higher involvement and “good” usability.
Comparative Analysis of Artefact Interaction and Manipulation Techniques in VR Museums: A Study of Performance and User Experience

For museums in Virtual Reality (VR), various interaction and manipulation techniques could be employed for users to engage with artefact interactions. This study examined four combinations of interaction (controller-based and hand-tracking) and manipulation (direct and indirect) techniques, assessing user performance and experience with these interaction techniques in a virtual museum environment. We conducted a within-subjects experiment and asked participants to perform a series of transform manipulation tasks using the four techniques. Participants’ task completion time was measured. They also provided feedback on acceptance, learnability, presence, sickness, and fatigue, and gave an overall ranking through post-experiment questionnaires and interviews. The results revealed that controller-based direct manipulation outperformed the other techniques in terms of task performance and user experience, with hand-tracking indirect manipulation being the least efficient and the least preferred option. The study offers insights for future research and development in refining interaction and manipulation techniques and designing more user-friendly VR museum experiences.
TouchRay: Towards Low-effort Object Selection at Any Distance in DeskVR

DeskVR allows users to experience Virtual Reality (VR) while sitting at a desk without requiring extensive movements. This makes it better suited for professional work environments where productivity over extended periods is essential. However, tasks that typically resort to mid-air gestures might not be suitable for DeskVR. In this paper, we focus on the fundamental task of object selection. We present TouchRay, an object selection technique conceived specifically for DeskVR that enables users to select objects at any distance while resting their hands on the desk. It also allows selecting objects’ sub-components by traversing their corresponding hierarchical trees. We conducted a user evaluation comparing TouchRay against state-of-the-art techniques targeted at traditional VR. Results revealed that participants could successfully select objects in different settings, with consistent times and on par with the baseline techniques in complex tasks, without requiring mid-air gestures.
Be Real in Scale: Swing for True Scale in Dual Camera Mode

Many mobile AR apps that use the front-facing camera can benefit significantly from knowing the metric scale of the user’s face. However, the true scale of the face is hard to measure because monocular vision suffers from a fundamental ambiguity in scale. The methods based on prior knowledge about the scene either have a large error or are not easily accessible. In this paper, we propose a new method to measure the face scale by a simple user interaction: the user only needs to swing the phone to capture two selfies while using the recently popular Dual Camera mode. This mode allows simultaneous streaming of the front camera and the rear cameras and has become a key feature in many social apps. A computer vision method is applied to first estimate the absolute motion of the phone from the images captured by two rear cameras, and then calculate the point cloud of the face by triangulation. We develop a prototype mobile app to validate the proposed method. Our user study shows that the proposed method is favored compared to existing methods because of its high accuracy and ease of use. Our method can be built into Dual Camera mode and can enable a wide range of applications (e.g., virtual try-on for online shopping, true-scale 3D face modeling, gaze tracking, and face anti-spoofing) by introducing true scale to smartphone-based XR. The code is available at https://github.com/ruiyu0/Swing-for-True-Scale.
Effects of Speed of a Collocated Virtual Walker and Proximity Toward a Static Virtual Character on Avoidance Movement Behavior

We explored the avoidance movement behaviors of study participants immersed in a virtual reality environment. We placed a static virtual character at the midpoint between the start and target spot for the avoidance task, and a virtual walker character in front of the starting spot and scripted it to reach the target spot. Participants were placed behind the virtual walker in order to measure its influence on participants’ behavior. We developed nine experimental conditions assigned to the virtual walker character by following a 3 (speed: slow vs. normal vs. fast walking speed) $\times 3$ (proximity: close vs. middle vs. far proximity to the static virtual character) study design. For this within-group study, we collected data from 22 study participants to explore how speed and proximity walking patterns assigned to a virtual walker character could impact participants’ avoidance movement behaviors and decisions. Our data revealed that 1) the speed factor impacted the participants’ avoidance movement behavior; 2) the proximity factor did not significantly impact the participants’ avoidance movement behavior; 3) the virtual walker character did not significantly impact participants’ avoidance decisions regarding the static virtual character; 4) in all examined conditions, the side-by-side distances between the participants and the static virtual character were inside the social space according to the proxemics model; and 5) in conditions in which a slow virtual walker character was present or in the condition of normal speed and far proximity, we observed an increased number of participants pass the virtual walker character.
MonoVAN: Visual Attention for Self-Supervised Monocular Depth Estimation

Depth estimation is crucial in various computer vision applications, including autonomous driving, robotics, and virtual and augmented reality. An accurate scene depth map is beneficial for localization, spatial registration, and tracking. It converts 2D images into precise 3D coordinates for accurate positioning, seamlessly aligns virtual and real objects in applications like AR, and enhances object tracking by distinguishing distances. The self-supervised monocular approach is particularly promising as it eliminates the need for complex and expensive data acquisition setups relying solely on a standard RGB camera. Recently, transformer-based architectures have become popular to solve this problem, but at high quality, they suffer from high computational cost and poor perception of small details as they focus more on global information. In this paper, we propose a novel fully convolutional network for monocular depth estimation, called MonoVAN, which incorporates the visual attention mechanism and applies super-resolution techniques in decoder to better capture fine-grained details in depth maps. To the best of our knowledge, this work pioneers the use of a convolutional visual attention in the context of depth estimation. Our experiments on outdoor KITTI benchmark and the indoor NYUv2 dataset show that our approach outperforms the most advanced self-supervised methods, including such state-of-the-art models as transformer-based VTDepth from ISMAR’22 and hybrid convolutional-transformer MonoFormer from AAAI’23, while having a comparable or even fewer number of parameters in our model than competitors. We also validate the impact of each proposed improvement in isolation, providing evidence of its significant contribution. Code and weights are available at https://github.com/IlyaInd/MonoVAN.
DEAMP: Dominant-Eye-Aware Foveated Rendering with Multi-Parameter optimization

The increasing use of high-resolution displays and the demand for interactive frame rates presents a major challenge to widespread adoption of virtual reality. Foveated rendering address this issue by lowering pixel sampling rate at the periphery of the display. How-ever, existing techniques do not fully exploit the feature of human binocular vision, i.e., the dominant eye. In this paper, we propose a Dominant-Eye-Aware foveated rendering method optimized with Multi-Parameter foveation (DEAMP). Specifically, we control the level of foveation for both eyes with two distinct sets of foveation parameters. To achieve this, each eye’s visual field is divided into three nested layers based on eccentricity. Multiple parameters govern the level of foveation of each layer, respectively. We conduct user studies to evaluate our method. Experimental results demonstrate that DEAMP is superior in terms of rendering time and reduces the disparity between pixel sampling rate and the visual acuity fall-off model while maintaining the perceptual quality.
Meta360: Exploring User-Specific and Robust Viewport Prediction in360-Degree Videos through Bi-Directional LSTM and Meta-Adaptation

Viewport prediction is a critical aspect of virtual reality (VR) video streaming, directly impacting user experience in adaptive streaming. However, most existing algorithms treat users as homogeneous entities and overlook the variations in user behaviors and video content. Additionally, they often struggle with long-term predictions and intense movement. Our research sheds light on the importance of considering user behavior variations and leveraging advanced techniques to optimize robust viewport prediction in VR video streaming. First, we address these limitations by conducting a comprehensive feature analysis on existing datasets to uncover distinctive user behaviors. Building upon these findings, we propose a novel approach that utilizes the power of Bidirectional Long Short-Term Memory (BiLSTM) networks and meta-learning. The BiLSTM architecture effectively captures long-term dependencies, which can strengthen the robustness of viewport prediction especially in longterm prediction and intense movement. Additionally, meta-learning enables personalized adaptation to individual users’ viewing behaviors. Through extensive evaluations on diverse datasets, our algorithm Meta360 demonstrates superior performance in terms of accuracy and robustness compared to state-of-the-art methods.
User Self-Motion Modulates the Perceptibility of Jitter for World-locked Objects in Augmented Reality

A key feature of augmented reality (AR) is the ability to display virtual content that appears stationary as users move throughout the physical world (‘world-locked rendering’). Imperfect world-locked rendering gives rise to perceptual artifacts that can negatively impact user experience. One example is random variation in the position of virtual objects that are intended to be stationary (jitter’). The human visual system is highly attuned to detect moving objects, and moreover it can disambiguate between the retinal velocities that arise from object motion and self-motion, respectively. In this study, we investigated how the perceptibility of AR object jitter varies as a function of user self-motion. Using a commercially available AR HMD to display a 3D textured cube, we measured sensitivity to added jitter versus a no-jitter reference using a two-interval forced choice task. Three user motion conditions (stationary, head rotation, and walking) and three object placement conditions (floating in free space, on a desk, and against a wall) were tested in a full factorial design. We hypothesized that (1) as users move their head and eyes during self-motion, their sensitivity to jitter will decrease, due to added retinal velocity; and (2) rendering virtual objects near physical surfaces will increase sensitivity to jitter, by providing proximal veridical visual cues. Psychometric thresholds indicated that users were significantly less sensitive to jitter during self-motion than when they were stationary, consistent with hypothesis (1). Users were also more sensitive to jitter in one of the two object placement conditions, providing partial support for hypothesis (2). To generalize beyond distinct user motion and object placement conditions, we also analyzed eye tracking data. The amount of retinal slip (i.e. how much gaze drifted across the virtual object) predicted jitter thresholds better than recorded head movements alone, suggesting a retinally-driven decrease in jitter sensitivity during self-motion. These results can be used to inform requirements for AR world-locked rendering systems, as well as how these may be updated dynamically using online measurement of user head and eye movements.
Towards a Framework for Validating XR Prototyping for Performance Evaluations of Simulated User Experiences

Extended Reality (XR) technology has matured in recent years, leading to increased use of XR simulations for prototyping novel human-centered interfaces, approximating advanced display hardware, or exploring future user experiences, before realising them in real-world scenarios. However, the validity of utilizing XR prototyping (XRP) as a method for gathering performance data on novel user experiences is still underexplored, i.e, it is not clear if results gathered in simulations can be transferred to a real experience. To address this gap, we propose a validation framework that supports establishing equivalence of performance measures gathered with real products and simulated products and, thus, improve ecological validity of XRP. To demonstrate the utility of the framework, we conduct an exemplary validation study using a Varjo XR-3, a state-of-the-art XR head-mounted display (HMD). The study focuses on steering a small drone and comparing it to interactions with its real-world counterpart. We identify functional fidelity, i.e., functional similarity between real and simulated product, as well as simulation overhead from wearing an HMD as major confounding factors for XRP.