Reviewerly/exHarmony huggingface dataset:
RC-SMPL: Real-time Cumulative SMPL-based Avatar Body Generation

We present a novel method for avatar body generation that cumulatively updates the texture and normal map in real-time. Multiple images or videos have been broadly adopted to create detailed 3D human models that capture more realistic user identities in both Augmented Reality (AR) and Virtual Reality (VR) environments. However, this approach has a higher spatiotemporal cost because it requires a complex camera setup and extensive computational resources. For lightweight reconstruction of personalized avatar bodies, we design a system that progressively captures the texture and normal values using a single RGBD camera to generate the widely-accepted 3D parametric body model, SMPL-X. Quantitatively, our system maintains real-time performance while delivering reconstruction quality comparable to the state-of-the-art method. Moreover, user studies reveal the benefits of real-time avatar creation and its applicability in various collaborative scenarios. By enabling the production of high-fidelity avatars at a lower cost, our method provides more general way to create personalized avatar in AR/VR applications, thereby fostering more expressive self-representation in the metaverse.
Exploring the Effects of Virtually-Augmented Display Sizes on Users’ Spatial Memory in Smartwatches

The small display size of the smartwatches makes it difficult to display large amounts of information on the device. Prior work explored leveraging a second device (e.g., Head-mounted displays) to extend the space where users can access large information space with virtual displays anchored on their wrists. Though researchers showed that having an additional virtual screen increased information bandwidth, little is known about the effect of virtual display sizes on users’ performance. In this paper, we examined the impact of display sizes on spatial memory, workload, and user experience to better understand the prospects of virtually-augmented displays for smartwatches. Results from a user study revealed that a 4.8 inches display size can be the “sweet spot” for the virtually-augmented displays to ensure improved spatial memory performance and better user experience with less workload. Finally, we provided a set of design guidelines focusing to display size, spatial memory, user experience, and workload for designing virtually augmented user interfaces for smartwatches.
Who Did What When? Discovering Complex Historical Interrelations in Immersive Virtual Reality

Traditional digital tools for exploring historical data mostly rely on conventional 2D visualizations, which often cannot reveal all relevant interrelationships between historical fragments (e.g., persons or events). In this paper, we present a novel interactive exploration tool for historical data in VR, which represents fragments as spheres in a 3D environment and arranges them around the user based on their temporal, geo, categorical and semantic similarity. Quantitative and qualitative results from a user study with 29 participants revealed that most participants considered the virtual space and the abstract fragment representation well-suited to explore historical data and to discover complex interrelationships. These results were particularly underlined by high usability scores in terms of attractiveness, stimulation, and novelty, while researching historical facts with our system did not impose unexpectedly high task loads. Additionally, the insights from our post-study interviews provided valuable suggestions for future developments to further expand the possibilities of our system.
Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation

Pre-captured immersive environments using omnidirectional cameras provide a wide range of virtual reality applications. Previous research has shown that manipulating the eye height in egocentric virtual environments can significantly affect distance perception and immersion. However, the influence of eye height in pre-captured real environments has received less attention due to the difficulty of altering the perspective after finishing the capture process. To explore this influence, we first propose a pilot study that captures real environments with multiple eye heights and asks participants to judge the egocentric distances and immersion. If a significant influence is confirmed, an effective image-based approach to adapt pre-captured real-world environments to the user’s eye height would be desirable. Motivated by the study, we propose a learning-based approach for synthesizing novel views for omnidirectional images with altered eye heights. This approach employs a multitask architecture that learns depth and semantic segmentation in two formats, and generates high-quality depth and semantic segmentation to facilitate the inpainting stage. With the improved omnidirectional-aware layered depth image, our approach synthesizes natural and realistic visuals for eye height adaptation. Quantitative and qualitative evaluation shows favorable results against state-of-the-art methods, and an extensive user study verifies improved perception and immersion for pre-captured real-world environments.
Exploring Effective Immersive Approaches to Visualizing WiFi

WiFi networks are essential to our daily lives, but their signals are not visible to us. Therefore, it is challenging to evaluate the health of a network or make changes to ensure an optimal configuration. Traditional visualization approaches, such as contour lines, are not intuitive and lead to challenges in the analysis and comprehension of networks. In this paper, we introduce two novel visualizations: Wavelines and Stacked Bars. We then compared these visualizations to the state-of-the-art visualization technique of contour lines. We carried out a user study with 32 participants to validate that our novel visualizations can improve user confidence, accuracy, and completion time for the tasks of router localization, ranking of signal strengths, channel interference, and router coverage. We selected these tasks after extensive discussions with domain experts. We believe that our findings will assist network analysts in visually understanding our increasingly rich signal environments.
Investigating Avatars’ Impact on Shopper’s Perception and Behaviour in an Immersive Virtual Store

Immersive Virtual Reality (iVR) is rapidly expanding worldwide. This paper investigates how social cues in the form of virtual avatars can affect users’ perceived telepresence, social presence, co-presence, and flow. We further investigated whether these user perceptions impact their behaviour and whether they are related to the system’s usability. A total of 54 participants were invited to experience and interact with the virtual store and complete a post-experimental questionnaire. Contrary to earlier research, our findings demonstrated that participants’ perceptions of telepresence, social presence, co-presence, and user behaviour were not significantly impacted by the mere presence of virtual avatars in the store. Our results highlight the need for further research that examines the effectiveness of intelligent avatars in enhancing people’s sense of presence in immersive VR environments.
State-Aware Configuration Detection for Augmented Reality Step-by-Step Tutorials

Presenting tutorials in augmented reality is a compelling application area, but previous attempts have been limited to objects with only a small numbers of parts. Scaling augmented reality tutorials to complex assemblies of a large number of parts is difficult, because it requires automatically discriminating many similar-looking object configurations, which poses a challenge for current object detection techniques. In this paper, we seek to lift this limitation. Our approach is inspired by the observation that, even though the number of assembly steps may be large, their order is typically highly restricted: Some actions can only be performed after others. To leverage this observation, we enhance a state-of-the-art object detector to predict the current assembly state by conditioning on the previous one, and to learn the constraints on consecutive states. This learned ‘consecutive state prior’ helps the detector disambiguate configurations that are otherwise too similar in terms of visual appearance to be reliably discriminated. Via the state prior, the detector is also able to improve the estimated probabilities that a state detection is correct. We experimentally demonstrate that our technique enhances the detection accuracy for assembly sequences with a large number of steps and on a variety of use cases, including furniture, Lego and origami. Additionally, we demonstrate the use of our algorithm in an interactive augmented reality application.
A Blockchain Based Fund Management System for Construction Projects - A Comprehensive Case Study in Xiong'an New Area China

As large-scale construction projects become increasingly complex, the use and integration of advanced technologies are being emphasized more and more. However, the construction industry often lags behind most industries in the application of digital technologies. In recent years, a decentralized, peer-to-peer blockchain technology has attracted widespread attention from academia and industry. This paper provides a solution that combines blockchain technology with construction project fund management. The system involves participants such as the owner's unit, construction companies, government departments, banks, etc., adopting the technical architecture of the “Xiong'an Blockchain Underlying System”. The core business and key logic processing are all implemented through smart contracts, ensuring the transparency and traceability of the fund payment process. The goal of ensuring investment quality, standardizing investment behavior, and strengthening cost control is achieved through blockchain technology. The application of this system in the management of Xiong'an construction projects has verified that blockchain technology plays a significant positive role in strengthening fund management, enhancing fund supervision, and ensuring fund safety in the construction process of engineering projects. It helps to eliminate the common problems of multi-party trust and transparent supervision in the industry and can further improve the investment benefits of government investment projects and improve the management system and operation mechanism of investment projects.
Giant Finger: A Novel Visuo-Somatosensory Approach to Simulating Lower Body Movements in Virtual Reality

Surreal experience in virtual reality (VR) occurs when visual experience is accompanied by congruent somatosensation. Thus, VR contents that require physical actions are often bounded to our physical capabilities to maintain somatosensory consistency. Alternatively, users often choose less immersive but safer interfaces that offer a wider action variability. In either case, this situation compromises the potential for a hyper-realistic experience. To address this, we introduce “Giant Finger,” a concept that replicates human lower body movements through two enlarged virtual fingers in VR. Through a user study, we affirmed Giant Finger’s ownership using proprioceptive drift and questionnaire responses. We also compared Giant Finger’s capability to perform a variety of tasks with existing methods. Despite its minimalistic approach, Giant Finger demonstrated a high level of efficacy in supporting lower body movements, with ownership and presence comparable to those of the body-leaning method with whole-body motion. Giant Finger can replace the sensations of real legs or support locomotion in confined spaces by providing proprioceptive illusions to the virtual lower body. The applications showcased in this paper suggest that Giant Finger can enable new forms of movement with high action variability and immersion in various fields such as gaming, industry, and accessibility.
Software Engineering Approaches Adopted By Blockchain Developers

The adoption of blockchain technology has witnessed a rapid surge in recent years, revolutionizing various industries. As blockchain-based solutions continue to gain momentum, understanding the software development approaches employed by blockchain developers becomes crucial. This research paper examines the experiences of blockchain developers with software engineering (SE) approaches in blockchain development. We deployed an online survey focused on topics such as the importance of SE approaches, preferred methodologies, and the significance of different stages within the Software Development Life Cycle (SDLC) for blockchain projects. Our findings reveal a consensus among participants regarding the importance of utilizing SE approaches in blockchain development to ensure secure, reliable, and scalable solutions. Agile methodologies, such as Scrum, were prominently favored for their adaptability and iterative nature. Furthermore, design emerged as the stage of primary importance within the SDLC for blockchain-based solutions. These insights provide valuable guidance for researchers, practitioners, and organizations involved in blockchain development, contributing to the advancement of SE practices in the context of blockchain technology,
Natural Language-Based Model-Checking Framework for Move Smart Contracts

Significant efforts have been dedicated to employing model-checking as a formal verification approach in the context of smart contracts. The utilization of these tools necessitates an in-depth knowledge on the part of the developer regarding both the programming language and the implementation of model-checking techniques. To provide accessibility to developers with basic language proficiency, we present a technique for developing a conversational application framework that can be seamlessly linked with any model-checking tool for the purpose of creating a smart contract. This architecture offers a robust and effective approach to the development of safe and dependable smart contracts. The utilization of natural language processing techniques in conjunction with neural networks is employed for this objective. Using this methodology, a prototype implementation for Move smart contracts has been created and is used with the VeriMove model-checking tool. Using the offered graphical user interface, we were able to successfully build, compile and test Move smart contracts across four different classes of smart contracts. This strategy effectively decreases the amount of time and effort needed for manual coding and debugging. In addition, the use of the VeriMove model-checking tool guarantees that the smart contracts produced are devoid of any potential vulnerabilities and flaws.
Human-Computer Interaction for Decision Making in Digital Twins with Various Display Environments

Human-Computer Interaction (HCI) is the study of how humans and computers interact to achieve a specific purpose. The applications of digital twins in building/facility management are of special interest in this work. Digital twins of buildings are created by compiling various data into their three-dimensional virtual replicas, which are then updated in real time with the state of the buildings. By employing this approach, facility managers can minimize the time required to assess risks and devise problem-solving strategies remotely, which in turn reduces the need for their physical presence in the facility. The purpose of this study is to investigate the effectiveness of three virtual display environments when the users must make decisions regarding various building-related issues. A single monitor, a tiled display consisting of multiple monitors, and a head-mounted display (often known as VR) make up the three virtual display environments that are key to us. The times it took participants to make decisions in three simulated scenarios-detecting water leakage, identifying and extinguishing fires, and measuring objects-were recorded. Due to participants' prior expertise with the single monitor, this display setting had the best average decision making time. However, upon closer examination of each task, we discovered that the head-mounted display placed second in multiple tasks, despite the fact that the majority of participants had never used it. Decision-making duties involving the collection of data from multiple perspectives and with a realistic sense of objects or surroundings will make the most of this display environment.
RenderFusion: Balancing Local and Remote Rendering for Interactive 3D Scenes

Many modern-day XR devices (e.g. mobile headsets, phones, etc.) lack the computing resources required to render complex 3D scenes in real-time. Typically, to render a high-resolution scene on a lightweight XR device, 3D designers arduously decimate and fine-tune the objects. As an alternative, remote rendering systems can utilize powerful nearby servers to stream rendering results to a client. While this is a promising solution, it can introduce a variety of latency and reliability issues, especially under variable network conditions. In this paper, we present a distributed rendering system that combines both remote rendering and on-device, “local” rendering to add robustness to network fluctuations and device workloads. To maximize user QoE, our approach dynamically swaps an object’s rendering medium, adjusting for client workload, low frame rates, and several perceptual characteristics. To model these characteristics, we perform a study under simulated conditions to measure how users perceive latency and complexity differences between objects in a scene. Using the results of the study, we then provide an algorithm for choosing the optimal object rendering medium, based on rendering complexity as well as network and latency models, ensuring that a target frame rate will be met. Finally, we evaluate this algorithm on a prototype implementation that can provide cross-platform split rendering using web technologies.
A Closer Look at Dynamic Medical Visualization Techniques

In navigated surgery, physicians perform complex tasks assisted by virtual representations of anatomical structures and surgical tools. Integrating Augmented Reality (AR) in these scenarios enriches the information presented to the surgeon through a range of visualization techniques. Their selection is a crucial task as they represent the primary interface between the system and the surgeon.In this work, we present a novel approach to conveying augmented content using dynamic visualization techniques, allowing users to gather depth and shape information from both pictorial and kinetic cues. We conducted user studies comparing two novel dynamic methods – Object Flow and Wave Propagation – and three state-of-the-art static visualization techniques among medical experts. Our studies provide a detailed comparison of the visualization techniques’ efficacy in conveying shape and depth information from medical data, as well as task load and usability reported by the participants and post hoc analyses. We found that kinetic cues can assist users in understanding complex anatomical structures in medical AR.
Adaptive Color Structured Light for Calibration and Shape Reconstruction

Color structured light (SL) plays an important role in spatial augmented reality and shape reconstruction. Compared to traditional non-color multi-shot SL, it has the advantage of fewer projections, and can even achieve single-shot. However, distortions caused by ambient light and imaging devices limit color SL’s applicability and accuracy. A common solution is to apply color adaptation techniques to cancel the disturbances. Previous studies focus on either robust fixed color patterns or adaptation approaches that may require preliminary geometric calibrations. In this paper, we propose an approach that can efficiently adapt color SL to arbitrary ambient light and imaging devices’ color responses, without device response function calibration or geometric calibration. First, we design a novel algorithm to quickly find the most distinct colors that are easily separable under a new environment and device setup. Then, we design a maximum a posteriori (MAP)-based color detection algorithm that can utilize ambient light and device priors to robustly detect the SL colors. In experiments, our adaptive color SL outperforms previous methods in both calibration and shape reconstruction tasks across a variety of setups.
CoLT: Enhancing Collaborative Literature Review Tasks with Synchronous and Asynchronous Awareness Across the Reality-Virtuality Continuum

Collaboration plays a vital role in both academia and industry whenever we need to browse through a big amount of data to extract meaningful insights. These collaborations often involve people living far from each other, with different levels of access to technology. Effective cross-border collaborations require reliable telepresence systems that provide support for communication, cooperation, and understanding of contextual cues. In the context of collaborative academic writing, while immersive technologies offer novel ways to enhance collaboration and enable efficient information exchange in a shared workspace, traditional devices such as laptops still offer better readability for longer articles. We propose the design of a hybrid cross-reality cross-device networked system that allows the users to harness the advantages of both worlds. Our system allows users to import documents from their personal computers (PC) to an immersive headset, facilitating document sharing and simultaneous collaboration with both colocated colleagues and remote colleagues. Our system also enables a user to seamlessly transition between Virtual Reality, Augmented Reality, and the traditional PC environment, all within a shared workspace. We present the real-world scenario of a global academic team conducting a comprehensive literature review, demonstrating its potential for enhancing cross-reality hybrid collaboration and productivity.
Multi-Focus Querying of the Human Genome Information on Desktop and in Virtual Reality: an Evaluation

The human genome is incredibly information-rich, consisting of approximately 25,000 protein-coding genes spread out over 3.2 billion nucleotide base pairs contained within 24 unique chromosomes. The genome is important in maintaining spatial context, which assists in understanding gene interactions and relationships. However, existing methods of genome visualization that utilize spatial awareness are inefficient and prone to limitations in presenting gene information and spatial context. This study proposed an innovative approach to genome visualization and exploration utilizing virtual reality. To determine the optimal placement of gene information and evaluate its essentiality in a VR environment, we implemented and conducted a user study with three different interaction methods. Two interaction methods were developed in virtual reality to determine if gene information is better suited to be embedded within the chromosome ideogram or separate from the ideogram. The final ideogram interaction method was performed on a desktop and served as a benchmark to evaluate the potential benefits associated with the use of VR. Our study findings reveal a preference for VR, despite longer task completion times. In addition, the placement of gene information within the visualization had a notable impact on the ability of a user to complete tasks. Specifically, gene information embedded within the chromosome ideogram was better suited for single target identification and summarization tasks, while separating gene information from the ideogram better supported region comparison tasks.