Reviewerly/exHarmony huggingface dataset:
Linearization Weight Compression and In-Situ Hardware-Based Decompression for Attention-Based Neural Machine Translation

As recent machine translation models are mostly based on the attention-based neural machine translation (NMT), many well-known models such as Transformer or bidirectional encoder representations from Transformers (BERT) have been proposed. Along with algorithmic advancements, hardware acceleration methods for those attention-based neural machine translation models have also been introduced. However, the size of the parameters for attention-based NMT is also becoming larger to guarantee the satisfactory machine translation quality. Among various weights, linearization weights ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{Q}$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{K}$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{V}$ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{O}$ </tex-math></inline-formula> ) account for a non-negligible portion (by up to 30%) among the entire parameters in the modern NMT models. In this paper, we propose a method for linearization weight compression and near-memory hardware decoder for fast and in-situ weight decompression. Our weight compression method exploits the fixed-point quantization along with Huffman coding which is selectively applied depending on the weight value distribution. Our hardware decoder decompresses the Huffman-coded weights near-memory to minimize the weight decoding latency. Our compression method shows 4.9–10.0 compression ratio with small NMT score drops across the five widely used attention-based NMT models (Transformer, Transformer-XL-base, Transformer-XL-large, BERT-base, and BERT-large). In addition, due to the reduced linearization weight size, our proposed method with near-memory decoding enables multi-head attention (MHA) execution latency reduction by 11.8%, on average, as compared to the baseline when considering the weight loading and initialization. In terms of the memory data transfer energy consumption, our proposed method leads to a memory energy saving of 16.1%, on average, as compared to the baseline.
ARCam: A User-Defined Camera for AR Photographic Art Creation

Photography in augmented reality can be challenging due to the restrictions of pre-defined settings. However, adjustable photography settings and real-time previews are significant for AR photographic creation as creators must adjust multiple camera properties to present unique visual effects. In this work, we designed an AR camera (ARCam) with various adjustable properties to give users a high degree of freedom for photographic art creation in real-time preview.
Manipulation Guidance Field for Collaborative Object Manipulation in VR

Object manipulation is a fundamental interaction in virtual reality (VR). Efficient and accurate manipulation is important for many VR applications, especially collaborative VR applications. We introduce a collaborative method based on the manipulation guidance field (MGF) to improve manipulation accuracy and efficiency. We first introduce the concept of MGF and its construction method. Then we propose two strategies to accelerate MGF updating process. After that, we propose a collaborative manipulation method to manipulate objects using the guidance of MGF.
Design and Development of a Mixed Reality Acupuncture Training System

This paper looks at how mixed reality can be used for the improvement and enhancement of Chinese acupuncture practice through the introduction of an acupuncture training simulator. A prototype system developed for our study allows practitioners to insert virtual needles using their bare hands into a full-scale 3D representation of the human body with labelled acupuncture points. This provides them with a safe and natural environment to develop their acupuncture skills simulating the actual physical process of acupuncture. It also helps them to develop their muscle memory for acupuncture and better develops their memory of acupuncture points through a more immersive learning experience. We describe some of the design decisions and technical challenges overcome in the development of our system. We also present the results of a comparative user evaluation with potential users aimed at assessing the viability of such a mixed reality system being used as part of their training and development. The results of our evaluation reveal the training system outperformed in the enhancement of spatial understanding as well as improved learning and dexterity in acupuncture practice. These results go some way to demonstrating the potential of mixed reality for improving practice in therapeutic medicine.
[DC] Investigating student's learning processes in a Virtual Reality learning environment by analyzing their interaction behavior

Educational researchers use physiological sensors and human ob-servers to collect data to conduct quantitative studies in VR Learn ing Environments (VRLE). The data collected using physiological sensors are used to understand the affective state of the learners. Whereas, human observers are the most widely used method to collect data to understand the behavior of the learners in a VRLE. However, the data collection using human observers is labor inten-sive and may get biased due to cognitive, social, and communicative causes. Moreover, the data provided by human observers need to satisfy the inter-rater reliability test in order to become valid. There has been little or no work done to automatically collect the data re-lated to the behavior of the learners in VRLE. To overcome the gaps mentioned above we have developed a data collection mechanism to automatically log all the interaction behavior of the learners happening in VRLE along with the time-stamp in real-time. The developed data collection mechanism can open up new opportunities for data collection when deployed in VRLEs. Also, the data collected using it can be useful to get more insights about the learners' behavior and in turn the learning processes happening in VRLE by applying AI and ML tools. This doctoral thesis aims at investigating the interaction behavior of the learners happening in VRLE and analyze the behavioral data to understand the behavioral pattern responsible for the higher learning gain. The analysis can also be further done for the early prediction of the learning outcomes based on the interaction behavior to provide scaffolds and adaptive feedback.
Design and User Experience Evaluation of 3D Product Information in XR Shopping Application

Current 3D product information pages are believed to enrich the online shopping experience since it provides a more immersive experience. However, existing solutions still retain the 2D UI elements in the product information presentation design, preventing users from fully immersing themselves in the virtual environment and further degrading the shopping experience. In order to evaluate the user experience of 3D product information in XR shopping applications, we first construct a design space based on previous design cases of product information presentation in virtual environments and produce nine new solutions by combining elements in the design space.
Creating Informal Learning and First Responder Training XR Experiences with the ImmersiveDeck

In recent years eXtended Reality (XR) technologies have matured and have become affordable, yet creating XR experiences for training and learning in many cases is still a time-consuming and costly process, hindering widespread adoption. One factor driving effort is that content and features commonly required by many applications get re-implemented for each experience, instead of sharing and reusing these resources by means of a common platform. In this paper we present two XR experiences in the context of informal learning and first responder training along with the shared platform they have been created with and the creation process. Furthermore, we have technically evaluated relevant parts of the platform for feasibility of use with experience requirements and confirmed ap-plicability. Finally, we present an informal expert evaluation of the content creation process's user experience for the informal learning experience along with guidelines derived from the findings.
Vitalizing cultural memory with immersive data storytelling

Through decades of cultural heritage and cultural memory digitization, a massive amount of data has been collected and organized. There is an increasing interest in and demand for making the data approachable, explorable, and usable to the public audience, vi-sualizing the data with multiple scales, and expressing humanity knowledge with aesthetically appealing images.
Virtual Resection Planning using Bezier Surface Interactions in Collaborative VR Environments

The use of virtual reality (VR) has been proposed for collaborative planning or training scenarios to aid surgeons in preparing surgical interventions. In this paper, we explore the usage of cubic Bezier surfaces and their respective interactions in collaborative VR environments to define atypical resection surfaces for planning abdominal organ tumor resections. We conducted a pilot study (n = 10) to evaluate the usability of the Bezier surface interaction compared to a free deformation approach using the example of liver surgery planning. Moreover, we showed the prototype to an expert and conducted an interview to collect clinical feedback. The results show potential benefits of both interaction techniques that could be essential for virtual resection planning and training. The expert highlighted that combining both techniques could speed up the workflow and provide various options for planning.
ADAPTIVEFUSION: Low Power Scene Reconstruction

Towards deploying scene reconstruction on mobile devices with a stringent power budget, we propose ADAPTIVEFUSION to meet the power envelope by dynamically changing the reconstruction frequency while maintaining acceptable quality. Compared to a standard 30 Hz baseline, ADAPTIVEFUSION reduces frequency, and thus power, by 4.9 × on average and is applicable to other fusion-based algorithms.
IEEE VR 2023 Workshop: Datasets for developing intelligent XR applications (DATA4XR)

The 2nd workshop on Datasets for Developing Intelligent XR Applications (DATA4XR) aims to address the challenges of public datasets and reproducibility in Extended Reality, also known as XR (Augmented Reality, Virtual Reality, and Mixed Reality) research. The workshop brings together experts to discuss the availability, privacy concerns, and ethics related to open-sourcing the datasets used in XR research for algorithm training and user behavior analysis. By examining the ethical, moral, and privacy concerns
Artistic Exploration of Stop-Motion Animation in Virtual Reality: Spatializing the Analog Techniques of 2D Replacement and Object Animation by Using Digital Cutout and Realtime Rendering

The art-based research project VRinMotion explores the implemen-tation of analog stop-motion animation in virtual reality. ExperiMotion 1, the first of four workshops, focused on 2D replacement and object animation in VR, presenting a unique challenge in creating a 3D virtual space with flat animation techniques. The research team discussed and artistically tested various new ideas and methods focusing, i.a., on haptic properties and the role of stop-motion loops. Furthermore, it developed frameworks in Unity3D and TouchDe-signer for prototyping and implementing the project requirements. Finally, the team explored how the use of sound may improve spatial perception in VR and how separating the artistic process into dis-tinct modules allows several individuals to collaborate on the same project.
Where to Render: Studying Renderability for IBR of Large-Scale Scenes

Image-based rendering (IBR) technique enables presenting real scenes interactively to viewers and hence is a key component for implementing VR telepresence. The quality of IBR results depends on the set of pre-captured views, the rendering algorithm used, and the camera parameters of the novel view to be synthesized. Numerous methods were proposed for optimizing the set of captured images and enhancing the rendering algorithms. However, from which regions IBR methods can synthesize satisfactory results is not yet well studied. In this work, we introduce the concept of renderability, which predicts the quality of IBR results at any given viewpoint and view direction. Consequently, the renderability values evaluated for the 5D camera parameter space form a field, which effectively guides viewpoint/trajectory selection for IBR, especially for challenging large-scale 3D scenes. To demonstrate this capability, we designed 2 VR applications: a path planner that allows users to navigate through sparsely captured scenes with controllable rendering quality and a view selector that provides an overview for a scene from diverse and high quality perspectives. We believe the renderability concept, the proposed evaluation method, and the suggested applications will motivate and facilitate the use of IBR in various interactive settings.
Designing Navigation Tool for Immersive Analytics in AR

As augmented and virtual reality technologies continue to mature, many studies have looked into navigation in 3D visualizations in recent years. However, most studies that examine navigation in 3D visualizations were conducted using a virtual reality environment (VR). Navigation in augmented reality (AR) may have different needs and challenges which have remained undiscovered. My research aims to investigate the elements that influence navigation performance in AR and how to improve the efficiency and user experience of navigation in AR by design, development, and user studies of a navigation tool EaseNav that supports users' navigation procedure.
Streamlining Epilepsy Surgery Planning Rounds with Virtual Reality

We introduce RealityFlow, a virtual reality (VR) system designed to assist neurologists in a clinical workflow of planning epilepsy surgery. RealityFlow offers 3D direct VR manipulation to prepare data, integrates spatial and temporal information pieces for demonstration in a 3D space, and introduces a novel visualization of seizure propagation to help neurologists analyze and present a seizure spread and its types.
Does realism of a virtual character influence arousal? Exploratory study with pupil size measurement

In this poster we discuss exploratory analyses seeking to test whether realism of a virtual character can influence arousal measured by pupil diameter change in response to viewing a virtual character. To do this we conducted a study in which 180 virtual characters were presented to 45 participants in a free-viewing procedure, while their physiological responses were recorded. We manipulated the realism of these characters in terms of their appearance, behavior and intractability. Results suggest that appearance and intractability can influence arousal of the participants.
Subjective Quality Assessment of User-Generated $360^{\circ}$ Videos

In this poster, we establish one of the largest virtual reality (VR) video database, containing 502 user-generated videos with rich content and commingled authentic distortions (often localized in space and time). We capture viewing behaviors (i.e., scanpaths) of 139 users, and collect their opinion scores of perceived qual-ity under four different viewing conditions (two starting points <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\times$</tex> two exploration times). We provide a thorough statistical analysis of recorded data, resulting in several interesting observations, which reveal how viewing conditions affect human behav-iors and perceived quality. The database is available at https://github.com/Yao-Yiru/VR-Video-Database.
Mixed Reality Guided Museum Tour: Digital Enhancement of Museum Experience

Museums as architectural structures rely on physical objects, which limit the content of their exhibits and the frequency of updates. The aim of the research was to create a richer museum visit experience by adding digital commentary and presentation to existing exhibits through a mixed reality-based guided tour system. We utilized headsets and visual positioning system to design contents that would enable a deeper understanding of the exhibits and an enjoyable experience. We found that more than 80 percent of participants to our experiment agreed with the effectiveness of the system and willingness to pay additional experience fees.
Visualsickness: A web application to record and organize cybersickness data

Organizing cybersickness data using a paper simulator sickness questionnaire (SSQ) is challenging for researchers. We developed a web application to make it easier to collect, store, organize, and report SSQ data. Using this, researchers can create studies, multiple sessions within a study, and SSQs at multiple time intervals within a session. In addition, we extended on SSQ by introducing a visual SSQ with emoji animations representing the SSQ's symptoms.
Sequential Eyelid Gestures for User Interfaces in VR

In this poster, we present user-defined sequential eyelid gestures to control UIs in VR. Previous works have proposed sequential eyelid gestures for VR interaction. However, they do not include squint or wide-open eyelid states. In contrast, we consider those eyelid states too, and we encouraged users to prioritize how well the gestures fitted the commands. To validate that the user-defined gestures are effective, we tested them in a user study (N = 17) with five different UI commands for VR interfaces. We use the Vive Pro Eye HMD to detect all our final gestures with low error rates.
Towards a Metaverse in Health Informatics: 3D Visualisation of Physical Activity from VR Gaming

Playing VR games has potential for contributing towards a player's physical activity, while engaging them through a fully-immersive experience. However, limited work has focused on the player's physical health data from playing VR games and how it can be represented to make the player aware of their exertion to meet their goals and prevent over-exertion. We present the design of a long-term 3D dashboard that presents the exertion data from VR gameplay, while being accessible outside of VR.
IEEE VR 2023 Workshops: Workshop: ARES - Augmented Reality Enabling Superhuman Sports &amp; Serious Games (2nd Annual Workshop)

In the last years, the usage of Augmented Reality (AR) advanced into new frontiers through the widespread adoption of affordable mobile devices. For example, in Superhuman Sports, AR provides the potential to be a driving factor by revolutionizing team sports or by introducing novel game mechanics. In the field of Serious Games, AR can bring 3D serious content into the physical world to enhance immersion and knowledge transfer. In both areas, the full potential of this vision-based technology is still limited through the lack of framework support and tangible controller feedback for purely virtual objects. We want to bring together experts from different computing and engineering disciplines to overcome the barriers of complex AR game concepts and to enhance interactions with augmented objects in a playful way in two important research fields: The goal of Superhuman Sports is to surpass the limitations of the human body by incorporating technology. Cognitive and physical capabilities are augmented by involving rich interactions through tangible and virtual game objects. Concepts of team sports based on competition and the exploration of new experiences through novel senses and reflexes, are helping to augment traditional sports or to path the way towards new game ideas. Serious Games combine the world of playing games for entertainment with the goal of learning in an almost subconsciously manner. Combining these two approaches requires meeting the demands of both. Learning content must be presented correctly and coherently, but on the other hand, it must be perfectly integrated into a game concept. Today, there are still various unsolved and related challenges for AR sports concepts and AR Serious Games alike, which should be spontaneously, anywhere, playable on mobile/wearable devices. Major challenges include, but are not limited to:
iARVis: Mobile AR Based Declarative Information Visualization Authoring, Exploring and Sharing

We present iARVis, a proof-of-concept toolkit for creating, experiencing, and sharing mobile AR-based information visualization environments. Over the past years, AR has emerged as a promising medium for information and data visualization beyond the physical media and the desktop, enabling interactivity and eliminating spatial limits. However, the creation of such environments remains difficult and frequently necessitates low-level programming expertise and lengthy hand encodings. We present a declarative approach for defining the augmented reality (AR) environment, including how information is automatically positioned, laid out, and interacted with, to improve the efficiency and flexibility of constructing AR-based information visualization environments. We provide fundamental layout and visual components such as the grid, rich text, images, and charts for the development of complex visualization widgets, as well as automatic targeting methods based on image and object tracking for the development of the AR environment. To increase design efficiency, we also provide features such as hot-reload and several creation levels for both novice and advanced users. We also investigate how the augmented reality-based visualization environment could persist and be shared through the internet and provide ways for storing, sharing, and restoring the environment to give a continuous and seamless experience. To demonstrate the viability and extensibility, we evaluate iARVis using a variety of use cases along with performance evaluation and expert reviews.
Designing Viewpoint Transition Techniques in Multiscale Virtual Environments

Viewpoint transitions have been shown to improve users' spatial orientation and help them build a cognitive map when they are navigating an unfamiliar virtual environment. Previous work has investigated transitions in single-scale virtual environments, focusing on trajectories and continuity. We extend this work with an in-depth investigation of transition techniques in multiscale virtual environments (MVEs). We identify challenges in navigating MVEs with nested structures and assess how different transition techniques affect spatial understanding and usability. Through two user studies, we investigated transition trajectories, interactive control of transition movement, and speed modulation in a nested MVE. We show that some types of viewpoint transitions enhance users' spatial awareness and confidence in their spatial orientation and reduce the need to revisit a target point of interest multiple times.
Give me some room please! Personal space bubbles for safety and performance

Personal space bubbles are implemented in virtual environments to protect users from physical harassment. When activated, an impermeable boundary encloses the user completely and complicates collaborative tasks, such as passing objects or performing social gestures. When personal space protection is not balanced with functionality, the personal space bubble becomes a gilded cage. In this paper, we raise the possibility of alternate designs for personal space bubbles and test their impact on task performance within a workplace training context. Our early findings suggest that alternate bubble designs have the potential to balance safety and performance metrics such as task completion.