Reviewerly/exHarmony huggingface dataset:
A Design Space of Multi-Display Spatial Interactions for Visualization Tasks

This paper presents a design space of spatial interaction techniques for multi-display visualizations. By analyzing 56 papers on multi-display tools and techniques, we identify the three dimensions and their associated design choices for different types of spatial interactions using multiple displays and devices. This work aims to provide guidance for designing interactive multi-display visualizations that meet different requirements of visual analysis and sensemaking tasks while also inspiring future design ideas. Visualization researchers can create new multi-display visualizations by using combinations of the design choices associated with different visualization tasks. Our design space also allows them to explore areas of spatial interactions that have been underutilized to their full potential or to understand the requirements for new spatial interaction techniques for multi-display visualizations.
Joint Action in Collaborative Mixed Reality: Effects of Immersion Type and Physical Location

Understanding how people effectively perform actions together is fundamental when designing Collaborative Mixed Reality (CMR) applications. While most of the studies on CMR mostly considered either how users are immersed in the CMR (e.g., in virtual or augmented reality), or how the physical workspace is shared by users (i.e., distributed or collocated), little is known about how their combination could influence user’s interaction in CMR. In this paper, we present a user study (n=23) that investigates the effect of the mixed reality setup on the user’s immersion and spatial interaction during a joint-action task. Groups of two participants had to perform two types of joint actions while carrying a virtual rope to maintain a certain distance: (1) Gate, where participants had to pass through a virtual aperture together and (2) Fruit, where participants had to use a rope to slice a virtual fruit moving in the CMR. Users were either in a distributed or collocated setup, and either immersed in virtual or augmented reality. Our results showed that users’ proxemics was altered by the immersion type and location setup, but also the user’s subjective experience. These results contribute to the understanding of joint action in CMR and they are discussed to improve the design of CMR applications.
Immersive Analysis of Spatiotemporal Racing Data

This paper presents an exploratory study of expert behavior while analysing spatio-temporal data in an immersive environment for the racing engineering domain. The analysis of moving-object data has become crucial in diverse fields, including sports analysis, logistics and autonomous navigation. This data provides valuable insights into events occurring at specific points in space and time, enabling professionals to make informed decisions. For example, in motorsports, race engineers rely on spatio-temporal data and vehicle telemetry to identify key events and optimize performance. However, conventional stacked line charts used by race engineers offer limited spatial association. Effective visualization techniques that incorporate the spatial context are essential to fully understand and interpret the complex nature of moving-object data. To address this, we propose novel immersive designs for analyzing motorsports data. Our study uses a critical task in racing engineering—comparing cornering performance of two drivers to document patterns of behavior of expert analysis. Experts used conventional charts, situated 3D graphs and a hybrid with both options within an Immersive Analytics environment. Results demonstrate the potential of our approach. All experts preferred the hybrid visualization combining conventional and situated approaches. In the short study period, experts found value in the situated approach to understand differences in distances between events.
Active Interactive Labelling Massive Samples for Object Detection

Aerial object detection is the process of detecting objects in remote sensing images, such as aerial or satellite imagery. However, due to the unique characteristics and challenges of remote sensing images, such as large image sizes and dense distribution of small objects, annotating the data is time-consuming and costly. Active learning methods can reduce the cost of labeling data and improve the model’s generalization ability by selecting the most informative and representative unlabeled samples. In this paper, we studied how to apply active learning techniques to remote sensing object detection tasks and found that traditional active learning frameworks are not suitable. Therefore, we designed a remote sensing task-oriented active learning framework that can more efficiently select representative samples and improve the performance of remote sensing object detection tasks. In addition, we proposed an adaptive weighting loss to further improve the generalization ability of the model in unlabeled areas. A large number of experiments conducted on the remote sensing dataset DOTA-v2.0 showed that applying various classical active learning methods to the new active learning framework can achieve better performance.
Minimizing Eye Movements and Distractions in Head-Mounted Augmented Reality through Eye-Gaze Adaptiveness

In augmented reality assistance systems for manual assembly, the user is forced to move the eyes to areas that are away from the performed assembly task in various situations which can cause inevitable physical effort and mental distraction. To counteract these movements and distraction potentials, we have adapted, developed, and evaluated approaches to minimize such movements and thus reduce distractions. Eye-Gaze Attention Markers can be used to direct the user’s attention back to the current point of interest. With Eye-Gaze Following, an eye-gaze-based extension of an available Mixed Reality Toolkit (MRTK) object following functionality, currently relevant information can be attached to the user’s eye gaze to reduce focus shifts. For these approaches, we present the design and their implementation, as well as a user study (N = 40) evaluating completion time, mental workload, eye movements, and focus shifts to determine the impact on eye movement and distraction minimization. Furthermore, we provide guidelines for the use and improvements of the approaches presented. With the proposed approaches, we were able to achieve improvements in terms of eye movements and distraction minimization and identify further potential for research.
VibAware: Context-Aware Tap and Swipe Gestures Using Bio-Acoustic Sensing

The use of microgestures has improved the robustness and naturalness of subtle hand interactions. However, the conventional approach often neglects the context in which users perform microgestures. We present VibAware, a context-aware tap and swipe gesture recognition using bio-acoustic sensing. We use both active and passive sensing approaches to recognize finger-based microgestures while also recognizing the associated interaction contexts, including Within-Hand, Surface, and Hand Grasp. We employ accelerometers and an active acoustic transmitter to form a bio-acoustic system with multiple bandpass filter processing. Through user studies, we validate the accuracy of context-aware tap and swipe gesture recognition. We propose a context-aware microgesture recognition pipeline to enable adaptive input controls for rich and affordable hand interactions.
In-the-Wild Experiences with an Interactive Glanceable AR System for Everyday Use

Augmented reality head-worn displays (AR HWDs) of the near future will be worn all day every day, delivering information to users anywhere and anytime. Recent research has explored how information can be presented on AR HWDs to facilitate easy acquisition without intruding on the user’s physical tasks. However, it remains unclear what users would like to do beyond passive viewing of information, and what are the best ways to interact with everyday content displayed in AR HWDs. To address this gap, our research focuses on the implementation of a functional prototype that leverages the concept of Glanceable AR while incorporating various interaction capabilities for users to take quick actions on their personal information. Instead of being overwhelmed and continuously attentive to virtual information, our system centers around the idea that virtual information should stay invisible and unobtrusive when not needed but is quickly accessible and interactable. Through an in-the-wild study involving three AR experts, our findings shed light on how to design interactions in AR HWDs to support everyday tasks, as well as how people perceive using feature-rich Glanceable AR interfaces during social encounters.
Simulating Human Visual System Based on Vision Transformer

The human visual system (HVS) is capable of responding in real-time to complex visual environments. During the process of freely observing visual scenes, predicting eye movements and visual fixations is a task known as scanpath prediction, which aims to simulate the HVS. In this paper, we propose a visual transformer-based model to study the attentional processes of the human visual system in analyzing visual scenes, thereby achieving scanpath prediction. This technology has important applications in human-computer interaction, virtual reality, augmented reality, and other fields. We have significantly simplified the workflow of scanpath prediction and the overall model architecture, achieving performance superior to existing methods.
Inconsistency Based Active Learning for temporal object detection.

As a promising direction to alleviate the issue of high cost in label annotation, active learning has made a remarkable progress in image recognition. Recently, some efforts have tried to brought it into more visual tasks such as object detection, human-object interaction and so on. Despite the effectiveness of existing methods, there still lacking research specifically for temporal scenes, which is common in real world vision task scenes like automatic driving. To fill this gap, we propose a inconsistency based active learning method specific for temporal object detection. Specifically, We design two novel active learning strategies to ensure that the selected samples are informative and contain less redundant information in temporal scenarios. In the first stage, we adopt temporal inconsistency strategy to avoid sampling too much redundant samples; in the second stage, we sample the most informative samples by utilizing the committee inconsistency strategy. Additionally, we conduct extensive experiments on Waymo dataset, the experimental results have demonstrate the effectiveness of our proposed method. Specifically, we achieve the same performance as training on the whole dataset when just sampling 20% samples, which is much better than entropy based method.
Machine Learning on Topological Constraint for Mismatching Removal

Image feature matching is a critical task in human-computer interaction, such as driverless cars, collaborative medical robots, aiming to establish correspondence between two images inputted from human or computer. To ensure the accuracy of matches, removing mismatches is of utmost importance. In recent years, machine learning has emerged as a new perspective for achieving effective mismatch removal. However, current learning-based methods suffer from a lack of generalizability due to their heavy reliance on extensive image data for training. Consequently, handling cases with a high mismatch ratio becomes challenging. In this paper, a novel approach is proposed that incorporates the triangular topology constraint into machine learning, which we called LTM. By summarizing topology constraints around the matching points and integrating the idea of sampling, we successfully address the mismatch removal task. Notably, our method stands out by requiring few parameters as input, enabling us to train it using just several image pairs from four sets. Remarkably, our method achieves outstanding results on diverse datasets using various machine learning techniques compared with many existing methods.
Visualization Placement for Outdoor Augmented Data Tours

We investigate the potential impact of visualization placement on enhancing the user experience for outdoor augmented data tours, the tours that rely on situated data visualization in augmented reality (AR). Little is known about the optimal placement of outdoor AR information visualizations. Inspired by an existing guided tour about the environmental impact of buildings, we investigated three potential placements for data visualizations: on buildings, on the floor in the user’s immediate area, and on a virtual map displayed in front of the user. For visualizations placed on the floor and on the map, we added a virtual link that connects the visualization with the corresponding real-world building. A user study with 18 participants indicates that visualizations placed on buildings are preferred, placing visualizations close to the user on a map results in the highest reading accuracy, and placing visualizations on buildings and on maps results in similar recall scores. Our study provides insight into the design of augmented data tours in outdoor environments: positioning AR visualizations close to participants seems to increase reading accuracy, and recalling information from visualizations placed on the floor seems to be more difficult than recalling information from visualizations placed on buildings.
Low-Fi VR Controller: Improved Mobile Virtual Reality Interaction via Camera-Based Tracking

Mobile virtual reality (VR) provides an accessible alternative to high-end VR, but currently offers only limited interaction, hindering its usability, the variety of VR experiences it can provide, and widespread adoption. To improve interaction on mobile VR, we present a novel input solution: the Low-Fi VR Controller. The controller uses the smartphone camera to track markers to provide 6DOF input. It costs virtually nothing as it is made of cost-effective accessible materials. We performed a user study based on Fitts’ law to evaluate the controller's performance in selection tasks, and to compare three selection activation methods (Instant, Dwell, Marker). Despite tracking issues, selection throughput with the Instant method was comparable to other similar ray-based selection techniques reported in other studies, at roughly 2.2 bps. Our results validate the controller as an acceptable 3D input device and will propose avenues to improve performance and user experience with the controller in future work.
Enhanced Transformer Interaction Components for Human-Object Interaction Detection

Human-Object Interaction task (HOI) aims to detect all <human, action, object> triplets in the image that exist interaction. General methods contain two-stage and one-stage algorithms. Recently, some works based on Transformer have been proposed. Transformer-based approaches directly detect HOI relationships in an end-to-end way, making the whole pipeline simpler. For the existing HOI detection model HOTR, we aim to optimize its internal structure of Transformer to better adapt to the HOI detection task. We use the residual structure to strengthen the components related to the interaction in the interaction detection, and input the enhanced results into the next decoder layer. Finally, the accuracy of the baseline method is improved to a certain extent, which is improved by 1.44% mAP and 3.21% mAP on the benchmark datasets V-COCO and HICO-DET respectively, and the accuracy exceeds most mainstream methods.
Positions of Wearable Keyboard in VR Environment

Wearable keyboards have been widely tried for text input with some set on the VR user. While the position of the wearable keyboard could yield some differences, such as ease of use and input speed, comparisons have not been comprehensive. Two experiments were conducted to investigate the impacts of the mounting positions and the user’s actions including standing and walking. Each position is almost the same in typing performance, but some are affected by user’s situations especially while walking. One interesting finding is that impressions about the position change before and after actual use. We recommend that the user should not rely too much on the first impression, and should actually try possible positions before use, the differences are not so large though.
A Study on Multi-User Interaction-based Redirected Walking

The most intuitive mode of locomotion in VR experiences is natural walking, which is constrained by the size of the walkable physical area. Perception-based locomotion techniques such as Redirected Walking (RDW) can circumvent this limitation by using artificial manipulations of the user’s position or rotation in VR. In this work, we focus on integrating redirection techniques into a continuous, uninterrupted VR experience and analyze the effects of interactions in VR between two remote users in a shared virtual space to hide discrete redirections from the user’s perception as much as possible. Our results show that discrete translations, which our subjects used to cover an average of 37.8% of their distance, were almost imperceptible during the execution of the collaborative interactions for 81% of our participants. Only 9.5% of our study participants noticed rotation gains. Based on these results, we provide guidelines on how developers can more effectively use RDW techniques in multi-user VR experiences.
"\"I consider VR Table Tennis to be my secret weapon!\": An Analysis of the VR Table Tennis Players' Experiences Outside the Lab"

Thanks to stand-alone Virtual Reality (VR) advances, users can play realistic simulations of real-life sports at their homes. In these game simulations, players control their avatars by doing the same movements as in real life (RL) while playing against a person or AI opponent, making VR sports attractive for the players. In this paper, we surveyed a popular VR table tennis game community, focusing on understanding their demographics, challenges, and experiences with skill transfers between VR and RL. Our results show that, on average, VR table tennis players are primarily men, live in Europe/Asia, and are 38 years old. We also found that the current state of VR technology affects the player’s experience and that players see VR as a convenient way to play matches but that RL is better for socialization. Finally, we identified skills like backhand and forehand strikes that players perceived to be transferred from VR to RL and vice versa. Our research findings have the potential to serve as a valuable resource for VR table tennis game developers seeking to integrate mid-air controllers into their future projects.
Sphere Saber: A Virtual Reality Exergame to Study Age-Related Differences in Selective Visual Attention and Information Processing

Virtual reality (VR) applications enable users to perform physical exercises from home. Due to the absence of a real trainer, visual cues are typically used to provide important information about game mechanics or the current tasks. In this paper, we investigate age-related differences regarding visual perception and processing of these game cues during a VR exergame. Therefore, we conducted a comparative user study with nine older and nine younger adults, in which participants had to hit balls of different colors, which approached them. Our results show that younger participants performed better in the game than older participants, especially if a dual-task with additional information was presented. This implies an effect of age on both, task performance, as well as information processing and memory. Eye-tracking data suggests that these differences are not caused by not seeing the visual information signs that were presented, but rather differences in the processing. Furthermore, older adults reported a higher feeling of presence and lower simulator sickness scores. These insights provide important implications for the development of VR exergames for different age groups, especially if additional game information needs to be displayed.
ARFit: Pose-based Exercise Feedback with Mobile AR

Fitness encompasses a diverse array of activities, including gym sessions, home workouts, and various other forms of physical exercise. The importance of proper muscle movement remains consistent across all these settings. Ensuring people do the physical movements correctly, avoid injury, gain insights and maintain motivation is traditionally accomplished with the help of an expert instructor. In the absence of an expert, the most common at-home training methods are books, videos, or apps but they provide limited feedback. In this work, we introduce ARFit, an augmented reality application that uses pose-tracking technology to capture user exercise movements and provide feedback to ensure accurate posture.
Blink Don't Wink: Exploring Blinks as Input for VR Games

Eye tracking enables several novel ways of interacting with virtual reality (VR) games, yet research on blink interaction techniques (BITs) in VR remains scarce. This short paper introduces a preliminary design space for BITs in VR and describes an exploratory user study comparing five games designed around five BITs. Results from questionnaires indicate that the BITs were satisfactory, and qualitative feedback highlight factors influencing players' preferences and the importance of considering fatigue from high blink rates and potential accessibility limitations of single-eyed interactions.
Gaze Cue Switcher for Joint Attention in VR Remote Tourism

In video remote communication, the Gaze Cue, which visualizes the direction of a partner’s gaze, effectively enhances topic sharing by facilitating mutual gaze between users. However, since this Gaze Cue appears within the user’s field of view, viewers may feel obstructed by that. Additionally, when users want to share a topic, they promote topic sharing by verbally expressing their intentions, such as saying, “Look at that". In this study, assuming VR remote tourism, we experimented with a school tour experience. We examined the relationship between the speed of joint attention and the type of Gaze Cue switched in response to voice.
Implementation of Interactive Mirror-Transcending Aerial Imaging System

Many methods have been proposed for presenting digital information in mirror space. However, most existing methods can only display digital information in mirror space. Therefore, we study the concept of MiTAI (Mirror-Transcending Aerial Imaging), which allows for the continuous movement of digital information between mirrors and physical space. MiTAI provides a novel interactive experience combining mirrors, physical space, and digital information. In this paper, we present the first systemization of MiTAI that users can experience, as well as an initial study toward a concept evaluation of MiTAI. We describe an implementation of the system in detail, including an optical follow-up of the previous study, the design of the control system, the presentation of content, and a method for interacting with the aerial image. The implemented system provided a MiTAI viewing experience for approximately 150 attendees through a hands-on demonstration exhibit and had a strong impact on them. We also report on user feedback from the exhibition and future improvements.
Volu-Me Fan: Turn Your Layered-fan into a Volumetric Display

This paper proposes Volu-Me, which displays multi-layer images by projecting onto a layered-fan rotating at high speed. In this system, each propeller part of the fan is projected by a high-speed projector. The rotation of the fan is measured by a sensor, and the projected image is generated in accordance with the rotation of the propeller section, which makes the system applicable to any fan. This system enables users to observe 3D layered images without wearing special equipment, and multiple users can observe at the same time. In this paper, we implement the prototype of the system and show the results of the projections, and discuss the potential and future extension of the system.