Reviewerly/exHarmony huggingface dataset:
Human Factors Research in Immersive Virtual Reality Firefighter Training: A Systematic Review

Immersive virtual reality (VR) shows a lot of potential for the training of professionals in the emergency response domain. Firefighters occupy a unique position among emergency personnel as the threats they encounter are mainly environmental. Immersive VR therefore represents a great opportunity to be utilized for firefighter training. This systematic review summarizes the existing literature of VR firefighting training that has a specific focus on human factors and learning outcomes, as opposed to literature that solely covers the system, or simulation, with little consideration given to its user. An extensive literature search followed by rigorous filtering of publications with narrowly defined criteria was performed to aggregate results from methodologically sound user studies. The included studies provide evidence that suggests the suitability of VR firefighter training, especially in search and rescue and commander training scenarios. Although the overall number of publications is small, the viability of VR as an ecologically valid analog to real-life training is promising. In the future, more work is needed to establish clear evidence and guidelines to optimize the effectiveness of VR training and to increase reliable data through appropriate research endeavors.
Asynchronous Reliability-Aware Multi-UAV Coverage Path Planning

Graceful degradation is a potential advantage of Multi-Robot Systems over Single-Robot Systems. In aerial robotics applications, such as infrastructure inspection, this trait is desirable as it would improve mission reliability despite the use of failure-prone low-cost drones. The Reliability-Aware Multi-Agent Coverage Path Planning (RA-MCPP) problem finds path plans for each robot to maximise the probability of mission completion by a given deadline. This paper proposes a path planner for RA-MCPP formulated in continuous time, enabling more complex realistic environments to be considered. The proposed method (i) extends a reliability evaluation framework to evaluate the Probability of Completion metric on asynchronous strategies on non-unit lattice graph environments, and (ii) introduces a greedy-genetic meta-heuristic optimisation method as a scalable and accurate RA-MCPP solver. This method is shown to provide plans with higher reliability when compared with existing approaches in three real inspection scenarios.
Design and Development of a Low-cost Device for Weight and Center of Gravity Simulation in Virtual Reality

With rapid advances in virtual reality (VR) technology, the use of haptics has become important to allow users to feel the physical properties of virtual objects. Current research has focused mainly on either weight variation or changing the center of gravity, which limits the simulation potential and may affect the feeling of immersion. This research explores the design and development of a device that can simulate both weight and center of gravity using low-cost components. Through an iterative design process and continuous testing with users, we arrived at a final prototype, FluidWeight, a device that can be attached to a typical VR handheld controller. FluidWeight uses fluid, which is transported from a central storage to a receptacle attached to the controller. A final experiment shows that users enjoyed using it because it could help increase the sense of realism in VR applications.
A SAR-to-Optical Image Translation Method Based on PIX2PIX

Optical remote sensing images are susceptible to adverse weather effects, such as cloud occlusion, which lead to low availability of optical image data. However, synthetic aperture radar (SAR) can well overcome these shortcomings of optical imaging because of SAR working in an all-weather environment. Due to the imaging mechanism of SAR image is essentially different from optical image, the interpretation of SAR image is a huge challenge. Inspired by the powerful image-to-image translation capability of Generative Adversarial Networks (GANs), this paper proposes an improved Pix2Pix network to achieve the translation task from SAR image to optical image. In order to better maintain the structural information after image translation, this paper introduces a constraint based on phase consistency as the consideration of the loss function. Experimental results show that the proposed method has obvious advantages for multimodal remote sensing data translation tasks in comparison with the current state-of-the-art methods.
Sports Enterprise Marketing and Financial Risk Management Based on Decision Tree and Data Mining

With the development of modern economy, traditional sports industry enterprises have also been further developed in the financial business. How to ensure information security and financial risk management is the problem faced by sports companies. Risk assessment is the use of mathematical models to calculate the risk factors established in the previous step to predict possible risks. In response to the above problems, we developed a sports enterprise marketing and financial risk management model based on decision trees and data mining. First, we have established a relevant evaluation index system and data samples through in-depth understanding of the actual marketing and financial problems of sports companies. Second, we use the decision tree algorithm to mine and explore related data samples and conduct risk assessment through related indicators. By using the model to calculate the probability of occurrence of the risk, analyze the degree of damage. Finally, the algorithm of this paper is analyzed and discussed through simulation experiments.
An Interpretable Machine Learning Model Enhanced Integrated CPU-GPU DVFS Governor

Modern heterogeneous CPU-GPU-based mobile architectures, which execute intensive mobile gaming/graphics applications, use software governors to achieve high performance with energy-efficiency. However, existing governors typically utilize simple statistical or heuristic models, assuming linear relationships using a small unbalanced dataset of mobile games; and the limitations result in high prediction errors for dynamic and diverse gaming workloads on heterogeneous platforms. To overcome these limitations, we propose an interpretable machine learning (ML) model enhanced integrated CPU-GPU governor: (1) It builds tree-based piecewise linear models (i.e., model trees) offline considering both high accuracy (low error) and interpretable ML models based on mathematical formulas using a simulatability operation counts quantitative metric. And then (2) it deploys the selected models for online estimation into an integrated CPU-GPU Dynamic Voltage Frequency Scaling governor. Our experiments on a test set of 20 mobile games exhibiting diverse characteristics show that our governor achieved significant energy efficiency gains of over 10% (up to 38%) improvements on average in energy-per-frame with a surprising-but-modest 3% improvement in Frames-per-Second performance, compared to a typical state-of-the-art governor that employs simple linear regression models.
The use of AR to preserve and popularize traditional Chinese musical instruments as part of the formation of the tourist attractiveness of the national art of Guizhou province

Abstract This article explores augmented reality (AR) possibilities for the preservation of national cultural art, popularization of Chinese musical instruments among the youth, and meeting the tourist interest in the cultural heritage of China in a fascinating interactive form. Using the generated database containing information on traditional musical instruments of the Guizhou province, the authors have developed an AR application adapted for educational and tourist purposes. Its central aim was to acquaint people with traditional Chinese musical instruments in the context of familiarization with the national culture of China. Testing of the developed application was carried out by hundred students from Guizhou Minzu University (College of Tourism and Steward, the College of Media, the College of Music and Dance, and the College of Data Science and Information Engineering) as part of the National Culture and Art study course. As a result, the created AR app was proved to expand the understanding of the culture and traditions of the Chinese people, intensify the perception of national musical identity, increase interactivity-caused inspiration and involvement (89%), and enhance interest in new knowledge (93%). Based on the obtained research outcomes, a model for the popularization of traditional Chinese musical instruments in the structure of the integrated national tourism product of Guizhou province was developed. The article is believed to be useful for individuals studying the national cultural heritage of China and educators seeking innovative solutions that improve the educational practice.
High-Speed Robot Navigation using Predicted Occupancy Maps

Safe and high-speed navigation is a key enabling capability for real world deployment of robotic systems. A significant limitation of existing approaches is the computational bottleneck associated with explicit mapping and the limited field of view (FOV) of existing sensor technologies. In this paper, we study algorithmic approaches that allow the robot to predict spaces extending beyond the sensor horizon for robust planning at high speeds. We accomplish this using a generative neural network trained from real-world data without requiring human annotated labels. Further, we extend our existing control algorithms to support leveraging the predicted spaces to improve collision-free planning and navigation at high speeds. Our experiments are conducted on a physical robot based on the MIT race car using an RGBD sensor where were able to demonstrate improved performance at 4 m/s compared to a controller not operating on predicted regions of the map.
DeepGame

Cloud gaming enables users to play games on virtually any device. This is achieved by offloading the game rendering and encoding to cloud datacenters. As game resolutions and frame rates increase, cloud gaming platforms face a major challenge to stream high quality games due to the high bandwidth and low latency requirements. In this paper, we propose a new video encoding pipeline, called DeepGame, for cloud gaming platforms to reduce the bandwidth requirements with limited to no impact on the player quality of experience. DeepGame learns the player's contextual interest in the game and the temporal correlation of that interest using a spatio-temporal deep neural network. Then, it encodes various areas in the video frames with different quality levels proportional to their contextual importance. DeepGame does not change the source code of the video encoder or the video game, and it does not require any additional hardware or software at the client side. We implemented DeepGame in an open-source cloud gaming platform and evaluated its performance using multiple popular games. We also conducted a subjective study with real players to demonstrate the potential gains achieved by DeepGame and its practicality. Our results show that DeepGame can reduce the bandwidth requirements by up to 36% compared to the baseline encoder, while maintaining the same level of perceived quality for players and running in real time.
Latent Effects for Reusable Language Components

The development of programming languages can be quite complicated and costly. Hence, much effort has been devoted to the modular definition of language features that can be reused in various combinations to define new languages and experiment with their semantics. A notable outcome of these efforts is the algebra-based “datatypes à la carte” (DTC) approach. When combined with algebraic effects, DTC can model a wide range of common language features. Unfortunately, the current state of the art does not cover modular definitions of advanced control-flow mechanisms that defer execution to an appropriate point, such as call-by-name and call-by-need evaluation, as well as (multi-)staging.
Combining Attention with Flow for Person Image Synthesis

Pose-guided person image synthesis aims to synthesize person images by transforming reference images into target poses. In this paper, we observe that the commonly used spatial transformation blocks have complementary advantages. We propose a novel model by combining the attention operation with the flow-based operation. Our model not only takes the advantage of the attention operation to generate accurate target structures but also uses the flow-based operation to sample realistic source textures. Both objective and subjective experiments demonstrate the superiority of our model. Meanwhile, comprehensive ablation studies verify our hypotheses and show the efficacy of the proposed modules. Besides, additional experiments on the portrait image editing task demonstrate the versatility of the proposed combination.
Multiple Objects-Aware Visual Question Generation

Visual question generation task aims to generate meaningful questions about an image according to a target answer. Existing studies mainly focus on merely one object related to the target answer in an image to generate a question. However, a target answer is often related to multiple key objects in an image, which focuses on only one object may mislead its model to generate questions that are only related to partial fragments of the answer. To address this problem, we propose a multi-objects aware generation model to capture all key objects related to an answer and generate the corresponding question. We first introduce a co-attention network to capture the relationship between each object in an image and the answer, and then extract the key objects that are related to the answer. Then, a graph network is introduced to capture the relationships between the key objects and other objects in the image that are not related to the answer, which helps generate questions that involve more visual content. Finally, the learned information from the graph network is fed into a standard decoder module to produce questions. Extensive experiments on the VQA v2.0 dataset show that the proposed model outperforms the state-of-the-art models.
State-aware Video Procedural Captioning

Video procedural captioning (VPC), which generates procedural text from instructional videos, is an essential task for scene understanding and real-world applications. The main challenge of VPC is to describe how to manipulate materials accurately. This paper focuses on this challenge by designing a new VPC task, generating a procedural text from the clip sequence of an instructional video and material list. In this task, the state of materials is sequentially changed by manipulations, yielding their state-aware visual representations (e.g., eggs are transformed into cracked, stirred, then fried forms). The essential difficulty is to convert such visual representations into textual representations; that is, a model should track the material states after manipulations to better associate the cross-modal relations. To achieve this, we propose a novel VPC method, which modifies an existing textual simulator for tracking material states as a visual simulator and incorporates it into a video captioning model. Our experimental results show the effectiveness of the proposed method, which outperforms state-of-the-art video captioning models. We further analyze the learned embedding of materials to demonstrate that the simulators capture their state transition. The code and dataset are available from https://github.com/misogil0116/svpc
Sample-efficient Reinforcement Learning in Robotic Table Tennis

Reinforcement learning (RL) has achieved some impressive recent successes in various computer games and simulations. Most of these successes are based on having large numbers of episodes from which the agent can learn. In typical robotic applications, however, the number of feasible attempts is very limited. In this paper we present a sample-efficient RL algorithm applied to the example of a table tennis robot. In table tennis every stroke is different, with varying placement, speed and spin. An accurate return therefore has to be found depending on a high-dimensional continuous state space. To make learning in few trials possible the method is embedded into our robot system. In this way we can use a one-step environment. The state space depends on the ball at hitting time (position, velocity, spin) and the action is the racket state (orientation, velocity) at hitting. An actor-critic based deterministic policy gradient algorithm was developed for accelerated learning. Our approach performs competitively both in a simulation and on the real robot in a number of challenging scenarios. Accurate results are obtained without pre-training in under 200 episodes of training. The video presenting our experiments is available at https://youtu.be/uRAtdoL6Wpw.
Quandary: An open-source C++ package for high-performance optimal control of open quantum systems

Quantum optimal control can be used to shape the control pulses for realizing unitary and non-unitary transformations of quantum states. These control pulses provide the fundamental interface between the quantum compiler and the quantum hardware. Most current software for quantum optimal control (e.g. Qutip [1] or Krotov [2]) is restricted to run on shared memory platforms, limiting their applicability to smaller quantum systems, in particular if interactions with the environment are taken into account. This paper gives an overview of the open-source code Quandary, which is designed to solve quantum control problems in larger open quantum systems modelled by Lindblad’s master equation. Implemented in C++, Quandary uses the message passing paradigm for distributed memory computers that enables scalability to large numbers of compute cores. Accompanied by numerical examples, this paper presents an overview on existing theoretical developments for open optimal quantum control realizing state-to-state transfer, unitary gate optimization as well as state-preparation, and presents the numerical tools and implementation aspect as realized in Quandary, for deployment on modern high-performance computing platforms.
Deep Self-Supervised t-SNE for Multi-modal Subspace Clustering

Existing multi-modal subspace clustering methods, aiming to exploit the correlation information between different modalities, have achieved promising preliminary results. However, these methods might be incapable of handling real problems with complex heterogeneous structures between different modalities, since the large heterogeneous structure makes it difficult to directly learn a discriminative shared self-representation for multi-modal clustering. To tackle this problem, in this paper, we propose a deep Self-supervised t-SNE method (StSNE) for multi-modal subspace clustering, which learns soft label features by multi-modal encoders and utilizes the common label feature to supervise soft label feature of each modal by adversarial training and reconstruction networks. Specifically, the proposed StSNE consists of four components: 1) multi-modal convolutional encoders; 2) a self-supervised t-SNE module; 3) a self-expressive layer; 4) multi-modal convolutional decoders. Multi-modal data are fed to encoders to obtain soft label features, for which the self-supervised t-SNE module is added to make full use of the label information among different modalities. Simultaneously, the latent representations given by encoders are constrained by a self-expressive layer to capture the hierarchical information of each modal, followed by decoders reconstructing the encoded features to preserve the structure of the original data. Experimental results on several public datasets demonstrate the superior clustering performance of the proposed method over state-of-the-art methods.
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

For a given video-based Human-Object Interaction scene, modeling the spatio-temporal relationship between humans and objects is the important cue to understand the contextual information presented in the video. With the efficient spatio-temporal relationship modeling, it is possible not only to uncover contextual information in each frame, but to directly capture inter-frame dependencies as well. Capturing the position changes of human and objects over the spatio-temporal dimension is more critical when significant changes in the appearance features may not occur over time. When utilizing appearance features, the spatial location and the semantic information are also the key to improve the video-based Human-Object Interaction recognition performance. In this paper, Spatio-Temporal Interaction Graph Parsing Networks (STIGPN) are constructed, which encode the videos with a graph composed of human and object nodes. These nodes are connected by two types of relations: (i) intra-frame relations: modeling the interactions between human and the interacted objects within each frame. (ii) inter-frame relations: capturing the long range dependencies between human and the interacted objects across frame. With the graph, STIGPN learn spatio-temporal features directly from the whole video-based Human-Object Interaction scenes. Multi-modal features and a multi-stream fusion strategy are used to enhance the reasoning capability of STIGPN. Two Human-Object Interaction video datasets, including CAD-120 and Something-Else, are used to evaluate the proposed architectures, and the state-of-the-art performance demonstrates the superiority of STIGPN. Code for STIGPN is available at https://github.com/GuangmingZhu/STIGPN.
An On-Line POMDP Solver for Continuous Observation Spaces

Planning under partial obervability is essential for autonomous robots. A principled way to address such planning problems is the Partially Observable Markov Decision Process (POMDP). Although solving POMDPs is computationally intractable, substantial advancements have been achieved in developing approximate POMDP solvers in the past two decades. However, computing robust solutions for problems with continuous observation spaces remains challenging. Most on-line solvers rely on discretising the observation space or artificially limiting the number of observations that are considered during planning to compute tractable policies. In this paper we propose a new on-line POMDP solver, called Lazy Belief Extraction for Continuous Observation POMDPs (LABECOP), that combines methods from Monte-Carlo-Tree-Search and particle filtering to construct a policy reprentation which doesn't require discretised observation spaces and avoids limiting the number of observations considered during planning. Experiments on three different problems involving continuous observation spaces indicate that LABECOP performs similar or better than state- of-the-art POMDP solvers.
Unsupervised Learned Kalman Filtering

In this paper we adapt KalmanNet, which is a recently proposed deep neural network (DNN)-aided system whose architecture follows the operation of the model-based Kalman filter (KF), to learn its mapping in an unsupervised manner, i.e., without requiring ground-truth states. The unsupervised adaptation is achieved by exploiting the hybrid model-based/data-driven architecture of KalmanNet, which internally predicts the next observation as the KF does. These internal features are then used to compute the loss rather than the state estimate at the output of the system. With the capability of unsupervised learning, one can use KalmanNet not only to track the hidden state, but also to adapt to variations in the state space (SS) model. We numerically demonstrate that when the noise statistics are unknown, unsupervised KalmanNet achieves a similar performance to KalmanNet with supervised learning. We also show that we can adapt a pre-trained KalmanNet to changing SS models without providing additional data thanks to the unsupervised capabilities.