OpenRaiser/Intern-Atlas huggingface dataset:
Fault-Aware Robust Control via Adversarial Reinforcement Learning
Venue: arXiv cs.RO 2020
Year: 2020

Robots have limited adaptation ability compared to humans and animals in the case of damage. However, robot damages are prevalent in real-world applications, especially for robots deployed in extreme environments. The fragility of robots greatly limits their widespread application. We propose an adversarial reinforcement learning framework, which significantly increases robot robustness over joint damage cases in both manipulation tasks and locomotion tasks. The agent is trained iteratively under the joint damage cases where it has poor performance. We validate our algorithm on a three-fingered robot hand and a quadruped robot. Our algorithm can be trained only in simulation and directly deployed on a real robot without any fine-tuning. It also demonstrates exceeding success rates over arbitrary joint damage cases.
Predicting Rigid Body Dynamics using Dual Quaternion Recurrent Neural Networks with Quaternion Attention
Venue: arXiv cs.LG 2020
Year: 2020

We propose a novel neural network architecture based on dual quaternions which allow for a compact representation of informations with a main focus on describing rigid body movements. To cover the dynamic behavior inherent to rigid body movements, we propose recurrent architectures in the neural network. To further model the interactions between individual rigid bodies as well as external inputs efficiently, we incorporate a novel attention mechanism employing dual quaternion algebra. The introduced architecture is trainable by means of gradient based algorithms. We apply our approach to a parcel prediction problem where a rigid body with an initial position, orientation, velocity and angular velocity moves through a fixed simulation environment which exhibits rich interactions between the parcel and the boundaries.
PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization
Venue: arXiv cs.CV 2020
Year: 2020

We present a new framework for Patch Distribution Modeling, PaDiM, to concurrently detect and localize anomalies in images in a one-class learning setting. PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding, and of multivariate Gaussian distributions to get a probabilistic representation of the normal class. It also exploits correlations between the different semantic levels of CNN to better localize anomalies. PaDiM outperforms current state-of-the-art approaches for both anomaly detection and localization on the MVTec AD and STC datasets. To match real-world visual industrial inspection, we extend the evaluation protocol to assess performance of anomaly localization algorithms on non-aligned dataset. The state-of-the-art performance and low complexity of PaDiM make it a good candidate for many industrial applications.
BONIK: A Blockchain Empowered Chatbot for Financial Transactions
Venue: arXiv cs.CR 2020
Year: 2020

A Chatbot is a popular platform to enable users to interact with a software or website to gather information or execute actions in an automated fashion. In recent years, chatbots are being used for executing financial transactions, however, there are a number of security issues, such as secure authentication, data integrity, system availability and transparency, that must be carefully handled for their wide-scale adoption. Recently, the blockchain technology, with a number of security advantages, has emerged as one of the foundational technologies with the potential to disrupt a number of application domains, particularly in the financial sector. In this paper, we forward the idea of integrating a chatbot with blockchain technology in the view to improve the security issues in financial chatbots. More specifically, we present BONIK, a blockchain empowered chatbot for financial transactions, and discuss its architecture and design choices. Furthermore, we explore the developed Proof-of-Concept (PoC), evaluate its performance, analyse how different security and privacy issues are mitigated using BONIK.
Ginkgo -- A Math Library designed for Platform Portability
Venue: arXiv cs.DC 2020
Year: 2020

The first associations to software sustainability might be the existence of a continuous integration (CI) framework; the existence of a testing framework composed of unit tests, integration tests, and end-to-end tests; and also the existence of software documentation. However, when asking what is a common deathblow for a scientific software product, it is often the lack of platform and performance portability. Against this background, we designed the Ginkgo library with the primary focus on platform portability and the ability to not only port to new hardware architectures, but also achieve good performance. In this paper we present the Ginkgo library design, radically separating algorithms from hardware-specific kernels forming the distinct hardware executors, and report our experience when adding execution backends for NVIDIA, AMD, and Intel GPUs. We also comment on the different levels of performance portability, and the performance we achieved on the distinct hardware backends.
Distributed Online Learning with Multiple Kernels
Venue: arXiv cs.LG 2020
Year: 2020

In the Internet-of-Things (IoT) systems, there are plenty of informative data provided by a massive number of IoT devices (e.g., sensors). Learning a function from such data is of great interest in machine learning tasks for IoT systems. Focusing on streaming (or sequential) data, we present a privacy-preserving distributed online learning framework with multiplekernels (named DOMKL). The proposed DOMKL is devised by leveraging the principles of an online alternating direction of multipliers (OADMM) and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret, implying that every learned function achieves the performance of the best function in hindsight as in the state-of-the-art centralized online learning method. Moreover, it is ensured that the learned functions of any two neighboring learners have a negligible difference as T grows, i.e., the so-called consensus constraints hold. Via experimental tests with various real datasets, we verify the effectiveness of the proposed DOMKL on regression and time-series prediction tasks.
A User's Guide to Calibrating Robotics Simulators
Venue: arXiv cs.LG 2020
Year: 2020

Simulators are a critical component of modern robotics research. Strategies for both perception and decision making can be studied in simulation first before deployed to real world systems, saving on time and costs. Despite significant progress on the development of sim-to-real algorithms, the analysis of different methods is still conducted in an ad-hoc manner, without a consistent set of tests and metrics for comparison. This paper fills this gap and proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms. We open-source the benchmark, training data, and trained models, which can be found at https://github.com/NVlabs/sim-parameter-estimation.
Statistical model-based evaluation of neural networks
Venue: arXiv cs.LG 2020
Year: 2020

Using a statistical model-based data generation, we develop an experimental setup for the evaluation of neural networks (NNs). The setup helps to benchmark a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds. This allows us to test the effects of training data size, data dimension, data geometry, noise, and mismatch between training and testing conditions. In the proposed setup, we use a Gaussian mixture distribution to generate data for training and testing a set of competing NNs. Our experiments show the importance of understanding the type and statistical conditions of data for appropriate application and design of NNs
Traffic Characteristics of Virtual Reality over Edge-enabled Wi-Fi Networks
Venue: arXiv cs.NI 2020
Year: 2020

Virtual reality (VR) is becoming prevalent with a plethora of applications in education, healthcare, entertainment, etc. To increase the user mobility, and to reduce the energy consumption and production cost of VR head mounted displays (HMDs), wireless VR with edge-computing has been the focus of both industry and academia. However, transferring large video frames of VR applications with their stringent Quality of Service (QoS) requirements over wireless network requires innovations and optimizations across different network layers. In order to develop efficient architectures, protocols and scheduling mechanisms, the traffic characteristics of various types of VR applications are required. In this paper, we first compute the theoretical throughput requirements of an ideal VR experience as well as a popular VR HMD. We then examine the traffic characteristics of a set of VR applications using an edge-enabled Wi-Fi network. Our results reveal interesting findings that can be considered in developing new optimizations, protocols, access mechanisms and scheduling algorithms.
Study of Opportunistic Relaying and Jamming Based on Secrecy-Rate Maximization for Buffer-Aided Relay Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we investigate opportunistic relaying and jamming techniques and develop relay selection algorithms that maximize the secrecy rate for multiuser buffer-aided relay networks. We develop an approach to maximize the secrecy rate of relay systems that does not require the channel state information (CSI) of the eavesdroppers. We also devise relaying and jamming function selection (RJFS) algorithms to select multiple relay nodes as well as multiple jamming nodes to assist the transmission. In the proposed RJFS algorithms inter-relay interference cancellation (IC) is taken into account. IC is first performed to improve the transmission rate to legitimate users and then inter-relay IC is applied to amplify the jamming signal to the eavesdroppers and enhance the secrecy rate. With the buffer-aided relays the jamming signal can be stored at the relay nodes and a buffer-aided RJFS (BF-RJFS) algorithm is proposed. Greedy RJFS and BF-RJFS algorithms are then developed for relay selection with reduced complexity. Simulation results show that the proposed RJFS and BF-RJFS algorithms can achieve a higher secrecy rate performance than previously reported techniques even in the absence of CSI of the eavesdroppers.
Liquid Warping GAN with Attention: A Unified Framework for Human Image Synthesis
Venue: arXiv cs.CV 2020
Year: 2020

We tackle human image synthesis, including human motion imitation, appearance transfer, and novel view synthesis, within a unified framework. It means that the model, once being trained, can be used to handle all these tasks. The existing task-specific methods mainly use 2D keypoints to estimate the human body structure. However, they only express the position information with no abilities to characterize the personalized shape of the person and model the limb rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape. It can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose an Attentional Liquid Warping GAN with Attentional Liquid Warping Block (AttLWB) that propagates the source information in both image and feature spaces to the synthesized reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method can support a more flexible warping from multiple sources. To further improve the generalization ability of the unseen source images, a one/few-shot adversarial learning is applied. In detail, it firstly trains a model in an extensive training set. Then, it finetunes the model by one/few-shot unseen image(s) in a self-supervised way to generate high-resolution (512 x 512 and 1024 x 1024) results. Also, we build a new dataset, namely iPER dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis. Extensive experiments demonstrate the effectiveness of our methods in terms of preserving face identity, shape consistency, and clothes details. All codes and dataset are available on https://impersonator.org/work/impersonator-plus-plus.html.
On the Performance of RIS-Assisted Dual-Hop Mixed RF-UWOC Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we investigate the performance of a reconfigurable intelligent surface (RIS)-assisted dual-hop mixed radio-frequency underwater wireless optical communication (RF-UWOC) system. An RIS is an emerging and low-cost technology that aims to enhance the strength of the received signal, thus improving the system performance. In the considered system setup, a ground source does not have a reliable direct link to a given marine buoy and communicates with it through an RIS installed on a building. In particular, the buoy acts as a relay that sends the signal to an underwater destination. In this context, analytical expressions for the outage probability (OP), average bit error rate (ABER), and average channel capacity (ACC) are derived assuming fixed-gain amplify-and-forward (AF) and decode-and-forward (DF) relaying protocols at the marine buoy. Moreover, asymptotic analyses of the OP and ABER are carried out in order to gain further insights from the analytical frameworks. In particular, the system diversity order is derived and it is shown to depend on the RF link parameters and on the detection schemes of the UWOC link. Finally, it is demonstrated that RIS-assisted systems can effectively improve the performance of mixed dual-hop RF-UWOC systems.
Distributed Injection-Locking in Analog Ising Machines to Solve Combinatorial Optimizations
Venue: arXiv cs.ET 2020
Year: 2020

The oscillator-based Ising machine (OIM) is a network of coupled CMOS oscillators that solves combinatorial optimization problems. In this paper, the distribution of the injection-locking oscillations throughout the circuit is proposed to accelerate the phase-locking of the OIM. The implications of the proposed technique theoretically investigated and verified by extensive simulations in EDA tools with a $130~nm$ PTM model. By distributing the injective signal of the super-harmonic oscillator, the speed is increased by $219.8\%$ with negligible increase in the power dissipation and phase-locking error of the device due to the distributed technique.
Tracking and Visualizing Signs of Degradation for an Early Failure Prediction of a Rolling Bearing
Venue: arXiv cs.RO 2020
Year: 2020

Predictive maintenance, i.e. predicting failure to be few steps ahead of the fault, is one of the pillars of Industry 4.0. An effective method for that is to track early signs of degradation before a failure happens. This paper presents an innovative failure predictive scheme for machines. The proposed scheme combines the use of full spectrum of the vibration data caused by the machines and data visualization technologies. This scheme is featured by no training data required and by quick start after installation. First, we propose to use full spectrum (as high-dimensional data vector) with no cropping and no complex feature extraction and to visualize data behavior by mapping the high dimensional vectors into a 2D map. We then can ensure the simplicity of process and less possibility of overlooking of important information as well as providing a human-friendly and human-understandable output. Second, we propose Real-Time Data Tracker (RTDT) which predicts the failure at an appropriate time with sufficient time for maintenance by plotting real-time frequency spectrum data of the target machine on the 2D map composed from normal data. Third, we show the test results of our proposal using vibration data of bearings from real-world test-to-failure measurements provided by the public dataset, the IMS dataset.
Combining Gesture and Voice Control for Mid-Air Manipulation of CAD Models in VR Environments
Venue: arXiv cs.HC 2020
Year: 2020

Modeling 3D objects in domains like Computer Aided Design (CAD) is time-consuming and comes with a steep learning curve needed to master the design process as well as tool complexities. In order to simplify the modeling process, we designed and implemented a prototypical system that leverages the strengths of Virtual Reality (VR) hand gesture recognition in combination with the expressiveness of a voice-based interface for the task of 3D modeling. Furthermore, we use the Constructive Solid Geometry (CSG) tree representation for 3D models within the VR environment to let the user manipulate objects from the ground up, giving an intuitive understanding of how the underlying basic shapes connect. The system uses standard mid-air 3D object manipulation techniques and adds a set of voice commands to help mitigate the deficiencies of current hand gesture recognition techniques. A user study was conducted to evaluate the proposed prototype. The combination of our hybrid input paradigm shows to be a promising step towards easier to use CAD modeling.
What is a Process Model Composed of? A Systematic Literature Review of Meta-Models in BPM
Venue: arXiv cs.SE 2020
Year: 2020

Business process modelling languages typically enable the representation of business process models by employing (graphical) symbols. These symbols can vary depending upon the verbosity of the language, the modelling paradigm, the focus of the language, and so on. To make explicit the different constructs and rules employed by a specific language, as well as bridge the gap across different languages, meta-models have been proposed in literature. These meta-models are a crucial source of knowledge on what state-of-the-art literature considers relevant to describe business processes. The goal of this work is to provide an extensive systematic literature review (SLR) of business process meta-models. This SLR aims at answering research questions concerning: (i) the kind of meta-models proposed in literature; (ii) the recurring constructs they contain; (iii) their purposes; and (iv) their evaluations.
Communication-Aware Energy Efficient Trajectory Planning with Limited Channel Knowledge
Venue: arXiv cs.RO 2020
Year: 2020

Wireless communications is nowadays an important aspect of robotics. There are many applications in which a robot must move to a certain goal point while transmitting information through a wireless channel which depends on the particular trajectory chosen by the robot to reach the goal point. In this context, we develop a method to generate optimum trajectories which allow the robot to reach the goal point using little mechanical energy while transmitting as much data as possible. This is done by optimizing the trajectory (path and velocity profile) so that the robot consumes less energy while also offering good wireless channel conditions. We consider a realistic wireless channel model as well as a realistic dynamic model for the mobile robot (considered here to be a drone). Simulations results illustrate the merits of the proposed method.
Res-GCNN: A Lightweight Residual Graph Convolutional Neural Networks for Human Trajectory Forecasting
Venue: arXiv cs.CV 2020
Year: 2020

Autonomous driving vehicles (ADVs) hold great hopes to solve traffic congestion problems and reduce the number of traffic accidents. Accurate trajectories prediction of other traffic agents around ADVs is of key importance to achieve safe and efficient driving. Pedestrians, particularly, are more challenging to forecast due to their complex social in-teractions and randomly moving patterns. We propose a Residual Graph Convolutional Neural Network (Res-GCNN), which models the interactive behaviors of pedes-trians by using the adjacent matrix of the constructed graph for the current scene. Though the proposed Res-GCNN is quite lightweight with only about 6.4 kilo parameters which outperforms all other methods in terms of parameters size, our experimental results show an improvement over the state of art by 13.3% on the Final Displacement Error (FDE) which reaches 0.65 meter. As for the Average Dis-placement Error (ADE), we achieve a suboptimal result (the value is 0.37 meter), which is also very competitive. The Res-GCNN is evaluated in the platform with an NVIDIA GeForce RTX1080Ti GPU, and its mean inference time of the whole dataset is only about 2.2 microseconds. Compared with other methods, the proposed method shows strong potential for onboard application accounting for forecasting accuracy and time efficiency. The code will be made publicly available on GitHub.
CGAP2: Context and gap aware predictive pose framework for early detection of gestures
Venue: arXiv cs.CV 2020
Year: 2020

With a growing interest in autonomous vehicles' operation, there is an equally increasing need for efficient anticipatory gesture recognition systems for human-vehicle interaction. Existing gesture-recognition algorithms have been primarily restricted to historical data. In this paper, we propose a novel context and gap aware pose prediction framework(CGAP2), which predicts future pose data for anticipatory recognition of gestures in an online fashion. CGAP2 implements an encoder-decoder architecture paired with a pose prediction module to anticipate future frames followed by a shallow classifier. CGAP2 pose prediction module uses 3D convolutional layers and depends on the number of pose frames supplied, the time difference between each pose frame, and the number of predicted pose frames. The performance of CGAP2 is evaluated on the Human3.6M dataset with the MPJPE metric. For pose prediction of 15 frames in advance, an error of 79.0mm is achieved. The pose prediction module consists of only 26M parameters and can run at 50 FPS on the NVidia RTX Titan. Furthermore, the ablation study indicates supplying higher context information to the pose prediction module can be detrimental for anticipatory recognition. CGAP2 has a 1-second time advantage compared to other gesture recognition systems, which can be crucial for autonomous vehicles.
Specifying Autonomy in the Internet of Things: The Autonomy Model and Notation
Venue: arXiv cs.HC 2020
Year: 2020

Driven by digitization in society and industry, automating behavior in an autonomous way substantially alters industrial value chains in the smart service world. As processes are enhanced with sensor and actuator technology, they become digitally interconnected and merge into an Internet of Things (IoT) to form cyber-physical systems (CPS). Using these automated systems, enterprises can improve the performance and quality of their operations. However, currently it is neither feasible nor reasonable to equip any machine with full autonomy when networking with other machines or people. It is necessary to specify rules for machine behavior that also determine an adequate degree of autonomy to realize the potential benefits of the IoT. Yet, there is a lack of methodologies and guidelines to support the design and implementation of machines as explicit autonomous agents such that many designs only consider autonomy implicitly. To address this research gap, we perform a comprehensive literature review to extract 12 requirements for the design of autonomous agents in the IoT. We introduce a set of constitutive characteristics for agents and introduce a classification framework for interactions in multi-agent systems. We integrate our findings by developing a conceptual modeling language consisting of a meta model and a notation that facilitates the specification and design of autonomous agents within the IoT as well as CPS: the Autonomy Model and Notation. We illustrate and discuss the approach and its limitations.