pwc-archive/papers-with-abstracts huggingface dataset:
Characterizing Attacks on Deep Reinforcement Learning
Authors: Xinlei Pan, Chaowei Xiao, Warren He, Shuang Yang, Jian Peng, MingJie Sun, JinFeng Yi, Zijiang Yang, Mingyan Liu, Bo Li, Dawn Song
Date: 2019-07-21 00:00:00
Tasks: Atari Games, Autonomous Driving, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent studies show that Deep Reinforcement Learning (DRL) models are vulnerable to adversarial attacks, which attack DRL models by adding small perturbations to the observations. However, some attacks assume full availability of the victim model, and some require a huge amount of computation, making them less feasible for real world applications. In this work, we make further explorations of the vulnerabilities of DRL by studying other aspects of attacks on DRL using realistic and efficient attacks. First, we adapt and propose efficient black-box attacks when we do not have access to DRL model parameters. Second, to address the high computational demands of existing attacks, we introduce efficient online sequential attacks that exploit temporal consistency across consecutive steps. Third, we explore the possibility of an attacker perturbing other aspects in the DRL setting, such as the environment dynamics. Finally, to account for imperfections in how an attacker would inject perturbations in the physical world, we devise a method for generating a robust physical perturbations to be printed. The attack is evaluated on a real-world robot under various conditions. We conduct extensive experiments both in simulation such as Atari games, robotics and autonomous driving, and on real-world robotics, to compare the effectiveness of the proposed attacks with baseline approaches. To the best of our knowledge, we are the first to apply adversarial attacks on DRL systems to physical robots.
A Learning-Based Two-Stage Spectrum Sharing Strategy with Multiple Primary Transmit Power Levels
Authors: Rui Zhang, Peng Cheng, Zhuo Chen, Yonghui Li, Branka Vucetic
Date: 2019-07-21 00:00:00
Tasks: Prediction, Reinforcement Learning

Multi-parameter cognition in a cognitive radio network (CRN) provides a more thorough understanding of the radio environments, and could potentially lead to far more intelligent and efficient spectrum usage for a secondary user. In this paper, we investigate the multi-parameter cognition problem for a CRN where the primary transmitter (PT) radiates multiple transmit power levels, and propose a learning-based two-stage spectrum sharing strategy. We first propose a data-driven/machine learning based multi-level spectrum sensing scheme, including the spectrum learning (Stage I) and prediction (the first part in Stage II). This fully blind sensing scheme does not require any prior knowledge of the PT power characteristics. Then, based on a novel normalized power level alignment metric, we propose two prediction-transmission structures, namely periodic and non-periodic, for spectrum access (the second part in Stage II), which enable the secondary transmitter (ST) to closely follow the PT power level variation. The periodic structure features a fixed prediction interval, while the non-periodic one dynamically determines the interval with a proposed reinforcement learning algorithm to further improve the alignment metric. Finally, we extend the prediction-transmission structure to an online scenario, where the number of PT power levels might change as a consequence of PT adapting to the environment fluctuation or quality of service variation. The simulation results demonstrate the effectiveness of the proposed strategy in various scenarios.
Heuristic solutions to robust variants of the minimum-cost integer flow problem
Authors: Marko Špoljarec, Robert Manger
Date: 2019-07-21 00:00:00
Tasks: ARC

This paper deals with robust optimization applied to network flows. Two robust variants of the minimum-cost integer flow problem are considered. Thereby, uncertainty in problem formulation is limited to arc unit costs and expressed by a finite set of explicitly given scenarios. It is shown that both problem variants are NP-hard. To solve the considered variants, several heuristics based on local search or evolutionary computing are proposed. The heuristics are experimentally evaluated on appropriate problem instances.
Direct Quantification for Coronary Artery Stenosis Using Multiview Learning
Authors: Dong Zhang, Guang Yang, Shu Zhao, Yanping Zhang, Heye Zhang, Shuo Li
Date: 2019-07-20 00:00:00
Tasks: Multiview Learning, regression

The quantification of the coronary artery stenosis is of significant clinical importance in coronary artery disease diagnosis and intervention treatment. It aims to quantify the morphological indices of the coronary artery lesions such as minimum lumen diameter, reference vessel diameter, lesion length, and these indices are the reference of the interventional stent placement. In this study, we propose a direct multiview quantitative coronary angiography (DMQCA) model as an automatic clinical tool to quantify the coronary artery stenosis from X-ray coronary angiography images. The proposed DMQCA model consists of a multiview module with two attention mechanisms, a key-frame module, and a regression module, to achieve direct accurate multiple-index estimation. The multi-view module comprehensively learns the Spatio-temporal features of coronary arteries through a three-dimensional convolution. The attention mechanisms of each view focus on the subtle feature of the lesion region and capture the important context information. The key-frame module learns the subtle features of the stenosis through successive dilated residual blocks. The regression module finally generates the indices estimation from multiple features.
Arena: a toolkit for Multi-Agent Reinforcement Learning
Authors: Qing Wang, Jiechao Xiong, Lei Han, Meng Fang, Xinghai Sun, Zhuobin Zheng, Peng Sun, Zhengyou Zhang
Date: 2019-07-20 00:00:00
Tasks: Multi-agent Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft, Starcraft II

We introduce Arena, a toolkit for multi-agent reinforcement learning (MARL) research. In MARL, it usually requires customizing observations, rewards and actions for each agent, changing cooperative-competitive agent-interaction, and playing with/against a third-party agent, etc. We provide a novel modular design, called Interface, for manipulating such routines in essentially two ways: 1) Different interfaces can be concatenated and combined, which extends the OpenAI Gym Wrappers concept to MARL scenarios. 2) During MARL training or testing, interfaces can be embedded in either wrapped OpenAI Gym compatible Environments or raw environment compatible Agents. We offer off-the-shelf interfaces for several popular MARL platforms, including StarCraft II, Pommerman, ViZDoom, Soccer, etc. The interfaces effectively support self-play RL and cooperative-competitive hybrid MARL. Also, Arena can be conveniently extended to your own favorite MARL platform.
VS-Net: Variable splitting network for accelerated parallel MRI reconstruction
Authors: Jinming Duan, Jo Schlemper, Chen Qin, Cheng Ouyang, Wenjia Bai, Carlo Biffi, Ghalib Bello, Ben Statton, Declan P. O'Regan, Daniel Rueckert
Date: 2019-07-19 00:00:00
Tasks: compressed sensing, Deep Learning, MRI Reconstruction, Rolling Shutter Correction

In this work, we propose a deep learning approach for parallel magnetic resonance imaging (MRI) reconstruction, termed a variable splitting network (VS-Net), for an efficient, high-quality reconstruction of undersampled multi-coil MR data. We formulate the generalized parallel compressed sensing reconstruction as an energy minimization problem, for which a variable splitting optimization method is derived. Based on this formulation we propose a novel, end-to-end trainable deep neural network architecture by unrolling the resulting iterative process of such variable splitting scheme. VS-Net is evaluated on complex valued multi-coil knee images for 4-fold and 6-fold acceleration factors. We show that VS-Net outperforms state-of-the-art deep learning reconstruction algorithms, in terms of reconstruction accuracy and perceptual quality. Our code is publicly available at https://github.com/j-duan/VS-Net.
An Actor-Critic-Attention Mechanism for Deep Reinforcement Learning in Multi-view Environments
Authors: Elaheh Barati, Xue-wen Chen
Date: 2019-07-19 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In reinforcement learning algorithms, leveraging multiple views of the environment can improve the learning of complicated policies. In multi-view environments, due to the fact that the views may frequently suffer from partial observability, their level of importance are often different. In this paper, we propose a deep reinforcement learning method and an attention mechanism in a multi-view environment. Each view can provide various representative information about the environment. Through our attention mechanism, our method generates a single feature representation of environment given its multiple views. It learns a policy to dynamically attend to each view based on its importance in the decision-making process. Through experiments, we show that our method outperforms its state-of-the-art baselines on TORCS racing car simulator and three other complex 3D environments with obstacles. We also provide experimental results to evaluate the performance of our method on noisy conditions and partial observation settings.
Detection of Real-world Driving-induced Affective State Using Physiological Signals and Multi-view Multi-task Machine Learning
Authors: Daniel Lopez-Martinez, Neska El-Haouij, Rosalind Picard
Date: 2019-07-19 00:00:00

Affective states have a critical role in driving performance and safety. They can degrade driver situation awareness and negatively impact cognitive processes, severely diminishing road safety. Therefore, detecting and assessing drivers' affective states is crucial in order to help improve the driving experience, and increase safety, comfort and well-being. Recent advances in affective computing have enabled the detection of such states. This may lead to empathic automotive user interfaces that account for the driver's emotional state and influence the driver in order to improve safety. In this work, we propose a multiview multi-task machine learning method for the detection of driver's affective states using physiological signals. The proposed approach is able to account for inter-drive variability in physiological responses while enabling interpretability of the learned models, a factor that is especially important in systems deployed in the real world. We evaluate the models on three different datasets containing real-world driving experiences. Our results indicate that accounting for drive-specific differences significantly improves model performance.
Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
Authors: Luca Biferale, Fabio Bonaccorso, Michele Buzzicotti, Patricio Clark Di Leoni, Kristian Gustavsson
Date: 2019-07-17 00:00:00
Tasks: Navigate, Reinforcement Learning, Reinforcement Learning (RL)

To find the path that minimizes the time to navigate between two given points in a fluid flow is known as Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel which has a slip velocity with fixed intensity, Vs , but variable direction and navigating in a 2D turbulent sea. We show that an Actor-Critic RL algorithm is able to find quasi-optimal solutions for both time-independent and chaotically evolving flow configurations. For the frozen case, we also compared the results with strategies obtained analytically from continuous Optimal Navigation (ON) protocols. We show that for our application, ON solutions are unstable for the typical duration of the navigation process, and are therefore not useful in practice. On the other hand, RL solutions are much more robust with respect to small changes in the initial conditions and to external noise, even when V s is much smaller than the maximum flow velocity. Furthermore, we show how the RL approach is able to take advantage of the flow properties in order to reach the target, especially when the steering speed is small.
DeepDA: LSTM-based Deep Data Association Network for Multi-Targets Tracking in Clutter
Authors: Huajun Liu, HUI ZHANG, Christoph Mertz
Date: 2019-07-16 00:00:00
Tasks: Combinatorial Optimization

The Long Short-Term Memory (LSTM) neural network based data association algorithm named as DeepDA for multi-target tracking in clutters is proposed to deal with the NP-hard combinatorial optimization problem in this paper. Different from the classical data association methods involving complex models and accurate prior knowledge on clutter density, filter covariance or associated gating etc, data-driven deep learning methods have been extensively researched for this topic. Firstly, data association mathematical problem for multitarget tracking on unknown target number, missed detection and clutter, which is beyond one-to-one mapping between observations and targets is redefined formally. Subsequently, an LSTM network is designed to learn the measurement-to-track association probability from radar noisy measurements and exist tracks. Moreover, an LSTM-based data-driven deep neural network after a supervised training through the BPTT and RMSprop optimization method can get the association probability directly. Experimental results on simulated data show a significant performance on association ratio, target ID switching and time-consuming for tracking multiple targets even they are crossing each other in the complicated clutter environment.
Vadere: An open-source simulation framework to promote interdisciplinary understanding
Authors: Benedikt Kleinmeier, Benedikt Zönnchen, Marion Gödel, Gerta Köster
Date: 2019-07-16 00:00:00

Pedestrian dynamics is an interdisciplinary field of research. Psychologists, sociologists, traffic engineers, physicists, mathematicians and computer scientists all strive to understand the dynamics of a moving crowd. In principle, computer simulations offer means to further this understanding. Yet, unlike for many classic dynamical systems in physics, there is no universally accepted locomotion model for crowd dynamics. On the contrary, a multitude of approaches, with very different characteristics, compete. Often only the experts in one special model type are able to assess the consequences these characteristics have on a simulation study. Therefore, scientists from all disciplines who wish to use simulations to analyze pedestrian dynamics need a tool to compare competing approaches. Developers, too, would profit from an easy way to get insight into an alternative modeling ansatz. Vadere meets this interdisciplinary demand by offering an open-source simulation framework that is lightweight in its approach and in its user interface while offering pre-implemented versions of the most widely spread models.
Ghost-free multi exposure image fusion technique using dense SIFT descriptor and guided filter
Authors: Naila Hayat, Muhammad Imran
Date: 2019-07-01 00:00:00
Proceeding: journal 2019 7
Tasks: Multi-Exposure Image Fusion

A ghost-free multi-exposure image fusion technique using the dense SIFT descriptor and the guided filter is proposed in this paper. The results suggest that the presented scheme produces high-quality images using ordinary cameras and that too without the ghosting artifact. To do so, the dense SIFT descriptor is used to extract the local contrast information from source images. Whereas, for the dynamic scenes, the histogram equalization and median filtering are used to calculate the color dissimilarity feature. Three weighting terms: local contrast, brightness, and color dissimilarity feature are used to estimate the initial weights. The estimated initial weights contain discontinuities. Therefore, the guided filter is used to remove the noise and discontinuity in initial weights. Finally, the fusion is performed using a pyramid decomposition method. Experimental results prove the superiority of the proposed technique over existing state-of-the-art methods in terms of both subjective and objective evaluation.
A Fine-Grained Spectral Perspective on Neural Networks
Authors: Greg Yang, Hadi Salman
Date: 2019-07-24 00:00:00

Are neural networks biased toward simple functions? Does depth always help learn more complex features? Is training the last layer of a network as good as training all layers? How to set the range for learning rate tuning? These questions seem unrelated at face value, but in this work we give all of them a common treatment from the spectral perspective. We will study the spectra of the *Conjugate Kernel, CK,* (also called the *Neural Network-Gaussian Process Kernel*), and the *Neural Tangent Kernel, NTK*. Roughly, the CK and the NTK tell us respectively "what a network looks like at initialization" and "what a network looks like during and after training." Their spectra then encode valuable information about the initial distribution and the training and generalization properties of neural networks. By analyzing the eigenvalues, we lend novel insights into the questions put forth at the beginning, and we verify these insights by extensive experiments of neural networks. We derive fast algorithms for computing the spectra of CK and NTK when the data is uniformly distributed over the boolean cube, and show this spectra is the same in high dimensions when data is drawn from isotropic Gaussian or uniformly over the sphere. Code replicating our results is available at github.com/thegregyang/NNspectra.
Robust and Communication-Efficient Collaborative Learning
Authors: Amirhossein Reisizadeh, Hossein Taheri, Aryan Mokhtari, Hamed Hassani, Ramtin Pedarsani
Date: 2019-07-24 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Quantization

We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized optimization schemes face two major system bottlenecks: stragglers' delay and communication overhead. In this paper, we tackle these bottlenecks by proposing a novel decentralized and gradient-based optimization algorithm named as QuanTimed-DSGD. Our algorithm stands on two main ideas: (i) we impose a deadline on the local gradient computations of each node at each iteration of the algorithm, and (ii) the nodes exchange quantized versions of their local models. The first idea robustifies to straggling nodes and the second alleviates communication efficiency. The key technical contribution of our work is to prove that with non-vanishing noises for quantization and stochastic gradients, the proposed method exactly converges to the global optimal for convex loss functions, and finds a first-order stationary point in non-convex scenarios. Our numerical evaluations of the QuanTimed-DSGD on training benchmark datasets, MNIST and CIFAR-10, demonstrate speedups of up to 3x in run-time, compared to state-of-the-art decentralized optimization methods.
Distilled Siamese Networks for Visual Tracking
Authors: Jianbing Shen, Yuanpei Liu, Xingping Dong, Xiankai Lu, Fahad Shahbaz Khan, Steven Hoi
Date: 2019-07-24 00:00:00
Tasks: Knowledge Distillation, Object Tracking, Visual Tracking

In recent years, Siamese network based trackers have significantly advanced the state-of-the-art in real-time tracking. Despite their success, Siamese trackers tend to suffer from high memory costs, which restrict their applicability to mobile devices with tight memory budgets. To address this issue, we propose a distilled Siamese tracking framework to learn small, fast and accurate trackers (students), which capture critical knowledge from large Siamese trackers (teachers) by a teacher-students knowledge distillation model. This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools. In particular, our model contains a single teacher-student distillation module and a student-student knowledge sharing mechanism. The former is designed using a tracking-specific distillation strategy to transfer knowledge from a teacher to students. The latter is utilized for mutual learning between students to enable in-depth knowledge understanding. Extensive empirical evaluations on several popular Siamese trackers demonstrate the generality and effectiveness of our framework. Moreover, the results on five tracking benchmarks show that the proposed distilled trackers achieve compression rates of up to 18$\times$ and frame-rates of $265$ FPS, while obtaining comparable tracking accuracy compared to base models.
A neural network based post-filter for speech-driven head motion synthesis
Authors: JinHong Lu, Hiroshi Shimodaira
Date: 2019-07-24 00:00:00
Tasks: Motion Synthesis

Despite the fact that neural networks are widely used for speech-driven head motion synthesis, it is well-known that the output of neural networks is noisy or discontinuous due to the limited capability of deep neural networks in predicting human motion. Thus, post-processing is required to obtain smooth head motion trajectories for animation. It is common to apply a linear filter or consider keyframes as post-processing. However, neither approach is optimal as there is always a trade-off between smoothness and accuracy. We propose to employ a neural network trained in a way that it is capable of reconstructing the head motions, in order to overcome this limitation. In the objective evaluation, this filter is proved to be good at de-noising data involving types of noise (dropout or Gaussian noise). Objective metrics also demonstrate the improvement of the joined head motion's smoothness after being processed by our proposed filter. A detailed analysis reveals that our proposed filter learns the characteristic of head motions. The subjective evaluation shows that participants were unable to distinguish the synthesised head motions with our proposed filter from ground truth, which was preferred over the Gaussian filter and moving average.
IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
Authors: Nirbhay Modhe, Prithvijit Chattopadhyay, Mohit Sharma, Abhishek Das, Devi Parikh, Dhruv Batra, Ramakrishna Vedantam
Date: 2019-07-24 00:00:00
Tasks: Decision Making, Hierarchical Reinforcement Learning, Sequential Decision Making

We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.
Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References
Authors: Prakhar Gupta, Shikib Mehri, Tiancheng Zhao, Amy Pavel, Maxine Eskenazi, Jeffrey P. Bigham
Date: 2019-07-24 00:00:00
Proceeding: WS 2019 9
Tasks: Dialogue Evaluation, Diversity, Open-Domain Dialog

The aim of this paper is to mitigate the shortcomings of automatic evaluation of open-domain dialog systems through multi-reference evaluation. Existing metrics have been shown to correlate poorly with human judgement, particularly in open-domain dialog. One alternative is to collect human annotations for evaluation, which can be expensive and time consuming. To demonstrate the effectiveness of multi-reference evaluation, we augment the test set of DailyDialog with multiple references. A series of experiments show that the use of multiple references results in improved correlation between several automatic metrics and human judgement for both the quality and the diversity of system output.