pwc-archive/papers-with-abstracts huggingface dataset:
Explore with Dynamic Map: Graph Structured Reinforcement Learning
Authors: Jiarui Jin, Sijin Zhou, Weinan Zhang, Rasool Fakoor, David Wipf, Tong He, Yong Yu, Zheng Zhang, Alex Smola
Date: 2021-01-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In reinforcement learning, a map with states and transitions built based on historical trajectories is often helpful in exploration and exploitation. Even so, learning and planning on such a map within a sparse environment remains a challenge. As a step towards this goal, we propose Graph Structured Reinforcement Learning (GSRL), which utilizes historical trajectories to slowly adjust exploration directions and learn related experiences while rapidly updating the value function estimation. GSRL constructs a dynamic graph on top of state transitions in the replay buffer based on historical trajectories, and develops an attention strategy on the map to select an appropriate goal direction, which decomposes the task of reaching a distant goal state into a sequence of easier tasks. We also leverage graph structure to sample related trajectories for efficient value learning. Results demonstrate that GSRL can outperform the state-of-the-art algorithms in terms of sample efficiency on benchmarks with sparse reward functions.
Learning a Max-Margin Classifier for Cross-Domain Sentiment Analysis
Authors: Mohammad Rostami, Aram Galstyan
Date: 2021-01-01 00:00:00
Tasks: Domain Adaptation, Marketing, Sentiment Analysis

 Sentiment analysis is a costly yet necessary task for enterprises to study the opinions of their costumers to improve their products and services and to determine optimal   marketing strategies. Due to existence of a wide range of domains across different products and services, cross-domain sentiment analysis methods have received significant attention in recent years. These methods mitigate the domain gap between different applications by training cross-domain generalizable classifiers which help to relax the need for individual data annotation per each domain. Most existing methods focus on learning domain-agnostic representations that are invariant with respect to both the source and the target domains. As a result, a classifier that is trained using  annotated data in a source domain, would generalize well in a related target domain. In this work, we introduce a new domain adaptation method which induces large margins between different classes in an embedding space based on the notion of prototypical distribution. This embedding space is trained to be domain-agnostic by matching the data distributions across the domains. Large margins in the source domain help to reduce the effect of ``domain shift'' on the performance of a trained classifier in the target domain. Theoreticaland empirical analysis are provided to demonstrate that the method is effective.
Activation-level uncertainty in deep neural networks
Authors: Pablo Morales-Alvarez, Daniel Hernández-Lobato, Rafael Molina, José Miguel Hernández-Lobato
Date: 2021-01-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Gaussian Processes

Current approaches for uncertainty estimation in deep learning often produce too confident results. Bayesian Neural Networks (BNNs) model uncertainty in the space of weights, which is usually high-dimensional and limits the quality of variational approximations. The more recent functional BNNs (fBNNs) address this only partially because, although the prior is specified in the space of functions, the posterior approximation is still defined in terms of stochastic weights. In this work we propose to move uncertainty from the weights (which are deterministic) to the activation function. Specifically, the activations are modelled with simple 1D Gaussian Processes (GP), for which a triangular kernel inspired by the ReLu non-linearity is explored. Our experiments show that activation-level stochasticity provides more reliable uncertainty estimates than BNN and fBNN, whereas it performs competitively in standard prediction tasks. We also study the connection with deep GPs, both theoretically and empirically. More precisely, we show that activation-level uncertainty requires fewer inducing points and is better suited for deep architectures.
RECONNAISSANCE FOR REINFORCEMENT LEARNING WITH SAFETY CONSTRAINTS
Authors: Shin-ichi Maeda, Hayato Watahiki, Yi Ouyang, Shintarou Okada, Masanori Koyama
Date: 2021-01-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Practical reinforcement learning problems are often formulated as constrained Markov decision process (CMDP) problems, in which the agent has to maximize the expected return while satisfying a set of prescribed safety constraints. In this study, we consider a situation in which the agent has access to the generative model which provides us with a next state sample for any given state-action pair, and propose a model to solve a CMDP problem by decomposing the CMDP into a pair of MDPs; \textit{reconnaissance} MDP (R-MDP) and \textit{planning} MDP (P-MDP). In R-MDP, we train threat function, the Q-function analogue of danger that can determine whether a given state-action pair is safe or not. In P-MDP, we train a reward-seeking policy while using a fixed threat function to determine the safeness of each action. With the help of generative model, we can efficiently train the threat function by preferentially sampling rare dangerous events. Once the threat function for a baseline policy is computed, we can solve other CMDP problems with different reward and different danger-constraint without the need to re-train the model. We also present an efficient approximation method for the threat function that can greatly reduce the difficulty of solving R-MDP. We will demonstrate the efficacy of our method over classical approaches in benchmark dataset and complex collision-free navigation tasks.
SkipW: Resource adaptable RNN with strict upper computational limit
Authors: Tsiry Mayet, Anne Lambert, Pascal Leguyadec, Francoise Le Bolzer, François Schnitzler
Date: 2021-01-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Activity Recognition, Human Activity Recognition

We introduce Skip-Window, a method to allow recurrent neural networks (RNNs) to trade off accuracy for computational cost during the analysis of a sequence. Similarly to existing approaches, Skip-Window extends existing RNN cells by adding a mechanism to encourage the model to process fewer inputs. Unlike existing approaches, Skip-Window is able to respect a strict computational budget, making this model more suitable for limited hardware. We evaluate this approach on two datasets: a human activity recognition task and adding task. Our results show that Skip-Window is able to exceed the accuracy of existing approaches for a lower computational cost while strictly limiting said cost.
Non-iterative Parallel Text Generation via Glancing Transformer
Authors: Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei LI
Date: 2021-01-01 00:00:00
Tasks: Language Modeling, Language Modelling, Text Generation

Although non-autoregressive models with one-iteration generation achieves remarkable inference speed-up, they still falls behind their autoregressive counterparts inprediction accuracy. The non-autoregressive models with best accuracy currently rely on multiple decoding iterations, which largely sacrifice the inference speed of non-autoregressive models.  Inspired by the way of learning word dependencies in autoregressive and iterative-decoding models, we propose Glancing Transformer (GLAT) with glancing language model, which learns to capture the word dependency in a gradual fashion. Experiments on three benchmarks demonstrate that our approach can significantly improve the accuracy of non-autoregressive models without multiple decoding iterations. In particular, GLAT achieves state-of-the-art results among non-iterative models and even outperforms top iterative counterparts in some specific benchmarks.
A Simple and Effective Baseline for Out-of-Distribution Detection using Abstention
Authors: Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, Jeff Bilmes
Date: 2021-01-01 00:00:00
Tasks: Deep Learning, Out-of-Distribution Detection, text-classification, Text Classification

Refraining from confidently predicting when faced with  categories of inputs different from those seen during training is an important requirement for the safe deployment of deep learning systems. While simple to state, this has been  a particularly challenging problem in deep learning, where models often end up making overconfident predictions in such situations. In this work we present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention: when encountering a sample from an unseen class, the desired behavior is to abstain from predicting. Our  approach uses a network with an extra abstention class and is trained on a dataset that  is augmented with an uncurated  set that consists of a large number of out-of-distribution (OoD) samples that are assigned the label of the abstention class; the model is then trained to learn an effective discriminator between in and out-of-distribution samples. 

 We  compare this relatively simple approach against a wide variety of more complex methods that have been proposed both for out-of-distribution detection as well as uncertainty modeling in deep learning, and empirically demonstrate its effectiveness on a wide variety of of benchmarks and deep architectures for image recognition and text classification, often outperforming existing approaches by significant margins. Given the simplicity and effectiveness of this method, we propose that this approach be used as a new additional baseline for future work in this domain.
Sparta: Spatially Attentive and Adversarially Robust Activations
Authors: Qing Guo, Felix Juefei-Xu, Changqing Zhou, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu
Date: 2021-01-01 00:00:00
Tasks: Denoising

Adversarial training has been demonstrated to be useful for improving the robustness of deep neural networks (DNNs). However, the impacts of basic network components (e.g., ReLU, the widely used activation function for DNNs) to adversarial training effectiveness received less attention and has not been comprehensively investigated so far. To fill this gap, in this work, we argue that the spatially-shared and input-independent activating properties of the ReLU make the DNNs under both standard training and adversarial training less robust to white-box adversarial attacks. To address such challenges, we design a novel activation function, i.e., Sparta: Spatially Attentive and Adversarially Robust Activation, which enables DNNs to achieve higher robustness (i.e., lower error rate on adversarial examples) and accuracy (i.e., lower error rate on clean examples) than the DNNs based on the state-of-the-art activation functions. We further investigate the relationships between our Sparta and the state-of-the-art search-based activation function, i.e., Swish, and feature denoising method, providing insights about the advantages of our method. Moreover, comprehensive evaluations have demonstrated two important properties of our method: First, superior transferability across DNNs. Our adversarially trained Sparta function for one DNN (e.g., ResNet-18) can be fixed to train another adversarially robust DNN (e.g., ResNet-34), achieving higher robustness than the one using vanilla ReLU as activation. Second, superior transferability across datasets. The Sparta function trained on one dataset (e.g., CIFAR-10) can be employed to train adversarially robust DNNs on another dataset (e.g., SVHN) and helps achieve higher robustness than DNNs with vanilla ReLU as activation.  These properties have highlighted the flexibility and versatility of Sparta. Accompanying code is also submitted.
Lipschitz-Bounded Equilibrium Networks
Authors: Max Revay, Ruigang Wang, Ian Manchester
Date: 2021-01-01 00:00:00
Tasks: Generalization Bounds, image-classification, Image Classification

This paper introduces new parameterizations of equilibrium neural networks, i.e.
networks defined by implicit equations. This model class includes standard multilayer
and residual networks as special cases. The new parameterization admits
a Lipschitz bound during training via unconstrained optimization, i.e. no projections
or barrier functions are required. Lipschitz bounds are a common proxy for
robustness and appear in many generalization bounds. Furthermore, compared to
previous works we show well-posedness (existence of solutions) under less restrictive
conditions on the network weights and more natural assumptions on the
activation functions: that they are monotone and slope restricted. These results
are proved by establishing novel connections with convex optimization, operator
splitting on non-Euclidean spaces, and contracting neural ODEs. In image classification
experiments we show that the Lipschitz bounds are very accurate and
improve robustness to adversarial attacks.
Optimal Designs of Gaussian Processes with Budgets for Hyperparameter Optimization
Authors: Yimin Huang, YuJun Li, Zhenguo Li, Zhihua Zhang
Date: 2021-01-01 00:00:00
Tasks: Gaussian Processes, Hyperparameter Optimization

The remarkable performance of modern deep learning methods depends critically on the optimization of their hyperparameters. One major challenge is that evaluating a single hyperparameter configuration on large datasets could nowadays easily exceed hours or days. For efficient sampling and fast evaluation, some previous works presented effective computing resource allocation schemes and built a Bayesian surrogate model to sample candidate hyperparameters. However, the model itself is not related to budgets which are set manually. To deal with this problem, a new Gaussian Process model involved in budgets is proposed. Further, for this model, an optimal design is constructed by the equivalence theorem to replace random search as an initial sampling strategy in the search space. Experiments demonstrate that the new model has the best performance among competing methods. Moreover, comparisons between different initial designs with the same model show the advantage of the proposed optimal design.
PODS: Policy Optimization via Differentiable Simulation
Authors: Miguel Angel Zamora Mora, Momchil Peychev, Sehoon Ha, Martin Vechev, Stelian Coros
Date: 2021-01-01 00:00:00
Tasks: Reinforcement Learning (RL)

Current reinforcement learning (RL) methods use simulation models as simple black-box oracles. In this paper, with the goal of improving the performance exhibited by RL algorithms, we explore a systematic way of leveraging the additional information provided by an emerging class of differentiable simulators. Building on concepts established by Deterministic Policy Gradients (DPG) methods, the neural network policies learned with our approach represent deterministic actions. In a departure from standard methodologies, however, learning these policy does not hinge on approximations of the value function that must be learned concurrently in an actor-critic fashion. Instead, we exploit differentiable simulators to directly compute the analytic gradient of a policy's value function with respect to the actions it outputs. This, in turn, allows us to efficiently perform locally optimal policy improvement iterations. Compared against other state-of-the-art RL methods, we show that with minimal hyper-parameter tuning our approach consistently leads to better asymptotic behavior across a set of payload manipulation tasks that demand high precision.
Regioned Episodic Reinforcement Learning
Authors: Jiarui Jin, Cong Chen, Ming Zhou, Weinan Zhang, Rasool Fakoor, David Wipf, Yong Yu, Jun Wang, Alex Smola
Date: 2021-01-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Goal-oriented reinforcement learning algorithms are often good at exploration, not exploitation, while episodic algorithms excel at exploitation, not exploration. As a result, neither of these approaches alone can lead to a sample efficient algorithm in complex environments with high dimensional state space and delayed rewards. Motivated by these observations and shortcomings, in this paper, we introduce Regioned Episodic Reinforcement Learning (RERL) that combines the episodic and goal-oriented learning strengths and leads to a more sample efficient and ef- fective algorithm. RERL achieves this by decomposing the space into several sub-space regions and constructing regions that lead to more effective exploration and high values trajectories. Extensive experiments on various benchmark tasks show that RERL outperforms existing methods in terms of sample efficiency and final rewards.
Memory Augmented Design of Graph Neural Networks
Authors: Tao Xiong, Liang Zhu, Ruofan Wu, Yuan Qi
Date: 2021-01-01 00:00:00
Tasks: Node Classification

The expressive power of graph neural networks (GNN) has drawn much interest recently. Most existent work focused on measuring the expressiveness of GNN through the task of distinguishing between graphs. In this paper, we inspect the representation limits of locally unordered messaging passing (LUMP) GNN architecture through the lens of \emph{node classification}. For GNNs based on permutation invariant local aggregators, we characterize graph-theoretic conditions under which such GNNs fail to discriminate simple instances, regardless of underlying architecture or network depth. To overcome this limitation, we propose a novel framework to augment GNNs with global graph information called \emph{memory augmentation}. Specifically, we allow every node in the original graph to interact with a group of memory nodes. For each node, information from all the other nodes in the graph can be gleaned through the relay of the memory nodes. For proper backbone architectures like GAT and GCN, memory augmented GNNs are theoretically shown to be more expressive than LUMP GNNs. Empirical evaluations demonstrate the significant improvement of memory augmentation. In particular, memory augmented GAT and GCN are shown to either outperform or closely match state-of-the-art performance across various benchmark datasets.
RETHINKING LOCAL LOW RANK MATRIX DETECTION:A MULTIPLE-FILTER BASED NEURAL NETWORK FRAMEWORK
Authors: Pengtao Dang, Wennan Chang, Haiqi Zhu, Changlin Wan, Tong Zhao, Tingbo Guo, Paul Salama, Sha Cao, Chi Zhang
Date: 2021-01-01 00:00:00
Tasks: Recommendation Systems

The matrix local low rank representation (MLLRR) is a critical technique used in recommender systems, text mining and computer vision.  In MLLRR, how to identify the row and column indices that form a distinct low rank sub-matrix is the main challenge.  Existing solutions on MLLRR all leverage problem-specific assumptions, which makes them lack generalizability and prohibits the detection of a substantial amount of true patterns. In this work, we first organize the general MLLRR problem into three subproblems based on different low rank properties ,and we argue that most of existing efforts focus on only one category, which leaves the other two unsolved. Based on our categorization, we develop a novel multiple filter based neural network framework, namely FLLRM, which is the first of its kind of method to solve all three MLLRR problems.  We systematically benchmark FLLRM with stat-of-the-art methods on an extensive set of synthetic data of the three subproblems, empowered by a robustness evaluation of parameters and theoretic discussions.  Experimental results show that FLLRM outperforms all existing methods and enables a general solution to the two unsolved sub problems.  Experiments on two real-world datasets also validate the effectiveness of FLLRM on identifying local low rank matrices corresponding to novel context specific knowledge.
ARMCMC: ONLINE MODEL PARAMETERS DENSITY ESTIMATION IN BAYESIAN PARADIGM
Authors: Pedram Agand, Mo Chen, Hamid D. Taghirad
Date: 2021-01-01 00:00:00
Tasks: Density Estimation

Although the Bayesian paradigm provides a rigorous framework to estimate the full probability distribution over unknown parameters,  its online implementation can be challenging due to heavy computational cost. This paper proposes Adaptive Recursive Markov Chain Monte Carlo (ARMCMC) which estimates full probability density of model parameters while alleviating  shortcomings of conventional online approaches. These shortcomings include only being able to account for Gaussian noise, only being applicable to systems with linear in the parameters (LIP) constraint, and having requirements on persistence excitation (PE). In ARMCMC, we propose a variable jump distribution, which depends on a temporal forgetting factor.  This allows one to adjust the trade off between exploitation and exploration, depending on whether there is an abrupt change to the parameter being estimated. We prove that ARMCMC requires fewer samples to achieve the same precision and reliability compared to conventional MCMC approaches.  We demonstrate our approach on a challenging benchmark:  estimation of parameters in the Hunt-Crossley dynamic model, which models both on/off contact forces applied to soft materials. Our method shows up to 75% improvement in parameter point estimation accuracy and approximately 40% reduction in tracking error compared with recursive least squares and conventional MCMC.
Learning to Solve Multi-Robot Task Allocation with a Covariant-Attention based Neural Architecture
Authors: Steve Paul, Payam Ghassemi, Souma Chowdhury
Date: 2021-01-01 00:00:00
Tasks: Combinatorial Optimization, Graph Learning

This paper presents a novel graph (reinforcement) learning method to solve an important class of multi-robot task allocation (MRTA) problems that involve tasks with deadlines, and robots with ferry range and payload constraints (thus requiring multiple tours per robot). While drawing motivation from recent graph learning methods that learn to solve combinatorial optimization problems of the mTSP/VRP type, this paper seeks to provide better convergence and tractable scalability (with an increasing number of tasks) in learning to solve MRTA. The proposed neural architecture, called Covariant Attention-based Model or CAM, includes two main components: 1) an encoder: a covariant node-based embedding model to represent each task as a learnable feature vector, and 2) a decoder: an attention-based model to facilitate a sequential output. In order to learn the feature vectors, a policy-gradient method based on REINFORCE is used. The new learning architecture is applied to a flood response problem (representative of the general class of Multi-task Robots, and Single-robot Tasks or MR-ST problems), where multiple unmanned aerial vehicles (UAVs) supply survival kits to victims spread out over an area. To perform a comparative analysis, the well-known attention-based approach (that is designed to solve mTSP/VRP problems) is extended and applied to the stated MRTA class of problems. A comparison is performed in terms of a cost function that considers both task completion rate and cumulative traveled distance. The results show that our primary method is superior not only in terms of the cost function (over various training and unseen test scenarios), but also provide significantly faster convergence and yields learnt policies that can be executed within 6 milliseconds per robot, thereby allowing real-time application.