pwc-archive/papers-with-abstracts huggingface dataset:
Learning Chess Blindfolded
Authors: Shubham Toshniwal, Sam Wiseman, Karen Livescu, Kevin Gimpel
Date: 2021-01-01 00:00:00
Tasks: Domain Probing, Game of Chess, Language Modeling, Language Modelling, Natural Language Understanding

Transformer language models have made tremendous strides in natural language
understanding. However, the complexity of natural language makes it challenging
to ascertain how accurately these models are tracking the world state underlying
the text. Motivated by this issue, we consider the task of language modeling for
the game of chess. Unlike natural language, chess notations describe a simple,
constrained, and deterministic domain. Probing for the world (board) state can be
done simply via language modeling prompts. Additionally, we have access to a vast
number of chess games coupled with the exact state at every move, allowing us to
measure the impact of various ways of including grounding during language model
training. Overall, we find that with enough training data, transformers can learn
to track pieces and predict legal moves when trained solely from move sequences.
However, in adverse circumstances (small training sets or prediction following long
move histories), providing access to board state information during training can
yield consistent improvements.
PriorityCut: Occlusion-aware Regularization for Image Animation
Authors: Wai Ting Cheung, Gyeongsu Chae
Date: 2021-01-01 00:00:00
Tasks: Data Augmentation, Image Animation, Image Generation

Image animation generates a video of a source image following the motion of a driving video. Self-supervised image animation approaches do not require explicit pose references as inputs, thus offering large flexibility in learning. State-of-the-art self-supervised image animation approaches mostly warp the source image according to the motion of the driving video, and recover the warping artifacts by inpainting. When the source and the driving images have large pose differences, heavy inpainting is necessary. Without guidance, heavily inpainted regions usually suffer from loss of details. While previous data augmentation techniques such as CutMix are effective in regularizing non-warp-based image generation, directly applying them to image animation ignores the difficulty of inpainting on the warped image. We propose PriorityCut, a novel augmentation approach that uses the top-$k$ percent occluded pixels of the foreground to regularize image animation. By taking into account the difficulty of inpainting, PriorityCut preserves better identity than vanilla CutMix and outperforms state-of-the-art image animation models in terms of the pixel-wise difference, low-level similarity, keypoint distance, and feature embedding distance.
Robust Imitation via Decision-Time Planning
Authors: Carl Qi, Pieter Abbeel, Aditya Grover
Date: 2021-01-01 00:00:00
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Zero-shot Generalization

The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal. A popular class of approaches infers the (unknown) reward function via inverse reinforcement learning (IRL) followed by maximizing this reward function via reinforcement learning (RL). The policies learned via these approaches are however very brittle in practice and deteriorate quickly even with small test-time perturbations due to compounding errors. We propose Imitation with Planning at Test-time (IMPLANT), a new algorithm for imitation learning that utilizes decision-time planning to correct for compounding errors of any base imitation policy. In contrast to existing approaches, we retain both the imitation policy and the rewards model at decision-time, thereby benefiting from the learning signal of the two components. Empirically, we demonstrate that IMPLANT significantly outperforms benchmark imitation learning approaches on standard control environments and excels at zero-shot generalization when subject to challenging perturbations in test-time dynamics.
Domain Knowledge in Exploration Noise in AlphaZero
Authors: Eric Weiner, George D Montañez, Aaron Trujillo, Abtin Molavi
Date: 2021-01-01 00:00:00

The AlphaZero algorithm has achieved remarkable success in a variety of sequential, perfect information games including Go, Shogi and Chess. In the original paper the only hyperparameter that is changed from game to game is the $\alpha$ parameter governing a search prior. In this paper we investigate the properties of this hyperparameter. First, we build a formal intuition for its behavior on a toy example meant to isolate the influence of $\alpha$. Then, by comparing performance of AlphaZero agents with different $\alpha$ values on Connect 4, we show that the performance of AlphaZero improves considerably with a good choice of $\alpha$. This all highlights the importance of $\alpha$ as an interpretable hyperparameter which allows for cross-game tuning that more opaque hyperparameters like model architecture may not.
ScheduleNet: Learn to Solve MinMax mTSP Using Reinforcement Learning with Delayed Reward
Authors: Junyoung Park, Sanzhar Bakhtiyarov, Jinkyoo Park
Date: 2021-01-01 00:00:00
Tasks: Combinatorial Optimization, Graph Neural Network, reinforcement-learning, Reinforcement Learning (RL)

Combinatorial Optimization (CO) problems are theoretically challenging yet crucial in practice. Numerous works used Reinforcement Learning (RL) to tackle these CO problems. As current approaches mainly focus on single-worker CO problems such as the famous Travelling Salesman Problem (TSP), we focus on more practical extension of TSP to multi-worker (salesmen) setting, specifically MinMax mTSP. From the RL perspective, Minmax mTSP raises several significant challenges, such as the cooperation of multiple workers and the need for a well-engineered reward function. In this paper, we present the RL framework with (1) worker-task heterograph and type-aware Graph Neural Network, and (2) the RL training method that is stable, has fast convergence speed, and directly optimizes the objective of MinMax mTSP in a delayed reward setting. We achieve comparable performance to a highly optimized meta-heuristic baseline, OR-Tools, and outperforms it in 10% of the cases, both on in-training and out-of-training problem distributions. Moreover, our problem formulation enables us to solve problems with any number of salesmen (workers) and cities.
WaNet - Imperceptible Warping-based Backdoor Attack
Authors: Tuan Anh Nguyen, Anh Tuan Tran
Date: 2021-01-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Backdoor Attack

With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years. A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears. However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans. In this paper, we instead propose using warping-based triggers. The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness. To make such models undetectable by machine defenders, we propose a novel training mode, called the ``noise mode. The trained networks successfully attack and bypass the state of the art defense methods on standard classification datasets, including MNIST, CIFAR-10, and GTSRB. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism's efficiency.
On the relationship between topology and gradient propagation in deep networks
Authors: Kartikeya Bhardwaj, Guihong Li, Radu Marculescu
Date: 2021-01-01 00:00:00

In this paper, we address two fundamental research questions in neural architecture design: (i) How does the architecture topology impact the gradient flow during training? (ii) Can certain topological characteristics of deep networks indicate a priori (i.e., without training) which models, with a different number of parameters/FLOPS/layers, achieve a similar accuracy? To this end, we formulate the problem of deep learning architecture design from a network science perspective and introduce a new metric called NN-Mass to quantify how effectively information flows through a given architecture. We establish a theoretical link between NN-Mass, a topological property of neural architectures, and gradient flow characteristics (e.g., Layerwise Dynamical Isometry). As such, NN-Mass can identify models with similar accuracy, despite having significantly different size/compute requirements. Detailed experiments on both synthetic and real datasets (e.g., MNIST, CIFAR-10, CIFAR-100, ImageNet) provide extensive evidence for our insights. Finally, we show that the closed-form equation of our theoretically grounded NN-Mass metric enables us to design efficient architectures directly without time-consuming training and searching.
Partial Rejection Control for Robust Variational Inference in Sequential Latent Variable Models
Authors: Rahul Sharma, Soumya Banerjee, Dootika Vats, Piyush Rai
Date: 2021-01-01 00:00:00
Tasks: Variational Inference

Effective variational inference crucially depends on a flexible variational family of distributions. Recent work has explored sequential Monte-Carlo (SMC) methods to construct variational distributions, which can, in principle, approximate the target posterior arbitrarily well, which is especially appealing for models with inherent sequential structure. However, SMC, which represents the posterior using a weighted set of particles, often suffers from particle weight degeneracy, leading to a large variance of the resulting estimators. To address this issue, we present a novel approach that leverages the idea of \emph{partial} rejection control (PRC) and enables us to develop a robust variational inference framework. Although PRC constructs a low variance estimator of the marginal likelihood, unbiased estimators are not available in the literature for arbitrary variational posteriors. We solve this issue by employing a \emph{dice-enterprise}: a generalization of the \emph{Bernoulli factory} to construct unbiased estimators for SMC-PRC. The resulting variational lower bound can be optimized efficiently with respect to the variational parameters. We show theoretical properties of the lower bound and report experiments on various sequential models, such as the Gaussian state-space model and variational RNN, on which our approach outperforms existing methods.
GraphCGAN: Convolutional Graph Neural Network with Generative Adversarial Networks
Authors: Sheng Zhang, Rui Song, Wenbin Lu
Date: 2021-01-01 00:00:00
Tasks: Graph Neural Network

Graph convolutional networks (GCN) achieved superior performances in graph-based semi-supervised learning (SSL) tasks. 
Generative adversarial networks (GAN) also show the ability to increase the performance in SSL.
However, there is still no good way to combine the GAN and GCN in graph-based SSL tasks.
In this work, we present GraphCGAN, a novel framework to incorporate adversarial learning with convolution-based graph neural networks, to operate on graph-structured data. 
In GraphCGAN, we show that generator can generate topology structure and features of fake nodes jointly and boost the performance of convolution-based graph neural networks classifier.
In a number of experiments on benchmark datasets, we show that the proposed GraphCGAN outperforms the baseline methods by a significant margin.
Outlier Preserving Distribution Mapping Autoencoders
Authors: Walter Gerych, Elke Rundensteiner, Emmanuel Agu
Date: 2021-01-01 00:00:00
Tasks: Outlier Detection

State-of-the-art deep outlier detection methods map data into a latent space with the aim of having  outliers far away from inliers in this space. Unfortunately, this often fails as  the divergence penalty they adopt pushes outliers into the same high-probability regions as inliers. We propose a novel method, OP-DMA, that successfully addresses the above problem. OP-DMA   succeeds in mapping outliers to low probability regions in the latent space by leveraging a novel Prior-Weighted Loss (PWL) that utilizes the insight that outliers are likely to have a higher reconstruction error than inliers. Building on this insight,  OP-DMA weights the reconstruction error of individual points by a multivariate Gaussian probability density function evaluated at each point's latent representation. We demonstrate and provide theoretical proof that this succeeds to map outliers to low-probability regions. Our experimental study shows that OP-DMA  consistently outperforms  state-of-art  methods on a rich variety of outlier detection benchmark datasets.
Revisiting Locally Supervised Training of Deep Neural Networks
Authors: Yulin Wang, Zanlin Ni, Shiji Song, Le Yang, Gao Huang
Date: 2021-01-01 00:00:00
Proceeding: ICLR 2021 1

Due to the need to store the intermediate activations for back-propagation, end-to-end (E2E) training of deep networks usually suffers from high GPUs memory footprint. This paper aims to address this problem by revisiting the locally supervised learning, where a network is split into gradient-isolated modules and trained with local supervision. We experimentally show that simply training local modules with E2E loss tends to collapse task-relevant information at early layers, and hence hurts the performance of the full model. To avoid this issue, we propose an information propagation (InfoPro) loss, which encourages local modules to preserve as much useful information as possible, while progressively discard task-irrelevant information. As InfoPro loss is difficult to compute in its original form, we derive a feasible upper bound as a surrogate optimization objective, yielding a simple but effective algorithm. In fact, we show that the proposed method boils down to minimizing the combination of a reconstruction loss and a normal cross-entropy/contrastive term. Extensive experiments on five datasets (i.e., CIFAR, SVHN, STL-10, ImageNet and Cityscapes) validate that our method is capable of achieving competitive performance with fewer than 40% memory footprint compared to E2E training, or enables training local modules asynchronously for potential training acceleration.
SGD on Neural Networks learns Robust Features before Non-Robust
Authors: Vikram Nitin
Date: 2021-01-01 00:00:00

Neural networks are known to be vulnerable to adversarial attacks - small, imperceptible perturbations that cause the network to misclassify an input. A recent line of work attempts to explain this behavior by positing the existence of non-robust features - well-generalizing but brittle features present in the data distribution that are learned by the network and can be perturbed to cause misclassification.

In this paper, we look at the dynamics of neural network training through the perspective of robust and non-robust features. We find that there are two very distinct pathways that neural network training can follow, depending on the hyperparameters used. In the first pathway, the network initially learns only predictive, robust features and weakly predictive non-robust features, and subsequently learns predictive, non-robust features. On the other hand, a network trained via the second pathway eschews predictive non-robust features altogether, and rapidly overfits the training data. We provide strong empirical evidence to corroborate this hypothesis, as well as theoretical analysis in a simplified setting. Key to our analysis is a better understanding of the relationship between predictive non-robust features and adversarial transferability. We present our findings in light of other recent results on the evolution of inductive biases learned by neural networks over the course of training.

Finally, we digress to show that rather than being quirks of the data distribution, predictive non-robust features might actually occur across datasets with different distributions drawn from independent sources, indicating that they perhaps possess some meaning in terms of human semantics.
Data augmentation for deep learning based accelerated MRI reconstruction
Authors: Zalan Fabian, Reinhard Heckel, Mahdi Soltanolkotabi
Date: 2021-01-01 00:00:00
Tasks: Data Augmentation, Deep Learning, Image Reconstruction, Image Restoration, MRI Reconstruction

Deep neural networks have emerged as very successful tools for image restoration and reconstruction tasks. These networks are often trained end-to-end to directly reconstruct an image from a noisy or corrupted measurement of that image. To achieve state-of-the-art performance, training on large and diverse sets of images is considered critical. However, it is often difficult and/or expensive to collect large amounts of training images. Inspired by the success of Data Augmentation (DA) for classification problems, in this paper, we propose a pipeline for data augmentation for image reconstruction tasks arising in medical imaging and explore its effectiveness at reducing the required training data in a variety of settings. We focus on accelerated magnetic resonance imaging, where the goal is to reconstruct an image from a few under-sampled linear measurements. Our DA pipeline is specifically designed to utilize the invariances present in medical imaging measurements as naive DA strategies that neglect the physics of the problem fail. We demonstrate the effectiveness of our data augmentation pipeline by showing that for some problem regimes, DA can achieve comparable performance to the state of the art on the FastMRI dataset while using significantly fewer training data. Specifically, for 8-fold acceleration we achieve performance comparable to the state of the art with only $10\%$ of the training data for multi-coil reconstruction and with only $33\%$ of the training data for single-coil reconstruction. Our findings show that in the low-data regime DA is beneficial, whereas in the high-data regime it has diminishing returns.
Play to Grade: Grading Interactive Coding Games as Classifying Markov Decision Process
Authors: Allen Nie, Emma Brunskill, Chris Piech
Date: 2021-01-01 00:00:00

Contemporary coding education often present students with the task of developing programs that have user interaction and complex dynamic systems, such as mouse based games. While pedagogically compelling, grading such student programs requires dynamic user inputs, therefore they are difficult to grade by unit tests. In this paper we formalize the challenge of grading interactive programs as a task of classifying Markov Decision Processes (MDPs). Each student's program fully specifies an MDP where the agent needs to operate and decide, under reasonable generalization, if the dynamics and reward model of the input MDP conforms to a set of latent MDPs. We demonstrate that by experiencing a handful of latent MDPs millions of times, we can use the agent to sample trajectories from the input MDP and use a classifier to determine membership. Our method drastically reduces the amount of data needed to train an automatic grading system for interactive code assignments and present a challenge to state-of-the-art reinforcement learning generalization methods. Together with Code.org, we curated a dataset of 700k student submissions, one of the largest dataset of anonymized student submissions to a single assignment. This Code.org assignment had no previous solution for automatically providing correctness feedback to students and as such this contribution could lead to meaningful improvement in educational experience.
A framework for learned sparse sketches
Authors: Simin Liu, Tianrui Liu, Ali Vakilian, Yulin Wan, David Woodruff
Date: 2021-01-01 00:00:00
Tasks: Clustering, regression

Sketching is a compression technique that can be applied to many problems to solve them quickly and approximately. The matrices used to project data to smaller dimensions are called sketches. In this work, we consider the problem of optimizing sketches to obtain low approximation error over a data distribution. 

We introduce a general framework for learning and applying input-independent sparse sketches. The sketch optimization procedure has two stages: one for optimizing the placement of the sketch's non-zero entries and another for optimizing their values. Next, we provide a way to apply learned sketches that has worst-case guarantees for approximation error. 

We instantiate this framework with three sketching applications: least-squares regression, low-rank approximation (LRA), and k-means clustering. Our experiments demonstrate that our approach substantially decreases approximation error compared to classical and naively learned sketches. 

Finally, we investigate the theoretical aspects of our approach. For regression and LRA, we show that our method obtains state-of-the art accuracy for fixed time complexity. For LRA, we prove that it is strictly better to include the first optimization stage. For k-means, we derive a more straightforward means of retaining approximation guarantees.
Temporal Attention Modules for Memory-Augmented Neural Networks
Authors: Rodolfo Palma, Alvaro Soto, Luis Martí, Nayat Sanchez-pi
Date: 2021-01-01 00:00:00

We introduce two temporal attention modules which can be plugged into traditional memory augmented recurrent neural networks to improve their performance in natural language processing tasks.
The temporal attention modules provide new inductive biases allowing the models to compute attention distributions over the different time steps of input sequences.
The values of these attention distributions can be inspected to identify the sequence's elements that the model considered relevant during the inference.
Using the Entity Network (Henaff et al., 2016) as the model backbone, experiments were made on the dataset bAbI tasks, a set of QA tasks.
Due to the addition of the temporal attention modules, the performance metric increased 26% when the temporal attention was supervised, and 13,5% when it wasn't.
Moreover, the usage of temporal attention modules proved useful at resolving reasoning tasks that the original model was unable to solve.
Dependency Structure Discovery from Interventions
Authors: Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Bernhard Schölkopf, Michael Curtis Mozer, Hugo Larochelle, Christopher Pal, Yoshua Bengio
Date: 2021-01-01 00:00:00

Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data. However, there are theoretical limitations on the identifiability of underlying structures obtained from observational data alone. Interventional data provides much richer information about the underlying data-generating process. However, the extension and application of methods designed for observational data to include interventions is not straightforward and remains an open problem. In this paper we provide a general framework based on continuous optimization and neural networks to create models for the combination of observational and interventional data. The proposed method is applicable even in the challenging and realistic case that the identity of the intervened upon variable is unknown. We examine the proposed method in the setting of graph recovery both de novo and from a partially-known edge set. We establish strong benchmark results on several structure learning tasks, including structure recovery of both synthetic graphs as well as standard graphs from the Bayesian Network Repository.