pwc-archive/papers-with-abstracts huggingface dataset:
Accelerating first order optimization algorithms
Authors: Ange tato, Roger nkambou
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Stochastic Optimization

There exist several stochastic optimization algorithms. However in most cases, it is difficult to tell for a particular problem which will be the best optimizer to choose as each of them are good. Thus, we present a simple and intuitive technique, when applied to first order optimization algorithms, is able to improve the speed of convergence and reaches a better minimum for the loss function compared to the original algorithms. The proposed solution modifies the update rule, based on the variation of the direction of the gradient during training. We conducted several tests with Adam and AMSGrad on two different  datasets. The preliminary results show that the proposed technique improves the performance of existing optimization algorithms and works well in practice.
An Analysis of Composite Neural Network Performance from Function Composition Perspective
Authors: Ming-Chuan Yang, Meng Chang Chen
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5

This work investigates the performance of a composite neural network, which is composed of pre-trained neural network models and non-instantiated neural network models, connected to form a rooted directed graph. A pre-trained neural network model is generally a well trained neural network model targeted for a specific function. The advantages of adopting such a pre-trained model in a composite neural network are two folds. One is to benefit from other's intelligence and diligence and the other is saving the efforts in data preparation and resources and time in training. However, the overall performance of composite neural network is still not clear. In this work, we prove that a composite neural network, with high probability, performs better than any of its pre-trained components under certain assumptions. In addition, if an extra pre-trained component is added to a composite network, with high probability the overall performance will be improved. In the empirical evaluations, distinctively different applications support the above findings.
Complexity of Training ReLU Neural Networks
Authors: Digvijay Boob, Santanu S. Dey, Guanghui Lan
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5

In this paper, we explore some basic questions on complexity of training Neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension d of the data is fixed then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.
A preconditioned accelerated stochastic gradient descent algorithm
Authors: Alexandru Onose, Seyed Iman Mossavat, Henk-Jan H. Smilde
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Stochastic Optimization

We propose a preconditioned accelerated stochastic gradient method suitable for large scale optimization. We derive sufficient convergence conditions for the minimization of convex functions using a generic class of diagonal preconditioners and provide a formal convergence proof based on a framework originally used for on-line learning. Inspired by recent popular adaptive per-feature algorithms, we propose a specific preconditioner based on the second moment of the gradient. The sufficient convergence conditions motivate a critical adaptation of the per-feature updates in order to ensure convergence. We show empirical results for the minimization of convex and non-convex cost functions, in the context of neural network training. The method compares favorably with respect to current, first order, stochastic optimization methods.
Attentive Task-Agnostic Meta-Learning for Few-Shot Text Classification
Authors: Xiang Jiang, Mohammad Havaei, Gabriel Chartrand, Hassan Chouaib, Thomas Vincent, Andrew Jesson, Nicolas Chapados, Stan Matwin
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Classification, Few-Shot Text Classification, General Classification, Inductive Bias, Meta-Learning, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Natural Language Understanding, Representation Learning, text-classification, Text Classification, Transfer Learning

Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural language understanding. Inspired by the Model-Agnostic Meta-Learning framework (MAML), we introduce the Attentive Task-Agnostic Meta-Learning (ATAML) algorithm for text classification. The proposed ATAML is designed to encourage task-agnostic representation learning by way of task-agnostic parameterization and facilitate task-specific adaptation via attention mechanisms. We provide evidence to show that the attention mechanism in ATAML has a synergistic effect on learning performance. Our experimental results reveal that, for few-shot text classification tasks, gradient-based meta-learning approaches ourperform popular transfer learning methods. In comparisons with models trained from random initialization, pretrained models and meta trained MAML, our proposed ATAML method generalizes better on single-label and multi-label classification tasks in miniRCV1 and miniReuters-21578 datasets.
Whitening and Coloring transform for GANs
Authors: Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5

Batch Normalization (BN) is a common technique used 
to speed-up and stabilize training. On the other hand, the learnable parameters of BN are commonly used in conditional Generative Adversarial Networks (cGANs)
for representing class-specific information using conditional Batch Normalization (cBN). In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization. 
We show that our conditional Coloring can represent categorical conditioning information which largely helps the cGAN qualitative results. Moreover, we show that full-feature whitening is important in a general GAN scenario in which the training process is known to be highly unstable.
We test our approach on different datasets and 
using different GAN networks and training protocols,
showing a consistent improvement in all the tested frameworks. Our CIFAR-10 supervised results are higher than all previous works on this dataset.
Autoencoder-based Music Translation
Authors: Noam Mor, Lior Wolf, Adam Polyak, Yaniv Taigman
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Translation

We present a method for translating music across musical instruments and styles. This method is based on unsupervised training of a multi-domain wavenet autoencoder, with a shared encoder and a domain-independent latent space that is trained end-to-end on waveforms. Employing a diverse training dataset and large net capacity, the single encoder allows us to translate also from musical domains that were not seen during training. We evaluate our method on a dataset collected from professional musicians, and achieve convincing translations. We also study the properties of the obtained translation and demonstrate translating even from a whistle, potentially enabling the creation of instrumental music by untrained humans.
NEURAL MALWARE CONTROL WITH DEEP REINFORCEMENT LEARNING
Authors: Yu Wang, Jack W. Stokes, Mady Marinescu
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Antimalware products are a key component in detecting malware attacks, and their engines typically execute unknown programs in a sandbox prior to running them on the native operating system. Files cannot be scanned indefinitely so the engine employs heuristics to determine when to halt execution. Previous research has investigated analyzing the sequence of system calls generated during this emulation process to predict if an unknown file is malicious, but these models require the emulation to be stopped after executing a fixed number of events from the beginning of the file. Also, these classifiers are not accurate enough to halt emulation in the middle of the file on their own. In this paper, we propose a novel algorithm which overcomes this limitation and learns the best time to halt the file's execution based on deep reinforcement learning (DRL). Because the new DRL-based system continues to emulate the unknown file until it can make a confident decision to stop, it prevents attackers from avoiding detection by initiating malicious activity after a fixed number of system calls. Results show that the proposed malware execution control model automatically halts emulation for 91.3\% of the files earlier than heuristics employed by the engine. Furthermore, classifying the files at that time improves the true positive rate by 61.5%, at a false positive rate of 1%, compared to a baseline classifier.
Beyond Games: Bringing Exploration to Robots in Real-world
Authors: Deepak Pathak, Dhiraj Gandhi, Abhinav Gupta
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Efficient Exploration, Reinforcement Learning

Exploration has been a long standing problem in both model-based and model-free learning methods for sensorimotor control. While there has been major advances over the years, most of these successes have been demonstrated in either video games or simulation environments. This is primarily because the rewards (even the intrinsic ones) are non-differentiable since they are function of the environment (which is a black-box). In this paper, we focus on the policy optimization aspect of the intrinsic reward function. Specifically, by using a local approximation, we formulate intrinsic reward as a differentiable function so as to perform policy optimization using likelihood maximization -- much like supervised learning instead of reinforcement learning. This leads to a significantly sample efficient exploration policy. Our experiments clearly show that our approach outperforms both on-policy and off-policy optimization approaches like REINFORCE and DQN respectively. But most importantly, we are able to implement an exploration policy on a robot which learns to interact with objects completely from scratch just using data collected via the differentiable exploration module. See project videos at https://doubleblindICLR.github.io/robot-exploration/
Deep reinforcement learning with relational inductive biases
Authors: Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew Botvinick, Oriol Vinyals, Peter Battaglia
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Relational Reasoning, Starcraft, Starcraft II

We introduce an approach for augmenting model-free deep reinforcement learning agents with a mechanism for relational reasoning over structured representations, which improves performance, learning efficiency, generalization, and interpretability. Our architecture encodes an image as a set of vectors, and applies an iterative message-passing procedure to discover and reason about relevant entities and relations in a scene. In six of seven StarCraft II Learning Environment mini-games, our agent achieved state-of-the-art performance, and surpassed human grandmaster-level on four. In a novel navigation and planning task, our agent's performance and learning efficiency far exceeded non-relational baselines, it was able to generalize to more complex scenes than it had experienced during training. Moreover, when we examined its learned internal representations, they reflected important structure about the problem and the agent's intentions. The main contribution of this work is to introduce techniques for representing and reasoning about states in model-free deep reinforcement learning agents via relational inductive biases. Our experiments show this approach can offer advantages in efficiency, generalization, and interpretability, and can scale up to meet some of the most challenging test environments in modern artificial intelligence.
Auxiliary Variational MCMC
Authors: Raza Habib, David Barber
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: regression, Variational Inference

We introduce Auxiliary Variational MCMC, a novel framework for learning MCMC kernels that combines recent advances in variational inference with insights drawn from traditional auxiliary variable MCMC methods such as Hamiltonian Monte Carlo. Our framework exploits low dimensional structure in the target distribution in order to learn a more efficient MCMC sampler. The resulting sampler is able to suppress random walk behaviour and mix between modes efficiently, without the need to compute gradients of the target distribution. We test our sampler on a number of challenging distributions, where the underlying structure is known, and on the task of posterior sampling in Bayesian logistic regression. Code to reproduce all experiments is available at https://github.com/AVMCMC/AuxiliaryVariationalMCMC .
Overcoming catastrophic forgetting through weight consolidation and long-term memory
Authors: Shixian Wen, Laurent Itti
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Task 2

Sequential learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby previously learned knowledge is erased during learning of new, disjoint knowledge. Here, we propose a new approach to sequential learning which leverages the recent discovery of adversarial examples. We use adversarial subspaces from previous tasks to enable learning of new tasks with less interference. We apply our method to sequentially learning to classify digits 0, 1, 2 (task 1), 4, 5, 6, (task 2), and 7, 8, 9 (task 3) in MNIST (disjoint MNIST task). We compare and combine our Adversarial Direction (AD) method with the recently proposed Elastic Weight Consolidation (EWC) method for sequential learning. We train each task for 20 epochs, which yields good initial performance (99.24% correct task 1 performance). After training task 2, and then task 3, both plain gradient descent (PGD) and EWC largely forget task 1 (task 1 accuracy 32.95% for PGD and 41.02% for EWC), while our combined approach (AD+EWC) still achieves 94.53% correct on task 1. We obtain similar results with a much more difficult disjoint CIFAR10 task (70.10% initial task 1 performance, 67.73% after learning tasks 2 and 3 for AD+EWC, while PGD and EWC both fall to chance level). We confirm qualitatively similar results for EMNIST with 5 tasks and under 3 variants of our approach. Our results suggest that AD+EWC can provide better sequential learning performance than either PGD or EWC.
Localized random projections challenge benchmarks for bio-plausible deep learning
Authors: Bernd Illing, Wulfram Gerstner, Johanni Brea
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, General Classification, Object Recognition

Similar to models of brain-like computation, artificial deep neural networks rely
on distributed coding, parallel processing and plastic synaptic weights. Training
deep neural networks with the error-backpropagation algorithm, however, is
considered bio-implausible. An appealing alternative to training deep neural networks
is to use one or a few hidden layers with fixed random weights or trained
with an unsupervised, local learning rule and train a single readout layer with a
supervised, local learning rule. We find that a network of leaky-integrate-andfire
neurons with fixed random, localized receptive fields in the hidden layer and
spike timing dependent plasticity to train the readout layer achieves 98.1% test
accuracy on MNIST, which is close to the optimal result achievable with error-backpropagation
in non-convolutional networks of rate neurons with one hidden
layer. To support the design choices of the spiking network, we systematically
compare the classification performance of rate networks with a single hidden
layer, where the weights of this layer are either random and fixed, trained with
unsupervised Principal Component Analysis or Sparse Coding, or trained with
the backpropagation algorithm. This comparison revealed, first, that unsupervised
learning does not lead to better performance than fixed random projections for
large hidden layers on digit classification (MNIST) and object recognition (CIFAR10);
second, networks with random projections and localized receptive fields
perform significantly better than networks with all-to-all connectivity and almost
reach the performance of networks trained with the backpropagation algorithm.
The performance of these simple random projection networks is comparable to
most current models of bio-plausible deep learning and thus provides an interesting
benchmark for future approaches.
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
Authors: Yunbo Wang, Lu Jiang, Ming-Hsuan Yang, Li-Jia Li, Mingsheng Long, Li Fei-Fei
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Activity Recognition, Video Prediction, Weather Forecasting

Spatiotemporal predictive learning, though long considered to be a promising self-supervised feature learning method, seldom shows its effectiveness beyond future video prediction. The reason is that it is difficult to learn good representations for both short-term frame dependency and long-term high-level relations. We present a new model, Eidetic 3D LSTM (E3D-LSTM), that integrates 3D convolutions into RNNs. The encapsulated 3D-Conv makes local perceptrons of RNNs motion-aware and enables the memory cell to store better short-term features. For long-term relations, we make the present memory state interact with its historical records via a gate-controlled self-attention module. We describe this memory transition mechanism eidetic as it is able to effectively recall the stored memories across multiple time stamps even after long periods of disturbance. We first evaluate the E3D-LSTM network on widely-used future video prediction datasets and achieve the state-of-the-art performance. Then we show that the E3D-LSTM network also performs well on the early activity recognition to infer what is happening or what will happen after observing only limited frames of video. This task aligns well with video prediction, as action intentions and tendency are important to achieve the state-of-the-art performance.
Learning Goal-Conditioned Value Functions with one-step Path rewards rather than Goal-Rewards
Authors: Vikas Dhiman, Shurjo Banerjee, Jeffrey M. Siskind, Jason J. Corso
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Multi-Goal Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Multi-goal reinforcement learning (MGRL) addresses tasks where the desired goal state can change for every trial. State-of-the-art algorithms model these problems such that the reward formulation depends on the goals, to associate them with high reward. This dependence introduces additional goal reward resampling steps in algorithms like Hindsight Experience Replay (HER) that reuse trials in which the agent fails to reach the goal by recomputing rewards as if reached states were psuedo-desired goals. We propose a reformulation of goal-conditioned value functions for MGRL that yields a similar algorithm, while removing the dependence of reward functions on the goal. Our formulation thus obviates the requirement of reward-recomputation that is needed by HER and its extensions. We also extend a closely related algorithm, Floyd-Warshall Reinforcement Learning, from tabular domains to deep neural networks for use as a baseline. Our results are competetive with HER while substantially improving sampling efficiency in terms of reward computation.
Unsupervised Hyper-alignment for Multilingual Word Embeddings
Authors: Jean Alaux, Edouard Grave, Marco Cuturi, Armand Joulin
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Multilingual Word Embeddings, Translation, Word Embeddings, Word Translation

We consider the problem of aligning continuous word representations, learned in multiple languages, to a common space. It was recently shown that, in the case of two languages, it is possible to learn such a mapping without supervision. This paper extends this line of work to the problem of aligning multiple languages to a common space. A solution is to independently map all languages to a pivot language. Unfortunately, this degrades the quality of indirect word translation. We thus propose a novel formulation that ensures composable mappings, leading to better alignments. We evaluate our method by jointly aligning word vectors in eleven languages, showing consistent improvement with indirect mappings while maintaining competitive performance on direct word translation.
Synthnet: Learning synthesizers end-to-end
Authors: Florin Schimbinschi, Christian Walder, Sarah Erfani, James Bailey
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5

Learning synthesizers and generating music in the raw audio domain is a challenging task. We investigate the learned representations of convolutional autoregressive generative models. Consequently, we show that mappings between musical notes and the harmonic style (instrument timbre) can be learned based on the raw audio music recording and the musical score (in binary piano roll format). Our proposed architecture, SynthNet uses minimal training data (9 minutes), is substantially better in quality and converges 6 times faster than the baselines. The quality of the generated waveforms (generation accuracy) is sufficiently high that they are almost identical to the ground truth. Therefore, we are able to directly measure generation error during training, based on the RMSE of the Constant-Q transform. Mean opinion scores are also provided. We validate our work using 7 distinct harmonic styles and also provide visualizations and links to all generated audio.
Diversity and Depth in Per-Example Routing Models
Authors: Prajit Ramachandran, Quoc V. Le
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Diversity, Multi-Task Learning

Routing models, a form of conditional computation where examples are routed through a subset of components in a larger network, have shown promising results in recent works. Surprisingly, routing models to date have lacked important properties, such as architectural diversity and large numbers of routing decisions. Both architectural diversity and routing depth can increase the representational power of a routing network. In this work, we address both of these deficiencies. We discuss the significance of architectural diversity in routing models, and explain the tradeoffs between capacity and optimization when increasing routing depth. In our experiments, we find that adding architectural diversity to routing models significantly improves performance, cutting the error rates of a strong baseline by 35% on an Omniglot setup. However, when scaling up routing depth, we find that modern routing techniques struggle with optimization. We conclude by discussing both the positive and negative results, and suggest directions for future research.