pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Defense by Learning to Attack
Authors: Zhehui Chen, Haoming Jiang, Yuyang Shi, Bo Dai, Tuo Zhao
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Adversarial Attack, Robust classification

Adversarial training provides a principled approach for training robust neural networks. From an optimization perspective, the adversarial training is essentially solving a minmax robust optimization problem. The outer minimization is trying to learn a robust classifier, while the inner maximization is trying to generate adversarial samples. Unfortunately, such a minmax problem is very difficult to solve due to the lack of convex-concave structure. This work proposes a new adversarial training method based on a general learning-to-learn framework. Specifically, instead of applying the existing hand-design algorithms for the inner problem, we learn an optimizer, which is parametrized as a convolutional neural network. At the same time, a robust classifier is learned to defense the adversarial attack generated by the learned optimizer. From the perspective of generative learning, our proposed method can be viewed as learning a deep generative model for generating adversarial samples, which is adaptive to the robust classification. Our experiments demonstrate that our proposed method significantly outperforms existing adversarial training methods on CIFAR-10 and CIFAR-100 datasets.
Understanding the Relation Between Maximum-Entropy Inverse Reinforcement Learning and Behaviour Cloning
Authors: Seyed Kamyar Seyed Ghasemipour, Shane Gu, Richard Zemel
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: continuous-control, Continuous Control, Decision Making, reinforcement-learning, Reinforcement Learning (RL), Relation

In many settings, it is desirable to learn decision-making and control policies through learning or from expert demonstrations. The most common approaches under this framework are Behaviour Cloning (BC), and Inverse Reinforcement Learning (IRL). Recent methods for IRL have demonstrated the capacity to learn effective policies with access to a very limited set of demonstrations, a scenario in which BC methods often fail. Unfortunately, directly comparing the algorithms for these methods does not provide adequate intuition for understanding this difference in performance. This is the motivating factor for our work. We begin by presenting $f$-MAX, a generalization of AIRL (Fu et al., 2018), a state-of-the-art IRL method. $f$-MAX provides grounds for more directly comparing the objectives for LfD. We demonstrate that $f$-MAX, and by inheritance AIRL, is a subset of the cost-regularized IRL framework laid out by Ho & Ermon (2016). We conclude by empirically evaluating the factors of difference between various LfD objectives in the continuous control domain.
Learning Deep Latent-variable MRFs with Amortized Bethe Free Energy Minimization
Authors: Sam Wiseman
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Variational Inference

While much recent work has targeted learning deep discrete latent variable models with variational inference, this setting remains challenging, and it is often necessary to make use of potentially high-variance gradient estimators in optimizing the ELBO. As an alternative, we propose to optimize a non-ELBO objective derived from the Bethe free energy approximation to an MRF's partition function. This objective gives rise to a saddle-point learning problem, which we train inference networks to approximately optimize. The derived objective requires no sampling, and can be efficiently computed for many MRFs of interest. We evaluate the proposed approach in learning high-order neural HMMs on text, and find that it often outperforms other approximate inference schemes in terms of true held-out log likelihood. At the same time, we find that all the approximate inference-based approaches to learning high-order neural HMMs we consider underperform learning with exact inference by a significant margin.
Compositional GAN (Extended Abstract): Learning Image-Conditional Binary Composition
Authors: Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, Trevor Darrell
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019

Generative Adversarial Networks (GANs) can produce images of surprising complexity and realism but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different objects in the world, including their relative scaling, spatial layout, occlusion, or viewpoint transformation is a challenging problem. In this work, we compose a pair of objects in a conditional GAN framework using a novel self-consistent composition-by-decomposition network. Given object images from two distinct distributions, our model can generate a realistic composite image from their joint distribution following the texture and shape of the input objects. Our results reveal that the learned model captures potential interactions between the two object domains, and can output their realistic composed scene at test time.
Deep Generative Models for Generating Labeled Graphs
Authors: Shuangfei Fan, Bert Huang
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Image Generation

As a new way to train generative models, generative adversarial networks (GANs) have achieved considerable success in image generation, and this framework has also recently been applied to data with graph structures. We identify the drawbacks of existing deep frameworks for generating graphs, and we propose labeled-graph generative adversarial networks (LGGAN) to train deep generative models for graph-structured data with node labels. We test the approach with different discriminative models as well as different GAN frameworks on various types of graph datasets, such as collections of citation networks and protein graphs. Experiment results show that our model can generate diverse labeled graphs that match the structural characteristics of the training data and outperforms all baselines in terms of quality, generality, and scalability.
DIVA: Domain Invariant Variational Autoencoder
Authors: Maximilian Ilse, Jakub M. Tomczak, Christos Louizos, Max Welling
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Domain Generalization, Rotated MNIST

We consider the problem of domain generalization, namely, how to learn representations
given data from a set of domains that generalize to data from a previously
unseen domain. We propose the domain invariant VAE (DIVA), a generative
model that tackles this problem by learning three independent latent subspaces,
one for the class, one for the domain and one for the object itself. In addition,
we highlight that due to the generative nature of our model we can also incorporate
unlabeled data from known or previously unseen domains. This property is
highly desirable in fields like medical imaging where labeled data is scarce. We
experimentally evaluate our model on the rotated MNIST benchmark where we
show that (i) the learned subspaces are indeed complementary to each other, (ii)
we improve upon recent works on this task and (iii) incorporating unlabelled data
can boost the performance even further.
HYPE: Human-eYe Perceptual Evaluation of Generative Models
Authors: Sharon Zhou, Mitchell Gordon, Ranjay Krishna, Austin Narcomey, Durim Morina, Michael S. Bernstein
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Image Generation, Unconditional Image Generation

Generative models often use human evaluations to determine and justify progress. Unfortunately, existing human evaluation methods are ad-hoc: there is currently no standardized, validated evaluation that: (1) measures perceptual fidelity, (2) is reliable, (3) separates models into clear rank order, and (4) ensures high-quality measurement without intractable cost. In response, we construct Human-eYe Perceptual Evaluation (HYPE), a human metric that is (1) grounded in psychophysics research in perception, (2) reliable across different sets of randomly sampled outputs from a model, (3) results in separable model performances, and (4) efficient in cost and time. We introduce two methods. The first, HYPE-Time, measures visual perception under adaptive time constraints to determine the minimum length of time (e.g., 250ms) that model output such as a generated face needs to be visible for people to distinguish it as real or fake. The second, HYPE-Infinity, measures human error rate on fake and real images with no time constraints, maintaining stability and drastically reducing time and cost. We test HYPE across four state-of-the-art generative adversarial networks (GANs) on unconditional image generation using two datasets, the popular CelebA and the newer higher-resolution FFHQ, and two sampling techniques of model outputs. By simulating HYPE's evaluation multiple times, we demonstrate consistent ranking of different models, identifying StyleGAN with truncation trick sampling (27.6% HYPE-Infinity deception rate, with roughly one quarter of images being misclassified by humans) as superior to StyleGAN without truncation (19.0%) on FFHQ.
Fully differentiable full-atom protein backbone generation
Authors: Namrata Anand, Raphael Eguchi, Po-Ssu Huang
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019

The fast generation and refinement of protein backbones would constitute a major advancement to current methodology for the design and development of de novo proteins. In this study, we train Generative Adversarial Networks (GANs) to generate fixed-length full-atom protein backbones, with the goal of sampling from the distribution of realistic 3-D backbone fragments. We represent protein structures by pairwise distances between all backbone atoms, and present a method for directly recovering and refining the corresponding backbone coordinates in a differentiable manner. We show that interpolations in the latent space of the generator correspond to smooth deformations of the output backbones, and that test set structures not seen by the generator during training exist in its image. Finally, we perform sequence design, relaxation, and ab initio folding of a subset of generated structures, and show that in some cases we can recover the generated folds after forward-folding. Together, these results suggest a mechanism for fast protein structure refinement and folding using external energy functions.
Visualizing and Understanding GANs
Authors: David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Object

We present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to object concepts with a segmentation-based network dissection method. Then, we examine the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. Finally, we examine the contextual relationship between these units and their surrounding by inserting the discovered object concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers and models, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in the scene.
Interactive Visual Exploration of Latent Space (IVELS) for peptide auto-encoder model selection
Authors: Tom Sercu, Sebastian Gehrmann, Hendrik Strobelt, Payel Das, Inkit Padhi, Cicero dos Santos, Kahini Wadhawan, Vijil Chenthamarakshan
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Model Selection

We present a tool for Interactive Visual Exploration of Latent Space (IVELS) for model selection.  Evaluating generative models of discrete sequences from a  continuous  latent  space  is  a  challenging  problem,  since  their  optimization involves multiple competing objective terms.  We introduce a model-selection pipeline  to  compare  and  filter  models  throughout  consecutive  stages  of  more complex and expensive metrics. We present the pipeline in an interactive visual tool to enable the exploration of the metrics, analysis of the learned latent space, and selection of the best model for a given task.  We focus specifically on the variational auto-encoder family in a case study of modeling peptide sequences, which are short sequences of amino acids. This task is especially interesting due to the presence of multiple attributes we want to model. We demonstrate how an interactive visual comparison can assist in evaluating how well an unsupervised auto-encoder meaningfully captures the attributes of interest in its latent space.
On the relationship between Normalising Flows and Variational- and Denoising Autoencoders
Authors: Alexey A. Gritsenko, Jasper Snoek, Tim Salimans
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Denoising, Normalising Flows, Representation Learning

Normalising Flows (NFs) are a class of likelihood-based generative models that have recently gained popularity. They are based on the idea of transforming a simple density into that of the data. We seek to better understand this class of models, and how they compare to previously proposed techniques for generative modeling and unsupervised representation learning. For this purpose we reinterpret NFs in the framework of Variational Autoencoders (VAEs), and present a new form of VAE that generalises normalising flows. The new generalised model also reveals a close connection to denoising autoencoders, and we therefore call our model the Variational Denoising Autoencoder (VDAE). Using our unified model, we systematically examine the model space between flows, variational autoencoders, and denoising autoencoders, in a set of preliminary experiments on the MNIST handwritten digits. The experiments shed light on the modeling assumptions implicit in these models, and they suggest multiple new directions for future research in this space.
WiSE-ALE: Wide Sample Estimator for Aggregate Latent Embedding
Authors: Shuyu Lin, Ronald Clark, Robert Birke, Niki Trigoni, Stephen Roberts
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019

In this paper, we present a new generative model for learning latent embeddings. Compared to the classical generative process, where each observed data point is generated from an individual latent variable, our approach assumes a global latent variable to generate the whole set of observed data points. We then propose a learning objective that is derived as an approximation to a lower bound to the data log likelihood, leading to our algorithm, WiSE-ALE. Compared to the standard ELBO objective, where the variational posterior for each data point is encouraged to match the prior distribution, the WiSE-ALE objective matches the averaged posterior, over all samples, with the prior, allowing the sample-wise posterior distributions to have a wider range of acceptable embedding mean and variance and leading to better reconstruction quality in the auto-encoding process. Through various examples and comparison to other state-of-the-art VAE models, we demonstrate that WiSE-ALE has excellent information embedding properties, whilst still retaining the ability to learn a smooth, compact representation.
DISENTANGLED STATE SPACE MODELS: UNSUPERVISED LEARNING OF DYNAMICS ACROSS HETEROGENEOUS ENVIRONMENTS
Authors: Ðorđe Miladinović, Waleed Gondal, Bernhard Schölkopf, Joachim M. Buhmann, Stefan Bauer
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Prediction, State Space Models

Sequential data often originates from diverse environments. Across them exist both shared regularities and environment specifics. To learn robust cross-environment descriptions of sequences we introduce disentangled state space models (DSSM). In the latent space of DSSM environment-invariant state dynamics is explicitly disentangled from environment-specific information governing that dynamics. We empirically show that such separation enables robust prediction, sequence manipulation and environment characterization. We also propose an unsupervised VAE-based training procedure to learn DSSM as Bayesian filters. In our experiments, we demonstrate state-of-the-art performance in controlled generation and prediction of bouncing ball video sequences across varying gravitational influences.
Generating Molecules via Chemical Reactions
Authors: John Bradshaw, Matt J. Kusner, Brooks Paige, Marwin H. S. Segler, José Miguel Hernández-Lobato
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Retrosynthesis, valid

Over the last few years exciting work in deep generative models has produced models able to suggest new organic molecules by generating strings, trees, and graphs representing their structure. While such models are able to generate molecules with desirable properties, their utility in practice is limited due to the difficulty in knowing how to synthesize these molecules. We therefore propose a new molecule generation model, mirroring a more realistic real-world process, where reactants are selected and combined to form more complex molecules. More specifically, our generative model proposes a bag of initial reactants (selected from a pool of commercially-available molecules) and uses a reaction model to predict how they react together to generate new molecules. Modeling the entire process of constructing a molecule during generation offers a number of advantages. First, we show that such a model has the ability to generate a wide, diverse set of valid and unique molecules due to the useful inductive biases of modeling reactions. Second, modeling synthesis routes rather than final molecules offers practical advantages to chemists who are not only interested in new molecules but also suggestions on stable and safe synthetic routes. Third, we demonstrate the capabilities of our model to also solve one-step retrosynthesis problems, predicting a set of reactants that can produce a target product.
Deep Random Splines for Point Process Intensity Estimation
Authors: Gabriel Loaiza-Ganem, John P. Cunningham
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Gaussian Processes, Point Processes

Gaussian processes are the leading class of distributions on random functions, but they suffer from well known issues including difficulty scaling and inflexibility with respect to certain shape constraints (such as nonnegativity). Here we propose Deep Random Splines, a flexible class of random functions obtained by transforming Gaussian noise through a deep neural network whose output are the parameters of a spline. Unlike Gaussian processes, Deep Random Splines allow us to readily enforce shape constraints while inheriting the richness and tractability of deep generative models. We also present an observational model for point process data which uses Deep Random Splines to model the intensity function of each point process and apply it to neuroscience data to obtain a low-dimensional representation of spiking activity. Inference is performed via a variational autoencoder that uses a novel recurrent encoder architecture that can handle multiple point processes as input.
FVD: A new Metric for Video Generation
Authors: Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, Sylvain Gelly
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Diversity, Representation Learning, Video Generation

Recent advances in deep generative models have lead to remarkable progress in synthesizing high quality images. Following their successful application in image processing and representation learning, an important next step is to consider videos. Learning generative models of video is a much harder task, requiring a model to capture the temporal dynamics of a scene, in addition to the visual presentation of objects. While recent generative models of video have had some success, current progress is hampered by the lack of qualitative metrics that consider visual quality, temporal coherence, and diversity of samples. To this extent we propose Fréchet Video Distance (FVD), a new metric for generative models of video based on FID. We contribute a large-scale human study, which confirms that FVD correlates well with qualitative human judgment of generated videos.
Unsupervised Demixing of Structured Signals from Their Superposition Using GANs
Authors: Mohammadreza Soltani, Swayambhoo Jain, Abhinav Sambasivan
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019

Recently, Generative Adversarial Networks (GANs) have emerged as a popular alternative for modeling complex high dimensional distributions. Most of the existing works implicitly assume that the clean samples from the target distribution are easily available. However, in many applications, this assumption is violated. In this paper, we consider the observation setting in which the samples from a target distribution are given by the superposition of two structured components, and leverage GANs for learning of the structure of the components. We propose a novel framework, demixing-GAN, which learns the distribution of two components at the same time. Through extensive numerical experiments, we demonstrate that the proposed framework can generate clean samples from unknown distributions, which further can be used in demixing of the unseen test images.
Understanding Posterior Collapse in Generative Latent Variable Models
Authors: James Lucas, George Tucker, Roger Grosse, Mohammad Norouzi
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Variational Inference

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational distribution closely matches the uninformative prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We identify how local maxima can emerge from the marginal log-likelihood of pPCA, which yields similar local maxima for the evidence lower bound (ELBO). We show that training a linear VAE with variational inference recovers a uniquely identifiable global maximum corresponding to the principal component directions. We provide empirical evidence that the presence of local maxima causes posterior collapse in deep non-linear VAEs. Our findings help to explain a wide range of heuristic approaches in the literature that attempt to diminish the effect of the KL term in the ELBO to reduce posterior collapse.
Bias Correction of Learned Generative Models via Likelihood-free Importance Weighting
Authors: Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric Horvitz, Stefano Ermon
Date: 2019-03-27 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Data Augmentation

A learned generative model often gives biased statistics relative to the underlying data distribution. A standard technique to correct this bias is by importance weighting samples from the model by the likelihood ratio under the model and true distributions. When the likelihood ratio is unknown, it can be estimated by training a probabilistic classifier to distinguish samples from the two distributions. In this paper, we employ this likelihood-free importance weighting framework to correct for the bias in using state-of-the-art deep generative models.We find that this technique consistently improves standard goodness-of-fit metrics for evaluating the sample quality of state-of-the-art generative models, suggesting reduced bias. Finally, we demonstrate its utility on representative applications in a) data augmentation for classification using generative adversarial networks, and b) model-based policy evaluation using off-policy data.