pwc-archive/papers-with-abstracts huggingface dataset:
On the Units of GANs (Extended Abstract)
Authors: David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba
Date: 2019-01-29 00:00:00

Generative Adversarial Networks (GANs) have achieved impressive results for many real-world applications. As an active research topic, many GAN variants have emerged with improvements in sample quality and training stability. However, visualization and understanding of GANs is largely missing. How does a GAN represent our visual world internally? What causes the artifacts in GAN results? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to concepts with a segmentation-based network dissection method. We quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. Finally, we examine the contextual relationship between these units and their surrounding by inserting the discovered concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in the scene. We will open source our interactive tools to help researchers and practitioners better understand their models.
Sample Complexity Bounds for Recurrent Neural Networks with Application to Combinatorial Graph Problems
Authors: Nil-Jana Akpinar, Bernhard Kratzwald, Stefan Feuerriegel
Date: 2019-01-29 00:00:00

Learning to predict solutions to real-valued combinatorial graph problems promises efficient approximations. As demonstrated based on the NP-hard edge clique cover number, recurrent neural networks (RNNs) are particularly suited for this task and can even outperform state-of-the-art heuristics. However, the theoretical framework for estimating real-valued RNNs is understood only poorly. As our primary contribution, this is the first work that upper bounds the sample complexity for learning real-valued RNNs. While such derivations have been made earlier for feed-forward and convolutional neural networks, our work presents the first such attempt for recurrent neural networks. Given a single-layer RNN with $a$ rectified linear units and input of length $b$, we show that a population prediction error of $\varepsilon$ can be realized with at most $\tilde{\mathcal{O}}(a^4b/\varepsilon^2)$ samples. We further derive comparable results for multi-layer RNNs. Accordingly, a size-adaptive RNN fed with graphs of at most $n$ vertices can be learned in $\tilde{\mathcal{O}}(n^6/\varepsilon^2)$, i.e., with only a polynomial number of samples. For combinatorial graph problems, this provides a theoretical foundation that renders RNNs competitive.
Structural Material Property Tailoring Using Deep Neural Networks
Authors: Oshin Olesegun, Ryan Noraas, Michael Giering, Nagendra Somanath
Date: 2019-01-29 00:00:00
Tasks: Management

Advances in robotics, artificial intelligence, and machine learning are
ushering in a new age of automation, as machines match or outperform human
performance. Machine intelligence can enable businesses to improve performance
by reducing errors, improving sensitivity, quality and speed, and in some cases
achieving outcomes that go beyond current resource capabilities. Relevant
applications include new product architecture design, rapid material
characterization, and life-cycle management tied with a digital strategy that
will enable efficient development of products from cradle to grave. In
addition, there are also challenges to overcome that must be addressed through
a major, sustained research effort that is based solidly on both inferential
and computational principles applied to design tailoring of functionally
optimized structures. Current applications of structural materials in the
aerospace industry demand the highest quality control of material
microstructure, especially for advanced rotational turbomachinery in aircraft
engines in order to have the best tailored material property. In this paper,
deep convolutional neural networks were developed to accurately predict
processing-structure-property relations from materials microstructures images,
surpassing current best practices and modeling efforts. The models
automatically learn critical features, without the need for manual
specification and/or subjective and expensive image analysis. Further, in
combination with generative deep learning models, a framework is proposed to
enable rapid material design space exploration and property identification and
optimization. The implementation must take account of real-time decision cycles
and the trade-offs between speed and accuracy.
High-Quality Self-Supervised Deep Image Denoising
Authors: Samuli Laine, Tero Karras, Jaakko Lehtinen, Timo Aila
Date: 2019-01-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Denoising, Image Denoising, Vocal Bursts Intensity Prediction

We describe a novel method for training high-quality image denoising models based on unorganized collections of corrupted images. The training does not need access to clean reference images, or explicit pairs of corrupted images, and can thus be applied in situations where such data is unacceptably expensive or impossible to acquire. We build on a recent technique that removes the need for reference data by employing networks with a "blind spot" in the receptive field, and significantly improve two key aspects: image quality and training efficiency. Our result quality is on par with state-of-the-art neural network denoisers in the case of i.i.d. additive Gaussian noise, and not far behind with Poisson and impulse noise. We also successfully handle cases where parameters of the noise model are variable and/or unknown in both training and evaluation data.
Differentially Private Markov Chain Monte Carlo
Authors: Mikko A. Heikkilä, Joonas Jälkö, Onur Dikmen, Antti Honkela
Date: 2019-01-29 00:00:00
Proceeding: NeurIPS 2019 12

Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC) algorithm whose privacy-guarantees are not subject to unrealistic assumptions on Markov chain convergence and that is applicable to posterior inference in arbitrary models. Our algorithm is based on a decomposition of the Barker acceptance test that allows evaluating the R\'enyi DP privacy cost of the accept-reject choice. We further show how to improve the DP guarantee through data subsampling and approximate acceptance tests.
Combined tract segmentation and orientation mapping for bundle-specific tractography
Authors: Jakob Wasserthal, Peter Neher, Dusan Hirjak, Klaus H. Maier-Hein
Date: 2019-01-29 00:00:00
Tasks: Clustering, Diffusion  MRI

While the major white matter tracts are of great interest to numerous studies in neuroscience and medicine, their manual dissection in larger cohorts from diffusion MRI tractograms is time-consuming, requires expert knowledge and is hard to reproduce. In previous work we presented tract orientation mapping (TOM) as a novel concept for bundle-specific tractography. It is based on a learned mapping from the original fiber orientation distribution function (FOD) peaks to tract specific peaks, called tract orientation maps. Each tract orientation map represents the voxel-wise principal orientation of one tract. Here, we present an extension of this approach that combines TOM with accurate segmentations of the tract outline and its start and end region. We also introduce a custom probabilistic tracking algorithm that samples from a Gaussian distribution with fixed standard deviation centered on each peak thus enabling more complete trackings on the tract orientation maps than deterministic tracking. These extensions enable the automatic creation of bundle-specific tractograms with previously unseen accuracy. We show for 72 different bundles on high quality, low quality and phantom data that our approach runs faster and produces more accurate bundle-specific tractograms than 7 state of the art benchmark methods while avoiding cumbersome processing steps like whole brain tractography, non-linear registration, clustering or manual dissection. Moreover, we show on 17 datasets that our approach generalizes well to datasets acquired with different scanners and settings as well as with pathologies. The code of our method is openly available at https://github.com/MIC-DKFZ/TractSeg.
Approximation of functions by neural networks
Authors: Andreas Thom
Date: 2019-01-29 00:00:00

We study the approximation of measurable functions on the hypercube by
functions arising from affine neural networks. Our main achievement is an
approximation of any measurable function $f \colon W_n \to [-1,1]$ up to a
prescribed precision $\varepsilon>0$ by a bounded number of neurons, depending
only on $\varepsilon$ and not on the function $f$ or $n \in \mathbb N$.
Implicit Diversity in Image Summarization
Authors: L. Elisa Celis, Vijay Keswani
Date: 2019-01-29 00:00:00
Tasks: Attribute, Diversity, Image Retrieval

Studies have shown that the people depicted in image search results tend to be of majority groups with respect to socially salient attributes. This skew goes beyond that which already exists in the world - e.g., Kay et al. showed that although 28% of CEOs in US are women, only 10% of the top 100 results for CEO in Google Image Search are women. Most existing approaches to correct for this kind of bias assume that the images of people include socially salient attribute labels. However, such labels are often unknown. Further, using automated techniques to infer these labels may often not be possible within acceptable accuracy ranges, and may not be desirable due to the additional biases this process could incur. We develop a novel approach that takes as input a visibly diverse control set of images and uses this set to select a set of images of people in response to a query. The goal is to have a resulting set that is more visibly diverse in a manner that emulates the diversity depicted in the control set. Importantly, this approach does not require images to be labelled at any point; effectively, it gives a way to implicitly diversify the set of images selected. We provide two variants of our approach: the first is a modification of the MMR algorithm to incorporate the diversity scores, and second is a more efficient variant that does not consider within-list redundancy. We evaluate these approaches empirically on two datasets 1) a new dataset containing top Google image results for 96 occupations, for which we evaluate gender and skin-tone diversity with respect to occupations and 2) the CelebA dataset for which we evaluate gender diversity with respect to facial features. Our approaches produce image sets that significantly improve the visible diversity of the results, compared to current Google search and other diverse image summarization algorithms, at a minimal cost to accuracy.
TiFi: Taxonomy Induction for Fictional Domains [Extended version]
Authors: Cuong Xuan Chu, Simon Razniewski, Gerhard Weikum
Date: 2019-01-29 00:00:00

Taxonomies are important building blocks of structured knowledge bases, and
their construction from text sources and Wikipedia has received much attention.
In this paper we focus on the construction of taxonomies for fictional domains,
using noisy category systems from fan wikis or text extraction as input. Such
fictional domains are archetypes of entity universes that are poorly covered by
Wikipedia, such as also enterprise-specific knowledge bases or highly
specialized verticals. Our fiction-targeted approach, called TiFi, consists of
three phases: (i) category cleaning, by identifying candidate categories that
truly represent classes in the domain of interest, (ii) edge cleaning, by
selecting subcategory relationships that correspond to class subsumption, and
(iii) top-level construction, by mapping classes onto a subset of high-level
WordNet categories. A comprehensive evaluation shows that TiFi is able to
construct taxonomies for a diverse range of fictional domains such as Lord of
the Rings, The Simpsons or Greek Mythology with very high precision and that it
outperforms state-of-the-art baselines for taxonomy induction by a substantial
margin.
RED-Attack: Resource Efficient Decision based Attack for Machine Learning
Authors: Faiq Khalid, Hassan Ali, Muhammad Abdullah Hanif, Semeen Rehman, Rehan Ahmed, Muhammad Shafique
Date: 2019-01-29 00:00:00
Tasks: Autonomous Vehicles, BIG-bench Machine Learning, General Classification, Traffic Sign Recognition

Due to data dependency and model leakage properties, Deep Neural Networks
(DNNs) exhibit several security vulnerabilities. Several security attacks
exploited them but most of them require the output probability vector. These
attacks can be mitigated by concealing the output probability vector. To
address this limitation, decision-based attacks have been proposed which can
estimate the model but they require several thousand queries to generate a
single untargeted attack image. However, in real-time attacks, resources and
attack time are very crucial parameters. Therefore, in resource-constrained
systems, e.g., autonomous vehicles where an untargeted attack can have a
catastrophic effect, these attacks may not work efficiently. To address this
limitation, we propose a resource efficient decision-based methodology which
generates the imperceptible attack, i.e., the RED-Attack, for a given black-box
model. The proposed methodology follows two main steps to generate the
imperceptible attack, i.e., classification boundary estimation and adversarial
noise optimization. Firstly, we propose a half-interval search-based algorithm
for estimating a sample on the classification boundary using a target image and
a randomly selected image from another class. Secondly, we propose an
optimization algorithm which first, introduces a small perturbation in some
randomly selected pixels of the estimated sample. Then to ensure
imperceptibility, it optimizes the distance between the perturbed and target
samples. For illustration, we evaluate it for CFAR-10 and German Traffic Sign
Recognition (GTSR) using state-of-the-art networks.
Understanding and Training Deep Diagonal Circulant Neural Networks
Authors: Alexandre Araujo, Benjamin Negrevergne, Yann Chevaleyre, Jamal Atif
Date: 2019-01-29 00:00:00
Tasks: Video Classification

In this paper, we study deep diagonal circulant neural networks, that is deep neural networks in which weight matrices are the product of diagonal and circulant ones. Besides making a theoretical analysis of their expressivity, we introduced principled techniques for training these models: we devise an initialization scheme and proposed a smart use of non-linearity functions in order to train deep diagonal circulant networks. Furthermore, we show that these networks outperform recently introduced deep networks with other types of structured layers. We conduct a thorough experimental study to compare the performance of deep diagonal circulant networks with state of the art models based on structured matrices and with dense models. We show that our models achieve better accuracy than other structured approaches while required 2x fewer weights as the next best approach. Finally we train deep diagonal circulant networks to build a compact and accurate models on a real world video classification dataset with over 3.8 million training examples.
Learning for Multi-Model and Multi-Type Fitting
Authors: Xun Xu, Loong-Fah Cheong, Zhuwen Li
Date: 2019-01-29 00:00:00
Tasks: Clustering, model, Model Selection, Vocal Bursts Type Prediction

Multi-model fitting has been extensively studied from the random sampling and
clustering perspectives. Most assume that only a single type/class of model is
present and their generalizations to fitting multiple types of
models/structures simultaneously are non-trivial. The inherent challenges
include choice of types and numbers of models, sampling imbalance and parameter
tuning, all of which render conventional approaches ineffective. In this work,
we formulate the multi-model multi-type fitting problem as one of learning deep
feature embedding that is clustering-friendly. In other words, points of the
same clusters are embedded closer together through the network. For inference,
we apply K-means to cluster the data in the embedded feature space and model
selection is enabled by analyzing the K-means residuals. Experiments are
carried out on both synthetic and real world multi-type fitting datasets,
producing state-of-the-art results. Comparisons are also made on single-type
multi-model fitting tasks with promising results as well.
Multi-Agent Reinforcement Learning with Multi-Step Generative Models
Authors: Orr Krupnik, Igor Mordatch, Aviv Tamar
Date: 2019-01-29 00:00:00
Tasks: continuous-control, Continuous Control, Decision Making, Model-based Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Manipulation

We consider model-based reinforcement learning (MBRL) in 2-agent, high-fidelity continuous control problems -- an important domain for robots interacting with other agents in the same workspace. For non-trivial dynamical systems, MBRL typically suffers from accumulating errors. Several recent studies have addressed this problem by learning latent variable models for trajectory segments and optimizing over behavior in the latent space. In this work, we investigate whether this approach can be extended to 2-agent competitive and cooperative settings. The fundamental challenge is how to learn models that capture interactions between agents, yet are disentangled to allow for optimization of each agent behavior separately. We propose such models based on a disentangled variational auto-encoder, and demonstrate our approach on a simulated 2-robot manipulation task, where one robot can either help or distract the other. We show that our approach has better sample efficiency than a strong model-free RL baseline, and can learn both cooperative and adversarial behavior from the same data.
Explicit topological priors for deep-learning based image segmentation using persistent homology
Authors: James R. Clough, Ilkay Oksuz, Nicholas Byrne, Julia A. Schnabel, Andrew P. King
Date: 2019-01-29 00:00:00
Tasks: Image Segmentation, Left Ventricle Segmentation, Segmentation, Semantic Segmentation, Topological Data Analysis

We present a novel method to explicitly incorporate topological prior
knowledge into deep learning based segmentation, which is, to our knowledge,
the first work to do so. Our method uses the concept of persistent homology, a
tool from topological data analysis, to capture high-level topological
characteristics of segmentation results in a way which is differentiable with
respect to the pixelwise probability of being assigned to a given class. The
topological prior knowledge consists of the sequence of desired Betti numbers
of the segmentation. As a proof-of-concept we demonstrate our approach by
applying it to the problem of left-ventricle segmentation of cardiac MR images
of 500 subjects from the UK Biobank dataset, where we show that it improves
segmentation performance in terms of topological correctness without
sacrificing pixelwise accuracy.
Automatic Whole-body Bone Age Assessment Using Deep Hierarchical Features
Authors: Hai-Duong Nguyen, Soo-Hyung Kim
Date: 2019-01-29 00:00:00
Tasks: Medical Image Analysis

Bone age assessment gives us evidence to analyze the children growth status
and the rejuvenation involved chronological and biological ages. All the
previous works consider left-hand X-ray image of a child in their works. In
this paper, we carry out a study on estimating human age using whole-body bone
CT images and a novel convolutional neural network. Our model with additional
connections shows an effective way to generate a massive number of vital
features while reducing overfitting influence on small training data in the
medical image analysis research area. A dataset and a comparison with common
deep architectures will be provided for future research in this field.
Representation Learning for Heterogeneous Information Networks via Embedding Events
Authors: Guoji Fu, Bo Yuan, Qiqi Duan, Xin Yao
Date: 2019-01-29 00:00:00
Tasks: Link Prediction, Node Classification, Object, Representation Learning

Network representation learning (NRL) has been widely used to help analyze
large-scale networks through mapping original networks into a low-dimensional
vector space. However, existing NRL methods ignore the impact of properties of
relations on the object relevance in heterogeneous information networks (HINs).
To tackle this issue, this paper proposes a new NRL framework, called
Event2vec, for HINs to consider both quantities and properties of relations
during the representation learning process. Specifically, an event (i.e., a
complete semantic unit) is used to represent the relation among multiple
objects, and both event-driven first-order and second-order proximities are
defined to measure the object relevance according to the quantities and
properties of relations. We theoretically prove how event-driven proximities
can be preserved in the embedding space by Event2vec, which utilizes event
embeddings to facilitate learning the object embeddings. Experimental studies
demonstrate the advantages of Event2vec over state-of-the-art algorithms on
four real-world datasets and three network analysis tasks (including network
reconstruction, link prediction, and node classification).
Reconstruction of 3D Porous Media From 2D Slices
Authors: Denis Volkhonskiy, Ekaterina Muravleva, Oleg Sudakov, Denis Orlov, Boris Belozerov, Evgeny Burnaev, Dmitry Koroteev
Date: 2019-01-29 00:00:00

In many branches of earth sciences, the problem of rock study on the micro-level arises. However, a significant number of representative samples is not always feasible. Thus the problem of the generation of samples with similar properties becomes actual. In this paper, we propose a novel deep learning architecture for three-dimensional porous media reconstruction from two-dimensional slices. We fit a distribution on all possible three-dimensional structures of a specific type based on the given dataset of samples. Then, given partial information (central slices), we recover the three-dimensional structure around such slices as the most probable one according to that constructed distribution. Technically, we implement this in the form of a deep neural network with encoder, generator and discriminator modules. Numerical experiments show that this method provides a good reconstruction in terms of Minkowski functionals.