pwc-archive/papers-with-abstracts huggingface dataset:
Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Losses
Authors: Ananya Uppal, Shashank Singh, Barnabás Póczos
Date: 2019-02-09 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Density Estimation

We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, $\mathcal{L}^p$ distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide both lower and upper bounds, identifying precisely how the choice of loss function and assumptions on the data interact to determine the minimax optimal convergence rate. We also show that linear distribution estimates, such as the empirical distribution or kernel density estimator, often fail to converge at the optimal rate. Our bounds generalize, unify, or improve several recent and classical results. Moreover, IPMs can be used to formalize a statistical model of generative adversarial networks (GANs). Thus, we show how our results imply bounds on the statistical error of a GAN, showing, for example, that GANs can strictly outperform the best linear estimator.
Assessing the Local Interpretability of Machine Learning Models
Authors: Dylan Slack, Sorelle A. Friedler, Carlos Scheidegger, Chitradeep Dutta Roy
Date: 2019-02-09 00:00:00
Tasks: BIG-bench Machine Learning

The increasing adoption of machine learning tools has led to calls for accountability via model interpretability. But what does it mean for a machine learning model to be interpretable by humans, and how can this be assessed? We focus on two definitions of interpretability that have been introduced in the machine learning literature: simulatability (a user's ability to run a model on a given input) and "what if" local explainability (a user's ability to correctly determine a model's prediction under local changes to the input, given knowledge of the model's original prediction). Through a user study with 1,000 participants, we test whether humans perform well on tasks that mimic the definitions of simulatability and "what if" local explainability on models that are typically considered locally interpretable. To track the relative interpretability of models, we employ a simple metric, the runtime operation count on the simulatability task. We find evidence that as the number of operations increases, participant accuracy on the local interpretability tasks decreases. In addition, this evidence is consistent with the common intuition that decision trees and logistic regression models are interpretable and are more interpretable than neural networks.
Multilingual Neural Machine Translation With Soft Decoupled Encoding
Authors: Xinyi Wang, Hieu Pham, Philip Arthur, Graham Neubig
Date: 2019-02-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Machine Translation, NMT, Translation

Multilingual training of neural machine translation (NMT) systems has led to
impressive accuracy improvements on low-resource languages. However, there are
still significant challenges in efficiently learning word representations in
the face of paucity of data. In this paper, we propose Soft Decoupled Encoding
(SDE), a multilingual lexicon encoding framework specifically designed to share
lexical-level information intelligently without requiring heuristic
preprocessing such as pre-segmenting the data. SDE represents a word by its
spelling through a character encoding, and its semantic meaning through a
latent embedding space shared by all languages. Experiments on a standard
dataset of four low-resource languages show consistent improvements over strong
multilingual NMT baselines, with gains of up to 2 BLEU on one of the tested
languages, achieving the new state-of-the-art on all four language pairs.
Space lower bounds for linear prediction in the streaming model
Authors: Yuval Dagan, Gil Kur, Ohad Shamir
Date: 2019-02-09 00:00:00
Tasks: regression

We show that fundamental learning tasks, such as finding an approximate linear separator or linear regression, require memory at least \emph{quadratic} in the dimension, in a natural streaming setting. This implies that such problems cannot be solved (at least in this setting) by scalable memory-efficient streaming algorithms. Our results build on a memory lower bound for a simple linear-algebraic problem -- finding orthogonal vectors -- and utilize the estimates on the packing of the Grassmannian, the manifold of all linear subspaces of fixed dimension.
Deep Algorithm Unrolling for Blind Image Deblurring
Authors: Yuelong Li, Mohammad Tofighi, Junyi Geng, Vishal Monga, Yonina C. Eldar
Date: 2019-02-09 00:00:00
Tasks: Blind Image Deblurring, Deblurring, Image Deblurring, Rolling Shutter Correction

Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed and the underlying structures are difficult to interpret. In recent years, a promising technique called algorithm unrolling has been developed that has helped connect iterative algorithms such as those for sparse coding to neural network architectures. However, such connections have not been made yet for blind image deblurring. In this paper, we propose a neural network architecture based on this idea. We first present an iterative algorithm that may be considered as a generalization of the traditional total-variation regularization method in the gradient domain. We then unroll the algorithm to construct a neural network for image deblurring which we refer to as Deep Unrolling for Blind Deblurring (DUBLID). Key algorithm parameters are learned with the help of training images. Our proposed deep network DUBLID achieves significant practical performance gains while enjoying interpretability at the same time. Extensive experimental results show that DUBLID outperforms many state-of-the-art methods and in addition is computationally faster.
Lumen boundary detection using neutrosophic c-means in IVOCT images
Authors: Mohammad Habibi, Ahmad Ayatollahi, Niyoosha Dallalazar, Ali Kermani
Date: 2019-02-09 00:00:00
Tasks: Boundary Detection

In this paper, a novel method for lumen boundary identification is proposed
using Neutrosophic c_means. This method clusters pixels of the intravascular
optical coherence tomography image into several clusters using indeterminacy
and Neutrosophic theory, which aims to detect the boundaries. Intravascular
optical coherence tomography images are cross-sectional and high-resolution
images which are taken from the coronary arterial wall. Coronary Artery Disease
cause a lot of death each year. The first step for diagnosing this kind of
diseases is to detect lumen boundary. Employing this approach, we obtained
0.972, 0.019, 0.076 mm2, 0.32 mm, and 0.985 as mean value for Jaccard measure
(JACC), the percentage of area difference (PAD), average distance (AD),
Hausdorff distance (HD), and dice index (DI), respectively. Based on our
results, this method enjoys high accuracy performance.
The Omniglot challenge: a 3-year progress report
Authors: Brenden M. Lake, Ruslan Salakhutdinov, Joshua B. Tenenbaum
Date: 2019-02-09 00:00:00
Tasks: General Classification, One-Shot Learning

Three years ago, we released the Omniglot dataset for one-shot learning, along with five challenge tasks and a computational model that addresses these tasks. The model was not meant to be the final word on Omniglot; we hoped that the community would build on our work and develop new approaches. In the time since, we have been pleased to see wide adoption of the dataset. There has been notable progress on one-shot classification, but researchers have adopted new splits and procedures that make the task easier. There has been less progress on the other four tasks. We conclude that recent approaches are still far from human-like concept learning on Omniglot, a challenge that requires performing many tasks with a single model.
Pure Exploration with Multiple Correct Answers
Authors: Rémy Degenne, Wouter M. Koolen
Date: 2019-02-09 00:00:00
Proceeding: NeurIPS 2019 12

We determine the sample complexity of pure exploration bandit problems with
multiple good answers. We derive a lower bound using a new game equilibrium
argument. We show how continuity and convexity properties of single-answer
problems ensures that the Track-and-Stop algorithm has asymptotically optimal
sample complexity. However, that convexity is lost when going to the
multiple-answer setting. We present a new algorithm which extends
Track-and-Stop to the multiple-answer case and has asymptotic sample complexity
matching the lower bound.
Depth-Map Generation using Pixel Matching in Stereoscopic Pair of Images
Authors: Asra Aslam, Mohd. Samar Ansari
Date: 2019-02-09 00:00:00
Tasks: 3D Depth Estimation

Modern day multimedia content generation and dissemination is moving towards the presentation of more and more `realistic' scenarios. The switch from 2-dimensional (2D) to 3-dimensional (3D) has been a major driving force in that direction. Over the recent past, a large number of approaches have been proposed for creating 3D images/videos most of which are based on the generation of depth-maps. This paper presents a new algorithm for obtaining depth information pertaining to a depicted scene from a set of available pair of stereoscopic images. The proposed algorithm performs a pixel-to-pixel matching of the two images in the stereo pair for estimation of depth. It is shown that the obtained depth-maps show improvements over the reported counterparts.
Synthetic Data Generators: Sequential and Private
Authors: Olivier Bousquet, Roi Livni, Shay Moran
Date: 2019-02-09 00:00:00
Tasks: Synthetic Data Generation

We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient). Previous work on synthetic data generators focused on the case that the query class $\mathcal{D}$ is finite and obtained sample complexity bounds that scale logarithmically with the size $|\mathcal{D}|$. Here we construct a private synthetic data generator whose sample complexity is independent of the domain size, and we replace finiteness with the assumption that $\mathcal{D}$ is privately PAC learnable (a formally weaker task, hence we obtain equivalence between the two tasks).
Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving
Authors: Jose Solomon, Francois Charette
Date: 2019-02-09 00:00:00

A novel hierarchical Deep Neural Network (DNN) model is presented to address the task of end-to-end driving. The model consists of a master classifier network which determines the driving task required from an input stereo image and directs said image to one of a set of subservient network regression models that perform inference and output a steering command. These subservient networks are designed and trained for a specific driving task: straightaway, swerve maneuver, tight turn, gradual turn, and chicane. Using this modular network strategy allows for two primary advantages: an overall reduction in the amount of data required to train the complete system, and for model tailoring where more complex models can be used for more challenging tasks while simplified networks can handle more mundane tasks. It is this latter facet of the model that makes the approach attractive to a number of applications beyond the current vehicle steering strategy.
Super-realtime facial landmark detection and shape fitting by deep regression of shape model parameters
Authors: Marcin Kopaczka, Justus Schock, Dorit Merhof
Date: 2019-02-09 00:00:00
Tasks: Facial Landmark Detection, Image Segmentation, Medical Image Segmentation, regression, Semantic Segmentation

We present a method for highly efficient landmark detection that combines
deep convolutional neural networks with well established model-based fitting
algorithms. Motivated by established model-based fitting methods such as active
shapes, we use a PCA of the landmark positions to allow generative modeling of
facial landmarks. Instead of computing the model parameters using iterative
optimization, the PCA is included in a deep neural network using a novel layer
type. The network predicts model parameters in a single forward pass, thereby
allowing facial landmark detection at several hundreds of frames per second.
Our architecture allows direct end-to-end training of a model-based landmark
detection method and shows that deep neural networks can be used to reliably
predict model parameters directly without the need for an iterative
optimization. The method is evaluated on different datasets for facial landmark
detection and medical image segmentation. PyTorch code is freely available at
https://github.com/justusschock/shapenet
Contextual Recurrent Neural Networks
Authors: Sam Wenke, Jim Fleming
Date: 2019-02-09 00:00:00
Tasks: Retrieval

There is an implicit assumption that by unfolding recurrent neural networks
(RNN) in finite time, the misspecification of choosing a zero value for the
initial hidden state is mitigated by later time steps. This assumption has been
shown to work in practice and alternative initialization may be suggested but
often overlooked. In this paper, we propose a method of parameterizing the
initial hidden state of an RNN. The resulting architecture, referred to as a
Contextual RNN, can be trained end-to-end. The performance on an associative
retrieval task is found to improve by conditioning the RNN initial hidden state
on contextual information from the input sequence. Furthermore, we propose a
novel method of conditionally generating sequences using the hidden state
parameterization of Contextual RNN.
Distance metric learning based on structural neighborhoods for dimensionality reduction and classification performance improvement
Authors: Mostafa Razavi Ghods, Mohammad Hossein Moattar, Yahya Forghani
Date: 2019-02-09 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, General Classification, Metric Learning

Distance metric learning can be viewed as one of the fundamental interests in pattern recognition and machine learning, which plays a pivotal role in the performance of many learning methods. One of the effective methods in learning such a metric is to learn it from a set of labeled training samples. The issue of data imbalance is the most important challenge of recent methods. This research tries not only to preserve the local structures but also covers the issue of imbalanced datasets. To do this, the proposed method first tries to extract a low dimensional manifold from the input data. Then, it learns the local neighborhood structures and the relationship of the data points in the ambient space based on the adjacencies of the same data points on the embedded low dimensional manifold. Using the local neighborhood relationships extracted from the manifold space, the proposed method learns the distance metric in a way which minimizes the distance between similar data and maximizes their distance from the dissimilar data points. The evaluations of the proposed method on numerous datasets from the UCI repository of machine learning, and also the KDDCup98 dataset as the most imbalance dataset, justify the supremacy of the proposed approach in comparison with other approaches especially when the imbalance factor is high.
3D Hand Shape and Pose from Images in the Wild
Authors: Adnane Boukhayma, Rodrigo de Bem, Philip H. S. Torr
Date: 2019-02-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Hand Pose Estimation, Decoder, Pose Prediction, valid

We present in this work the first end-to-end deep learning based method that
predicts both 3D hand shape and pose from RGB images in the wild. Our network
consists of the concatenation of a deep convolutional encoder, and a fixed
model-based decoder. Given an input image, and optionally 2D joint detections
obtained from an independent CNN, the encoder predicts a set of hand and view
parameters. The decoder has two components: A pre-computed articulated mesh
deformation hand model that generates a 3D mesh from the hand parameters, and a
re-projection module controlled by the view parameters that projects the
generated hand into the image domain. We show that using the shape and pose
prior knowledge encoded in the hand model within a deep learning framework
yields state-of-the-art performance in 3D pose prediction from images on
standard benchmarks, and produces geometrically valid and plausible 3D
reconstructions. Additionally, we show that training with weak supervision in
the form of 2D joint annotations on datasets of images in the wild, in
conjunction with full supervision in the form of 3D joint annotations on
limited available datasets allows for good generalization to 3D shape and pose
predictions on images in the wild.
Venn GAN: Discovering Commonalities and Particularities of Multiple Distributions
Authors: Yasin Yazici, Bruno Lecouat, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, Vijay Chandrasekhar
Date: 2019-02-09 00:00:00

We propose a GAN design which models multiple distributions effectively and
discovers their commonalities and particularities. Each data distribution is
modeled with a mixture of $K$ generator distributions. As the generators are
partially shared between the modeling of different true data distributions,
shared ones captures the commonality of the distributions, while non-shared
ones capture unique aspects of them. We show the effectiveness of our method on
various datasets (MNIST, Fashion MNIST, CIFAR-10, Omniglot, CelebA) with
compelling results.
Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
Authors: Michal Uricar, Pavel Krizek, David Hurych, Ibrahim Sobh, Senthil Yogamani, Patrick Denny
Date: 2019-02-09 00:00:00
Tasks: Autonomous Driving, BIG-bench Machine Learning, Data Augmentation

Generative Adversarial Networks (GAN) have gained a lot of popularity from their introduction in 2014 till present. Research on GAN is rapidly growing and there are many variants of the original GAN focusing on various aspects of deep learning. GAN are perceived as the most impactful direction of machine learning in the last decade. This paper focuses on the application of GAN in autonomous driving including topics such as advanced data augmentation, loss function learning, semi-supervised learning, etc. We formalize and review key applications of adversarial techniques and discuss challenges and open problems to be addressed.
Simulating extrapolated dynamics with parameterization networks
Authors: James P. L. Tan
Date: 2019-02-09 00:00:00
Tasks: Time Series, Time Series Analysis

An artificial neural network architecture, parameterization networks, is
proposed for simulating extrapolated dynamics beyond observed data in dynamical
systems. Parameterization networks are used to ensure the long term integrity
of extrapolated dynamics, while careful tuning of model hyperparameters against
validation errors controls overfitting. A parameterization network is
demonstrated on the logistic map, where chaos and other nonlinear phenomena
consistent with the underlying model can be extrapolated from non-chaotic
training time series with good fidelity. The stated results are a lot less
fantastical than they appear to be because the neural network is only
extrapolating between quadratic return maps. Nonetheless, the results do
suggest that successful extrapolation of qualitatively different behaviors
requires learning to occur on a level of abstraction where the corresponding
behaviors are more similar in nature.
Clustering Bioactive Molecules in 3D Chemical Space with Unsupervised Deep Learning
Authors: Chu Qin, Ying Tan, Shang Ying Chen, Xian Zeng, Xingxing Qi, Tian Jin, Huan Shi, Yiwei Wan, Yu Chen, Jingfeng Li, Weidong He, Yali Wang, Peng Zhang, Feng Zhu, Hongping Zhao, Yuyang Jiang, Yuzong Chen
Date: 2019-02-09 00:00:00
Tasks: Clustering, Diversity, Drug Discovery

Unsupervised clustering has broad applications in data stratification,
pattern investigation and new discovery beyond existing knowledge. In
particular, clustering of bioactive molecules facilitates chemical space
mapping, structure-activity studies, and drug discovery. These tasks,
conventionally conducted by similarity-based methods, are complicated by data
complexity and diversity. We ex-plored the superior learning capability of deep
autoencoders for unsupervised clustering of 1.39 mil-lion bioactive molecules
into band-clusters in a 3-dimensional latent chemical space. These
band-clusters, displayed by a space-navigation simulation software, band
molecules of selected bioactivity classes into individual band-clusters
possessing unique sets of common sub-structural features beyond structural
similarity. These sub-structural features form the frameworks of the
literature-reported pharmacophores and privileged fragments. Within each
band-cluster, molecules are further banded into selected sub-regions with
respect to their bioactivity target, sub-structural features and molecular
scaffolds. Our method is potentially applicable for big data clustering tasks
of different fields.
Low-pass filtering as Bayesian inference
Authors: Cristobal Valenzuela, Felipe Tobar
Date: 2019-02-09 00:00:00
Tasks: Bayesian Inference, Gaussian Processes, Time Series, Time Series Analysis

We propose a Bayesian nonparametric method for low-pass filtering that can
naturally handle unevenly-sampled and noise-corrupted observations. The
proposed model is constructed as a latent-factor model for time series, where
the latent factors are Gaussian processes with non-overlapping spectra. With
this construction, the low-pass version of the time series can be identified as
the low-frequency latent component, and therefore it can be found by means of
Bayesian inference. We show that the model admits exact training and can be
implemented with minimal numerical approximations. Finally, the proposed model
is validated against standard linear filters on synthetic and real-world time
series.
Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels
Authors: Jörg Stork, Martin Zaefferer, Thomas Bartz-Beielstein
Date: 2019-02-09 00:00:00
Tasks: Evolutionary Algorithms, Reinforcement Learning

In NeuroEvolution, the topologies of artificial neural networks are optimized
with evolutionary algorithms to solve tasks in data regression, data
classification, or reinforcement learning. One downside of NeuroEvolution is
the large amount of necessary fitness evaluations, which might render it
inefficient for tasks with expensive evaluations, such as real-time learning.
For these expensive optimization tasks, surrogate model-based optimization is
frequently applied as it features a good evaluation efficiency. While a
combination of both procedures appears as a valuable solution, the definition
of adequate distance measures for the surrogate modeling process is difficult.
In this study, we will extend cartesian genetic programming of artificial
neural networks by the use of surrogate model-based optimization. We propose
different distance measures and test our algorithm on a replicable benchmark
task. The results indicate that we can significantly increase the evaluation
efficiency and that a phenotypic distance, which is based on the behavior of
the associated neural networks, is most promising.