pwc-archive/papers-with-abstracts huggingface dataset:
Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients
Authors: Sai Praneeth Karimireddy, Sebastian U. Stich, Martin Jaggi
Date: 2018-06-01 00:00:00
Tasks: regression

We show that Newton's method converges globally at a linear rate for
objective functions whose Hessians are stable. This class of problems includes
many functions which are not strongly convex, such as logistic regression. Our
linear convergence result is (i) affine-invariant, and holds even if an (ii)
approximate Hessian is used, and if the subproblems are (iii) only solved
approximately. Thus we theoretically demonstrate the superiority of Newton's
method over first-order methods, which would only achieve a sublinear
$O(1/t^2)$ rate under similar conditions.
Adapted and Oversegmenting Graphs: Application to Geometric Deep Learning
Authors: Alberto Gomez, Veronika A. Zimmer, Bishesh Khanal, Nicolas Toussaint, Julia A. Schnabel
Date: 2018-06-01 00:00:00
Tasks: Clustering, Deep Learning, General Classification, image-classification, Image Classification

We propose a novel iterative method to adapt a a graph to d-dimensional image data. The method drives the nodes of the graph towards image features. The adaptation process naturally lends itself to a measure of feature saliency which can then be used to retain meaningful nodes and edges in the graph. From the adapted graph, we also propose the computation of a dual graph, which inherits the saliency measure from the adapted graph, and whose edges run along image features, hence producing an oversegmenting graph. The proposed method is computationally efficient and fully parallelisable. We propose two distance measures to find image saliency along graph edges, and evaluate the performance on synthetic images and on natural images from publicly available databases. In both cases, the most salient nodes of the graph achieve average boundary recall over 90%. We also apply our method to image classification on the MNIST hand-written digit dataset, using a recently proposed Deep Geometric Learning architecture, and achieving state-of-the-art classification accuracy, for a graph-based method, of 97.86%.
Inverting Supervised Representations with Autoregressive Neural Density Models
Authors: Charlie Nash, Nate Kushman, Christopher K. I. Williams
Date: 2018-06-01 00:00:00
Tasks: Density Estimation

We present a method for feature interpretation that makes use of recent
advances in autoregressive density estimation models to invert model
representations. We train generative inversion models to express a distribution
over input features conditioned on intermediate model representations. Insights
into the invariances learned by supervised models can be gained by viewing
samples from these inversion models. In addition, we can use these inversion
models to estimate the mutual information between a model's inputs and its
intermediate representations, thus quantifying the amount of information
preserved by the network at different stages. Using this method we examine the
types of information preserved at different layers of convolutional neural
networks, and explore the invariances induced by different architectural
choices. Finally we show that the mutual information between inputs and network
layers decreases over the course of training, supporting recent work by
Shwartz-Ziv and Tishby (2017) on the information bottleneck theory of deep
learning.
Radio Galaxy Morphology Generation Using DNN Autoencoder and Gaussian Mixture Models
Authors: Zhixian Ma, Jie Zhu, Weitian Li, Haiguang Xu
Date: 2018-06-01 00:00:00
Tasks: Decoder

The morphology of a radio galaxy is highly affected by its central active
galactic nuclei (AGN), which is studied to reveal the evolution of the super
massive black hole (SMBH). In this work, we propose a morphology generation
framework for two typical radio galaxies namely Fanaroff-Riley type-I (FRI) and
type-II (FRII) with deep neural network based autoencoder (DNNAE) and Gaussian
mixture models (GMMs). The encoder and decoder subnets in the DNNAE are
symmetric aside a fully-connected layer namely code layer hosting the extracted
feature vectors. By randomly generating the feature vectors later with a
three-component Gaussian Mixture models, new FRI or FRII radio galaxy
morphologies are simulated. Experiments were demonstrated on real radio galaxy
images, where we discussed the length of feature vectors, selection of lost
functions, and made comparisons on batch normalization and dropout techniques
for training the network. The results suggest a high efficiency and performance
of our morphology generation framework. Code is available at:
https://github.com/myinxd/dnnae-gmm.
Image-to-image translation for cross-domain disentanglement
Authors: Abel Gonzalez-Garcia, Joost Van de Weijer, Yoshua Bengio
Date: 2018-05-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Disentanglement, Image-to-Image Translation, Retrieval, Translation

Deep image translation methods have recently shown excellent results,
outputting high-quality images covering multiple modes of the data
distribution. There has also been increased interest in disentangling the
internal representations learned by deep methods to further improve their
performance and achieve a finer control. In this paper, we bridge these two
objectives and introduce the concept of cross-domain disentanglement. We aim to
separate the internal representation into three parts. The shared part contains
information for both domains. The exclusive parts, on the other hand, contain
only factors of variation that are particular to each domain. We achieve this
through bidirectional image translation based on Generative Adversarial
Networks and cross-domain autoencoders, a novel network component. Our model
offers multiple advantages. We can output diverse samples covering multiple
modes of the distributions of both domains, perform domain-specific image
transfer and interpolation, and cross-domain retrieval without the need of
labeled data, only paired images. We compare our model to the state-of-the-art
in multi-modal image translation and achieve better results for translation on
challenging datasets as well as for cross-domain retrieval on realistic
datasets.
Persistence paths and signature features in topological data analysis
Authors: Ilya Chevyrev, Vidit Nanda, Harald Oberhauser
Date: 2018-06-01 00:00:00
Tasks: General Classification, tensor algebra, Topological Data Analysis

We introduce a new feature map for barcodes that arise in persistent homology
computation. The main idea is to first realize each barcode as a path in a
convenient vector space, and to then compute its path signature which takes
values in the tensor algebra of that vector space. The composition of these two
operations - barcode to path, path to tensor series - results in a feature map
that has several desirable properties for statistical learning, such as
universality and characteristicness, and achieves state-of-the-art results on
common classification benchmarks.
Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design
Authors: Aleksandar Nikolov, Mohit Singh, Uthaipon Tao Tantipongpipat
Date: 2018-02-22 00:00:00
Tasks: Clustering, feature selection

We study the optimal design problems where the goal is to choose a set of
linear measurements to obtain the most accurate estimate of an unknown vector
in $d$ dimensions. We study the $A$-optimal design variant where the objective
is to minimize the average variance of the error in the maximum likelihood
estimate of the vector being measured. The problem also finds applications in
sensor placement in wireless networks, sparse least squares regression, feature
selection for $k$-means clustering, and matrix approximation. In this paper, we
introduce proportional volume sampling to obtain improved approximation
algorithms for $A$-optimal design. Our main result is to obtain improved
approximation algorithms for the $A$-optimal design problem by introducing the
proportional volume sampling algorithm. Our results nearly optimal bounds in
the asymptotic regime when the number of measurements done, $k$, is
significantly more than the dimension $d$. We also give first approximation
algorithms when $k$ is small including when $k=d$. The proportional
volume-sampling algorithm also gives approximation algorithms for other optimal
design objectives such as $D$-optimal design and generalized ratio objective
matching or improving previous best known results. Interestingly, we show that
a similar guarantee cannot be obtained for the $E$-optimal design problem. We
also show that the $A$-optimal design problem is NP-hard to approximate within
a fixed constant when $k=d$.
Nonlinear Acceleration of CNNs
Authors: Damien Scieur, Edouard Oyallon, Alexandre d'Aspremont, Francis Bach
Date: 2018-06-01 00:00:00

The Regularized Nonlinear Acceleration (RNA) algorithm is an acceleration
method capable of improving the rate of convergence of many optimization
schemes such as gradient descend, SAGA or SVRG. Until now, its analysis is
limited to convex problems, but empirical observations shows that RNA may be
extended to wider settings. In this paper, we investigate further the benefits
of RNA when applied to neural networks, in particular for the task of image
recognition on CIFAR10 and ImageNet. With very few modifications of exiting
frameworks, RNA improves slightly the optimization process of CNNs, after
training.
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms
Authors: Cédric Colas, Olivier Sigaud, Pierre-Yves Oudeyer
Date: 2018-02-14 00:00:00
Proceeding: ICML 2018 7
Tasks: Deep Reinforcement Learning, Diversity, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In continuous action domains, standard deep reinforcement learning algorithms
like DDPG suffer from inefficient exploration when facing sparse or deceptive
reward problems. Conversely, evolutionary and developmental methods focusing on
exploration like Novelty Search, Quality-Diversity or Goal Exploration
Processes explore more robustly but are less efficient at fine-tuning policies
using gradient descent. In this paper, we present the GEP-PG approach, taking
the best of both worlds by sequentially combining a Goal Exploration Process
and two variants of DDPG. We study the learning performance of these components
and their combination on a low dimensional deceptive reward problem and on the
larger Half-Cheetah benchmark. We show that DDPG fails on the former and that
GEP-PG improves over the best DDPG variant in both environments. Supplementary
videos and discussion can be found at http://frama.link/gep_pg, the code at
http://github.com/flowersteam/geppg.
Accurate and Efficient Similarity Search for Large Scale Face Recognition
Authors: Ce Qi, Zhi-Zhong Liu, Fei Su
Date: 2018-06-01 00:00:00
Tasks: Classification, Face Recognition, Face Verification, General Classification, image-classification, Image Classification

Face verification is a relatively easy task with the help of discriminative
features from deep neural networks. However, it is still a challenge to
recognize faces on millions of identities while keeping high performance and
efficiency. The challenge 2 of MS-Celeb-1M is a classification task. However,
the number of identities is too large and it is not that elegant to treat the
task as an image classification task. We treat the classification task as
similarity search and do experiments on different similarity search strategies.
Similarity search strategy accelerates the speed of searching and boosts the
accuracy of final results. The model used for extracting features is a single
deep neural network pretrained on CASIA-Webface, which is not trained on the
base set or novel set offered by official. Finally, we rank \textbf{3rd}, while
the speed of searching is 1ms/image.
Domain Adaptation for MRI Organ Segmentation using Reverse Classification Accuracy
Authors: Vanya V. Valindria, Ioannis Lavdas, Wenjia Bai, Konstantinos Kamnitsas, Eric O. Aboagye, Andrea G. Rockall, Daniel Rueckert, Ben Glocker
Date: 2018-06-01 00:00:00
Tasks: Classification, Domain Adaptation, General Classification, Organ Segmentation, Segmentation, Transfer Learning

The variations in multi-center data in medical imaging studies have brought
the necessity of domain adaptation. Despite the advancement of machine learning
in automatic segmentation, performance often degrades when algorithms are
applied on new data acquired from different scanners or sequences than the
training data. Manual annotation is costly and time consuming if it has to be
carried out for every new target domain. In this work, we investigate automatic
selection of suitable subjects to be annotated for supervised domain adaptation
using the concept of reverse classification accuracy (RCA). RCA predicts the
performance of a trained model on data from the new domain and different
strategies of selecting subjects to be included in the adaptation via transfer
learning are evaluated. We perform experiments on a two-center MR database for
the task of organ segmentation. We show that subject selection via RCA can
reduce the burden of annotation of new data for the target domain.
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset
Authors: Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue-Nkoutche, Pavan Kapanipathi, Nicholas Mattei, Ryan Musa, Kartik Talamadupula, Michael Witbrock
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 7
Tasks: AI2 Reasoning Challenge, ARC, General Classification, Information Retrieval, Reading Comprehension, Retrieval

The recent work of Clark et al. introduces the AI2 Reasoning Challenge (ARC)
and the associated ARC dataset that partitions open domain, complex science
questions into an Easy Set and a Challenge Set. That paper includes an analysis
of 100 questions with respect to the types of knowledge and reasoning required
to answer them; however, it does not include clear definitions of these types,
nor does it offer information about the quality of the labels. We propose a
comprehensive set of definitions of knowledge and reasoning types necessary for
answering the questions in the ARC dataset. Using ten annotators and a
sophisticated annotation interface, we analyze the distribution of labels
across the Challenge Set and statistics related to them. Additionally, we
demonstrate that although naive information retrieval methods return sentences
that are irrelevant to answering the query, sufficient supporting text is often
present in the (ARC) corpus. Evaluating with human-selected relevant sentences
improves the performance of a neural machine comprehension model by 42 points.
Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers
Authors: Sandro Pezzelle, Shane Steinert-Threlkeld, Raffaela Bernardi, Jakub Szymanik
Date: 2018-06-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Sentence

We study the role of linguistic context in predicting quantifiers (`few',
`all'). We collect crowdsourced data from human participants and test various
models in a local (single-sentence) and a global context (multi-sentence)
condition. Models significantly out-perform humans in the former setting and
are only slightly better in the latter. While human performance improves with
more linguistic context (especially on proportional quantifiers), model
performance suffers. Models are very effective in exploiting lexical and
morpho-syntactic patterns; humans are better at genuinely understanding the
meaning of the (global) context.
Producing radiologist-quality reports for interpretable artificial intelligence
Authors: William Gale, Luke Oakden-Rayner, Gustavo Carneiro, Andrew P. Bradley, Lyle J. Palmer
Date: 2018-06-01 00:00:00
Tasks: Decision Making, Deep Learning, Descriptive

Current approaches to explaining the decisions of deep learning systems for
medical tasks have focused on visualising the elements that have contributed to
each decision. We argue that such approaches are not enough to "open the black
box" of medical decision making systems because they are missing a key
component that has been used as a standard communication tool between doctors
for centuries: language. We propose a model-agnostic interpretability method
that involves training a simple recurrent neural network model to produce
descriptive sentences to clarify the decision of deep learning classifiers.
  We test our method on the task of detecting hip fractures from frontal pelvic
x-rays. This process requires minimal additional labelling despite producing
text containing elements that the original deep learning classification model
was not specifically trained to detect.
  The experimental results show that: 1) the sentences produced by our method
consistently contain the desired information, 2) the generated sentences are
preferred by doctors compared to current tools that create saliency maps, and
3) the combination of visualisations and generated text is better than either
alone.
Structured Local Optima in Sparse Blind Deconvolution
Authors: Yuqian Zhang, Han-Wen Kuo, John Wright
Date: 2018-06-01 00:00:00

Blind deconvolution is a ubiquitous problem of recovering two unknown signals from their convolution. Unfortunately, this is an ill-posed problem in general. This paper focuses on the {\em short and sparse} blind deconvolution problem, where the one unknown signal is short and the other one is sparsely and randomly supported. This variant captures the structure of the unknown signals in several important applications. We assume the short signal to have unit $\ell^2$ norm and cast the blind deconvolution problem as a nonconvex optimization problem over the sphere. We demonstrate that (i) in a certain region of the sphere, every local optimum is close to some shift truncation of the ground truth, and (ii) for a generic short signal of length $k$, when the sparsity of activation signal $\theta\lesssim k^{-2/3}$ and number of measurements $m\gtrsim poly(k)$, a simple initialization method together with a descent algorithm which escapes strict saddle points recovers a near shift truncation of the ground truth kernel.
A Reinforcement Learning Approach to Age of Information in Multi-User Networks
Authors: Elif Tuğçe Ceran, Deniz Gündüz, András György
Date: 2018-06-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Scheduling the transmission of time-sensitive data to multiple users over
error-prone communication channels is studied with the goal of minimizing the
long-term average age of information (AoI) at the users under a constraint on
the average number of transmissions at the source node. After each
transmission, the source receives an instantaneous ACK/NACK feedback from the
intended receiver and decides on what time and to which user to transmit the
next update. The optimal scheduling policy is first studied under different
feedback mechanisms when the channel statistics are known; in particular, the
standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols are
considered. Then a reinforcement learning (RL) approach is introduced, which
does not assume any a priori information on the random processes governing the
channel states. Different RL methods are verified and compared through
numerical simulations.
Sockeye: A Toolkit for Neural Machine Translation
Authors: Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, Matt Post
Date: 2017-12-15 00:00:00
Tasks: Decoder, Machine Translation, NMT, Translation

We describe Sockeye (version 1.12), an open-source sequence-to-sequence
toolkit for Neural Machine Translation (NMT). Sockeye is a production-ready
framework for training and applying models as well as an experimental platform
for researchers. Written in Python and built on MXNet, the toolkit offers
scalable training and inference for the three most prominent encoder-decoder
architectures: attentional recurrent neural networks, self-attentional
transformers, and fully convolutional networks. Sockeye also supports a wide
range of optimizers, normalization and regularization techniques, and inference
improvements from current NMT literature. Users can easily run standard
training recipes, explore different model settings, and incorporate new ideas.
In this paper, we highlight Sockeye's features and benchmark it against other
NMT toolkits on two language arcs from the 2017 Conference on Machine
Translation (WMT): English-German and Latvian-English. We report competitive
BLEU scores across all three architectures, including an overall best score for
Sockeye's transformer implementation. To facilitate further comparison, we
release all system outputs and training scripts used in our experiments. The
Sockeye toolkit is free software released under the Apache 2.0 license.
NetGAN: Generating Graphs via Random Walks
Authors: Aleksandar Bojchevski, Oleksandr Shchur, Daniel Zügner, Stephan Günnemann
Date: 2018-03-02 00:00:00
Proceeding: ICML 2018 7
Tasks: Graph Generation, Link Prediction

We propose NetGAN - the first implicit generative model for graphs able to
mimic real-world networks. We pose the problem of graph generation as learning
the distribution of biased random walks over the input graph. The proposed
model is based on a stochastic neural network that generates discrete output
samples and is trained using the Wasserstein GAN objective. NetGAN is able to
produce graphs that exhibit well-known network patterns without explicitly
specifying them in the model definition. At the same time, our model exhibits
strong generalization properties, as highlighted by its competitive link
prediction performance, despite not being trained specifically for this task.
Being the first approach to combine both of these desirable properties, NetGAN
opens exciting avenues for further research.
Learning convex bounds for linear quadratic control policy synthesis
Authors: Jack Umenberger, Thomas B. Schön
Date: 2018-06-01 00:00:00
Proceeding: NeurIPS 2018 12

Learning to make decisions from observed data in dynamic environments remains
a problem of fundamental importance in a number of fields, from artificial
intelligence and robotics, to medicine and finance. This paper concerns the
problem of learning control policies for unknown linear dynamical systems so as
to maximize a quadratic reward function. We present a method to optimize the
expected value of the reward over the posterior distribution of the unknown
system parameters, given data. The algorithm involves sequential convex
programing, and enjoys reliable local convergence and robust stability
guarantees. Numerical simulations and stabilization of a real-world inverted
pendulum are used to demonstrate the approach, with strong performance and
robustness properties observed in both.
Multi-layer Kernel Ridge Regression for One-class Classification
Authors: Chandan Gautam, Aruna Tiwari, Sundaram Suresh, Alexandros Iosifidis
Date: 2018-05-20 00:00:00
Tasks: Classification, General Classification, One-Class Classification, One-class classifier, regression

In this paper, a multi-layer architecture (in a hierarchical fashion) by
stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class
classification is proposed and is referred as MKOC. MKOC has many layers of
Auto-Encoders to project the input features into new feature space and the last
layer was regression based one class classifier. The Auto-Encoders use an
unsupervised approach of learning and the final layer uses semi-supervised
(trained by only positive samples) approach of learning. The proposed MKOC is
experimentally evaluated on 15 publicly available benchmark datasets.
Experimental results verify the effectiveness of the proposed approach over 11
existing state-of-the-art kernel-based one-class classifiers. Friedman test is
also performed to verify the statistical significance of the claim of the
superiority of the proposed one-class classifiers over the existing
state-of-the-art methods.