pwc-archive/papers-with-abstracts huggingface dataset:
Distribution Matching in Variational Inference
Authors: Mihaela Rosca, Balaji Lakshminarayanan, Shakir Mohamed
Date: 2018-02-19 00:00:00
Tasks: Variational Inference

With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces. We show this to be a consequence of learning by matching conditional distributions, and the limitations of explicit model and posterior distributions. It is popular to consider Generative Adversarial Networks (GANs) as a means of overcoming these limitations, leading to hybrids of VAEs and GANs. We perform a large-scale evaluation of several VAE-GAN hybrids and analyze the implications of class probability estimation for learning distributions. While promising, we conclude that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs.
A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
Authors: Xin Bing, Florentina Bunea, Marten Wegkamp
Date: 2018-05-17 00:00:00
Tasks: Topic Models, valid

We propose a new method of estimation in topic models, that is not a variation on the existing simplex finding algorithms, and that estimates the number of topics K from the observed data. We derive new finite sample minimax lower bounds for the estimation of A, as well as new upper bounds for our proposed estimator. We describe the scenarios where our estimator is minimax adaptive. Our finite sample analysis is valid for any number of documents (n), individual document length (N_i), dictionary size (p) and number of topics (K), and both p and K are allowed to increase with n, a situation not handled well by previous analyses. We complement our theoretical results with a detailed simulation study. We illustrate that the new algorithm is faster and more accurate than the current ones, although we start out with a computational and theoretical disadvantage of not knowing the correct number of topics K, while we provide the competing methods with the correct value in our simulations.
VITON: An Image-based Virtual Try-on Network
Authors: Xintong Han, Zuxuan Wu, Zhe Wu, Ruichi Yu, Larry S. Davis
Date: 2017-11-22 00:00:00
Proceeding: CVPR 2018 6
Tasks: Descriptive, Virtual Try-on

We present an image-based VIirtual Try-On Network (VITON) without using 3D
information in any form, which seamlessly transfers a desired clothing item
onto the corresponding region of a person using a coarse-to-fine strategy.
Conditioned upon a new clothing-agnostic yet descriptive person representation,
our framework first generates a coarse synthesized image with the target
clothing item overlaid on that same person in the same pose. We further enhance
the initial blurry clothing area with a refinement network. The network is
trained to learn how much detail to utilize from the target clothing item, and
where to apply to the person in order to synthesize a photo-realistic image in
which the target item deforms naturally with clear visual patterns. Experiments
on our newly collected Zalando dataset demonstrate its promise in the
image-based virtual try-on task over state-of-the-art generative models.
Adaptive Three Operator Splitting
Authors: Fabian Pedregosa, Gauthier Gidel
Date: 2018-04-06 00:00:00
Proceeding: ICML 2018 7

We propose and analyze an adaptive step-size variant of the Davis-Yin three
operator splitting. This method can solve optimization problems composed by a
sum of a smooth term for which we have access to its gradient and an arbitrary
number of potentially non-smooth terms for which we have access to their
proximal operator. The proposed method sets the step-size based on local
information of the objective --hence allowing for larger step-sizes--, only
requires two extra function evaluations per iteration and does not depend on
any step-size hyperparameter besides an initial estimate. We provide an
iteration complexity analysis that matches the best known results for the
non-adaptive variant: sublinear convergence for general convex functions and
linear convergence under strong convexity of the smooth term and smoothness of
one of the proximal terms. Finally, an empirical comparison with related
methods on 6 different problems illustrates the computational advantage of the
proposed method.
Convolutional Neural Networks for Aircraft Noise Monitoring
Authors: Nicholas Heller, Derek Anderson, Matt Baker, Brad Juffer, Nikolaos Papanikolopoulos
Date: 2018-06-12 00:00:00

Air travel is one of the fastest growing modes of transportation, however,
the effects of aircraft noise on populations surrounding airports is hindering
its growth. In an effort to study and ultimately mitigate the impact that this
noise has, many airports continuously monitor the aircraft noise in their
surrounding communities. Noise monitoring and analysis is complicated by the
fact that aircraft are not the only source of noise. In this work, we show that
a Convolutional Neural Network is well-suited for the task of identifying noise
events which are not caused by aircraft. Our system achieves an accuracy of
0.970 when trained on 900 manually labeled noise events. Our training data and
a TensorFlow implementation of our model are available at
https://github.com/neheller/aircraftnoise.
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information
Authors: Sudha Rao, Hal Daumé III
Date: 2018-05-12 00:00:00
Proceeding: ACL 2018 7

Inquiry is fundamental to communication, and machines cannot effectively
collaborate with humans unless they can ask questions. In this work, we build a
neural network model for the task of ranking clarification questions. Our model
is inspired by the idea of expected value of perfect information: a good
question is one whose expected answer will be useful. We study this problem
using data from StackExchange, a plentiful online resource in which people
routinely ask clarifying questions to posts so that they can better offer
assistance to the original poster. We create a dataset of clarification
questions consisting of ~77K posts paired with a clarification question (and
answer) from three domains of StackExchange: askubuntu, unix and superuser. We
evaluate our model on 500 samples of this dataset against expert human
judgments and demonstrate significant improvements over controlled baselines.
Early Stopping for Nonparametric Testing
Authors: Meimei Liu, Guang Cheng
Date: 2018-05-25 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: General Classification

Early stopping of iterative algorithms is an algorithmic regularization
method to avoid over-fitting in estimation and classification. In this paper,
we show that early stopping can also be applied to obtain the minimax optimal
testing in a general non-parametric setup. Specifically, a Wald-type test
statistic is obtained based on an iterated estimate produced by functional
gradient descent algorithms in a reproducing kernel Hilbert space. A notable
contribution is to establish a "sharp" stopping rule: when the number of
iterations achieves an optimal order, testing optimality is achievable;
otherwise, testing optimality becomes impossible. As a by-product, a similar
sharpness result is also derived for minimax optimal estimation under early
stopping studied in [11] and [19]. All obtained results hold for various kernel
classes, including Sobolev smoothness classes and Gaussian kernel classes.
Static Malware Detection & Subterfuge: Quantifying the Robustness of Machine Learning and Current Anti-Virus
Authors: William Fleshman, Edward Raff, Richard Zak, Mark McLean, Charles Nicholas
Date: 2018-06-12 00:00:00
Tasks: BIG-bench Machine Learning, Malware Detection

As machine-learning (ML) based systems for malware detection become more
prevalent, it becomes necessary to quantify the benefits compared to the more
traditional anti-virus (AV) systems widely used today. It is not practical to
build an agreed upon test set to benchmark malware detection systems on pure
classification performance. Instead we tackle the problem by creating a new
testing methodology, where we evaluate the change in performance on a set of
known benign & malicious files as adversarial modifications are performed. The
change in performance combined with the evasion techniques then quantifies a
system's robustness against that approach. Through these experiments we are
able to show in a quantifiable way how purely ML based systems can be more
robust than AV products at detecting malware that attempts evasion through
modification, but may be slower to adapt in the face of significantly novel
attacks.
Deep Sparse Coding for Invariant Multimodal Halle Berry Neurons
Authors: Edward Kim, Darryl Hannan, Garrett Kenyon
Date: 2017-11-21 00:00:00
Proceeding: CVPR 2018 6
Tasks: BIG-bench Machine Learning

Deep feed-forward convolutional neural networks (CNNs) have become ubiquitous
in virtually all machine learning and computer vision challenges; however,
advancements in CNNs have arguably reached an engineering saturation point
where incremental novelty results in minor performance gains. Although there is
evidence that object classification has reached human levels on narrowly
defined tasks, for general applications, the biological visual system is far
superior to that of any computer. Research reveals there are numerous missing
components in feed-forward deep neural networks that are critical in mammalian
vision. The brain does not work solely in a feed-forward fashion, but rather
all of the neurons are in competition with each other; neurons are integrating
information in a bottom up and top down fashion and incorporating expectation
and feedback in the modeling process. Furthermore, our visual cortex is working
in tandem with our parietal lobe, integrating sensory information from various
modalities.
  In our work, we sought to improve upon the standard feed-forward deep
learning model by augmenting them with biologically inspired concepts of
sparsity, top-down feedback, and lateral inhibition. We define our model as a
sparse coding problem using hierarchical layers. We solve the sparse coding
problem with an additional top-down feedback error driving the dynamics of the
neural network. While building and observing the behavior of our model, we were
fascinated that multimodal, invariant neurons naturally emerged that mimicked,
"Halle Berry neurons" found in the human brain. Furthermore, our sparse
representation of multimodal signals demonstrates qualitative and quantitative
superiority to the standard feed-forward joint embedding in common vision and
machine learning tasks.
Improving Optimization for Models With Continuous Symmetry Breaking
Authors: Robert Bamler, Stephan Mandt
Date: 2018-03-08 00:00:00
Tasks: Representation Learning, Time Series, Time Series Analysis, Word Embeddings

Many loss functions in representation learning are invariant under a
continuous symmetry transformation. For example, the loss function of word
embeddings (Mikolov et al., 2013) remains unchanged if we simultaneously rotate
all word and context embedding vectors. We show that representation learning
models for time series possess an approximate continuous symmetry that leads to
slow convergence of gradient descent. We propose a new optimization algorithm
that speeds up convergence using ideas from gauge theory in physics. Our
algorithm leads to orders of magnitude faster convergence and to more
interpretable representations, as we show for dynamic extensions of matrix
factorization and word embedding models. We further present an example
application of our proposed algorithm that translates modern words into their
historic equivalents.
Hierarchical Long-term Video Prediction without Supervision
Authors: Nevan Wichers, Ruben Villegas, Dumitru Erhan, Honglak Lee
Date: 2018-06-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Decoder, Prediction, Video Prediction

Much of recent research has been devoted to video prediction and generation,
yet most of the previous works have demonstrated only limited success in
generating videos on short-term horizons. The hierarchical video prediction
method by Villegas et al. (2017) is an example of a state-of-the-art method for
long-term video prediction, but their method is limited because it requires
ground truth annotation of high-level structures (e.g., human joint landmarks)
at training time. Our network encodes the input frame, predicts a high-level
encoding into the future, and then a decoder with access to the first frame
produces the predicted image from the predicted encoding. The decoder also
produces a mask that outlines the predicted foreground object (e.g., person) as
a by-product. Unlike Villegas et al. (2017), we develop a novel training method
that jointly trains the encoder, the predictor, and the decoder together
without highlevel supervision; we further improve upon this by using an
adversarial loss in the feature space to train the predictor. Our method can
predict about 20 seconds into the future and provides better results compared
to Denton and Fergus (2018) and Finn et al. (2016) on the Human 3.6M dataset.
Learning Mixtures of Linear Regressions with Nearly Optimal Complexity
Authors: Yuanzhi Li, YIngyu Liang
Date: 2018-02-22 00:00:00

Mixtures of Linear Regressions (MLR) is an important mixture model with many applications. In this model, each observation is generated from one of the several unknown linear regression components, where the identity of the generated component is also unknown. Previous works either assume strong assumptions on the data distribution or have high complexity. This paper proposes a fixed parameter tractable algorithm for the problem under general conditions, which achieves global convergence and the sample complexity scales nearly linearly in the dimension. In particular, different from previous works that require the data to be from the standard Gaussian, the algorithm allows the data from Gaussians with different covariances. When the conditional number of the covariances and the number of components are fixed, the algorithm has nearly optimal sample complexity $N = \tilde{O}(d)$ as well as nearly optimal computational complexity $\tilde{O}(Nd)$, where $d$ is the dimension of the data space. To the best of our knowledge, this approach provides the first such recovery guarantee for this general setting.
Fully Convolutional Network for Melanoma Diagnostics
Authors: Adon Phillips, Iris Teo, Jochen Lang
Date: 2018-06-12 00:00:00
Tasks: Anatomy, BIG-bench Machine Learning

This work seeks to determine how modern machine learning techniques may be
applied to the previously unexplored topic of melanoma diagnostics using
digital pathology. We curated a new dataset of 50 patient cases of cutaneous
melanoma using digital pathology. We provide gold standard annotations for
three tissue types (tumour, epidermis, and dermis) which are important for the
prognostic measurements known as Breslow thickness and Clark level. Then, we
devised a novel multi-stride fully convolutional network (FCN) architecture
that outperformed other networks trained and evaluated using the same data
according to standard metrics. Finally, we trained a model to detect and
localize the target tissue types. When processing previously unseen cases, our
model's output is qualitatively very similar to the gold standard. In addition
to the standard metrics computed as a baseline for our approach, we asked three
additional pathologists to measure the Breslow thickness on the network's
output. Their responses were diagnostically equivalent to the ground truth
measurements, and when removing cases where a measurement was not appropriate,
inter-rater reliability (IRR) between the four pathologists was 75.0%. Given
the qualitative and quantitative results, it is possible to overcome the
discriminative challenges of the skin and tumour anatomy for segmentation using
modern machine learning techniques, though more work is required to improve the
network's performance on dermis segmentation. Further, we show that it is
possible to achieve a level of accuracy required to manually perform the
Breslow thickness measurement.
INFERNO: Inference-Aware Neural Optimisation
Authors: Pablo de Castro, Tommaso Dorigo
Date: 2018-06-12 00:00:00

Complex computer simulations are commonly required for accurate data
modelling in many scientific disciplines, making statistical inference
challenging due to the intractability of the likelihood evaluation for the
observed data. Furthermore, sometimes one is interested on inference drawn over
a subset of the generative model parameters while taking into account model
uncertainty or misspecification on the remaining nuisance parameters. In this
work, we show how non-linear summary statistics can be constructed by
minimising inference-motivated losses via stochastic gradient descent such they
provided the smallest uncertainty for the parameters of interest. As a use
case, the problem of confidence interval estimation for the mixture coefficient
in a multi-dimensional two-component mixture model (i.e. signal vs background)
is considered, where the proposed technique clearly outperforms summary
statistics based on probabilistic classification, which are a commonly used
alternative but do not account for the presence of nuisance parameters.
Pricing Engine: Estimating Causal Impacts in Real World Business Settings
Authors: Matt Goldman, Brian Quistorff
Date: 2018-06-08 00:00:00

We introduce the Pricing Engine package to enable the use of Double ML
estimation techniques in general panel data settings. Customization allows the
user to specify first-stage models, first-stage featurization, second stage
treatment selection and second stage causal-modeling. We also introduce a
DynamicDML class that allows the user to generate dynamic treatment-aware
forecasts at a range of leads and to understand how the forecasts will vary as
a function of causally estimated treatment parameters. The Pricing Engine is
built on Python 3.5 and can be run on an Azure ML Workbench environment with
the addition of only a few Python packages. This note provides high-level
discussion of the Double ML method, describes the packages intended use and
includes an example Jupyter notebook demonstrating application to some publicly
available data. Installation of the package and additional technical
documentation is available at
$\href{https://github.com/bquistorff/pricingengine}{github.com/bquistorff/pricingengine}$.
Delta-encoder: an effective sample synthesis method for few-shot object recognition
Authors: Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Rogerio Feris, Abhishek Kumar, Raja Giryes, Alex M. Bronstein
Date: 2018-06-12 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Few-Shot Image Classification, Few-Shot Learning, Object Recognition

Learning to classify new categories based on just one or a few examples is a
long-standing challenge in modern computer vision. In this work, we proposes a
simple yet effective method for few-shot (and one-shot) object recognition. Our
approach is based on a modified auto-encoder, denoted Delta-encoder, that
learns to synthesize new samples for an unseen category just by seeing few
examples from it. The synthesized samples are then used to train a classifier.
The proposed approach learns to both extract transferable intra-class
deformations, or "deltas", between same-class pairs of training examples, and
to apply those deltas to the few provided examples of a novel class (unseen
during training) in order to efficiently synthesize samples from that new
class. The proposed method improves over the state-of-the-art in one-shot
object-recognition and compares favorably in the few-shot case. Upon acceptance
code will be made available.
Some variations on Ensembled Random Survival Forest with application to Cancer Research
Authors: Arabin Kumar Dey, Suhas N., Talasila Sai Teja, Anshul Juneja
Date: 2017-09-16 00:00:00

In this paper we describe a novel implementation of adaboost for prediction
of survival function. We take different variations of the algorithm and compare
the algorithms based on system run time and root mean square error. Our
construction includes right censoring data and competing risk data too. We take
different data set to illustrate the performance of the algorithms.
Accurate Detection of Inner Ears in Head CTs Using a Deep Volume-to-Volume Regression Network with False Positive Suppression and a Shape-Based Constraint
Authors: Dongqing Zhang, Jianing Wang, Jack H. Noble, Benoit M. Dawant
Date: 2018-06-12 00:00:00
Tasks: Anatomy

Cochlear implants (CIs) are neural prosthetics which are used to treat
patients with hearing loss. CIs use an array of electrodes which are surgically
inserted into the cochlea to stimulate the auditory nerve endings. After
surgery, CIs need to be programmed. Studies have shown that the spatial
relationship between the intra-cochlear anatomy and electrodes derived from
medical images can guide CI programming and lead to significant improvement in
hearing outcomes. However, clinical head CT images are usually obtained from
scanners of different brands with different protocols. The field of view thus
varies greatly and visual inspection is needed to document their content prior
to applying algorithms for electrode localization and intra-cochlear anatomy
segmentation. In this work, to determine the presence/absence of inner ears and
to accurately localize them in head CTs, we use a volume-to-volume
convolutional neural network which can be trained end-to-end to map a raw CT
volume to probability maps which indicate inner ear positions. We incorporate a
false positive suppression strategy in training and apply a shape-based
constraint. We achieve a labeling accuracy of 98.59% and a localization error
of 2.45mm. The localization error is significantly smaller than a random
forest-based approach that has been proposed recently to perform the same task.
Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents
Authors: Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth O. Stanley, Jeff Clune
Date: 2017-12-18 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Reinforcement Learning, Policy Gradient Methods, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Evolution strategies (ES) are a family of black-box optimization algorithms
able to train deep neural networks roughly as well as Q-learning and policy
gradient methods on challenging deep reinforcement learning (RL) problems, but
are much faster (e.g. hours vs. days) because they parallelize better. However,
many RL problems require directed exploration because they have reward
functions that are sparse or deceptive (i.e. contain local optima), and it is
unknown how to encourage such exploration with ES. Here we show that algorithms
that have been invented to promote directed exploration in small-scale evolved
neural networks via populations of exploring agents, specifically novelty
search (NS) and quality diversity (QD) algorithms, can be hybridized with ES to
improve its performance on sparse or deceptive deep RL tasks, while retaining
scalability. Our experiments confirm that the resultant new algorithms, NS-ES
and two QD algorithms, NSR-ES and NSRA-ES, avoid local optima encountered by ES
to achieve higher performance on Atari and simulated robots learning to walk
around a deceptive trap. This paper thus introduces a family of fast, scalable
algorithms for reinforcement learning that are capable of directed exploration.
It also adds this new family of exploration algorithms to the RL toolbox and
raises the interesting possibility that analogous algorithms with multiple
simultaneous paths of exploration might also combine well with existing RL
algorithms outside ES.