pwc-archive/papers-with-abstracts huggingface dataset:
Modular meta-learning
Authors: Ferran Alet, Tomás Lozano-Pérez, Leslie P. Kaelbling
Date: 2018-06-26 00:00:00
Tasks: Meta-Learning

Many prediction problems, such as those that arise in the context of robotics, have a simplifying underlying structure that, if known, could accelerate learning. In this paper, we present a strategy for learning a set of neural network modules that can be combined in different ways. We train different modular structures on a set of related tasks and generalize to new tasks by composing the learned modules in new ways. By reusing modules to generalize we achieve combinatorial generalization, akin to the "infinite use of finite means" displayed in language. Finally, we show this improves performance in two robotics-related problems.
The challenge of realistic music generation: modelling raw audio at scale
Authors: Sander Dieleman, Aäron van den Oord, Karen Simonyan
Date: 2018-06-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Music Generation

Realistic music generation is a challenging task. When building generative
models of music that are learnt from data, typically high-level representations
such as scores or MIDI are used that abstract away the idiosyncrasies of a
particular performance. But these nuances are very important for our perception
of musicality and realism, so in this work we embark on modelling music in the
raw audio domain. It has been shown that autoregressive models excel at
generating raw audio waveforms of speech, but when applied to music, we find
them biased towards capturing local signal structure at the expense of
modelling long-range correlations. This is problematic because music exhibits
structure at many different timescales. In this work, we explore autoregressive
discrete autoencoders (ADAs) as a means to enable autoregressive models to
capture long-range correlations in waveforms. We find that they allow us to
unconditionally generate piano music directly in the raw audio domain, which
shows stylistic consistency across tens of seconds.
Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels
Authors: Shujian Yu, Xiaoyang Wang, Jose C. Principe
Date: 2018-06-25 00:00:00
Tasks: Attribute, Drift Detection, General Classification, Two-sample testing

One important assumption underlying common classification models is the
stationarity of the data. However, in real-world streaming applications, the
data concept indicated by the joint distribution of feature and label is not
stationary but drifting over time. Concept drift detection aims to detect such
drifts and adapt the model so as to mitigate any deterioration in the model's
predictive performance. Unfortunately, most existing concept drift detection
methods rely on a strong and over-optimistic condition that the true labels are
available immediately for all already classified instances. In this paper, a
novel Hierarchical Hypothesis Testing framework with Request-and-Reverify
strategy is developed to detect concept drifts by requesting labels only when
necessary. Two methods, namely Hierarchical Hypothesis Testing with
Classification Uncertainty (HHT-CU) and Hierarchical Hypothesis Testing with
Attribute-wise "Goodness-of-fit" (HHT-AG), are proposed respectively under the
novel framework. In experiments with benchmark datasets, our methods
demonstrate overwhelming advantages over state-of-the-art unsupervised drift
detectors. More importantly, our methods even outperform DDM (the widely used
supervised drift detector) when we use significantly fewer labels.
A Proof of the Front-Door Adjustment Formula
Authors: Mohammad Ali Javidian, Marco Valtorta
Date: 2018-06-25 00:00:00

We provide a proof of the the Front-Door adjustment formula using the
do-calculus.
Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter
Authors: Chaitanya Mitash, Abdeslam Boularias, Kostas Bekris
Date: 2018-06-25 00:00:00
Tasks: object-detection, Object Detection, Object Recognition, Pose Estimation, Self-Learning

This paper focuses on vision-based pose estimation for multiple rigid objects
placed in clutter, especially in cases involving occlusions and objects resting
on each other. Progress has been achieved recently in object recognition given
advancements in deep learning. Nevertheless, such tools typically require a
large amount of training data and significant manual effort to label objects.
This limits their applicability in robotics, where solutions must scale to a
large number of objects and variety of conditions. Moreover, the combinatorial
nature of the scenes that could arise from the placement of multiple objects is
hard to capture in the training dataset. Thus, the learned models might not
produce the desired level of precision required for tasks, such as robotic
manipulation. This work proposes an autonomous process for pose estimation that
spans from data generation to scene-level reasoning and self-learning. In
particular, the proposed framework first generates a labeled dataset for
training a Convolutional Neural Network (CNN) for object detection in clutter.
These detections are used to guide a scene-level optimization process, which
considers the interactions between the different objects present in the clutter
to output pose estimates of high precision. Furthermore, confident estimates
are used to label online real images from multiple views and re-train the
process in a self-learning pipeline. Experimental results indicate that this
process is quickly able to identify in cluttered scenes physically-consistent
object poses that are more precise than the ones found by reasoning over
individual instances of objects. Furthermore, the quality of pose estimates
increases over time given the self-learning process.
Speech Denoising with Deep Feature Losses
Authors: Francois G. Germain, Qifeng Chen, Vladlen Koltun
Date: 2018-06-27 00:00:00
Tasks: Audio Tagging, Denoising, Speech Denoising

We present an end-to-end deep learning approach to denoising speech signals
by processing the raw waveform directly. Given input audio containing speech
corrupted by an additive background signal, the system aims to produce a
processed signal that contains only the speech content. Recent approaches have
shown promising results using various deep network architectures. In this
paper, we propose to train a fully-convolutional context aggregation network
using a deep feature loss. That loss is based on comparing the internal feature
activations in a different network, trained for acoustic environment detection
and domestic audio tagging. Our approach outperforms the state-of-the-art in
objective speech quality metrics and in large-scale perceptual experiments with
human listeners. It also outperforms an identical network trained using
traditional regression losses. The advantage of the new approach is
particularly pronounced for the hardest data with the most intrusive background
noise, for which denoising is most needed and most challenging.
Predicting CEFRL levels in learner English on the basis of metrics and full texts
Authors: Taylor Arnold, Nicolas Ballier, Thomas Gaillat, Paula Lissòn
Date: 2018-06-28 00:00:00

This paper analyses the contribution of language metrics and, potentially, of
linguistic structures, to classify French learners of English according to
levels of the Common European Framework of Reference for Languages (CEFRL). The
purpose is to build a model for the prediction of learner levels as a function
of language complexity features. We used the EFCAMDAT corpus, a database of one
million written assignments by learners. After applying language complexity
metrics on the texts, we built a representation matching the language metrics
of the texts to their assigned CEFRL levels. Lexical and syntactic metrics were
computed with LCA, LSA, and koRpus. Several supervised learning models were
built by using Gradient Boosted Trees and Keras Neural Network methods and by
contrasting pairs of CEFRL levels. Results show that it is possible to
implement pairwise distinctions, especially for levels ranging from A1 to B1
(A1=>A2: 0.916 AUC and A2=>B1: 0.904 AUC). Model explanation reveals
significant linguistic features for the predictiveness in the corpus. Word
tokens and word types appear to play a significant role in determining levels.
This shows that levels are highly dependent on specific semantic profiles.
Recovering Trees with Convex Clustering
Authors: Eric C. Chi, Stefan Steinerberger
Date: 2018-06-28 00:00:00
Tasks: Clustering

Convex clustering refers, for given $\left\{x_1, \dots, x_n\right\} \subset
\mathbb{R}^p$, to the minimization of \begin{eqnarray*} u(\gamma) & = &
\underset{u_1, \dots, u_n }{\arg\min}\;\sum_{i=1}^{n}{\lVert x_i - u_i
\rVert^2} + \gamma \sum_{i,j=1}^{n}{w_{ij} \lVert u_i - u_j\rVert},\\
\end{eqnarray*} where $w_{ij} \geq 0$ is an affinity that quantifies the
similarity between $x_i$ and $x_j$. We prove that if the affinities $w_{ij}$
reflect a tree structure in the $\left\{x_1, \dots, x_n\right\}$, then the
convex clustering solution path reconstructs the tree exactly. The main
technical ingredient implies the following combinatorial byproduct: for every
set $\left\{x_1, \dots, x_n \right\} \subset \mathbb{R}^p$ of $n \geq 2$
distinct points, there exist at least $n/6$ points with the property that for
any of these points $x$ there is a unit vector $v \in \mathbb{R}^p$ such that,
when viewed from $x$, `most' points lie in the direction $v$ \begin{eqnarray*}
\frac{1}{n-1}\sum_{i=1 \atop x_i \neq x}^{n}{ \left\langle \frac{x_i -
x}{\lVert x_i - x \rVert}, v \right\rangle} & \geq & \frac{1}{4}.
\end{eqnarray*}
A probabilistic constrained clustering for transfer learning and image category discovery
Authors: Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, Zsolt Kira
Date: 2018-06-28 00:00:00
Tasks: Clustering, Constrained Clustering, Deep Clustering, Ecg Risk Stratification, Transfer Learning

Neural network-based clustering has recently gained popularity, and in
particular a constrained clustering formulation has been proposed to perform
transfer learning and image category discovery using deep learning. The core
idea is to formulate a clustering objective with pairwise constraints that can
be used to train a deep clustering network; therefore the cluster assignments
and their underlying feature representations are jointly optimized end-to-end.
In this work, we provide a novel clustering formulation to address scalability
issues of previous work in terms of optimizing deeper networks and larger
amounts of categories. The proposed objective directly minimizes the negative
log-likelihood of cluster assignment with respect to the pairwise constraints,
has no hyper-parameters, and demonstrates improved scalability and performance
on both supervised learning and unsupervised transfer learning.
Direct Acceleration of SAGA using Sampled Negative Momentum
Authors: Kaiwen Zhou
Date: 2018-06-28 00:00:00
Tasks: Stochastic Optimization

Variance reduction is a simple and effective technique that accelerates
convex (or non-convex) stochastic optimization. Among existing variance
reduction methods, SVRG and SAGA adopt unbiased gradient estimators and are the
most popular variance reduction methods in recent years. Although various
accelerated variants of SVRG (e.g., Katyusha and Acc-Prox-SVRG) have been
proposed, the direct acceleration of SAGA still remains unknown. In this paper,
we propose a directly accelerated variant of SAGA using a novel Sampled
Negative Momentum (SSNM), which achieves the best known oracle complexity for
strongly convex problems (with known strong convexity parameter). Consequently,
our work fills the void of directly accelerated SAGA.
Deep Semi Supervised Generative Learning for Automated PD-L1 Tumor Cell Scoring on NSCLC Tissue Needle Biopsies
Authors: Ansh Kapil, Armin Meier, Aleksandra Zuraw, Keith Steele, Marlon Rebelatto, Günter Schmidt, Nicolas Brieu
Date: 2018-06-28 00:00:00

The level of PD-L1 expression in immunohistochemistry (IHC) assays is a key
biomarker for the identification of Non-Small-Cell-Lung-Cancer (NSCLC) patients
that may respond to anti PD-1/PD-L1 treatments. The quantification of PD-L1
expression currently includes the visual estimation of a Tumor Cell (TC) score
by a pathologist and consists of evaluating the ratio of PD-L1 positive and
PD-L1 negative tumor cells. Known challenges like differences in positivity
estimation around clinically relevant cut-offs and sub-optimal quality of
samples makes visual scoring tedious and subjective, yielding a scoring
variability between pathologists. In this work, we propose a novel deep
learning solution that enables the first automated and objective scoring of
PD-L1 expression in late stage NSCLC needle biopsies. To account for the low
amount of tissue available in biopsy images and to restrict the amount of
manual annotations necessary for training, we explore the use of
semi-supervised approaches against standard fully supervised methods. We
consolidate the manual annotations used for training as well the visual TC
scores used for quantitative evaluation with multiple pathologists. Concordance
measures computed on a set of slides unseen during training provide evidence
that our automatic scoring method matches visual scoring on the considered
dataset while ensuring repeatability and objectivity.
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates
Authors: Kaiwen Zhou, Fanhua Shang, James Cheng
Date: 2018-06-28 00:00:00
Proceeding: ICML 2018 7

Recent years have witnessed exciting progress in the study of stochastic
variance reduced gradient methods (e.g., SVRG, SAGA), their accelerated
variants (e.g, Katyusha) and their extensions in many different settings (e.g.,
online, sparse, asynchronous, distributed). Among them, accelerated methods
enjoy improved convergence rates but have complex coupling structures, which
makes them hard to be extended to more settings (e.g., sparse and asynchronous)
due to the existence of perturbation. In this paper, we introduce a simple
stochastic variance reduced algorithm (MiG), which enjoys the best-known
convergence rates for both strongly convex and non-strongly convex problems.
Moreover, we also present its efficient sparse and asynchronous variants, and
theoretically analyze its convergence rates in these settings. Finally,
extensive experiments for various machine learning problems such as logistic
regression are given to illustrate the practical improvement in both serial and
asynchronous settings.
Bayesian optimization of the PC algorithm for learning Gaussian Bayesian networks
Authors: Irene Córdoba, Eduardo C. Garrido-Merchán, Daniel Hernández-Lobato, Concha Bielza, Pedro Larrañaga
Date: 2018-06-28 00:00:00
Tasks: Bayesian Optimization

The PC algorithm is a popular method for learning the structure of Gaussian
Bayesian networks. It carries out statistical tests to determine absent edges
in the network. It is hence governed by two parameters: (i) The type of test,
and (ii) its significance level. These parameters are usually set to values
recommended by an expert. Nevertheless, such an approach can suffer from human
bias, leading to suboptimal reconstruction results. In this paper we consider a
more principled approach for choosing these parameters in an automatic way. For
this we optimize a reconstruction score evaluated on a set of different
Gaussian Bayesian networks. This objective is expensive to evaluate and lacks a
closed-form expression, which means that Bayesian optimization (BO) is a
natural choice. BO methods use a model to guide the search and are hence able
to exploit smoothness properties of the objective surface. We show that the
parameters found by a BO method outperform those found by a random search
strategy and the expert recommendation. Importantly, we have found that an
often overlooked statistical test provides the best over-all reconstruction
results.
Robust pose tracking with a joint model of appearance and shape
Authors: Yuliang Guo, Lakshmi Narasimhan Govindarajan, Benjamin Kimia, Thomas Serre
Date: 2018-06-28 00:00:00
Tasks: Pose Tracking

We present a novel approach for estimating the 2D pose of an articulated
object with an application to automated video analysis of small laboratory
animals. We have found that deformable part models developed for humans,
exemplified by the flexible mixture of parts (FMP) model, typically fail on
challenging animal poses. We argue that beyond encoding appearance and spatial
relations, shape is needed to overcome the lack of distinctive landmarks on
laboratory animal bodies. In our approach, a shape consistent FMP (scFMP) model
computes promising pose candidates after a standard FMP model is used to
rapidly discard false part detections. This "cascaded" approach combines the
relative strengths of spatial-relations, appearance and shape representations
and is shown to yield significant improvements over the original FMP model as
well as a representative deep neural network baseline.
Modeling Spatio-Temporal Human Track Structure for Action Localization
Authors: Guilhem Chéron, Anton Osokin, Ivan Laptev, Cordelia Schmid
Date: 2018-06-28 00:00:00
Tasks: Action Localization, Human Detection, Optical Flow Estimation, Spatio-Temporal Action Localization, Temporal Action Localization, Temporal Localization

This paper addresses spatio-temporal localization of human actions in video.
In order to localize actions in time, we propose a recurrent localization
network (RecLNet) designed to model the temporal structure of actions on the
level of person tracks. Our model is trained to simultaneously recognize and
localize action classes in time and is based on two layer gated recurrent units
(GRU) applied separately to two streams, i.e. appearance and optical flow
streams. When used together with state-of-the-art person detection and
tracking, our model is shown to improve substantially spatio-temporal action
localization in videos. The gain is shown to be mainly due to improved temporal
localization. We evaluate our method on two recent datasets for spatio-temporal
action localization, UCF101-24 and DALY, demonstrating a significant
improvement of the state of the art.
Learning Implicit Generative Models with the Method of Learned Moments
Authors: Suman Ravuri, Shakir Mohamed, Mihaela Rosca, Oriol Vinyals
Date: 2018-06-28 00:00:00
Proceeding: ICML 2018 7

We propose a method of moments (MoM) algorithm for training large-scale
implicit generative models. Moment estimation in this setting encounters two
problems: it is often difficult to define the millions of moments needed to
learn the model parameters, and it is hard to determine which properties are
useful when specifying moments. To address the first issue, we introduce a
moment network, and define the moments as the network's hidden units and the
gradient of the network's output with the respect to its parameters. To tackle
the second problem, we use asymptotic theory to highlight desiderata for
moments -- namely they should minimize the asymptotic variance of estimated
model parameters -- and introduce an objective to learn better moments. The
sequence of objectives created by this Method of Learned Moments (MoLM) can
train high-quality neural image samplers. On CIFAR-10, we demonstrate that
MoLM-trained generators achieve significantly higher Inception Scores and lower
Frechet Inception Distances than those trained with gradient
penalty-regularized and spectrally-normalized adversarial objectives. These
generators also achieve nearly perfect Multi-Scale Structural Similarity Scores
on CelebA, and can create high-quality samples of 128x128 images.
High Diversity Attribute Guided Face Generation with GANs
Authors: Evgeny Izutov
Date: 2018-06-28 00:00:00
Tasks: Attribute, Diversity, Face Generation, Vocal Bursts Intensity Prediction

In this work we focused on GAN-based solution for the attribute guided face
synthesis. Previous works exploited GANs for generation of photo-realistic face
images and did not pay attention to the question of diversity of the resulting
images. The proposed solution in its turn introducing novel latent space of
unit complex numbers is able to provide the diversity on the "birthday paradox"
score 3 times higher than the size of the training dataset. It is important to
emphasize that our result is shown on relatively small dataset (20k samples vs
200k) while preserving photo-realistic properties of generated faces on
significantly higher resolution (128x128 in comparison to 32x32 of previous
works).
Automatic Exploration of Machine Learning Experiments on OpenML
Authors: Daniel Kühn, Philipp Probst, Janek Thomas, Bernd Bischl
Date: 2018-06-28 00:00:00
Tasks: BIG-bench Machine Learning

Understanding the influence of hyperparameters on the performance of a
machine learning algorithm is an important scientific topic in itself and can
help to improve automatic hyperparameter tuning procedures. Unfortunately,
experimental meta data for this purpose is still rare. This paper presents a
large, free and open dataset addressing this problem, containing results on 38
OpenML data sets, six different machine learning algorithms and many different
hyperparameter configurations. Results where generated by an automated random
sampling strategy, termed the OpenML Random Bot. Each algorithm was
cross-validated up to 20.000 times per dataset with different hyperparameters
settings, resulting in a meta dataset of around 2.5 million experiments
overall.
Deep learning for dehazing: Comparison and analysis
Authors: A. Benoit, Leonel Cuevas, Jean-Baptiste Thomas
Date: 2018-06-28 00:00:00
Tasks: Deep Learning

We compare a recent dehazing method based on deep learning, Dehazenet, with
traditional state-of-the-art approaches , on benchmark data with reference.
Dehazenet estimates the depth map from transmission factor on a single color
image, which is used to inverse the Koschmieder model of imaging in the
presence of haze. In this sense, the solution is still attached to the
Koschmieder model. We demonstrate that the transmission is very well estimated
by the network, but also that this method exhibits the same limitation than
others due to the use of the same imaging model.
ResNet with one-neuron hidden layers is a Universal Approximator
Authors: Hongzhou Lin, Stefanie Jegelka
Date: 2018-06-28 00:00:00
Proceeding: NeurIPS 2018 12

We demonstrate that a very deep ResNet with stacked modules with one neuron
per hidden layer and ReLU activation functions can uniformly approximate any
Lebesgue integrable function in $d$ dimensions, i.e. $\ell_1(\mathbb{R}^d)$.
Because of the identity mapping inherent to ResNets, our network has
alternating layers of dimension one and $d$. This stands in sharp contrast to
fully connected networks, which are not universal approximators if their width
is the input dimension $d$ [Lu et al, 2017; Hanin and Sellke, 2017]. Hence, our
result implies an increase in representational power for narrow deep networks
by the ResNet architecture.