pwc-archive/papers-with-abstracts huggingface dataset:
Some Requests for Machine Learning Research from the East African Tech Scene
Authors: Milan Cvitkovic
Date: 2018-10-25 00:00:00
Tasks: BIG-bench Machine Learning

Based on 46 in-depth interviews with scientists, engineers, and CEOs, this
document presents a list of concrete machine research problems, progress on
which would directly benefit tech ventures in East Africa.
A Text Classification Application: Poet Detection from Poetry
Authors: Durmus Ozkan Sahin, Oguz Emre Kural, Erdal Kilic, Armagan Karabina
Date: 2018-10-24 00:00:00
Tasks: Classification, feature selection, General Classification, Text Categorization, text-classification, Text Classification

With the widespread use of the internet, the size of the text data increases
day by day. Poems can be given as an example of the growing text. In this
study, we aim to classify poetry according to poet. Firstly, data set
consisting of three different poetry of poets written in English have been
constructed. Then, text categorization techniques are implemented on it.
Chi-Square technique are used for feature selection. In addition, five
different classification algorithms are tried. These algorithms are Sequential
minimal optimization, Naive Bayes, C4.5 decision tree, Random Forest and
k-nearest neighbors. Although each classifier showed very different results,
over the 70% classification success rate was taken by sequential minimal
optimization technique.
CatBoost: gradient boosting with categorical features support
Authors: Anna Veronika Dorogush, Vasily Ershov, Andrey Gulin
Date: 2018-10-24 00:00:00
Tasks: Clustering, CPU, Dimensionality Reduction, General Classification, GPU

In this paper we present CatBoost, a new open-sourced gradient boosting
library that successfully handles categorical features and outperforms existing
publicly available implementations of gradient boosting in terms of quality on
a set of popular publicly available datasets. The library has a GPU
implementation of learning algorithm and a CPU implementation of scoring
algorithm, which are significantly faster than other gradient boosting
libraries on ensembles of similar sizes.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Authors: Richard Yuanzhe Pang, Kevin Gimpel
Date: 2018-10-28 00:00:00
Proceeding: WS 2019 11
Tasks: Sentence

We consider the problem of automatically generating textual paraphrases with modified attributes or properties, focusing on the setting without parallel data (Hu et al., 2017; Shen et al., 2017). This setting poses challenges for evaluation. We show that the metric of post-transfer classification accuracy is insufficient on its own, and propose additional metrics based on semantic preservation and fluency as well as a way to combine them into a single overall score. We contribute new loss functions and training strategies to address the different metrics. Semantic preservation is addressed by adding a cyclic consistency loss and a loss based on paraphrase pairs, while fluency is improved by integrating losses based on style-specific language models. We experiment with a Yelp sentiment dataset and a new literature dataset that we propose, using multiple models that extend prior work (Shen et al., 2017). We demonstrate that our metrics correlate well with human judgments, at both the sentence-level and system-level. Automatic and manual evaluation also show large improvements over the baseline method of Shen et al. (2017). We hope that our proposed metrics can speed up system development for new textual transfer tasks while also encouraging the community to address our three complementary aspects of transfer quality.
gpuRIR: A python library for Room Impulse Response simulation with GPU acceleration
Authors: David Diaz-Guerra, Antonio Miguel, Jose R. Beltran
Date: 2018-10-26 00:00:00
Tasks: CPU, GPU, Room Impulse Response (RIR)

The Image Source Method (ISM) is one of the most employed techniques to
calculate acoustic Room Impulse Responses (RIRs), however, its computational
complexity grows fast with the reverberation time of the room and its
computation time can be prohibitive for some applications where a huge number
of RIRs are needed. In this paper, we present a new implementation that
dramatically improves the computation speed of the ISM by using Graphic
Processing Units (GPUs) to parallelize both the simulation of multiple RIRs and
the computation of the images inside each RIR. We provide a Python library
under GNU license that can be easily used without any knowledge about GPU
programming and we show that it is about 100 times faster than other state of
the art CPU libraries.
Agent-Based Model Calibration using Machine Learning Surrogates
Authors: Francesco Lamperti, Andrea Roventini, Amir Sani
Date: 2017-03-30 00:00:00
Tasks: BIG-bench Machine Learning, model

Taking agent-based models (ABM) closer to the data is an open challenge. This
paper explicitly tackles parameter space exploration and calibration of ABMs
combining supervised machine-learning and intelligent sampling to build a
surrogate meta-model. The proposed approach provides a fast and accurate
approximation of model behaviour, dramatically reducing computation time. In
that, our machine-learning surrogate facilitates large scale explorations of
the parameter-space, while providing a powerful filter to gain insights into
the complex functioning of agent-based models. The algorithm introduced in this
paper merges model simulation and output analysis into a surrogate meta-model,
which substantially ease ABM calibration. We successfully apply our approach to
the Brock and Hommes (1998) asset pricing model and to the "Island" endogenous
growth model (Fagiolo and Dosi, 2003). Performance is evaluated against a
relatively large out-of-sample set of parameter combinations, while employing
different user-defined statistical tests for output analysis. The results
demonstrate the capacity of machine learning surrogates to facilitate fast and
precise exploration of agent-based models' behaviour over their often rugged
parameter spaces.
Compressive Sampling Approach for Image Acquisition with Lensless Endoscope
Authors: Stéphanie Guérit, Siddharth Sivankutty, Camille Scotté, John Alto Lee, Hervé Rigneault, Laurent Jacques
Date: 2018-10-29 00:00:00

The lensless endoscope is a promising device designed to image tissues in
vivo at the cellular scale. The traditional acquisition setup consists in
raster scanning during which the focused light beam from the optical fiber
illuminates sequentially each pixel of the field of view (FOV). The calibration
step to focus the beam and the sampling scheme both take time. In this
preliminary work, we propose a scanning method based on compressive sampling
theory. The method does not rely on a focused beam but rather on the random
illumination patterns generated by the single-mode fibers. Experiments are
performed on synthetic data for different compression rates (from 10 to 100% of
the FOV).
Scaling Gaussian Process Regression with Derivatives
Authors: David Eriksson, Kun Dong, Eric Hans Lee, David Bindel, Andrew Gordon Wilson
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Optimization, Dimensionality Reduction, Gaussian Processes, regression, Surface Reconstruction

Gaussian processes (GPs) with derivatives are useful in many applications,
including Bayesian optimization, implicit surface reconstruction, and terrain
reconstruction. Fitting a GP to function values and derivatives at $n$ points
in $d$ dimensions requires linear solves and log determinants with an ${n(d+1)
\times n(d+1)}$ positive definite matrix -- leading to prohibitive
$\mathcal{O}(n^3d^3)$ computations for standard direct methods. We propose
iterative solvers using fast $\mathcal{O}(nd)$ matrix-vector multiplications
(MVMs), together with pivoted Cholesky preconditioning that cuts the iterations
to convergence by several orders of magnitude, allowing for fast kernel
learning and prediction. Our approaches, together with dimensionality
reduction, enables Bayesian optimization with derivatives to scale to
high-dimensional problems and large evaluation budgets.
Assessing Generalization in Deep Reinforcement Learning
Authors: Charles Packer, Katelyn Gao, Jernej Kos, Philipp Krähenbühl, Vladlen Koltun, Dawn Song
Date: 2018-10-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Reinforcement Learning, Out-of-Distribution Generalization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (RL) has achieved breakthrough results on many
tasks, but agents often fail to generalize beyond the environment they were
trained in. As a result, deep RL algorithms that promote generalization are
receiving increasing attention. However, works in this area use a wide variety
of tasks and experimental setups for evaluation. The literature lacks a
controlled assessment of the merits of different generalization schemes. Our
aim is to catalyze community-wide progress on generalization in deep RL. To
this end, we present a benchmark and experimental protocol, and conduct a
systematic empirical study. Our framework contains a diverse set of
environments, our methodology covers both in-distribution and
out-of-distribution generalization, and our evaluation includes deep RL
algorithms that specifically tackle generalization. Our key finding is that
`vanilla' deep RL algorithms generalize better than specialized schemes that
were proposed specifically to tackle generalization.
Three Mechanisms of Weight Decay Regularization
Authors: Guodong Zhang, Chaoqi Wang, Bowen Xu, Roger Grosse
Date: 2018-10-29 00:00:00
Proceeding: ICLR 2019 5

Weight decay is one of the standard tricks in the neural network toolbox, but
the reasons for its regularization effect are poorly understood, and recent
results have cast doubt on the traditional interpretation in terms of $L_2$
regularization. Literal weight decay has been shown to outperform $L_2$
regularization for optimizers for which they differ. We empirically investigate
weight decay for three optimization algorithms (SGD, Adam, and K-FAC) and a
variety of network architectures. We identify three distinct mechanisms by
which weight decay exerts a regularization effect, depending on the particular
optimization algorithm and architecture: (1) increasing the effective learning
rate, (2) approximately regularizing the input-output Jacobian norm, and (3)
reducing the effective damping coefficient for second-order optimization. Our
results provide insight into how to improve the regularization of neural
networks.
Towards Principled Uncertainty Estimation for Deep Neural Networks
Authors: Richard Harang, Ethan M. Rudd
Date: 2018-10-29 00:00:00
Tasks: Bayesian Inference

When the cost of misclassifying a sample is high, it is useful to have an
accurate estimate of uncertainty in the prediction for that sample. There are
also multiple types of uncertainty which are best estimated in different ways,
for example, uncertainty that is intrinsic to the training set may be
well-handled by a Bayesian approach, while uncertainty introduced by shifts
between training and query distributions may be better-addressed by
density/support estimation. In this paper, we examine three types of
uncertainty: model capacity uncertainty, intrinsic data uncertainty, and open
set uncertainty, and review techniques that have been derived to address each
one. We then introduce a unified hierarchical model, which combines methods
from Bayesian inference, invertible latent density inference, and
discriminative classification in a single end-to-end deep neural network
topology to yield efficient per-sample uncertainty estimation in a detection
context. This approach addresses all three uncertainty types and can readily
accommodate prior/base rates for binary detection. We then discuss how to
extend this model to a more generic multiclass recognition context.
Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization
Authors: James Vuckovic
Date: 2018-10-29 00:00:00
Tasks: BIG-bench Machine Learning, Stochastic Optimization, Variational Inference

We introduce Kalman Gradient Descent, a stochastic optimization algorithm
that uses Kalman filtering to adaptively reduce gradient variance in stochastic
gradient descent by filtering the gradient estimates. We present both a
theoretical analysis of convergence in a non-convex setting and experimental
results which demonstrate improved performance on a variety of machine learning
areas including neural networks and black box variational inference. We also
present a distributed version of our algorithm that enables large-dimensional
optimization, and we extend our algorithm to SGD with momentum and RMSProp.
Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution
Authors: Dimitrios I. Diochnos, Saeed Mahloujifar, Mohammad Mahmoody
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12

We study adversarial perturbations when the instances are uniformly
distributed over $\{0,1\}^n$. We study both "inherent" bounds that apply to any
problem and any classifier for such a problem as well as bounds that apply to
specific problems and specific hypothesis classes.
  As the current literature contains multiple definitions of adversarial risk
and robustness, we start by giving a taxonomy for these definitions based on
their goals, we identify one of them as the one guaranteeing misclassification
by pushing the instances to the error region. We then study some classic
algorithms for learning monotone conjunctions and compare their adversarial
risk and robustness under different definitions by attacking the hypotheses
using instances drawn from the uniform distribution. We observe that sometimes
these definitions lead to significantly different bounds. Thus, this study
advocates for the use of the error-region definition, even though other
definitions, in other contexts, may coincide with the error-region definition.
  Using the error-region definition of adversarial perturbations, we then study
inherent bounds on risk and robustness of any classifier for any classification
problem whose instances are uniformly distributed over $\{0,1\}^n$. Using the
isoperimetric inequality for the Boolean hypercube, we show that for initial
error $0.01$, there always exists an adversarial perturbation that changes
$O(\sqrt{n})$ bits of the instances to increase the risk to $0.5$, making
classifier's decisions meaningless. Furthermore, by also using the central
limit theorem we show that when $n\to \infty$, at most $c \cdot \sqrt{n}$ bits
of perturbations, for a universal constant $c< 1.17$, suffice for increasing
the risk to $0.5$, and the same $c \cdot \sqrt{n} $ bits of perturbations on
average suffice to increase the risk to $1$, hence bounding the robustness by
$c \cdot \sqrt{n}$.
The problem with probabilistic DAG automata for semantic graphs
Authors: Ieva Vasiljeva, Sorcha Gilroy, Adam Lopez
Date: 2018-10-29 00:00:00
Proceeding: NAACL 2019 6

Semantic representations in the form of directed acyclic graphs (DAGs) have
been introduced in recent years, and to model them, we need probabilistic
models of DAGs. One model that has attracted some attention is the DAG
automaton, but it has not been studied as a probabilistic model. We show that
some DAG automata cannot be made into useful probabilistic models by the nearly
universal strategy of assigning weights to transitions. The problem affects
single-rooted, multi-rooted, and unbounded-degree variants of DAG automata, and
appears to be pervasive. It does not affect planar variants, but these are
problematic for other reasons.
Learning Comment Generation by Leveraging User-Generated Data
Authors: Zhaojiang Lin, Genta Indra Winata, Pascale Fung
Date: 2018-10-29 00:00:00
Tasks: Comment Generation, Information Retrieval, Retrieval

Existing models on open-domain comment generation are difficult to train, and
they produce repetitive and uninteresting responses. The problem is due to
multiple and contradictory responses from a single article, and by the rigidity
of retrieval methods. To solve this problem, we propose a combined approach to
retrieval and generation methods. We propose an attentive scorer to retrieve
informative and relevant comments by leveraging user-generated data. Then, we
use such comments, together with the article, as input for a
sequence-to-sequence model with copy mechanism. We show the robustness of our
model and how it can alleviate the aforementioned issue by using a large scale
comment generation dataset. The result shows that the proposed generative model
significantly outperforms strong baseline such as Seq2Seq with attention and
Information Retrieval models by around 27 and 30 BLEU-1 points respectively.
Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
Authors: David Reeb, Andreas Doerr, Sebastian Gerwinn, Barbara Rakitsch
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Gaussian Processes, Generalization Bounds, regression

Gaussian Processes (GPs) are a generic modelling tool for supervised
learning. While they have been successfully applied on large datasets, their
use in safety-critical applications is hindered by the lack of good performance
guarantees. To this end, we propose a method to learn GPs and their sparse
approximations by directly optimizing a PAC-Bayesian bound on their
generalization performance, instead of maximizing the marginal likelihood.
Besides its theoretical appeal, we find in our evaluation that our learning
method is robust and yields significantly better generalization guarantees than
other common GP approaches on several regression benchmark datasets.
Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
Authors: Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, Douglas Eck
Date: 2018-10-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Music Generation, Music Modeling, Music Transcription, Piano Music Modeling

Generating musical audio directly with neural networks is notoriously
difficult because it requires coherently modeling structure at many different
timescales. Fortunately, most music is also highly structured and can be
represented as discrete note events played on musical instruments. Herein, we
show that by using notes as an intermediate representation, we can train a
suite of models capable of transcribing, composing, and synthesizing audio
waveforms with coherent musical structure on timescales spanning six orders of
magnitude (~0.1 ms to ~100 s), a process we call Wave2Midi2Wave. This large
advance in the state of the art is enabled by our release of the new MAESTRO
(MIDI and Audio Edited for Synchronous TRacks and Organization) dataset,
composed of over 172 hours of virtuosic piano performances captured with fine
alignment (~3 ms) between note labels and audio waveforms. The networks and the
dataset together present a promising approach toward creating new expressive
and interpretable neural models of music.
Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
Authors: Arnab Kumar Mondal, Jose Dolz, Christian Desrosiers
Date: 2018-10-29 00:00:00
Tasks: 3D Medical Imaging Segmentation, Brain Image Segmentation, Brain Segmentation, Few-Shot Semantic Segmentation, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Semi-Supervised Semantic Segmentation

We address the problem of segmenting 3D multi-modal medical images in
scenarios where very few labeled examples are available for training.
Leveraging the recent success of adversarial learning for semi-supervised
segmentation, we propose a novel method based on Generative Adversarial
Networks (GANs) to train a segmentation model with both labeled and unlabeled
images. The proposed method prevents over-fitting by learning to discriminate
between true and fake patches obtained by a generator network. Our work extends
current adversarial learning approaches, which focus on 2D single-modality
images, to the more challenging context of 3D volumes of multiple modalities.
The proposed method is evaluated on the problem of segmenting brain MRI from
the iSEG-2017 and MRBrainS 2013 datasets. Significant performance improvement
is reported, compared to state-of-art segmentation networks trained in a
fully-supervised manner. In addition, our work presents a comprehensive
analysis of different GAN architectures for semi-supervised segmentation,
showing recent techniques like feature matching to yield a higher performance
than conventional adversarial training approaches. Our code is publicly
available at https://github.com/arnab39/FewShot_GAN-Unet3D
Approximate Bayesian Computation via Population Monte Carlo and Classification
Authors: Charlie Rogers-Smith, Henri Pesonen, Samuel Kaski
Date: 2018-10-29 00:00:00
Tasks: Classification, General Classification

Approximate Bayesian computation (ABC) methods can be used to sample from posterior distributions when the likelihood function is unavailable or intractable, as is often the case in biological systems. ABC methods suffer from inefficient particle proposals in high dimensions, and subjectivity in the choice of summary statistics, discrepancy measure, and error tolerance. Sequential Monte Carlo (SMC) methods have been combined with ABC to improve the efficiency of particle proposals, but suffer from subjectivity and require many simulations from the likelihood function. Likelihood-Free Inference by Ratio Estimation (LFIRE) leverages classification to estimate the posterior density directly but does not explore the parameter space efficiently. This work proposes a classification approach that approximates population Monte Carlo (PMC), where model class probabilities from classification are used to update particle weights. This approach, called Classification-PMC, blends adaptive proposals and classification, efficiently producing samples from the posterior without subjectivity. We show through a simulation study that Classification-PMC outperforms two state-of-the-art methods: ratio estimation and SMC ABC when it is computationally difficult to simulate from the likelihood.
Leveraging Gaussian Process and Voting-Empowered Many-Objective Evaluation for Fault Identification
Authors: Pei Cao, Qi Shuai, Jiong Tang
Date: 2018-10-29 00:00:00
Tasks: Structural Health Monitoring

Using piezoelectric impedance/admittance sensing for structural health
monitoring is promising, owing to the simplicity in circuitry design as well as
the high-frequency interrogation capability. The actual identification of fault
location and severity using impedance/admittance measurements, nevertheless,
remains to be an extremely challenging task. A first-principle based structural
model using finite element discretization requires high dimensionality to
characterize the high-frequency response. As such, direct inversion using the
sensitivity matrix usually yields an under-determined problem. Alternatively,
the identification problem may be cast into an optimization framework in which
fault parameters are identified through repeated forward finite element
analysis which however is oftentimes computationally prohibitive. This paper
presents an efficient data-assisted optimization approach for fault
identification without using finite element model iteratively. We formulate a
many-objective optimization problem to identify fault parameters, where
response surfaces of impedance measurements are constructed through Gaussian
process-based calibration. To balance between solution diversity and
convergence, an -dominance enabled many-objective simulated annealing algorithm
is established. As multiple solutions are expected, a voting score calculation
procedure is developed to further identify those solutions that yield better
implications regarding structural health condition. The effectiveness of the
proposed approach is demonstrated by systematic numerical and experimental case
studies.