pwc-archive/papers-with-abstracts huggingface dataset:
Enhancing Genetic Algorithms using Multi Mutations
Authors: Ahmad B. A. Hassanat, Esra'a Alkafaween, Nedal A. Al-Nawaiseh, Mohammad A. Abbadi, Mouhammd Al-kasassbeh, Mahmoud B. Alhasanat
Date: 2016-02-26 00:00:00

Mutation is one of the most important stages of the genetic algorithm because
of its impact on the exploration of global optima, and to overcome premature
convergence. There are many types of mutation, and the problem lies in
selection of the appropriate type, where the decision becomes more difficult
and needs more trial and error. This paper investigates the use of more than
one mutation operator to enhance the performance of genetic algorithms. Novel
mutation operators are proposed, in addition to two selection strategies for
the mutation operators, one of which is based on selecting the best mutation
operator and the other randomly selects any operator. Several experiments on
some Travelling Salesman Problems (TSP) were conducted to evaluate the proposed
methods, and these were compared to the well-known exchange mutation and
rearrangement mutation. The results show the importance of some of the proposed
methods, in addition to the significant enhancement of the genetic algorithm's
performance, particularly when using more than one mutation operator.
Stochastic Gradient Monomial Gamma Sampler
Authors: Yizhe Zhang, Changyou Chen, Zhe Gan, Ricardo Henao, Lawrence Carin
Date: 2017-06-05 00:00:00
Proceeding: ICML 2017 8

Recent advances in stochastic gradient techniques have made it possible to
estimate posterior distributions from large datasets via Markov Chain Monte
Carlo (MCMC). However, when the target posterior is multimodal, mixing
performance is often poor. This results in inadequate exploration of the
posterior distribution. A framework is proposed to improve the sampling
efficiency of stochastic gradient MCMC, based on Hamiltonian Monte Carlo. A
generalized kinetic function is leveraged, delivering superior stationary
mixing, especially for multimodal distributions. Techniques are also discussed
to overcome the practical issues introduced by this generalization. It is shown
that the proposed approach is better at exploring complex multimodal posterior
distributions, as demonstrated on multiple applications and in comparison with
other stochastic gradient MCMC methods.
Towards Unifying Hamiltonian Monte Carlo and Slice Sampling
Authors: Yizhe Zhang, Xiangyu Wang, Changyou Chen, Ricardo Henao, Kai Fan, Lawrence Carin
Date: 2016-02-25 00:00:00
Proceeding: NeurIPS 2016 12

We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling,
demonstrating their connection via the Hamiltonian-Jacobi equation from
Hamiltonian mechanics. This insight enables extension of HMC and slice sampling
to a broader family of samplers, called Monomial Gamma Samplers (MGS). We
provide a theoretical analysis of the mixing performance of such samplers,
proving that in the limit of a single parameter, the MGS draws decorrelated
samples from the desired target distribution. We further show that as this
parameter tends toward this limit, performance gains are achieved at a cost of
increasing numerical difficulty and some practical convergence issues. Our
theoretical results are validated with synthetic data and real-world
applications.
Sentence Object Notation: Multilingual sentence notation based on Wordnet
Authors: Abdelkrime Aries, Djamel Eddine Zegour, Walid Khaled Hidouci
Date: 2018-01-03 00:00:00
Tasks: Object, Sentence, Text Summarization

The representation of sentences is a very important task. It can be used as a
way to exchange data inter-applications. One main characteristic, that a
notation must have, is a minimal size and a representative form. This can
reduce the transfer time, and hopefully the processing time as well.
  Usually, sentence representation is associated to the processed language. The
grammar of this language affects how we represent the sentence. To avoid
language-dependent notations, we have to come up with a new representation
which don't use words, but their meanings. This can be done using a lexicon
like wordnet, instead of words we use their synsets. As for syntactic
relations, they have to be universal as much as possible.
  Our new notation is called STON "SenTences Object Notation", which somehow
has similarities to JSON. It is meant to be minimal, representative and
language-independent syntactic representation. Also, we want it to be readable
and easy to be created. This simplifies developing simple automatic generators
and creating test banks manually. Its benefit is to be used as a medium between
different parts of applications like: text summarization, language translation,
etc. The notation is based on 4 languages: Arabic, English, Franch and
Japanese; and there are some cases where these languages don't agree on one
representation. Also, given the diversity of grammatical structure of different
world languages, this annotation may fail for some languages which allows more
future improvements.
Focus: Querying Large Video Datasets with Low Latency and Low Cost
Authors: Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodik, Paramvir Bahl, Matthai Philipose, Phillip B. Gibbons, Onur Mutlu
Date: 2018-01-10 00:00:00
Tasks: GPU

Large volumes of videos are continuously recorded from cameras deployed for
traffic control and surveillance with the goal of answering "after the fact"
queries: identify video frames with objects of certain classes (cars, bags)
from many days of recorded video. While advancements in convolutional neural
networks (CNNs) have enabled answering such queries with high accuracy, they
are too expensive and slow. We build Focus, a system for low-latency and
low-cost querying on large video datasets. Focus uses cheap ingestion
techniques to index the videos by the objects occurring in them. At
ingest-time, it uses compression and video-specific specialization of CNNs.
Focus handles the lower accuracy of the cheap CNNs by judiciously leveraging
expensive CNNs at query-time. To reduce query time latency, we cluster similar
objects and hence avoid redundant processing. Using experiments on video
streams from traffic, surveillance and news channels, we see that Focus uses
58X fewer GPU cycles than running expensive ingest processors and is 37X faster
than processing all the video at query time.
MIZAN: A Large Persian-English Parallel Corpus
Authors: Omid Kashefi
Date: 2018-01-07 00:00:00
Tasks: Machine Translation, Sentence, Translation

One of the most major and essential tasks in natural language processing is machine translation that is now highly dependent upon multilingual parallel corpora. Through this paper, we introduce the biggest Persian-English parallel corpus with more than one million sentence pairs collected from masterpieces of literature. We also present acquisition process and statistics of the corpus, and experiment a base-line statistical machine translation system using the corpus.
MilkQA: a Dataset of Consumer Questions for the Task of Answer Selection
Authors: Marcelo Criscuolo, Erick Rocha Fonseca, Sandra Maria Aluísio, Ana Carolina Sperança-Criscuolo
Date: 2018-01-10 00:00:00
Tasks: Answer Selection, Question Answering

We introduce MilkQA, a question answering dataset from the dairy domain
dedicated to the study of consumer questions. The dataset contains 2,657 pairs
of questions and answers, written in the Portuguese language and originally
collected by the Brazilian Agricultural Research Corporation (Embrapa). All
questions were motivated by real situations and written by thousands of authors
with very different backgrounds and levels of literacy, while answers were
elaborated by specialists from Embrapa's customer service. Our dataset was
filtered and anonymized by three human annotators. Consumer questions are a
challenging kind of question that is usually employed as a form of seeking
information. Although several question answering datasets are available, most
of such resources are not suitable for research on answer selection models for
consumer questions. We aim to fill this gap by making MilkQA publicly
available. We study the behavior of four answer selection models on MilkQA: two
baseline models and two convolutional neural network archictetures. Our results
show that MilkQA poses real challenges to computational models, particularly
due to linguistic characteristics of its questions and to their unusually
longer lengths. Only one of the experimented models gives reasonable results,
at the cost of high computational requirements.
Estimation and Inference on Heterogeneous Treatment Effects in High-Dimensional Dynamic Panels under Weak Dependence
Authors: Vira Semenova, Matt Goldman, Victor Chernozhukov, Matt Taddy
Date: 2017-12-28 00:00:00
Tasks: Causal Inference, Model Selection, regression, Time Series Analysis

This paper provides estimation and inference methods for a conditional average treatment effects (CATE) characterized by a high-dimensional parameter in both homogeneous cross-sectional and unit-heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable with explanatory variables. The first step of our procedure is orthogonalization, where we partial out the controls and unit effects from the outcome and the base treatment and take the cross-fitted residuals. This step uses a novel generic cross-fitting method we design for weakly dependent time series and panel data. This method "leaves out the neighbors" when fitting nuisance components, and we theoretically power it by using Strassen's coupling. As a result, we can rely on any modern machine learning method in the first step, provided it learns the residuals well enough. Second, we construct an orthogonal (or residual) learner of CATE -- the Lasso CATE -- that regresses the outcome residual on the vector of interactions of the residualized treatment with explanatory variables. If the complexity of CATE function is simpler than that of the first-stage regression, the orthogonal learner converges faster than the single-stage regression-based learner. Third, we perform simultaneous inference on parameters of the CATE function using debiasing. We also can use ordinary least squares in the last two steps when CATE is low-dimensional. In heterogeneous panel data settings, we model the unobserved unit heterogeneity as a weakly sparse deviation from Mundlak (1978)'s model of correlated unit effects as a linear function of time-invariant covariates and make use of L1-penalization to estimate these models. We demonstrate our methods by estimating price elasticities of groceries based on scanner data. We note that our results are new even for the cross-sectional (i.i.d) case.
Real-to-Virtual Domain Unification for End-to-End Autonomous Driving
Authors: Luona Yang, Xiaodan Liang, Tairui Wang, Eric Xing
Date: 2018-01-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: Autonomous Driving

In the spectrum of vision-based autonomous driving, vanilla end-to-end models
are not interpretable and suboptimal in performance, while mediated perception
models require additional intermediate representations such as segmentation
masks or detection bounding boxes, whose annotation can be prohibitively
expensive as we move to a larger scale. More critically, all prior works fail
to deal with the notorious domain shift if we were to merge data collected from
different sources, which greatly hinders the model generalization ability. In
this work, we address the above limitations by taking advantage of virtual data
collected from driving simulators, and present DU-drive, an unsupervised
real-to-virtual domain unification framework for end-to-end autonomous driving.
It first transforms real driving data to its less complex counterpart in the
virtual domain and then predicts vehicle control commands from the generated
virtual image. Our framework has three unique advantages: 1) it maps driving
data collected from a variety of source distributions into a unified domain,
effectively eliminating domain shift; 2) the learned virtual representation is
simpler than the input real image and closer in form to the "minimum sufficient
statistic" for the prediction task, which relieves the burden of the
compression phase while optimizing the information bottleneck tradeoff and
leads to superior prediction performance; 3) it takes advantage of annotated
virtual data which is unlimited and free to obtain. Extensive experiments on
two public driving datasets and two driving simulators demonstrate the
performance superiority and interpretive capability of DU-drive.
Approximation beats concentration? An approximation view on inference with smooth radial kernels
Authors: Mikhail Belkin
Date: 2018-01-10 00:00:00

Positive definite kernels and their associated Reproducing Kernel Hilbert
Spaces provide a mathematically compelling and practically competitive
framework for learning from data.
  In this paper we take the approximation theory point of view to explore
various aspects of smooth kernels related to their inferential properties. We
analyze eigenvalue decay of kernels operators and matrices, properties of
eigenfunctions/eigenvectors and "Fourier" coefficients of functions in the
kernel space restricted to a discrete set of data points. We also investigate
the fitting capacity of kernels, giving explicit bounds on the fat shattering
dimension of the balls in Reproducing Kernel Hilbert spaces. Interestingly, the
same properties that make kernels very effective approximators for functions in
their "native" kernel space, also limit their capacity to represent arbitrary
functions. We discuss various implications, including those for gradient
descent type methods.
  It is important to note that most of our bounds are measure independent.
Moreover, at least in moderate dimension, the bounds for eigenvalues are much
tighter than the bounds which can be obtained from the usual matrix
concentration results. For example, we see that the eigenvalues of kernel
matrices show nearly exponential decay with constants depending only on the
kernel and the domain. We call this "approximation beats concentration"
phenomenon as even when the data are sampled from a probability distribution,
some of their aspects are better understood in terms of approximation theory.
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Authors: Ruth Fong, Andrea Vedaldi
Date: 2017-04-11 00:00:00
Proceeding: ICCV 2017 10
Tasks: Interpretable Machine Learning

As machine learning algorithms are increasingly applied to high impact yet high risk tasks, such as medical diagnosis or autonomous driving, it is critical that researchers can explain how such algorithms arrived at their predictions. In recent years, a number of image saliency methods have been developed to summarize where highly complex neural networks "look" in an image for evidence for their predictions. However, these techniques are limited by their heuristic nature and architectural constraints. In this paper, we make two main contributions: First, we propose a general framework for learning different kinds of explanations for any black box algorithm. Second, we specialise the framework to find the part of an image most responsible for a classifier decision. Unlike previous works, our method is model-agnostic and testable because it is grounded in explicit and interpretable image perturbations.
Inferring a Third Spatial Dimension from 2D Histological Images
Authors: Maxime W. Lafarge, Josien P. W. Pluim, Koen A. J. Eppenhof, Pim Moeskops, Mitko Veta
Date: 2018-01-10 00:00:00
Tasks: Data Augmentation

Histological images are obtained by transmitting light through a tissue
specimen that has been stained in order to produce contrast. This process
results in 2D images of the specimen that has a three-dimensional structure. In
this paper, we propose a method to infer how the stains are distributed in the
direction perpendicular to the surface of the slide for a given 2D image in
order to obtain a 3D representation of the tissue. This inference is achieved
by decomposition of the staining concentration maps under constraints that
ensure realistic decomposition and reconstruction of the original 2D images.
Our study shows that it is possible to generate realistic 3D images making this
method a potential tool for data augmentation when training deep learning
models.
A First Derivative Potts Model for Segmentation and Denoising Using ILP
Authors: Ruobing Shen, Gerhard Reinelt, Stéphane Canu
Date: 2017-09-21 00:00:00
Tasks: Denoising, Image Segmentation, Segmentation, Semantic Segmentation, Unsupervised Image Segmentation

Unsupervised image segmentation and denoising are two fundamental tasks in
image processing. Usually, graph based models such as multicut are used for
segmentation and variational models are employed for denoising. Our approach
addresses both problems at the same time. We propose a novel ILP formulation of
the first derivative Potts model with the $\ell_1$ data term, where binary
variables are introduced to deal with the $\ell_0$ norm of the regularization
term. The ILP is then solved by a standard off-the-shelf MIP solver. Numerical
experiments are compared with the multicut problem.
A Smoothed Analysis of the Greedy Algorithm for the Linear Contextual Bandit Problem
Authors: Sampath Kannan, Jamie Morgenstern, Aaron Roth, Bo Waggoner, Zhiwei Steven Wu
Date: 2018-01-10 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Fairness

Bandit learning is characterized by the tension between long-term exploration
and short-term exploitation. However, as has recently been noted, in settings
in which the choices of the learning algorithm correspond to important
decisions about individual people (such as criminal recidivism prediction,
lending, and sequential drug trials), exploration corresponds to explicitly
sacrificing the well-being of one individual for the potential future benefit
of others. This raises a fairness concern. In such settings, one might like to
run a "greedy" algorithm, which always makes the (myopically) optimal decision
for the individuals at hand - but doing this can result in a catastrophic
failure to learn. In this paper, we consider the linear contextual bandit
problem and revisit the performance of the greedy algorithm. We give a smoothed
analysis, showing that even when contexts may be chosen by an adversary, small
perturbations of the adversary's choices suffice for the algorithm to achieve
"no regret", perhaps (depending on the specifics of the setting) with a
constant amount of initial training data. This suggests that "generically"
(i.e. in slightly perturbed environments), exploration and exploitation need
not be in conflict in the linear setting.
CoPaSul Manual - Contour-based parametric and superpositional intonation stylization
Authors: Uwe D. Reichel
Date: 2016-12-14 00:00:00
Tasks: Clustering

The purposes of the CoPaSul toolkit are (1) automatic prosodic annotation and
(2) prosodic feature extraction from syllable to utterance level. CoPaSul
stands for contour-based, parametric, superpositional intonation stylization.
In this framework intonation is represented as a superposition of global and
local contours that are described parametrically in terms of polynomial
coefficients. On the global level (usually associated but not necessarily
restricted to intonation phrases) the stylization serves to represent register
in terms of time-varying F0 level and range. On the local level (e.g. accent
groups), local contour shapes are described. From this parameterization several
features related to prosodic boundaries and prominence can be derived.
Furthermore, by coefficient clustering prosodic contour classes can be obtained
in a bottom-up way. Next to the stylization-based feature extraction also
standard F0 and energy measures (e.g. mean and variance) as well as rhythmic
aspects can be calculated. At the current state automatic annotation comprises:
segmentation into interpausal chunks, syllable nucleus extraction, and
unsupervised localization of prosodic phrase boundaries and prominent
syllables. F0 and partly also energy feature sets can be derived for: standard
measurements (as median and IQR), register in terms of F0 level and range,
prosodic boundaries, local contour shapes, bottom-up derived contour classes,
Gestalt of accent groups in terms of their deviation from higher level prosodic
units, as well as for rhythmic aspects quantifying the relation between F0 and
energy contours and prosodic event rates.
Blessing of dimensionality: mathematical foundations of the statistical physics of data
Authors: A. N. Gorban, I. Y. Tyukin
Date: 2018-01-10 00:00:00
Tasks: BIG-bench Machine Learning

The concentration of measure phenomena were discovered as the mathematical
background of statistical mechanics at the end of the XIX - beginning of the XX
century and were then explored in mathematics of the XX-XXI centuries. At the
beginning of the XXI century, it became clear that the proper utilisation of
these phenomena in machine learning might transform the curse of dimensionality
into the blessing of dimensionality.
  This paper summarises recently discovered phenomena of measure concentration
which drastically simplify some machine learning problems in high dimension,
and allow us to correct legacy artificial intelligence systems. The classical
concentration of measure theorems state that i.i.d. random points are
concentrated in a thin layer near a surface (a sphere or equators of a sphere,
an average or median level set of energy or another Lipschitz function, etc.).
  The new stochastic separation theorems describe the thin structure of these
thin layers: the random points are not only concentrated in a thin layer but
are all linearly separable from the rest of the set, even for exponentially
large random sets. The linear functionals for separation of points can be
selected in the form of the linear Fisher's discriminant.
  All artificial intelligence systems make errors. Non-destructive correction
requires separation of the situations (samples) with errors from the samples
corresponding to correct behaviour by a simple and robust classifier. The
stochastic separation theorems provide us by such classifiers and a
non-iterative (one-shot) procedure for learning.
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
Authors: Ludovic Trottier, Philippe Giguère, Brahim Chaib-Draa
Date: 2016-05-30 00:00:00
Tasks: Object Recognition, Scene Understanding

Object recognition is an important task for improving the ability of visual
systems to perform complex scene understanding. Recently, the Exponential
Linear Unit (ELU) has been proposed as a key component for managing bias shift
in Convolutional Neural Networks (CNNs), but defines a parameter that must be
set by hand. In this paper, we propose learning a parameterization of ELU in
order to learn the proper activation shape at each layer in the CNNs. Our
results on the MNIST, CIFAR-10/100 and ImageNet datasets using the NiN,
Overfeat, All-CNN and ResNet networks indicate that our proposed Parametric ELU
(PELU) has better performances than the non-parametric ELU. We have observed as
much as a 7.28% relative error improvement on ImageNet with the NiN network,
with only 0.0003% parameter increase. Our visual examination of the non-linear
behaviors adopted by Vgg using PELU shows that the network took advantage of
the added flexibility by learning different activations at different layers.