pwc-archive/papers-with-abstracts huggingface dataset:
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction
Authors: Edward Schmerling, Karen Leung, Wolf Vollprecht, Marco Pavone
Date: 2017-10-25 00:00:00
Tasks: Decision Making

This paper presents a method for constructing human-robot interaction
policies in settings where multimodality, i.e., the possibility of multiple
highly distinct futures, plays a critical role in decision making. We are
motivated in this work by the example of traffic weaving, e.g., at highway
on-ramps/off-ramps, where entering and exiting cars must swap lanes in a short
distance---a challenging negotiation even for experienced drivers due to the
inherent multimodal uncertainty of who will pass whom. Our approach is to learn
multimodal probability distributions over future human actions from a dataset
of human-human exemplars and perform real-time robot policy construction in the
resulting environment model through massively parallel sampling of human
responses to candidate robot action sequences. Direct learning of these
distributions is made possible by recent advances in the theory of conditional
variational autoencoders (CVAEs), whereby we learn action distributions
simultaneously conditioned on the present interaction history, as well as
candidate future robot actions in order to take into account response dynamics.
We demonstrate the efficacy of this approach with a human-in-the-loop
simulation of a traffic weaving scenario.
High Five: Improving Gesture Recognition by Embracing Uncertainty
Authors: Diman Zad Tootaghaj, Adrian Sampson, Todd Mytkowicz, Kathryn S. McKinley
Date: 2017-10-25 00:00:00
Tasks: Classification, General Classification, Gesture Recognition, Probabilistic Programming, Quantization, Vocal Bursts Intensity Prediction

Sensors on mobile devices---accelerometers, gyroscopes, pressure meters, and
GPS---invite new applications in gesture recognition, gaming, and fitness
tracking. However, programming them remains challenging because human gestures
captured by sensors are noisy. This paper illustrates that noisy gestures
degrade training and classification accuracy for gesture recognition in
state-of-the-art deterministic Hidden Markov Models (HMM). We introduce a new
statistical quantization approach that mitigates these problems by (1) during
training, producing gesture-specific codebooks, HMMs, and error models for
gesture sequences; and (2) during classification, exploiting the error model to
explore multiple feasible HMM state sequences. We implement classification in
Uncertain<t>, a probabilistic programming system that encapsulates HMMs and
error models and then automates sampling and inference in the runtime.
Uncertain<T> developers directly express a choice of application-specific
trade-off between recall and precision at gesture recognition time, rather than
at training time. We demonstrate benefits in configurability, precision,
recall, and recognition on two data sets with 25 gestures from 28 people and
4200 total gestures. Incorporating gesture error more accurately in modeling
improves the average recognition rate of 20 gestures from 34\% in prior work to
62\%. Incorporating the error model during classification further improves the
average gesture recognition rate to 71\%. As far as we are aware, no prior work
shows how to generate an HMM error model during training and use it to improve
classification rates.
Malware Detection by Eating a Whole EXE
Authors: Edward Raff, Jon Barker, Jared Sylvester, Robert Brandon, Bryan Catanzaro, Charles Nicholas
Date: 2017-10-25 00:00:00
Tasks: Malware Detection

In this work we introduce malware detection from raw byte sequences as a
fruitful research area to the larger machine learning community. Building a
neural network for such a problem presents a number of interesting challenges
that have not occurred in tasks such as image processing or NLP. In particular,
we note that detection from raw bytes presents a sequence problem with over two
million time steps and a problem where batch normalization appear to hinder the
learning process. We present our initial work in building a solution to tackle
this problem, which has linear complexity dependence on the sequence length,
and allows for interpretable sub-regions of the binary to be identified. In
doing so we will discuss the many challenges in building a neural network to
process data at this scale, and the methods we used to work around them.
A Markov Chain Theory Approach to Characterizing the Minimax Optimality of Stochastic Gradient Descent (for Least Squares)
Authors: Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, Venkata Krishna Pillutla, Aaron Sidford
Date: 2017-10-25 00:00:00

This work provides a simplified proof of the statistical minimax optimality
of (iterate averaged) stochastic gradient descent (SGD), for the special case
of least squares. This result is obtained by analyzing SGD as a stochastic
process and by sharply characterizing the stationary covariance matrix of this
process. The finite rate optimality characterization captures the constant
factors and addresses model mis-specification.
DPCA: Dimensionality Reduction for Discriminative Analytics of Multiple Large-Scale Datasets
Authors: Gang Wang, Jia Chen, Georgios B. Giannakis
Date: 2017-10-25 00:00:00
Tasks: Dimensionality Reduction

Principal component analysis (PCA) has well-documented merits for data
extraction and dimensionality reduction. PCA deals with a single dataset at a
time, and it is challenged when it comes to analyzing multiple datasets. Yet in
certain setups, one wishes to extract the most significant information of one
dataset relative to other datasets. Specifically, the interest may be on
identifying, namely extracting features that are specific to a single target
dataset but not the others. This paper develops a novel approach for such
so-termed discriminative data analysis, and establishes its optimality in the
least-squares (LS) sense under suitable data modeling assumptions. The
criterion reveals linear combinations of variables by maximizing the ratio of
the variance of the target data to that of the remainders. The novel approach
solves a generalized eigenvalue problem by performing SVD just once. Numerical
tests using synthetic and real datasets showcase the merits of the proposed
approach relative to its competing alternatives.
Graph Convolutional Matrix Completion
Authors: Rianne van den Berg, Thomas N. Kipf, Max Welling
Date: 2017-06-07 00:00:00
Tasks: Collaborative Filtering, Link Prediction, Matrix Completion, Recommendation Systems

We consider matrix completion for recommender systems from the point of view
of link prediction on graphs. Interaction data such as movie ratings can be
represented by a bipartite user-item graph with labeled edges denoting observed
ratings. Building on recent progress in deep learning on graph-structured data,
we propose a graph auto-encoder framework based on differentiable message
passing on the bipartite interaction graph. Our model shows competitive
performance on standard collaborative filtering benchmarks. In settings where
complimentary feature information or structured data such as a social network
is available, our framework outperforms recent state-of-the-art methods.
Frequency Based Index Estimating the Subclusters' Connection Strength
Authors: Lukas Pastorek
Date: 2017-10-19 00:00:00
Tasks: Clustering, Management

In this paper, a frequency coefficient based on the Sen-Shorrocks-Thon (SST)
poverty index notion is proposed. The clustering SST index can be used as the
method for determination of the connection between similar neighbor
sub-clusters. Consequently, connections can reveal existence of natural
homogeneous. Through estimation of the connection strength, we can also verify
information about the estimated number of natural clusters that is necessary
assumption of efficient market segmentation and campaign management and
financial decisions. The index can be used as the complementary tool for the
U-matrix visualization. The index is tested on an artificial dataset with known
parameters and compared with results obtained by the Unified-distance matrix
method.
Real-Time Automatic Fetal Brain Extraction in Fetal MRI by Deep Learning
Authors: Seyed Sadegh Mohseni Salehi, Seyed Raein Hashemi, Clemente Velasco-Annis, Abdelhakim Ouaalam, Judy A. Estroff, Deniz Erdogmus, Simon K. Warfield, Ali Gholipour
Date: 2017-10-25 00:00:00
Tasks: 3D Reconstruction, Brain Segmentation, Motion Detection, Segmentation

Brain segmentation is a fundamental first step in neuroimage analysis. In the
case of fetal MRI, it is particularly challenging and important due to the
arbitrary orientation of the fetus, organs that surround the fetal head, and
intermittent fetal motion. Several promising methods have been proposed but are
limited in their performance in challenging cases and in real-time
segmentation. We aimed to develop a fully automatic segmentation method that
independently segments sections of the fetal brain in 2D fetal MRI slices in
real-time. To this end, we developed and evaluated a deep fully convolutional
neural network based on 2D U-net and autocontext, and compared it to two
alternative fast methods based on 1) a voxelwise fully convolutional network
and 2) a method based on SIFT features, random forest and conditional random
field. We trained the networks with manual brain masks on 250 stacks of
training images, and tested on 17 stacks of normal fetal brain images as well
as 18 stacks of extremely challenging cases based on extreme motion, noise, and
severely abnormal brain shape. Experimental results show that our U-net
approach outperformed the other methods and achieved average Dice metrics of
96.52% and 78.83% in the normal and challenging test sets, respectively. With
an unprecedented performance and a test run time of about 1 second, our network
can be used to segment the fetal brain in real-time while fetal MRI slices are
being acquired. This can enable real-time motion tracking, motion detection,
and 3D reconstruction of fetal brain MRI.
A unified framework for manifold landmarking
Authors: Hongteng Xu, Licheng Yu, Mark Davenport, Hongyuan Zha
Date: 2017-10-25 00:00:00

The success of semi-supervised manifold learning is highly dependent on the
quality of the labeled samples. Active manifold learning aims to select and
label representative landmarks on a manifold from a given set of samples to
improve semi-supervised manifold learning. In this paper, we propose a novel
active manifold learning method based on a unified framework of manifold
landmarking. In particular, our method combines geometric manifold landmarking
methods with algebraic ones. We achieve this by using the Gershgorin circle
theorem to construct an upper bound on the learning error that depends on the
landmarks and the manifold's alignment matrix in a way that captures both the
geometric and algebraic criteria. We then attempt to select landmarks so as to
minimize this bound by iteratively deleting the Gershgorin circles
corresponding to the selected landmarks. We also analyze the complexity,
scalability, and robustness of our method through simulations, and demonstrate
its superiority compared to existing methods. Experiments in regression and
classification further verify that our method performs better than its
competitors.
A Minimax Optimal Algorithm for Crowdsourcing
Authors: Thomas Bonald, Richard Combes
Date: 2016-06-01 00:00:00
Proceeding: NeurIPS 2017 12

We consider the problem of accurately estimating the reliability of workers
based on noisy labels they provide, which is a fundamental question in
crowdsourcing. We propose a novel lower bound on the minimax estimation error
which applies to any estimation procedure. We further propose Triangular
Estimation (TE), an algorithm for estimating the reliability of workers. TE has
low complexity, may be implemented in a streaming setting when labels are
provided by workers in real time, and does not rely on an iterative procedure.
We further prove that TE is minimax optimal and matches our lower bound. We
conclude by assessing the performance of TE and other state-of-the-art
algorithms on both synthetic and real-world data sets.
Exploring the Use of Text Classification in the Legal Domain
Authors: Octavia-Maria Sulea, Marcos Zampieri, Shervin Malmasi, Mihaela Vela, Liviu P. Dinu, Josef van Genabith
Date: 2017-10-25 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

In this paper, we investigate the application of text classification methods
to support law professionals. We present several experiments applying machine
learning techniques to predict with high accuracy the ruling of the French
Supreme Court and the law area to which a case belongs to. We also investigate
the influence of the time period in which a ruling was made on the form of the
case description and the extent to which we need to mask information in a full
case ruling to automatically obtain training and test data that resembles case
descriptions. We developed a mean probability ensemble system combining the
output of multiple SVM classifiers. We report results of 98% average F1 score
in predicting a case ruling, 96% F1 score for predicting the law area of a
case, and 87.07% F1 score on estimating the date of a ruling.
Feature learning in feature-sample networks using multi-objective optimization
Authors: Filipe Alves Neto Verri, Renato Tinós, Liang Zhao
Date: 2017-10-25 00:00:00
Tasks: BIG-bench Machine Learning

Data and knowledge representation are fundamental concepts in machine
learning. The quality of the representation impacts the performance of the
learning model directly. Feature learning transforms or enhances raw data to
structures that are effectively exploited by those models. In recent years,
several works have been using complex networks for data representation and
analysis. However, no feature learning method has been proposed for such
category of techniques. Here, we present an unsupervised feature learning
mechanism that works on datasets with binary features. First, the dataset is
mapped into a feature--sample network. Then, a multi-objective optimization
process selects a set of new vertices to produce an enhanced version of the
network. The new features depend on a nonlinear function of a combination of
preexisting features. Effectively, the process projects the input data into a
higher-dimensional space. To solve the optimization problem, we design two
metaheuristics based on the lexicographic genetic algorithm and the improved
strength Pareto evolutionary algorithm (SPEA2). We show that the enhanced
network contains more information and can be exploited to improve the
performance of machine learning methods. The advantages and disadvantages of
each optimization strategy are discussed.
Gated Orthogonal Recurrent Units: On Learning to Forget
Authors: Li Jing, Caglar Gulcehre, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljačić, Yoshua Bengio
Date: 2017-06-08 00:00:00
Tasks: Denoising, Question Answering

We present a novel recurrent neural network (RNN) based model that combines
the remembering ability of unitary RNNs with the ability of gated RNNs to
effectively forget redundant/irrelevant information in its memory. We achieve
this by extending unitary RNNs with a gating mechanism. Our model is able to
outperform LSTMs, GRUs and Unitary RNNs on several long-term dependency
benchmark tasks. We empirically both show the orthogonal/unitary RNNs lack the
ability to forget and also the ability of GORU to simultaneously remember long
term dependencies while forgetting irrelevant information. This plays an
important role in recurrent neural networks. We provide competitive results
along with an analysis of our model on many natural sequential tasks including
the bAbI Question Answering, TIMIT speech spectrum prediction, Penn TreeBank,
and synthetic tasks that involve long-term dependencies such as algorithmic,
parenthesis, denoising and copying tasks.
Dynamic Integration of Background Knowledge in Neural NLU Systems
Authors: Dirk Weissenborn, Tomáš Kočiský, Chris Dyer
Date: 2017-06-08 00:00:00
Proceeding: ICLR 2018 1
Tasks: Common Sense Reasoning, Natural Language Inference, Natural Language Understanding, Question Answering, RTE

Common-sense and background knowledge is required to understand natural
language, but in most neural natural language understanding (NLU) systems, this
knowledge must be acquired from training corpora during learning, and then it
is static at test time. We introduce a new architecture for the dynamic
integration of explicit background knowledge in NLU models. A general-purpose
reading module reads background knowledge in the form of free-text statements
(together with task-specific text inputs) and yields refined word
representations to a task-specific NLU architecture that reprocesses the task
inputs with these representations. Experiments on document question answering
(DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness
and flexibility of the approach. Analysis shows that our model learns to
exploit knowledge in a semantically appropriate way.
How priors of initial hyperparameters affect Gaussian process regression models
Authors: Zexun Chen, Bo wang
Date: 2016-05-25 00:00:00
Tasks: GPR, regression

The hyperparameters in Gaussian process regression (GPR) model with a
specified kernel are often estimated from the data via the maximum marginal
likelihood. Due to the non-convexity of marginal likelihood with respect to the
hyperparameters, the optimization may not converge to the global maxima. A
common approach to tackle this issue is to use multiple starting points
randomly selected from a specific prior distribution. As a result the choice of
prior distribution may play a vital role in the predictability of this
approach. However, there exists little research in the literature to study the
impact of the prior distributions on the hyperparameter estimation and the
performance of GPR. In this paper, we provide the first empirical study on this
problem using simulated and real data experiments. We consider different types
of priors for the initial values of hyperparameters for some commonly used
kernels and investigate the influence of the priors on the predictability of
GPR models. The results reveal that, once a kernel is chosen, different priors
for the initial hyperparameters have no significant impact on the performance
of GPR prediction, despite that the estimates of the hyperparameters are very
different to the true values in some cases.
Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoning
Authors: Shoaib Jameel, Steven Schockaert
Date: 2016-02-18 00:00:00
Tasks: Entity Embeddings

Conceptual spaces are geometric representations of conceptual knowledge, in
which entities correspond to points, natural properties correspond to convex
regions, and the dimensions of the space correspond to salient features. While
conceptual spaces enable elegant models of various cognitive phenomena, the
lack of automated methods for constructing such representations have so far
limited their application in artificial intelligence. To address this issue, we
propose a method which learns a vector-space embedding of entities from
Wikipedia and constrains this embedding such that entities of the same semantic
type are located in some lower-dimensional subspace. We experimentally
demonstrate the usefulness of these subspaces as (approximate) conceptual space
representations by showing, among others, that important features can be
modelled as directions and that natural properties tend to correspond to convex
regions.
Supervised Classification: Quite a Brief Overview
Authors: Marco Loog
Date: 2017-10-25 00:00:00
Tasks: Classification, General Classification

The original problem of supervised classification considers the task of
automatically assigning objects to their respective classes on the basis of
numerical measurements derived from these objects. Classifiers are the tools
that implement the actual functional mapping from these measurements---also
called features or inputs---to the so-called class label---or output. The
fields of pattern recognition and machine learning study ways of constructing
such classifiers. The main idea behind supervised methods is that of learning
from examples: given a number of example input-output relations, to what extent
can the general mapping be learned that takes any new and unseen feature vector
to its correct class? This chapter provides a basic introduction to the
underlying ideas of how to come to a supervised classification problem. In
addition, it provides an overview of some specific classification techniques,
delves into the issues of object representation and classifier evaluation, and
(very) briefly covers some variations on the basic supervised classification
task that may also be of interest to the practitioner.
MEBoost: Variable Selection in the Presence of Measurement Error
Authors: Benjamin Brown, Timothy Weaver, Julian Wolfson
Date: 2017-01-09 00:00:00
Tasks: Nutrition, Variable Selection

We present a novel method for variable selection in regression models when
covariates are measured with error. The iterative algorithm we propose,
MEBoost, follows a path defined by estimating equations that correct for
covariate measurement error. Via simulation, we evaluated our method and
compare its performance to the recently-proposed Convex Conditioned Lasso
(CoCoLasso) and to the "naive" Lasso which does not correct for measurement
error. Increasing the degree of measurement error increased prediction error
and decreased the probability of accurate covariate selection, but this loss of
accuracy was least pronounced when using MEBoost. We illustrate the use of
MEBoost in practice by analyzing data from the Box Lunch Study, a clinical
trial in nutrition where several variables are based on self-report and hence
measured with error.
The Heterogeneous Ensembles of Standard Classification Algorithms (HESCA): the Whole is Greater than the Sum of its Parts
Authors: James Large, Jason Lines, Anthony Bagnall
Date: 2017-10-25 00:00:00
Tasks: Decision Making, General Classification

Building classification models is an intrinsically practical exercise that
requires many design decisions prior to deployment. We aim to provide some
guidance in this decision making process. Specifically, given a classification
problem with real valued attributes, we consider which classifier or family of
classifiers should one use. Strong contenders are tree based homogeneous
ensembles, support vector machines or deep neural networks. All three families
of model could claim to be state-of-the-art, and yet it is not clear when one
is preferable to the others. Our extensive experiments with over 200 data sets
from two distinct archives demonstrate that, rather than choose a single family
and expend computing resources on optimising that model, it is significantly
better to build simpler versions of classifiers from each family and ensemble.
We show that the Heterogeneous Ensembles of Standard Classification Algorithms
(HESCA), which ensembles based on error estimates formed on the train data, is
significantly better (in terms of error, balanced error, negative log
likelihood and area under the ROC curve) than its individual components,
picking the component that is best on train data, and a support vector machine
tuned over 1089 different parameter configurations. We demonstrate HESCA+,
which contains a deep neural network, a support vector machine and two decision
tree forests, is significantly better than its components, picking the best
component, and HESCA. We analyse the results further and find that HESCA and
HESCA+ are of particular value when the train set size is relatively small and
the problem has multiple classes. HESCA is a fast approach that is, on average,
as good as state-of-the-art classifiers, whereas HESCA+ is significantly better
than average and represents a strong benchmark for future research.