pwc-archive/papers-with-abstracts huggingface dataset:
Machine Learning for the Geosciences: Challenges and Opportunities
Authors: Anuj Karpatne, Imme Ebert-Uphoff, Sai Ravela, Hassan Ali Babaie, Vipin Kumar
Date: 2017-11-13 00:00:00
Tasks: BIG-bench Machine Learning

Geosciences is a field of great societal relevance that requires solutions to
several urgent problems facing our humanity and the planet. As geosciences
enters the era of big data, machine learning (ML) -- that has been widely
successful in commercial domains -- offers immense potential to contribute to
problems in geosciences. However, problems in geosciences have several unique
challenges that are seldom found in traditional applications, requiring novel
problem formulations and methodologies in machine learning. This article
introduces researchers in the machine learning (ML) community to these
challenges offered by geoscience problems and the opportunities that exist for
advancing both machine learning and geosciences. We first highlight typical
sources of geoscience data and describe their properties that make it
challenging to use traditional machine learning techniques. We then describe
some of the common categories of geoscience problems where machine learning can
play a role, and discuss some of the existing efforts and promising directions
for methodological development in machine learning. We conclude by discussing
some of the emerging research themes in machine learning that are applicable
across all problems in the geosciences, and the importance of a deep
collaboration between machine learning and geosciences for synergistic
advancements in both disciplines.
Person Recognition using Smartphones' Accelerometer Data
Authors: Thingom Bishal Singha, Rajsekhar Kumar Nath, A. V. Narsimhadhan
Date: 2017-11-13 00:00:00
Tasks: Person Recognition

Smartphones have become quite pervasive in various aspects of our daily
lives. They have become important links to a host of important data and
applications, which if compromised, can lead to disastrous results. Due to
this, today's smartphones are equipped with multiple layers of authentication
modules. However, there still lies the need for a viable and unobtrusive layer
of security which can perform the task of user authentication using resources
which are cost-efficient and widely available on smartphones. In this work, we
propose a method to recognize users using data from a phone's embedded
accelerometer sensors. Features encapsulating information from both time and
frequency domains are extracted from walking data samples, and are used to
build a Random Forest ensemble classification model. Based on the experimental
results, the resultant model delivers an accuracy of 0.9679 and Area under
Curve (AUC) of 0.9822.
Training Quantized Nets: A Deeper Understanding
Authors: Hao Li, Soham De, Zheng Xu, Christoph Studer, Hanan Samet, Tom Goldstein
Date: 2017-06-07 00:00:00
Proceeding: NeurIPS 2017 12

Currently, deep neural networks are deployed on low-power portable devices by
first training a full-precision model using powerful hardware, and then
deriving a corresponding low-precision model for efficient inference on such
systems. However, training models directly with coarsely quantized weights is a
key step towards learning on embedded platforms that have limited computing
resources, memory capacity, and power consumption. Numerous recent publications
have studied methods for training quantized networks, but these studies have
mostly been empirical. In this work, we investigate training methods for
quantized neural networks from a theoretical viewpoint. We first explore
accuracy guarantees for training methods under convexity assumptions. We then
look at the behavior of these algorithms for non-convex problems, and show that
training algorithms that exploit high-precision representations have an
important greedy search phase that purely quantized training methods lack,
which explains the difficulty of training using low-precision arithmetic.
Weightless: Lossy Weight Encoding For Deep Neural Network Compression
Authors: Brandon Reagen, Udit Gupta, Robert Adolf, Michael M. Mitzenmacher, Alexander M. Rush, Gu-Yeon Wei, David Brooks
Date: 2017-11-13 00:00:00
Tasks: Model Compression, Neural Network Compression, Quantization

The large memory requirements of deep neural networks limit their deployment
and adoption on many devices. Model compression methods effectively reduce the
memory requirements of these models, usually through applying transformations
such as weight pruning or quantization. In this paper, we present a novel
scheme for lossy weight encoding which complements conventional compression
techniques. The encoding is based on the Bloomier filter, a probabilistic data
structure that can save space at the cost of introducing random errors.
Leveraging the ability of neural networks to tolerate these imperfections and
by re-training around the errors, the proposed technique, Weightless, can
compress DNN weights by up to 496x with the same model accuracy. This results
in up to a 1.51x improvement over the state-of-the-art.
Tensor Decompositions for Modeling Inverse Dynamics
Authors: Stephan Baier, Volker Tresp
Date: 2017-11-13 00:00:00
Tasks: Knowledge Graphs, Multi-class Classification, regression, Relational Reasoning, Tensor Decomposition

Modeling inverse dynamics is crucial for accurate feedforward robot control.
The model computes the necessary joint torques, to perform a desired movement.
The highly non-linear inverse function of the dynamical system can be
approximated using regression techniques. We propose as regression method a
tensor decomposition model that exploits the inherent three-way interaction of
positions x velocities x accelerations. Most work in tensor factorization has
addressed the decomposition of dense tensors. In this paper, we build upon the
decomposition of sparse tensors, with only small amounts of nonzero entries.
The decomposition of sparse tensors has successfully been used in relational
learning, e.g., the modeling of large knowledge graphs. Recently, the approach
has been extended to multi-class classification with discrete input variables.
Representing the data in high dimensional sparse tensors enables the
approximation of complex highly non-linear functions. In this paper we show how
the decomposition of sparse tensors can be applied to regression problems.
Furthermore, we extend the method to continuous inputs, by learning a mapping
from the continuous inputs to the latent representations of the tensor
decomposition, using basis functions. We evaluate our proposed model on a
dataset with trajectories from a seven degrees of freedom SARCOS robot arm. Our
experimental results show superior performance of the proposed functional
tensor model, compared to challenging state-of-the art methods.
Attention-based Information Fusion using Multi-Encoder-Decoder Recurrent Neural Networks
Authors: Stephan Baier, Sigurd Spieckermann, Volker Tresp
Date: 2017-11-13 00:00:00
Tasks: Decoder

With the rising number of interconnected devices and sensors, modeling
distributed sensor networks is of increasing interest. Recurrent neural
networks (RNN) are considered particularly well suited for modeling sensory and
streaming data. When predicting future behavior, incorporating information from
neighboring sensor stations is often beneficial. We propose a new RNN based
architecture for context specific information fusion across multiple spatially
distributed sensor stations. Hereby, latent representations of multiple local
models, each modeling one sensor station, are jointed and weighted, according
to their importance for the prediction. The particular importance is assessed
depending on the current context using a separate attention function. We
demonstrate the effectiveness of our model on three different real-world sensor
network datasets.
Model Criticism in Latent Space
Authors: Sohan Seth, Iain Murray, Christopher K. I. Williams
Date: 2017-11-13 00:00:00
Tasks: Gaussian Processes, model

Model criticism is usually carried out by assessing if replicated data
generated under the fitted model looks similar to the observed data, see e.g.
Gelman, Carlin, Stern, and Rubin [2004, p. 165]. This paper presents a method
for latent variable models by pulling back the data into the space of latent
variables, and carrying out model criticism in that space. Making use of a
model's structure enables a more direct assessment of the assumptions made in
the prior and likelihood. We demonstrate the method with examples of model
criticism in latent space applied to factor analysis, linear dynamical systems
and Gaussian processes.
Doubly Stochastic Variational Inference for Deep Gaussian Processes
Authors: Hugh Salimbeni, Marc Deisenroth
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Gaussian Processes, General Classification, regression, Variational Inference

Gaussian processes (GPs) are a good choice for function approximation as they
are flexible, robust to over-fitting, and provide well-calibrated predictive
uncertainty. Deep Gaussian processes (DGPs) are multi-layer generalisations of
GPs, but inference in these models has proved challenging. Existing approaches
to inference in DGP models assume approximate posteriors that force
independence between the layers, and do not work well in practice. We present a
doubly stochastic variational inference algorithm, which does not force
independence between layers. With our method of inference we demonstrate that a
DGP model can be used effectively on data ranging in size from hundreds to a
billion points. We provide strong empirical evidence that our inference scheme
for DGPs works well in practice in both classification and regression.
PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference
Authors: Jonathan H. Huggins, Ryan P. Adams, Tamara Broderick
Date: 2017-09-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression

Generalized linear models (GLMs) -- such as logistic regression, Poisson
regression, and robust regression -- provide interpretable models for diverse
data types. Probabilistic approaches, particularly Bayesian ones, allow
coherent estimates of uncertainty, incorporation of prior information, and
sharing of power across experiments via hierarchical models. In practice,
however, the approximate Bayesian methods necessary for inference have either
failed to scale to large data sets or failed to provide theoretical guarantees
on the quality of inference. We propose a new approach based on constructing
polynomial approximate sufficient statistics for GLMs (PASS-GLM). We
demonstrate that our method admits a simple algorithm as well as trivial
streaming and distributed extensions that do not compound error across
computations. We provide theoretical guarantees on the quality of point (MAP)
estimates, the approximate posterior, and posterior mean and uncertainty
estimates. We validate our approach empirically in the case of logistic
regression using a quadratic approximation and show competitive performance
with stochastic gradient descent, MCMC, and the Laplace approximation in terms
of speed and multiple measures of accuracy -- including on an advertising data
set with 40 million data points and 20,000 covariates.
Provably efficient neural network representation for image classification
Authors: Yichen Huang
Date: 2017-11-13 00:00:00
Tasks: Classification, Efficient Neural Network, General Classification, image-classification, Image Classification

The state-of-the-art approaches for image classification are based on neural
networks. Mathematically, the task of classifying images is equivalent to
finding the function that maps an image to the label it is associated with. To
rigorously establish the success of neural network methods, we should first
prove that the function has an efficient neural network representation, and
then design provably efficient training algorithms to find such a
representation. Here, we achieve the first goal based on a set of assumptions
about the patterns in the images. The validity of these assumptions is very
intuitive in many image classification problems, including but not limited to,
recognizing handwritten digits.
Convolutional neural networks pretrained on large face recognition datasets for emotion classification from video
Authors: Boris Knyazev, Roman Shvetsov, Natalia Efremova, Artem Kuharenko
Date: 2017-11-13 00:00:00
Tasks: Emotion Classification, Emotion Recognition, Face Recognition, General Classification

In this paper we describe a solution to our entry for the emotion recognition
challenge EmotiW 2017. We propose an ensemble of several models, which capture
spatial and audio features from videos. Spatial features are captured by
convolutional neural networks, pretrained on large face recognition datasets.
We show that usage of strong industry-level face recognition networks increases
the accuracy of emotion recognition. Using our ensemble we improve on the
previous best result on the test set by about 1 %, achieving a 60.03 %
classification accuracy without any use of visual temporal information.
Vertebral body segmentation with GrowCut: Initial experience, workflow and practical application
Authors: Jan Egger, Christopher Nimsky, Xiaojun Chen
Date: 2017-11-13 00:00:00
Tasks: Segmentation

In this contribution, we used the GrowCut segmentation algorithm publicly
available in three-dimensional Slicer for three-dimensional segmentation of
vertebral bodies. To the best of our knowledge, this is the first time that the
GrowCut method has been studied for the usage of vertebral body segmentation.
In brief, we found that the GrowCut segmentation times were consistently less
than the manual segmentation times. Hence, GrowCut provides an alternative to a
manual slice-by-slice segmentation process.
Deep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework
Authors: Qianli Ma, Lifeng Shen, Garrison W. Cottrell
Date: 2017-11-13 00:00:00
Tasks: Time Series, Time Series Analysis

As an efficient recurrent neural network (RNN) model, reservoir computing
(RC) models, such as Echo State Networks, have attracted widespread attention
in the last decade. However, while they have had great success with time series
data [1], [2], many time series have a multiscale structure, which a
single-hidden-layer RC model may have difficulty capturing. In this paper, we
propose a novel hierarchical reservoir computing framework we call Deep Echo
State Networks (Deep-ESNs). The most distinctive feature of a Deep-ESN is its
ability to deal with time series through hierarchical projections.
Specifically, when an input time series is projected into the high-dimensional
echo-state space of a reservoir, a subsequent encoding layer (e.g., a PCA,
autoencoder, or a random projection) can project the echo-state representations
into a lower-dimensional space. These low-dimensional representations can then
be processed by another ESN. By using projection layers and encoding layers
alternately in the hierarchical framework, a Deep-ESN can not only attenuate
the effects of the collinearity problem in ESNs, but also fully take advantage
of the temporal kernel property of ESNs to explore multiscale dynamics of time
series. To fuse the multiscale representations obtained by each reservoir, we
add connections from each encoding layer to the last output layer. Theoretical
analyses prove that stability of a Deep-ESN is guaranteed by the echo state
property (ESP), and the time complexity is equivalent to a conventional ESN.
Experimental results on some artificial and real world time series demonstrate
that Deep-ESNs can capture multiscale dynamics, and outperform both standard
ESNs and previous hierarchical ESN-based models.
SVSGAN: Singing Voice Separation via Generative Adversarial Network
Authors: Zhe-Cheng Fan, Yen-Lin Lai, Jyh-Shing Roger Jang
Date: 2017-10-31 00:00:00
Tasks: Generative Adversarial Network

Separating two sources from an audio mixture is an important task with many
applications. It is a challenging problem since only one signal channel is
available for analysis. In this paper, we propose a novel framework for singing
voice separation using the generative adversarial network (GAN) with a
time-frequency masking function. The mixture spectra is considered to be a
distribution and is mapped to the clean spectra which is also considered a
distribtution. The approximation of distributions between mixture spectra and
clean spectra is performed during the adversarial training process. In contrast
with current deep learning approaches for source separation, the parameters of
the proposed framework are first initialized in a supervised setting and then
optimized by the training procedure of GAN in an unsupervised setting.
Experimental results on three datasets (MIR-1K, iKala and DSD100) show that
performance can be improved by the proposed framework consisting of
conventional networks.
Phonemic and Graphemic Multilingual CTC Based Speech Recognition
Authors: Markus Müller, Sebastian Stüker, Alex Waibel
Date: 2017-11-13 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Training automatic speech recognition (ASR) systems requires large amounts of
data in the target language in order to achieve good performance. Whereas large
training corpora are readily available for languages like English, there exists
a long tail of languages which do suffer from a lack of resources. One method
to handle data sparsity is to use data from additional source languages and
build a multilingual system. Recently, ASR systems based on recurrent neural
networks (RNNs) trained with connectionist temporal classification (CTC) have
gained substantial research interest. In this work, we extended our previous
approach towards training CTC-based systems multilingually. Our systems feature
a global phone set, based on the joint phone sets of each source language. We
evaluated the use of different language combinations as well as the addition of
Language Feature Vectors (LFVs). As contrastive experiment, we built systems
based on graphemes as well. Systems having a multilingual phone set are known
to suffer in performance compared to their monolingual counterparts. With our
proposed approach, we could reduce the gap between these mono- and multilingual
setups, using either graphemes or phonemes.
Solving the Resource Constrained Project Scheduling Problem Using the Parallel Tabu Search Designed for the CUDA Platform
Authors: Libor Bukata, Premysl Sucha, Zdenek Hanzalek
Date: 2017-11-13 00:00:00
Tasks: CPU, GPU, Scheduling

In the paper, a parallel Tabu Search algorithm for the Resource Constrained
Project Scheduling Problem is proposed. To deal with this NP-hard combinatorial
problem many optimizations have been performed. For example, a resource
evaluation algorithm is selected by a heuristic and an effective Tabu List was
designed. In addition to that, a capacity-indexed resource evaluation algorithm
was proposed and the GPU (Graphics Processing Unit) version uses a homogeneous
model to reduce the required communication bandwidth. According to the
experiments, the GPU version outperforms the optimized parallel CPU version
with respect to the computational time and the quality of solutions. In
comparison with other existing heuristics, the proposed solution often gives
better quality solutions.
Kernel Two-Sample Hypothesis Testing Using Kernel Set Classification
Authors: Hamed Masnadi-Shirazi
Date: 2017-06-18 00:00:00
Tasks: Classification, General Classification, Two-sample testing, Vocal Bursts Valence Prediction

The two-sample hypothesis testing problem is studied for the challenging
scenario of high dimensional data sets with small sample sizes. We show that
the two-sample hypothesis testing problem can be posed as a one-class set
classification problem. In the set classification problem the goal is to
classify a set of data points that are assumed to have a common class. We prove
that the average probability of error given a set is less than or equal to the
Bayes error and decreases as a power of $n$ number of sample data points in the
set. We use the positive definite Set Kernel for directly mapping sets of data
to an associated Reproducing Kernel Hilbert Space, without the need to learn a
probability distribution. We specifically solve the two-sample hypothesis
testing problem using a one-class SVM in conjunction with the proposed Set
Kernel. We compare the proposed method with the Maximum Mean Discrepancy,
F-Test and T-Test methods on a number of challenging simulated high dimensional
and small sample size data. We also perform two-sample hypothesis testing
experiments on six cancer gene expression data sets and achieve zero type-I and
type-II error results on all data sets.
The identity of information: how deterministic dependencies constrain information synergy and redundancy
Authors: Daniel Chicharro, Giuseppe Pica, Stefano Panzeri
Date: 2017-11-13 00:00:00

Understanding how different information sources together transmit information
is crucial in many domains. For example, understanding the neural code requires
characterizing how different neurons contribute unique, redundant, or
synergistic pieces of information about sensory or behavioral variables.
Williams and Beer (2010) proposed a partial information decomposition (PID)
which separates the mutual information that a set of sources contains about a
set of targets into nonnegative terms interpretable as these pieces.
Quantifying redundancy requires assigning an identity to different information
pieces, to assess when information is common across sources. Harder et al.
(2013) proposed an identity axiom stating that there cannot be redundancy
between two independent sources about a copy of themselves. However,
Bertschinger et al. (2012) showed that with a deterministically related
sources-target copy this axiom is incompatible with ensuring PID nonnegativity.
Here we study systematically the effect of deterministic target-sources
dependencies. We introduce two synergy stochasticity axioms that generalize the
identity axiom, and we derive general expressions separating stochastic and
deterministic PID components. Our analysis identifies how negative terms can
originate from deterministic dependencies and shows how different assumptions
on information identity, implicit in the stochasticity and identity axioms,
determine the PID structure. The implications for studying neural coding are
discussed.
Learning to Attend, Copy, and Generate for Session-Based Query Suggestion
Authors: Mostafa Dehghani, Sascha Rothe, Enrique Alfonseca, Pascal Fleury
Date: 2017-08-11 00:00:00
Tasks: Decoder

Users try to articulate their complex information needs during search
sessions by reformulating their queries. To make this process more effective,
search engines provide related queries to help users in specifying the
information need in their search process. In this paper, we propose a
customized sequence-to-sequence model for session-based query suggestion. In
our model, we employ a query-aware attention mechanism to capture the structure
of the session context. is enables us to control the scope of the session from
which we infer the suggested next query, which helps not only handle the noisy
data but also automatically detect session boundaries. Furthermore, we observe
that, based on the user query reformulation behavior, within a single session a
large portion of query terms is retained from the previously submitted queries
and consists of mostly infrequent or unseen terms that are usually not included
in the vocabulary. We therefore empower the decoder of our model to access the
source words from the session context during decoding by incorporating a copy
mechanism. Moreover, we propose evaluation metrics to assess the quality of the
generative models for query suggestion. We conduct an extensive set of
experiments and analysis. e results suggest that our model outperforms the
baselines both in terms of the generating queries and scoring candidate queries
for the task of query suggestion.