pwc-archive/papers-with-abstracts huggingface dataset:
How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models
Authors: Kathrin Grosse, David Pfaff, Michael Thomas Smith, Michael Backes
Date: 2017-11-17 00:00:00
Tasks: Bayesian Inference, Gaussian Processes

Machine learning models are vulnerable to Adversarial Examples: minor
perturbations to input samples intended to deliberately cause
misclassification. Current defenses against adversarial examples, especially
for Deep Neural Networks (DNN), are primarily derived from empirical
developments, and their security guarantees are often only justified
retroactively. Many defenses therefore rely on hidden assumptions that are
subsequently subverted by increasingly elaborate attacks. This is not
surprising: deep learning notoriously lacks a comprehensive mathematical
framework to provide meaningful guarantees.
  In this paper, we leverage Gaussian Processes to investigate adversarial
examples in the framework of Bayesian inference. Across different models and
datasets, we find deviating levels of uncertainty reflect the perturbation
introduced to benign samples by state-of-the-art attacks, including novel
white-box attacks on Gaussian Processes. Our experiments demonstrate that even
unoptimized uncertainty thresholds already reject adversarial examples in many
scenarios.
  Comment: Thresholds can be broken in a modified attack, which was done in
arXiv:1812.02606 (The limitations of model uncertainty in adversarial
settings).
Constrained Convolutional-Recurrent Networks to Improve Speech Quality with Low Impact on Recognition Accuracy
Authors: Rasool Fakoor, Xiaodong He, Ivan Tashev, Shuayb Zarar
Date: 2018-02-16 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, Speech Enhancement

For a speech-enhancement algorithm, it is highly desirable to simultaneously
improve perceptual quality and recognition rate. Thanks to computational costs
and model complexities, it is challenging to train a model that effectively
optimizes both metrics at the same time. In this paper, we propose a method for
speech enhancement that combines local and global contextual structures
information through convolutional-recurrent neural networks that improves
perceptual quality. At the same time, we introduce a new constraint on the
objective function using a language model/decoder that limits the impact on
recognition rate. Based on experiments conducted with real user data, we
demonstrate that our new context-augmented machine-learning approach for speech
enhancement improves PESQ and WER by an additional 24.5% and 51.3%,
respectively, when compared to the best-performing methods in the literature.
Online Machine Learning in Big Data Streams
Authors: András A. Benczúr, Levente Kocsis, Róbert Pálovics
Date: 2018-02-16 00:00:00
Tasks: BIG-bench Machine Learning, Recommendation Systems

The area of online machine learning in big data streams covers algorithms
that are (1) distributed and (2) work from data streams with only a limited
possibility to store past data. The first requirement mostly concerns software
architectures and efficient algorithms. The second one also imposes nontrivial
theoretical restrictions on the modeling methods: In the data stream model,
older data is no longer available to revise earlier suboptimal modeling
decisions as the fresh data arrives.
  In this article, we provide an overview of distributed software architectures
and libraries as well as machine learning models for online learning. We
highlight the most important ideas for classification, regression,
recommendation, and unsupervised modeling from streaming data, and we show how
they are implemented in various distributed data stream processing systems.
  This article is a reference material and not a survey. We do not attempt to
be comprehensive in describing all existing methods and solutions; rather, we
give pointers to the most important resources in the field. All related
sub-fields, online algorithms, online learning, and distributed data processing
are hugely dominant in current research and development with conceptually new
research results and software components emerging at the time of writing. In
this article, we refer to several survey results, both for distributed data
processing and for online machine learning. Compared to past surveys, our
article is different because we discuss recommender systems in extended detail.
Articulatory information and Multiview Features for Large Vocabulary Continuous Speech Recognition
Authors: Vikramjit Mitra, Wen Wang, Chris Bartels, Horacio Franco, Dimitra Vergyri
Date: 2018-02-16 00:00:00
Tasks: speech-recognition, Speech Recognition

This paper explores the use of multi-view features and their discriminative
transforms in a convolutional deep neural network (CNN) architecture for a
continuous large vocabulary speech recognition task. Mel-filterbank energies
and perceptually motivated forced damped oscillator coefficient (DOC) features
are used after feature-space maximum-likelihood linear regression (fMLLR)
transforms, which are combined and fed as a multi-view feature to a single CNN
acoustic model. Use of multi-view feature representation demonstrated
significant reduction in word error rates (WERs) compared to the use of
individual features by themselves. In addition, when articulatory information
was used as an additional input to a fused deep neural network (DNN) and CNN
acoustic model, it was found to demonstrate further reduction in WER for the
Switchboard subset and the CallHome subset (containing partly non-native
accented speech) of the NIST 2000 conversational telephone speech test set,
reducing the error rate by 12% relative to the baseline in both cases. This
work shows that multi-view features in association with articulatory
information can improve speech recognition robustness to spontaneous and
non-native speech.
Interpreting DNN output layer activations: A strategy to cope with unseen data in speech recognition
Authors: Vikramjit Mitra, Horacio Franco
Date: 2018-02-16 00:00:00
Tasks: speech-recognition, Speech Recognition

Unseen data can degrade performance of deep neural net acoustic models. To
cope with unseen data, adaptation techniques are deployed. For unlabeled unseen
data, one must generate some hypothesis given an existing model, which is used
as the label for model adaptation. However, assessing the goodness of the
hypothesis can be difficult, and an erroneous hypothesis can lead to poorly
trained models. In such cases, a strategy to select data having reliable
hypothesis can ensure better model adaptation. This work proposes a
data-selection strategy for DNN model adaptation, where DNN output layer
activations are used to ascertain the goodness of a generated hypothesis. In a
DNN acoustic model, the output layer activations are used to generate target
class probabilities. Under unseen data conditions, the difference between the
most probable target and the next most probable target is decreased compared to
the same for seen data, indicating that the model may be uncertain while
generating its hypothesis. This work proposes a strategy to assess a model's
performance by analyzing the output layer activations by using a distance
measure between the most likely target and the next most likely target, which
is used for data selection for performing unsupervised adaptation.
Deep Learning with Topological Signatures
Authors: Christoph Hofer, Roland Kwitt, Marc Niethammer, Andreas Uhl
Date: 2017-07-13 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: BIG-bench Machine Learning, Deep Learning, Topological Data Analysis

Inferring topological and geometrical information from data can offer an
alternative perspective on machine learning problems. Methods from topological
data analysis, e.g., persistent homology, enable us to obtain such information,
typically in the form of summary representations of topological features.
However, such topological signatures often come with an unusual structure
(e.g., multisets of intervals) that is highly impractical for most machine
learning techniques. While many strategies have been proposed to map these
topological signatures into machine learning compatible representations, they
suffer from being agnostic to the target learning task. In contrast, we propose
a technique that enables us to input topological signatures to deep neural
networks and learn a task-optimal representation during training. Our approach
is realized as a novel input layer with favorable theoretical properties.
Classification experiments on 2D object shapes and social network graphs
demonstrate the versatility of the approach and, in case of the latter, we even
outperform the state-of-the-art by a large margin.
Train on Validation: Squeezing the Data Lemon
Authors: Guy Tennenholtz, Tom Zahavy, Shie Mannor
Date: 2018-02-16 00:00:00
Tasks: Model Selection

Model selection on validation data is an essential step in machine learning.
While the mixing of data between training and validation is considered taboo,
practitioners often violate it to increase performance. Here, we offer a
simple, practical method for using the validation set for training, which
allows for a continuous, controlled trade-off between performance and
overfitting of model selection. We define the notion of
on-average-validation-stable algorithms as one in which using small portions of
validation data for training does not overfit the model selection process. We
then prove that stable algorithms are also validation stable. Finally, we
demonstrate our method on the MNIST and CIFAR-10 datasets using stable
algorithms as well as state-of-the-art neural networks. Our results show
significant increase in test performance with a minor trade-off in bias
admitted to the model selection process.
Inferring relevant features: from QFT to PCA
Authors: Cédric Bény
Date: 2018-02-16 00:00:00
Tasks: General Classification

In many-body physics, renormalization techniques are used to extract aspects
of a statistical or quantum state that are relevant at large scale, or for low
energy experiments. Recent works have proposed that these features can be
formally identified as those perturbations of the states whose
distinguishability most resist coarse-graining. Here, we examine whether this
same strategy can be used to identify important features of an unlabeled
dataset. This approach indeed results in a technique very similar to kernel PCA
(principal component analysis), but with a kernel function that is
automatically adapted to the data, or "learned". We test this approach on
handwritten digits, and find that the most relevant features are significantly
better for classification than those obtained from a simple gaussian kernel.
Flexible Network Binarization with Layer-wise Priority
Authors: Lixue Zhuang, Yi Xu, Bingbing Ni, Hongteng Xu
Date: 2017-09-13 00:00:00
Tasks: Binarization, Pedestrian Detection

How to effectively approximate real-valued parameters with binary codes plays
a central role in neural network binarization. In this work, we reveal an
important fact that binarizing different layers has a widely-varied effect on
the compression ratio of network and the loss of performance. Based on this
fact, we propose a novel and flexible neural network binarization method by
introducing the concept of layer-wise priority which binarizes parameters in
inverse order of their layer depth. In each training step, our method selects a
specific network layer, minimizes the discrepancy between the original
real-valued weights and its binary approximations, and fine-tunes the whole
network accordingly. During the iteration of the above process, it is
significant that we can flexibly decide whether to binarize the remaining
floating layers or not and explore a trade-off between the loss of performance
and the compression ratio of model. The resulting binary network is applied for
efficient pedestrian detection. Extensive experimental results on several
benchmarks show that under the same compression ratio, our method achieves much
lower miss rate and faster detection speed than the state-of-the-art neural
network binarization method.
Neural Granger Causality
Authors: Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaie, Emily Fox
Date: 2018-02-16 00:00:00
Tasks: Time Series Analysis

While most classical approaches to Granger causality detection assume linear dynamics, many interactions in real-world applications, like neuroscience and genomics, are inherently nonlinear. In these cases, using linear models may lead to inconsistent estimation of Granger causal interactions. We propose a class of nonlinear methods by applying structured multilayer perceptrons (MLPs) or recurrent neural networks (RNNs) combined with sparsity-inducing penalties on the weights. By encouraging specific sets of weights to be zero--in particular, through the use of convex group-lasso penalties--we can extract the Granger causal structure. To further contrast with traditional approaches, our framework naturally enables us to efficiently capture long-range dependencies between series either via our RNNs or through an automatic lag selection in the MLP. We show that our neural Granger causality methods outperform state-of-the-art nonlinear Granger causality methods on the DREAM3 challenge data. This data consists of nonlinear gene expression and regulation time courses with only a limited number of time points. The successes we show in this challenging dataset provide a powerful example of how deep learning can be useful in cases that go beyond prediction on large datasets. We likewise illustrate our methods in detecting nonlinear interactions in a human motion capture dataset.
Rapid Bayesian optimisation for synthesis of short polymer fiber materials
Authors: Cheng Li, David Rubin de Celis Leal, Santu Rana, Sunil Gupta, Alessandra Sutti, Stewart Greenhill, Teo Slezak, Murray Height, Svetha Venkatesh
Date: 2018-02-16 00:00:00
Tasks: Bayesian Optimisation, BIG-bench Machine Learning

The discovery of processes for the synthesis of new materials involves many
decisions about process design, operation, and material properties.
Experimentation is crucial but as complexity increases, exploration of
variables can become impractical using traditional combinatorial approaches. We
describe an iterative method which uses machine learning to optimise process
development, incorporating multiple qualitative and quantitative objectives. We
demonstrate the method with a novel fluid processing platform for synthesis of
short polymer fibers, and show how the synthesis process can be efficiently
directed to achieve material and process objectives.
An Anytime Algorithm for Task and Motion MDPs
Authors: Siddharth Srivastava, Nishant Desai, Richard Freedman, Shlomo Zilberstein
Date: 2018-02-16 00:00:00
Tasks: Decision Making, Motion Planning, Sequential Decision Making, Task and Motion Planning

Integrated task and motion planning has emerged as a challenging problem in
sequential decision making, where a robot needs to compute high-level strategy
and low-level motion plans for solving complex tasks. While high-level
strategies require decision making over longer time-horizons and scales, their
feasibility depends on low-level constraints based upon the geometries and
continuous dynamics of the environment. The hybrid nature of this problem makes
it difficult to scale; most existing approaches focus on deterministic, fully
observable scenarios. We present a new approach where the high-level decision
problem occurs in a stochastic setting and can be modeled as a Markov decision
process. In contrast to prior efforts, we show that complete MDP policies, or
contingent behaviors, can be computed effectively in an anytime fashion. Our
algorithm continuously improves the quality of the solution and is guaranteed
to be probabilistically complete. We evaluate the performance of our approach
on a challenging, realistic test problem: autonomous aircraft inspection. Our
results show that we can effectively compute consistent task and motion
policies for the most likely execution-time outcomes using only a fraction of
the computation required to develop the complete task and motion policy.
Convolutional Sparse Representations with Gradient Penalties
Authors: Brendt Wohlberg
Date: 2017-05-12 00:00:00
Tasks: Form, Image Reconstruction

While convolutional sparse representations enjoy a number of useful
properties, they have received limited attention for image reconstruction
problems. The present paper compares the performance of block-based and
convolutional sparse representations in the removal of Gaussian white noise.
While the usual formulation of the convolutional sparse coding problem is
slightly inferior to the block-based representations in this problem, the
performance of the convolutional form can be boosted beyond that of the
block-based form by the inclusion of suitable penalties on the gradients of the
coefficient maps.
A Comparison of Constraint Handling Techniques for Dynamic Constrained Optimization Problems
Authors: Maria-Yaneli Ameca-Alducin, Maryam Hasani-Shoreh, Wilson Blaikie, Frank Neumann, Efren Mezura-Montes
Date: 2018-02-16 00:00:00
Tasks: Change Detection

Dynamic constrained optimization problems (DCOPs) have gained researchers
attention in recent years because a vast majority of real world problems change
over time. There are studies about the effect of constrained handling
techniques in static optimization problems. However, there lacks any
substantial study in the behavior of the most popular constraint handling
techniques when dealing with DCOPs. In this paper we study the four most
popular used constraint handling techniques and apply a simple Differential
Evolution (DE) algorithm coupled with a change detection mechanism to observe
the behavior of these techniques. These behaviors were analyzed using a common
benchmark to determine which techniques are suitable for the most prevalent
types of DCOPs. For the purpose of analysis, common measures in static
environments were adapted to suit dynamic environments. While an overall
superior technique could not be determined, certain techniques outperformed
others in different aspects like rate of optimization or reliability of
solutions.
Matrix completion with deterministic pattern - a geometric perspective
Authors: Alexander Shapiro, Yao Xie, Rui Zhang
Date: 2018-01-31 00:00:00
Tasks: Matrix Completion

We consider the matrix completion problem with a deterministic pattern of
observed entries. In this setting, we aim to answer the question: under what
condition there will be (at least locally) unique solution to the matrix
completion problem, i.e., the underlying true matrix is identifiable. We answer
the question from a certain point of view and outline a geometric perspective.
We give an algebraically verifiable sufficient condition, which we call the
well-posedness condition, for the local uniqueness of MRMC solutions. We argue
that this condition is necessary for local stability of MRMC solutions, and we
show that the condition is generic using the characteristic rank. We also argue
that the low-rank approximation approaches are more stable than MRMC and
further propose a sequential statistical testing procedure to determine the
"true" rank from observed entries. Finally, we provide numerical examples aimed
at verifying validity of the presented theory.
Auto-Encoding Total Correlation Explanation
Authors: Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan
Date: 2018-02-16 00:00:00
Tasks: Disentanglement, Representation Learning

Advances in unsupervised learning enable reconstruction and generation of
samples from complex distributions, but this success is marred by the
inscrutability of the representations learned. We propose an
information-theoretic approach to characterizing disentanglement and dependence
in representation learning using multivariate mutual information, also called
total correlation. The principle of total Cor-relation Ex-planation (CorEx) has
motivated successful unsupervised learning applications across a variety of
domains, but under some restrictive assumptions. Here we relax those
restrictions by introducing a flexible variational lower bound to CorEx.
Surprisingly, we find that this lower bound is equivalent to the one in
variational autoencoders (VAE) under certain conditions. This
information-theoretic view of VAE deepens our understanding of hierarchical VAE
and motivates a new algorithm, AnchorVAE, that makes latent codes more
interpretable through information maximization and enables generation of richer
and more realistic samples.
Learning Latent Features with Pairwise Penalties in Low-Rank Matrix Completion
Authors: Kaiyi Ji, Jian Tan, Jinfeng Xu, Yuejie Chi
Date: 2018-02-16 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

Low-rank matrix completion has achieved great success in many real-world data applications. A matrix factorization model that learns latent features is usually employed and, to improve prediction performance, the similarities between latent variables can be exploited by pairwise learning using the graph regularized matrix factorization (GRMF) method. However, existing GRMF approaches often use the squared loss to measure the pairwise differences, which may be overly influenced by dissimilar pairs and lead to inferior prediction. To fully empower pairwise learning for matrix completion, we propose a general optimization framework that allows a rich class of (non-)convex pairwise penalty functions. A new and efficient algorithm is developed to solve the proposed optimization problem, with a theoretical convergence guarantee under mild assumptions. In an important situation where the latent variables form a small number of subgroups, its statistical guarantee is also fully considered. In particular, we theoretically characterize the performance of the complexity-regularized maximum likelihood estimator, as a special case of our framework, which is shown to have smaller errors when compared to the standard matrix completion framework without pairwise penalties. We conduct extensive experiments on both synthetic and real datasets to demonstrate the superior performance of this general framework.
Disentangling Aspect and Opinion Words in Target-based Sentiment Analysis using Lifelong Learning
Authors: Shuai Wang, Mianwei Zhou, Sahisnu Mazumder, Bing Liu, Yi Chang
Date: 2018-02-16 00:00:00
Tasks: Lifelong learning, Sentiment Analysis

Given a target name, which can be a product aspect or entity, identifying its
aspect words and opinion words in a given corpus is a fine-grained task in
target-based sentiment analysis (TSA). This task is challenging, especially
when we have no labeled data and we want to perform it for any given domain. To
address it, we propose a general two-stage approach. Stage one extracts/groups
the target-related words (call t-words) for a given target. This is relatively
easy as we can apply an existing semantics-based learning technique. Stage two
separates the aspect and opinion words from the grouped t-words, which is
challenging because we often do not have enough word-level aspect and opinion
labels. In this work, we formulate this problem in a PU learning setting and
incorporate the idea of lifelong learning to solve it. Experimental results
show the effectiveness of our approach.