pwc-archive/papers-with-abstracts huggingface dataset:
Sparse Signal Recovery for Binary Compressed Sensing by Majority Voting Neural Networks
Authors: Daisuke Ito, Tadashi Wadayama
Date: 2016-10-29 00:00:00
Tasks: compressed sensing

In this paper, we propose majority voting neural networks for sparse signal
recovery in binary compressed sensing. The majority voting neural network is
composed of several independently trained feedforward neural networks employing
the sigmoid function as an activation function. Our empirical study shows that
a choice of a loss function used in training processes for the network is of
prime importance. We found a loss function suitable for sparse signal recovery,
which includes a cross entropy-like term and an $L_1$ regularized term. From
the experimental results, we observed that the majority voting neural network
achieves excellent recovery performance, which is approaching the optimal
performance as the number of component nets grows. The simple architecture of
the majority voting neural networks would be beneficial for both software and
hardware implementations.
Building Energy Load Forecasting using Deep Neural Networks
Authors: Daniel L. Marino, Kasun Amarasinghe, Milos Manic
Date: 2016-10-29 00:00:00
Tasks: Decision Making, energy management, Load Forecasting, Management

Ensuring sustainability demands more efficient energy management with
minimized energy wastage. Therefore, the power grid of the future should
provide an unprecedented level of flexibility in energy management. To that
end, intelligent decision making requires accurate predictions of future energy
demand/load, both at aggregate and individual site level. Thus, energy load
forecasting have received increased attention in the recent past, however has
proven to be a difficult problem. This paper presents a novel energy load
forecasting methodology based on Deep Neural Networks, specifically Long Short
Term Memory (LSTM) algorithms. The presented work investigates two variants of
the LSTM: 1) standard LSTM and 2) LSTM-based Sequence to Sequence (S2S)
architecture. Both methods were implemented on a benchmark data set of
electricity consumption data from one residential customer. Both architectures
where trained and tested on one hour and one-minute time-step resolution
datasets. Experimental results showed that the standard LSTM failed at
one-minute resolution data while performing well in one-hour resolution data.
It was shown that S2S architecture performed well on both datasets. Further, it
was shown that the presented methods produced comparable results with the other
deep learning methods for energy forecasting in literature.
Selective De-noising of Sparse-Coloured Images
Authors: Arjun Chaudhuri
Date: 2016-10-29 00:00:00
Tasks: Astronomy

Since time immemorial, noise has been a constant source of disturbance to the
various entities known to mankind. Noise models of different kinds have been
developed to study noise in more detailed fashion over the years. Image
processing, particularly, has extensively implemented several algorithms to
reduce noise in photographs and pictorial documents to alleviate the effect of
noise. Images with sparse colours-lesser number of distinct colours in them-are
common nowadays, especially in astronomy and astrophysics where black and white
colours form the main components. Additive noise of Gaussian type is the most
common form of noise to be studied and analysed in majority of communication
channels, namely-satellite links, mobile base station to local cellular tower
communication channel,et. al. Most of the time, we encounter images from
astronomical sources being distorted with noise maximally as they travel long
distance from telescopes in outer space to Earth. Considering Additive White
Gaussian Noise(AWGN) to be the common noise in these long distance channels,
this paper provides an insight and an algorithmic approach to pixel-specific
de-noising of sparse-coloured images affected by AWGN. The paper concludes with
some essential future avenues and applications of this de-noising method in
industry and academia.
KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics
Authors: Evan R. Sparks, Shivaram Venkataraman, Tomer Kaftan, Michael J. Franklin, Benjamin Recht
Date: 2016-10-29 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, image-classification, Image Classification

Modern advanced analytics applications make use of machine learning
techniques and contain multiple steps of domain-specific and general-purpose
processing with high resource requirements. We present KeystoneML, a system
that captures and optimizes the end-to-end large-scale machine learning
applications for high-throughput training in a distributed environment with a
high-level API. This approach offers increased ease of use and higher
performance over existing systems for large scale learning. We demonstrate the
effectiveness of KeystoneML in achieving high quality statistical accuracy and
scalable training using real world datasets in several domains. By optimizing
execution KeystoneML achieves up to 15x training throughput over unoptimized
execution on a real image classification application.
The p-filter: multi-layer FDR control for grouped hypotheses
Authors: Rina Foygel Barber, Aaditya Ramdas
Date: 2015-12-10 00:00:00
Tasks: Two-sample testing

In many practical applications of multiple hypothesis testing using the False
Discovery Rate (FDR), the given hypotheses can be naturally partitioned into
groups, and one may not only want to control the number of false discoveries
(wrongly rejected null hypotheses), but also the number of falsely discovered
groups of hypotheses (we say a group is falsely discovered if at least one
hypothesis within that group is rejected, when in reality the group contains
only nulls). In this paper, we introduce the p-filter, a procedure which
unifies and generalizes the standard FDR procedure by Benjamini and Hochberg
and global null testing procedure by Simes. We first prove that our proposed
method can simultaneously control the overall FDR at the finest level
(individual hypotheses treated separately) and the group FDR at coarser levels
(when such groups are user-specified). We then generalize the p-filter
procedure even further to handle multiple partitions of hypotheses, since that
might be natural in many applications. For example, in neuroscience
experiments, we may have a hypothesis for every (discretized) location in the
brain, and at every (discretized) timepoint: does the stimulus correlate with
activity in location x at time t after the stimulus was presented? In this
setting, one might want to group hypotheses by location and by time.
Importantly, our procedure can handle multiple partitions which are
nonhierarchical (i.e. one partition may arrange p-values by voxel, and another
partition arranges them by time point; neither one is nested inside the other).
We prove that our procedure controls FDR simultaneously across these multiple
lay- ers, under assumptions that are standard in the literature: we do not need
the hypotheses to be independent, but require a nonnegative dependence
condition known as PRDS.
Beyond Exchangeability: The Chinese Voting Process
Authors: Moontae Lee, Seok Hyun Jin, David Mimno
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Position

Many online communities present user-contributed responses such as reviews of
products and answers to questions. User-provided helpfulness votes can
highlight the most useful responses, but voting is a social process that can
gain momentum based on the popularity of responses and the polarity of existing
votes. We propose the Chinese Voting Process (CVP) which models the evolution
of helpfulness votes as a self-reinforcing process dependent on position and
presentation biases. We evaluate this model on Amazon product reviews and more
than 80 StackExchange forums, measuring the intrinsic quality of individual
responses and behavioral coefficients of different communities.
Dynamic matrix recovery from incomplete observations under an exact low-rank constraint
Authors: Liangbei Xu, Mark A. Davenport
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Matrix Completion, Recommendation Systems, Topic Models

Low-rank matrix factorizations arise in a wide variety of applications --
including recommendation systems, topic models, and source separation, to name
just a few. In these and many other applications, it has been widely noted that
by incorporating temporal information and allowing for the possibility of
time-varying models, significant improvements are possible in practice.
However, despite the reported superior empirical performance of these dynamic
models over their static counterparts, there is limited theoretical
justification for introducing these more complex models. In this paper we aim
to address this gap by studying the problem of recovering a dynamically
evolving low-rank matrix from incomplete observations. First, we propose the
locally weighted matrix smoothing (LOWEMS) framework as one possible approach
to dynamic matrix recovery. We then establish error bounds for LOWEMS in both
the {\em matrix sensing} and {\em matrix completion} observation models. Our
results quantify the potential benefits of exploiting dynamic constraints both
in terms of recovery accuracy and sample complexity. To illustrate these
benefits we provide both synthetic and real-world experimental results.
Probabilistic Model Checking for Complex Cognitive Tasks -- A case study in human-robot interaction
Authors: Sebastian Junges, Nils Jansen, Joost-Pieter Katoen, Ufuk Topcu
Date: 2016-10-28 00:00:00

This paper proposes to use probabilistic model checking to synthesize optimal
robot policies in multi-tasking autonomous systems that are subject to
human-robot interaction. Given the convincing empirical evidence that human
behavior can be related to reinforcement models, we take as input a
well-studied Q-table model of the human behavior for flexible scenarios. We
first describe an automated procedure to distill a Markov decision process
(MDP) for the human in an arbitrary but fixed scenario. The distinctive issue
is that -- in contrast to existing models -- under-specification of the human
behavior is included. Probabilistic model checking is used to predict the
human's behavior. Finally, the MDP model is extended with a robot model.
Optimal robot policies are synthesized by analyzing the resulting two-player
stochastic game. Experimental results with a prototypical implementation using
PRISM show promising results.
A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
Authors: Steven Cheng-Xian Li, Benjamin Marlin
Date: 2016-06-14 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

We present a general framework for classification of sparse and
irregularly-sampled time series. The properties of such time series can result
in substantial uncertainty about the values of the underlying temporal
processes, while making the data difficult to deal with using standard
classification methods that assume fixed-dimensional feature spaces. To address
these challenges, we propose an uncertainty-aware classification framework
based on a special computational layer we refer to as the Gaussian process
adapter that can connect irregularly sampled time series data to any black-box
classifier learnable using gradient descent. We show how to scale up the
required computations based on combining the structured kernel interpolation
framework and the Lanczos approximation method, and how to discriminatively
train the Gaussian process adapter in combination with a number of classifiers
end-to-end using backpropagation.
Detecting Breast Cancer using a Compressive Sensing Unmixing Algorithm
Authors: Richard Obermeier, Jose Angel Martinez-Lorenzo
Date: 2016-10-28 00:00:00
Tasks: Compressive Sensing

Traditional breast cancer imaging methods using microwave Nearfield Radar
Imaging (NRI) seek to recover the complex permittivity of the tissues at each
voxel in the imaging region. This approach is suboptimal, in that it does not
directly consider the permittivity values that healthy and cancerous breast
tissues typically have. In this paper, we describe a novel unmixing algorithm
for detecting breast cancer. In this approach, the breast tissue is separated
into three components, low water content (LWC), high water content (HWC), and
cancerous tissues, and the goal of the optimization procedure is to recover the
mixture proportions for each component. By utilizing this approach in a hybrid
DBT / NRI system, the unmixing reconstruction process can be posed as a sparse
recovery problem, such that compressive sensing (CS) techniques can be
employed. A numerical analysis is performed, which demonstrates that cancerous
lesions can be detected from their mixture proportion under the appropriate
conditions.
Algorithms for Fitting the Constrained Lasso
Authors: Brian R. Gaines, Hua Zhou
Date: 2016-10-28 00:00:00

We compare alternative computing strategies for solving the constrained lasso
problem. As its name suggests, the constrained lasso extends the widely-used
lasso to handle linear constraints, which allow the user to incorporate prior
information into the model. In addition to quadratic programming, we employ the
alternating direction method of multipliers (ADMM) and also derive an efficient
solution path algorithm. Through both simulations and real data examples, we
compare the different algorithms and provide practical recommendations in terms
of efficiency and accuracy for various sizes of data. We also show that, for an
arbitrary penalty matrix, the generalized lasso can be transformed to a
constrained lasso, while the converse is not true. Thus, our methods can also
be used for estimating a generalized lasso, which has wide-ranging
applications. Code for implementing the algorithms is freely available in the
Matlab toolbox SparseReg.
Supervised Learning in Spiking Neural Networks for Precise Temporal Encoding
Authors: Brian Gardner, André Grüning
Date: 2016-01-14 00:00:00

Precise spike timing as a means to encode information in neural networks is
biologically supported, and is advantageous over frequency-based codes by
processing input features on a much shorter time-scale. For these reasons, much
recent attention has been focused on the development of supervised learning
rules for spiking neural networks that utilise a temporal coding scheme.
However, despite significant progress in this area, there still lack rules that
have a theoretical basis, and yet can be considered biologically relevant. Here
we examine the general conditions under which synaptic plasticity most
effectively takes place to support the supervised learning of a precise
temporal code. As part of our analysis we examine two spike-based learning
methods: one of which relies on an instantaneous error signal to modify
synaptic weights in a network (INST rule), and the other one on a filtered
error signal for smoother synaptic weight modifications (FILT rule). We test
the accuracy of the solutions provided by each rule with respect to their
temporal encoding precision, and then measure the maximum number of input
patterns they can learn to memorise using the precise timings of individual
spikes as an indication of their storage capacity. Our results demonstrate the
high performance of FILT in most cases, underpinned by the rule's
error-filtering mechanism, which is predicted to provide smooth convergence
towards a desired solution during learning. We also find FILT to be most
efficient at performing input pattern memorisations, and most noticeably when
patterns are identified using spikes with sub-millisecond temporal precision.
In comparison with existing work, we determine the performance of FILT to be
consistent with that of the highly efficient E-learning Chronotron, but with
the distinct advantage that FILT is also implementable as an online method for
increased biological realism.
Rényi Divergence Variational Inference
Authors: Yingzhen Li, Richard E. Turner
Date: 2016-02-06 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Variational Inference

This paper introduces the variational R\'enyi bound (VR) that extends
traditional variational inference to R\'enyi's alpha-divergences. This new
family of variational methods unifies a number of existing approaches, and
enables a smooth interpolation from the evidence lower-bound to the log
(marginal) likelihood that is controlled by the value of alpha that
parametrises the divergence. The reparameterization trick, Monte Carlo
approximation and stochastic optimisation methods are deployed to obtain a
tractable and unified framework for optimisation. We further consider negative
alpha values and propose a novel variational inference method as a new special
case in the proposed framework. Experiments on Bayesian neural networks and
variational auto-encoders demonstrate the wide applicability of the VR bound.
Real-time Online Action Detection Forests using Spatio-temporal Contexts
Authors: Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Date: 2016-10-28 00:00:00
Tasks: Action Detection, Online Action Detection

Online action detection (OAD) is challenging since 1) robust yet
computationally expensive features cannot be straightforwardly used due to the
real-time processing requirements and 2) the localization and classification of
actions have to be performed even before they are fully observed. We propose a
new random forest (RF)-based online action detection framework that addresses
these challenges. Our algorithm uses computationally efficient skeletal joint
features. High accuracy is achieved by using robust convolutional neural
network (CNN)-based features which are extracted from the raw RGBD images, plus
the temporal relationships between the current frame of interest, and the past
and future frames. While these high-quality features are not available in
real-time testing scenario, we demonstrate that they can be effectively
exploited in training RF classifiers: We use these spatio-temporal contexts to
craft RF's new split functions improving RFs' leaf node statistics. Experiments
with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our
algorithm significantly improves the accuracy over the state-of-the-art online
action detection algorithms while achieving the real-time efficiency of
existing skeleton-based RF classifiers.
Word Embeddings for the Construction Domain
Authors: Antoine J. -P. Tixier, Michalis Vazirgiannis, Matthew R. Hallowell
Date: 2016-10-28 00:00:00
Tasks: Benchmarking, General Classification, Word Embeddings

We introduce word vectors for the construction domain. Our vectors were
obtained by running word2vec on an 11M-word corpus that we created from scratch
by leveraging freely-accessible online sources of construction-related text. We
first explore the embedding space and show that our vectors capture meaningful
construction-specific concepts. We then evaluate the performance of our vectors
against that of ones trained on a 100B-word corpus (Google News) within the
framework of an injury report classification task. Without any parameter
tuning, our embeddings give competitive results, and outperform the Google News
vectors in many cases. Using a keyword-based compression of the reports also
leads to a significant speed-up with only a limited loss in performance. We
release our corpus and the data set we created for the classification task as
publicly available, in the hope that they will be used by future studies for
benchmarking and building on our work.
Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
Authors: Antoine Gautier, Quynh Nguyen, Matthias Hein
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Polynomial Neural Networks

The optimization problem behind neural networks is highly non-convex.
Training with stochastic gradient descent and variants requires careful
parameter tuning and provides no guarantee to achieve the global optimum. In
contrast we show under quite weak assumptions on the data that a particular
class of feedforward neural networks can be trained globally optimal with a
linear convergence rate with our nonlinear spectral method. Up to our knowledge
this is the first practically feasible method which achieves such a guarantee.
While the method can in principle be applied to deep networks, we restrict
ourselves for simplicity in this paper to one and two hidden layer networks.
Our experiments confirm that these models are rich enough to achieve good
performance on a series of real-world datasets.
Character Sequence Models for ColorfulWords
Authors: Kazuya Kawakami, Chris Dyer, Bryan R. Routledge, Noah A. Smith
Date: 2016-09-28 00:00:00

We present a neural network architecture to predict a point in color space
from the sequence of characters in the color's name. Using large scale
color--name pairs obtained from an online color design forum, we evaluate our
model on a "color Turing test" and find that, given a name, the colors
predicted by our model are preferred by annotators to color names created by
humans. Our datasets and demo system are available online at colorlab.us.
Scaling Factorial Hidden Markov Models: Stochastic Variational Inference without Messages
Authors: Yin Cheng Ng, Pawel Chilinski, Ricardo Silva
Date: 2016-08-12 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Variational Inference

Factorial Hidden Markov Models (FHMMs) are powerful models for sequential
data but they do not scale well with long sequences. We propose a scalable
inference and learning algorithm for FHMMs that draws on ideas from the
stochastic variational inference, neural network and copula literatures. Unlike
existing approaches, the proposed algorithm requires no message passing
procedure among latent variables and can be distributed to a network of
computers to speed up learning. Our experiments corroborate that the proposed
algorithm does not introduce further approximation bias compared to the proven
structured mean-field algorithm, and achieves better performance with long
sequences and large FHMMs.
PAC Reinforcement Learning with Rich Observations
Authors: Akshay Krishnamurthy, Alekh Agarwal, John Langford
Date: 2016-02-08 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Decision Making, Multi-Armed Bandits, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

We propose and study a new model for reinforcement learning with rich
observations, generalizing contextual bandits to sequential decision making.
These models require an agent to take actions based on observations (features)
with the goal of achieving long-term performance competitive with a large set
of policies. To avoid barriers to sample-efficient learning associated with
large observation spaces and general POMDPs, we focus on problems that can be
summarized by a small number of hidden states and have long-term rewards that
are predictable by a reactive function class. In this setting, we design and
analyze a new reinforcement learning algorithm, Least Squares Value Elimination
by Exploration. We prove that the algorithm learns near optimal behavior after
a number of episodes that is polynomial in all relevant parameters, logarithmic
in the number of policies, and independent of the size of the observation
space. Our result provides theoretical justification for reinforcement learning
with function approximation.