pwc-archive/papers-with-abstracts huggingface dataset:
Properties and Bayesian fitting of restricted Boltzmann machines
Authors: Andee Kaplan, Daniel Nordman, Stephen Vardeman
Date: 2016-12-04 00:00:00
Tasks: Uncertainty Quantification

A restricted Boltzmann machine (RBM) is an undirected graphical model
constructed for discrete or continuous random variables, with two layers, one
hidden and one visible, and no conditional dependency within a layer. In recent
years, RBMs have risen to prominence due to their connection to deep learning.
By treating a hidden layer of one RBM as the visible layer in a second RBM, a
deep architecture can be created. RBMs are thought to thereby have the ability
to encode very complex and rich structures in data, making them attractive for
supervised learning. However, the generative behavior of RBMs is largely
unexplored and typical fitting methodology does not easily allow for
uncertainty quantification in addition to point estimates. In this paper, we
discuss the relationship between RBM parameter specification in the binary case
and model properties such as degeneracy, instability and uninterpretability. We
also describe the associated difficulties that can arise with likelihood-based
inference and further discuss the potential Bayes fitting of such (highly
flexible) models, especially as Gibbs sampling (quasi-Bayes) methods are often
advocated for the RBM model structure.
CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition
Authors: Dogancan Temel, Gukyeong Kwon, Mohit Prabhushankar, Ghassan AlRegib
Date: 2017-12-07 00:00:00
Tasks: Data Augmentation, Traffic Sign Recognition

In this paper, we investigate the robustness of traffic sign recognition
algorithms under challenging conditions. Existing datasets are limited in terms
of their size and challenging condition coverage, which motivated us to
generate the Challenging Unreal and Real Environments for Traffic Sign
Recognition (CURE-TSR) dataset. It includes more than two million traffic sign
images that are based on real-world and simulator data. We benchmark the
performance of existing solutions in real-world scenarios and analyze the
performance variation with respect to challenging conditions. We show that
challenging conditions can decrease the performance of baseline methods
significantly, especially if these challenging conditions result in loss or
misplacement of spatial information. We also investigate the effect of data
augmentation and show that utilization of simulator data along with real-world
data enhance the average recognition performance in real-world scenarios. The
dataset is publicly available at https://ghassanalregib.com/cure-tsr/.
Complex-valued Gaussian Process Regression for Time Series Analysis
Authors: Luca Ambrogioni, Eric Maris
Date: 2016-11-30 00:00:00
Tasks: regression, Time Series, Time Series Analysis

The construction of synthetic complex-valued signals from real-valued
observations is an important step in many time series analysis techniques. The
most widely used approach is based on the Hilbert transform, which maps the
real-valued signal into its quadrature component. In this paper, we define a
probabilistic generalization of this approach. We model the observable
real-valued signal as the real part of a latent complex-valued Gaussian
process. In order to obtain the appropriate statistical relationship between
its real and imaginary parts, we define two new classes of complex-valued
covariance functions. Through an analysis of simulated chirplets and stochastic
oscillations, we show that the resulting Gaussian process complex-valued signal
provides a better estimate of the instantaneous amplitude and frequency than
the established approaches. Furthermore, the complex-valued Gaussian process
regression allows to incorporate prior information about the structure in
signal and noise and thereby to tailor the analysis to the features of the
signal. As a example, we analyze the non-stationary dynamics of brain
oscillations in the alpha band, as measured using magneto-encephalography.
Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization
Authors: Yang Wang, Lin Wu
Date: 2017-09-05 00:00:00
Tasks: Clustering

Multi-view data clustering attracts more attention than their single view
counterparts due to the fact that leveraging multiple independent and
complementary information from multi-view feature spaces outperforms the single
one. Multi-view Spectral Clustering aims at yielding the data partition
agreement over their local manifold structures by seeking
eigenvalue-eigenvector decompositions. However, as we observed, such classical
paradigm still suffers from (1) overlooking the flexible local manifold
structure, caused by (2) enforcing the low-rank data correlation agreement
among all views; worse still, (3) LRR is not intuitively flexible to capture
the latent data clustering structures. In this paper, we present the structured
LRR by factorizing into the latent low-dimensional data-cluster
representations, which characterize the data clustering structure for each
view. Upon such representation, (b) the laplacian regularizer is imposed to be
capable of preserving the flexible local manifold structure for each view. (c)
We present an iterative multi-view agreement strategy by minimizing the
divergence objective among all factorized latent data-cluster representations
during each iteration of optimization process, where such latent representation
from each view serves to regulate those from other views, such intuitive
process iteratively coordinates all views to be agreeable. (d) We remark that
such data-cluster representation can flexibly encode the data clustering
structure from any view with adaptive input cluster number. To this end, (e) a
novel non-convex objective function is proposed via the efficient alternating
minimization strategy. The complexity analysis are also presented. The
extensive experiments conducted against the real-world multi-view datasets
demonstrate the superiority over state-of-the-arts.
Integral Transforms from Finite Data: An Application of Gaussian Process Regression to Fourier Analysis
Authors: Luca Ambrogioni, Eric Maris
Date: 2017-04-10 00:00:00
Tasks: regression

Computing accurate estimates of the Fourier transform of analog signals from
discrete data points is important in many fields of science and engineering.
The conventional approach of performing the discrete Fourier transform of the
data implicitly assumes periodicity and bandlimitedness of the signal. In this
paper, we use Gaussian process regression to estimate the Fourier transform (or
any other integral transform) without making these assumptions. This is
possible because the posterior expectation of Gaussian process regression maps
a finite set of samples to a function defined on the whole real line, expressed
as a linear combination of covariance functions. We estimate the covariance
function from the data using an appropriately designed gradient ascent method
that constrains the solution to a linear combination of tractable kernel
functions. This procedure results in a posterior expectation of the analog
signal whose Fourier transform can be obtained analytically by exploiting
linearity. Our simulations show that the new method leads to sharper and more
precise estimation of the spectral density both in noise-free and
noise-corrupted signals. We further validate the method in two real-world
applications: the analysis of the yearly fluctuation in atmospheric CO2 level
and the analysis of the spectral content of brain signals.
Posterior Sampling for Large Scale Reinforcement Learning
Authors: Georgios Theocharous, Zheng Wen, Yasin Abbasi-Yadkori, Nikos Vlassis
Date: 2017-11-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a practical non-episodic PSRL algorithm that unlike recent
state-of-the-art PSRL algorithms uses a deterministic, model-independent
episode switching schedule. Our algorithm termed deterministic schedule PSRL
(DS-PSRL) is efficient in terms of time, sample, and space complexity. We prove
a Bayesian regret bound under mild assumptions. Our result is more generally
applicable to multiple parameters and continuous state action problems. We
compare our algorithm with state-of-the-art PSRL algorithms on standard
discrete and continuous problems from the literature. Finally, we show how the
assumptions of our algorithm satisfy a sensible parametrization for a large
class of problems in sequential recommendations.
"I know it when I see it". Visualization and Intuitive Interpretability
Authors: Fabian Offert
Date: 2017-11-20 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction

Most research on the interpretability of machine learning systems focuses on
the development of a more rigorous notion of interpretability. I suggest that a
better understanding of the deficiencies of the intuitive notion of
interpretability is needed as well. I show that visualization enables but also
impedes intuitive interpretability, as it presupposes two levels of technical
pre-interpretation: dimensionality reduction and regularization. Furthermore, I
argue that the use of positive concepts to emulate the distributed semantic
structure of machine learning models introduces a significant human bias into
the model. As a consequence, I suggest that, if intuitive interpretability is
needed, singular representations of internal model states should be avoided.
HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks
Authors: Dimitrios Stamoulis, Ermao Cai, Da-Cheng Juan, Diana Marculescu
Date: 2017-12-06 00:00:00
Tasks: Bayesian Optimization

While selecting the hyper-parameters of Neural Networks (NNs) has been so far
treated as an art, the emergence of more complex, deeper architectures poses
increasingly more challenges to designers and Machine Learning (ML)
practitioners, especially when power and memory constraints need to be
considered. In this work, we propose HyperPower, a framework that enables
efficient Bayesian optimization and random search in the context of power- and
memory-constrained hyper-parameter optimization for NNs running on a given
hardware platform. HyperPower is the first work (i) to show that power
consumption can be used as a low-cost, a priori known constraint, and (ii) to
propose predictive models for the power and memory of NNs executing on GPUs.
Thanks to HyperPower, the number of function evaluations and the best test
error achieved by a constraint-unaware method are reached up to 112.99x and
30.12x faster, respectively, while never considering invalid configurations.
HyperPower significantly speeds up the hyper-parameter optimization, achieving
up to 57.20x more function evaluations compared to constraint-unaware methods
for a given time interval, effectively yielding significant accuracy
improvements by up to 67.6%.
Forecasting Hands and Objects in Future Frames
Authors: Chenyou Fan, JangWon Lee, Michael S. Ryoo
Date: 2017-05-20 00:00:00
Tasks: Object, object-detection, Object Detection, Object Recognition

This paper presents an approach to forecast future presence and location of
human hands and objects. Given an image frame, the goal is to predict what
objects will appear in the future frame (e.g., 5 seconds later) and where they
will be located at, even when they are not visible in the current frame. The
key idea is that (1) an intermediate representation of a convolutional object
recognition model abstracts scene information in its frame and that (2) we can
predict (i.e., regress) such representations corresponding to the future frames
based on that of the current frame. We design a new two-stream convolutional
neural network (CNN) architecture for videos by extending the state-of-the-art
convolutional object detection network, and present a new fully convolutional
regression network for predicting future scene representations. Our experiments
confirm that combining the regressed future representation with our detection
network allows reliable estimation of future hands and objects in videos. We
obtain much higher accuracy compared to the state-of-the-art future object
presence forecast method on a public dataset.
A Novel Model for Arbitration between Planning and Habitual Control Systems
Authors: Farzaneh S. Fard, Thomas P. Trappenberg
Date: 2017-12-06 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Reinforcement Learning

It is well established that humans decision making and instrumental control
uses multiple systems, some which use habitual action selection and some which
require deliberate planning. Deliberate planning systems use predictions of
action-outcomes using an internal model of the agent's environment, while
habitual action selection systems learn to automate by repeating previously
rewarded actions. Habitual control is computationally efficient but may be
inflexible in changing environments. Conversely, deliberate planning may be
computationally expensive, but flexible in dynamic environments. This paper
proposes a general architecture comprising both control paradigms by
introducing an arbitrator that controls which subsystem is used at any time.
This system is implemented for a target-reaching task with a simulated
two-joint robotic arm that comprises a supervised internal model and deep
reinforcement learning. Through permutation of target-reaching conditions, we
demonstrate that the proposed is capable of rapidly learning kinematics of the
system without a priori knowledge, and is robust to (A) changing environmental
reward and kinematics, and (B) occluded vision. The arbitrator model is
compared to exclusive deliberate planning with the internal model and exclusive
habitual control instances of the model. The results show how such a model can
harness the benefits of both systems, using fast decisions in reliable
circumstances while optimizing performance in changing environments. In
addition, the proposed model learns very fast. Finally, the system which
includes internal models is able to reach the target under the visual
occlusion, while the pure habitual system is unable to operate sufficiently
under such conditions.
Sparse learning of stochastic dynamic equations
Authors: Lorenzo Boninsegna, Feliks Nüske, Cecilia Clementi
Date: 2017-12-06 00:00:00
Tasks: Denoising, Sparse Learning

With the rapid increase of available data for complex systems, there is great
interest in the extraction of physically relevant information from massive
datasets. Recently, a framework called Sparse Identification of Nonlinear
Dynamics (SINDy) has been introduced to identify the governing equations of
dynamical systems from simulation data. In this study, we extend SINDy to
stochastic dynamical systems, which are frequently used to model biophysical
processes. We prove the asymptotic correctness of stochastics SINDy in the
infinite data limit, both in the original and projected variables. We discuss
algorithms to solve the sparse regression problem arising from the practical
implementation of SINDy, and show that cross validation is an essential tool to
determine the right level of sparsity. We demonstrate the proposed methodology
on two test systems, namely, the diffusion in a one-dimensional potential, and
the projected dynamics of a two-dimensional diffusion process.
High performance ultra-low-precision convolutions on mobile devices
Authors: Andrew Tulloch, Yangqing Jia
Date: 2017-12-06 00:00:00
Tasks: CPU, Deep Learning, Vocal Bursts Intensity Prediction

Many applications of mobile deep learning, especially real-time computer
vision workloads, are constrained by computation power. This is particularly
true for workloads running on older consumer phones, where a typical device
might be powered by a single- or dual-core ARMv7 CPU. We provide an open-source
implementation and a comprehensive analysis of (to our knowledge) the state of
the art ultra-low-precision (<4 bit precision) implementation of the core
primitives required for modern deep learning workloads on ARMv7 devices, and
demonstrate speedups of 4x-20x over our additional state-of-the-art float32 and
int8 baselines.
Tomographic Reconstruction using Global Statistical Prior
Authors: Preeti Gopal, Ritwick Chaudhry, Sharat Chandran, Imants Svalbe, Ajit Rajwade
Date: 2017-12-06 00:00:00
Tasks: Anatomy, Compressive Sensing

Recent research in tomographic reconstruction is motivated by the need to
efficiently recover detailed anatomy from limited measurements. One of the ways
to compensate for the increasingly sparse sets of measurements is to exploit
the information from templates, i.e., prior data available in the form of
already reconstructed, structurally similar images. Towards this, previous work
has exploited using a set of global and patch based dictionary priors. In this
paper, we propose a global prior to improve both the speed and quality of
tomographic reconstruction within a Compressive Sensing framework.
  We choose a set of potential representative 2D images referred to as
templates, to build an eigenspace; this is subsequently used to guide the
iterative reconstruction of a similar slice from sparse acquisition data. Our
experiments across a diverse range of datasets show that reconstruction using
an appropriate global prior, apart from being faster, gives a much lower
reconstruction error when compared to the state of the art.
r-BTN: Cross-domain Face Composite and Synthesis from Limited Facial Patches
Authors: Yang Song, Zhifei Zhang, Hairong Qi
Date: 2017-06-02 00:00:00
Tasks: Facial Inpainting, Transfer Learning

We start by asking an interesting yet challenging question, "If an eyewitness
can only recall the eye features of the suspect, such that the forensic artist
can only produce a sketch of the eyes (e.g., the top-left sketch shown in Fig.
1), can advanced computer vision techniques help generate the whole face
image?" A more generalized question is that if a large proportion (e.g., more
than 50%) of the face/sketch is missing, can a realistic whole face
sketch/image still be estimated. Existing face completion and generation
methods either do not conduct domain transfer learning or can not handle large
missing area. For example, the inpainting approach tends to blur the generated
region when the missing area is large (i.e., more than 50%). In this paper, we
exploit the potential of deep learning networks in filling large missing region
(e.g., as high as 95% missing) and generating realistic faces with
high-fidelity in cross domains. We propose the recursive generation by
bidirectional transformation networks (r-BTN) that recursively generates a
whole face/sketch from a small sketch/face patch. The large missing area and
the cross domain challenge make it difficult to generate satisfactory results
using a unidirectional cross-domain learning structure. On the other hand, a
forward and backward bidirectional learning between the face and sketch domains
would enable recursive estimation of the missing region in an incremental
manner (Fig. 1) and yield appealing results. r-BTN also adopts an adversarial
constraint to encourage the generation of realistic faces/sketches. Extensive
experiments have been conducted to demonstrate the superior performance from
r-BTN as compared to existing potential solutions.
Wasserstein GAN
Authors: Martin Arjovsky, Soumith Chintala, Léon Bottou
Date: 2017-01-26 00:00:00
Tasks: Image Generation, Synthetic Data Generation

We introduce a new algorithm named WGAN, an alternative to traditional GAN
training. In this new model, we show that we can improve the stability of
learning, get rid of problems like mode collapse, and provide meaningful
learning curves useful for debugging and hyperparameter searches. Furthermore,
we show that the corresponding optimization problem is sound, and provide
extensive theoretical work highlighting the deep connections to other distances
between distributions.
Limitations on Variance-Reduction and Acceleration Schemes for Finite Sum Optimization
Authors: Yossi Arjevani
Date: 2017-06-06 00:00:00
Proceeding: NeurIPS 2017

We study the conditions under which one is able to efficiently apply
variance-reduction and acceleration schemes on finite sum optimization
problems. First, we show that, perhaps surprisingly, the finite sum structure
by itself, is not sufficient for obtaining a complexity bound of
$\tilde{\cO}((n+L/\mu)\ln(1/\epsilon))$ for $L$-smooth and $\mu$-strongly
convex individual functions - one must also know which individual function is
being referred to by the oracle at each iteration. Next, we show that for a
broad class of first-order and coordinate-descent finite sum algorithms
(including, e.g., SDCA, SVRG, SAG), it is not possible to get an `accelerated'
complexity bound of $\tilde{\cO}((n+\sqrt{n L/\mu})\ln(1/\epsilon))$, unless
the strong convexity parameter is given explicitly. Lastly, we show that when
this class of algorithms is used for minimizing $L$-smooth and convex finite
sums, the optimal complexity bound is $\tilde{\cO}(n+L/\epsilon)$, assuming
that (on average) the same update rule is used in every iteration, and
$\tilde{\cO}(n+\sqrt{nL/\epsilon})$, otherwise.
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
Authors: Andrea Soltoggio, Kenneth O. Stanley, Sebastian Risi
Date: 2017-03-30 00:00:00

Biological plastic neural networks are systems of extraordinary computational
capabilities shaped by evolution, development, and lifetime learning. The
interplay of these elements leads to the emergence of adaptive behavior and
intelligence. Inspired by such intricate natural phenomena, Evolved Plastic
Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed
plastic neural networks with a large variety of dynamics, architectures, and
plasticity rules: these artificial systems are composed of inputs, outputs, and
plastic components that change in response to experiences in an environment.
These systems may autonomously discover novel adaptive algorithms, and lead to
hypotheses on the emergence of biological adaptation. EPANNs have seen
considerable progress over the last two decades. Current scientific and
technological advances in artificial neural networks are now setting the
conditions for radically new approaches and results. In particular, the
limitations of hand-designed networks could be overcome by more flexible and
innovative solutions. This paper brings together a variety of inspiring ideas
that define the field of EPANNs. The main methods and results are reviewed.
Finally, new opportunities and developments are presented.
SGAN: An Alternative Training of Generative Adversarial Networks
Authors: Tatjana Chavdarova, François Fleuret
Date: 2017-12-06 00:00:00
Proceeding: CVPR 2018 6

The Generative Adversarial Networks (GANs) have demonstrated impressive
performance for data synthesis, and are now used in a wide range of computer
vision tasks. In spite of this success, they gained a reputation for being
difficult to train, what results in a time-consuming and human-involved
development process to use them.
  We consider an alternative training process, named SGAN, in which several
adversarial "local" pairs of networks are trained independently so that a
"global" supervising pair of networks can be trained against them. The goal is
to train the global pair with the corresponding ensemble opponent for improved
performances in terms of mode coverage. This approach aims at increasing the
chances that learning will not stop for the global pair, preventing both to be
trapped in an unsatisfactory local minimum, or to face oscillations often
observed in practice. To guarantee the latter, the global pair never affects
the local ones.
  The rules of SGAN training are thus as follows: the global generator and
discriminator are trained using the local discriminators and generators,
respectively, whereas the local networks are trained with their fixed local
opponent.
  Experimental results on both toy and real-world problems demonstrate that
this approach outperforms standard training in terms of better mitigating mode
collapse, stability while converging and that it surprisingly, increases the
convergence speed as well.