pwc-archive/papers-with-abstracts huggingface dataset:
A Unified Tensor-based Active Appearance Face Model
Authors: Zhen-Hua Feng, Josef Kittler, William Christmas, Xiao-Jun Wu
Date: 2016-12-30 00:00:00
Tasks: Face Model, model

Appearance variations result in many difficulties in face image analysis. To
deal with this challenge, we present a Unified Tensor-based Active Appearance
Model (UT-AAM) for jointly modelling the geometry and texture information of 2D
faces. For each type of face information, namely shape and texture, we
construct a unified tensor model capturing all relevant appearance variations.
This contrasts with the variation-specific models of the classical tensor AAM.
To achieve the unification across pose variations, a strategy for dealing with
self-occluded faces is proposed to obtain consistent shape and texture
representations of pose-varied faces. In addition, our UT-AAM is capable of
constructing the model from an incomplete training dataset, using tensor
completion methods. Last, we use an effective cascaded-regression-based method
for UT-AAM fitting. With these advancements, the utility of UT-AAM in practice
is considerably enhanced. As an example, we demonstrate the improvements in
training facial landmark detectors through the use of UT-AAM to synthesise a
large number of virtual samples. Experimental results obtained using the
Multi-PIE and 300-W face datasets demonstrate the merits of the proposed
approach.
Prediction of Muscle Activations for Reaching Movements using Deep Neural Networks
Authors: Najeeb Khan, Ian Stavness
Date: 2017-06-13 00:00:00

The motor control problem involves determining the time-varying muscle
activation trajectories required to accomplish a given movement. Muscle
redundancy makes motor control a challenging task: there are many possible
activation trajectories that accomplish the same movement. Despite this
redundancy, most movements are accomplished in highly stereotypical ways. For
example, point-to-point reaching movements are almost universally performed
with very similar smooth trajectories. Optimization methods are commonly used
to predict muscle forces for measured movements. However, these approaches
require computationally expensive simulations and are sensitive to the chosen
optimality criteria and regularization. In this work, we investigate deep
autoencoders for the prediction of muscle activation trajectories for
point-to-point reaching movements. We evaluate our DNN predictions with
simulated reaches and two methods to generate the muscle activations: inverse
dynamics (ID) and optimal control (OC) criteria. We also investigate optimal
network parameters and training criteria to improve the accuracy of the
predictions.
Why is Posterior Sampling Better than Optimism for Reinforcement Learning?
Authors: Ian Osband, Benjamin Van Roy
Date: 2016-07-01 00:00:00
Proceeding: ICML 2017 8
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Computational results demonstrate that posterior sampling for reinforcement
learning (PSRL) dramatically outperforms algorithms driven by optimism, such as
UCRL2. We provide insight into the extent of this performance boost and the
phenomenon that drives it. We leverage this insight to establish an
$\tilde{O}(H\sqrt{SAT})$ Bayesian expected regret bound for PSRL in
finite-horizon episodic Markov decision processes, where $H$ is the horizon,
$S$ is the number of states, $A$ is the number of actions and $T$ is the time
elapsed. This improves upon the best previous bound of $\tilde{O}(H S
\sqrt{AT})$ for any reinforcement learning algorithm.
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
Authors: Luke de Oliveira, Michela Paganini, Benjamin Nachman
Date: 2017-01-20 00:00:00
Tasks: Generative Adversarial Network

We provide a bridge between generative modeling in the Machine Learning
community and simulated physical processes in High Energy Particle Physics by
applying a novel Generative Adversarial Network (GAN) architecture to the
production of jet images -- 2D representations of energy depositions from
particles interacting with a calorimeter. We propose a simple architecture, the
Location-Aware Generative Adversarial Network, that learns to produce realistic
radiation patterns from simulated high energy particle collisions. The pixel
intensities of GAN-generated images faithfully span over many orders of
magnitude and exhibit the desired low-dimensional physical properties (i.e.,
jet mass, n-subjettiness, etc.). We shed light on limitations, and provide a
novel empirical validation of image quality and validity of GAN-produced
simulations of the natural world. This work provides a base for further
explorations of GANs for use in faster simulation in High Energy Particle
Physics.
Tensor Balancing on Statistical Manifold
Authors: Mahito Sugiyama, Hiroyuki Nakahara, Koji Tsuda
Date: 2017-02-27 00:00:00
Proceeding: ICML 2017 8

We solve tensor balancing, rescaling an Nth order nonnegative tensor by
multiplying N tensors of order N - 1 so that every fiber sums to one. This
generalizes a fundamental process of matrix balancing used to compare matrices
in a wide range of applications from biology to economics. We present an
efficient balancing algorithm with quadratic convergence using Newton's method
and show in numerical experiments that the proposed algorithm is several orders
of magnitude faster than existing ones. To theoretically prove the correctness
of the algorithm, we model tensors as probability distributions in a
statistical manifold and realize tensor balancing as projection onto a
submanifold. The key to our algorithm is that the gradient of the manifold,
used as a Jacobian matrix in Newton's method, can be analytically obtained
using the Moebius inversion formula, the essential of combinatorial
mathematics. Our model is not limited to tensor balancing, but has a wide
applicability as it includes various statistical and machine learning models
such as weighted DAGs and Boltzmann machines.
Joint Max Margin and Semantic Features for Continuous Event Detection in Complex Scenes
Authors: Iman Abbasnejad, Sridha Sridharan, Simon Denman, Clinton Fookes, Simon Lucey
Date: 2017-06-13 00:00:00
Tasks: Action Recognition, Event Detection, Temporal Action Localization

In this paper the problem of complex event detection in the continuous domain
(i.e. events with unknown starting and ending locations) is addressed. Existing
event detection methods are limited to features that are extracted from the
local spatial or spatio-temporal patches from the videos. However, this makes
the model vulnerable to the events with similar concepts e.g. "Open drawer" and
"Open cupboard". In this work, in order to address the aforementioned
limitations we present a novel model based on the combination of semantic and
temporal features extracted from video frames. We train a max-margin classifier
on top of the extracted features in an adaptive framework that is able to
detect the events with unknown starting and ending locations. Our model is
based on the Bidirectional Region Neural Network and large margin Structural
Output SVM. The generality of our model allows it to be simply applied to
different labeled and unlabeled datasets. We finally test our algorithm on
three challenging datasets, "UCF 101-Action Recognition", "MPII Cooking
Activities" and "Hollywood", and we report state-of-the-art performance.
Robust Budget Allocation via Continuous Submodular Functions
Authors: Matthew Staib, Stefanie Jegelka
Date: 2017-02-28 00:00:00
Proceeding: ICML 2017 8

The optimal allocation of resources for maximizing influence, spread of
information or coverage, has gained attention in the past years, in particular
in machine learning and data mining. But in applications, the parameters of the
problem are rarely known exactly, and using wrong parameters can lead to
undesirable outcomes. We hence revisit a continuous version of the Budget
Allocation or Bipartite Influence Maximization problem introduced by Alon et
al. (2012) from a robust optimization perspective, where an adversary may
choose the least favorable parameters within a confidence set. The resulting
problem is a nonconvex-concave saddle point problem (or game). We show that
this nonconvex problem can be solved exactly by leveraging connections to
continuous submodular functions, and by solving a constrained submodular
minimization problem. Although constrained submodular minimization is hard in
general, here, we establish conditions under which such a problem can be solved
to arbitrary precision $\epsilon$.
Spectral-graph Based Classifications: Linear Regression for Classification and Normalized Radial Basis Function Network
Authors: Zhenfang Hu, Gang Pan, Zhaohui Wu
Date: 2017-05-19 00:00:00
Tasks: General Classification, Model Selection, regression

Spectral graph theory has been widely applied in unsupervised and
semi-supervised learning. In this paper, we find for the first time, to our
knowledge, that it also plays a concrete role in supervised classification. It
turns out that two classifiers are inherently related to the theory: linear
regression for classification (LRC) and normalized radial basis function
network (nRBFN), corresponding to linear and nonlinear kernel respectively. The
spectral graph theory provides us with a new insight into a fundamental aspect
of classification: the tradeoff between fitting error and overfitting risk.
With the theory, ideal working conditions for LRC and nRBFN are presented,
which ensure not only zero fitting error but also low overfitting risk. For
quantitative analysis, two concepts, the fitting error and the spectral risk
(indicating overfitting), have been defined. Their bounds for nRBFN and LRC are
derived. A special result shows that the spectral risk of nRBFN is lower
bounded by the number of classes and upper bounded by the size of radial basis.
When the conditions are not met exactly, the classifiers will pursue the
minimum fitting error, running into the risk of overfitting. It turns out that
$\ell_2$-norm regularization can be applied to control overfitting. Its effect
is explored under the spectral context. It is found that the two terms in the
$\ell_2$-regularized objective are one-one correspondent to the fitting error
and the spectral risk, revealing a tradeoff between the two quantities.
Concerning practical performance, we devise a basis selection strategy to
address the main problem hindering the applications of (n)RBFN. With the
strategy, nRBFN is easy to implement yet flexible. Experiments on 14 benchmark
data sets show the performance of nRBFN is comparable to that of SVM, whereas
the parameter tuning of nRBFN is much easier, leading to reduction of model
selection time.
Zero-Shot Relation Extraction via Reading Comprehension
Authors: Omer Levy, Minjoon Seo, Eunsol Choi, Luke Zettlemoyer
Date: 2017-06-13 00:00:00
Proceeding: CONLL 2017 8
Tasks: Reading Comprehension, Relation, Relation Extraction, slot-filling, Slot Filling, Zero-shot Generalization, Zero-Shot Learning

We show that relation extraction can be reduced to answering simple reading
comprehension questions, by associating one or more natural-language questions
with each relation slot. This reduction has several advantages: we can (1)
learn relation-extraction models by extending recent neural
reading-comprehension techniques, (2) build very large training sets for those
models by combining relation-specific crowd-sourced questions with distant
supervision, and even (3) do zero-shot learning by extracting new relation
types that are only specified at test-time, for which we have no labeled
training examples. Experiments on a Wikipedia slot-filling task demonstrate
that the approach can generalize to new questions for known relation types with
high accuracy, and that zero-shot generalization to unseen relation types is
possible, at lower accuracy levels, setting the bar for future work on this
task.
Technical Report: Implementation and Validation of a Smart Health Application
Authors: Fran Casino, Constantinos Patsakis, Antoni Martinez-Balleste, Frederic Borras, Edgar Batista
Date: 2017-06-13 00:00:00

In this article, we explain in detail the internal structures and databases
of a smart health application. Moreover, we describe how to generate a
statistically sound synthetic dataset using real-world medical data.
Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlations
Authors: Yuanzhi Li, YIngyu Liang
Date: 2017-06-13 00:00:00
Proceeding: ICML 2017 8

Non-negative matrix factorization is a basic tool for decomposing data into
the feature and weight matrices under non-negativity constraints, and in
practice is often solved in the alternating minimization framework. However, it
is unclear whether such algorithms can recover the ground-truth feature matrix
when the weights for different features are highly correlated, which is common
in applications. This paper proposes a simple and natural alternating gradient
descent based algorithm, and shows that with a mild initialization it provably
recovers the ground-truth in the presence of strong correlations. In most
interesting cases, the correlation can be in the same order as the highest
possible. Our analysis also reveals its several favorable features including
robustness to noise. We complement our theoretical results with empirical
studies on semi-synthetic datasets, demonstrating its advantage over several
popular methods in recovering the ground-truth.
Interaction-Based Distributed Learning in Cyber-Physical and Social Networks
Authors: Francesco Sasso, Angelo Coluccia, Giuseppe Notarstefano
Date: 2017-06-13 00:00:00
Tasks: Anomaly Detection

In this paper we consider a network scenario in which agents can evaluate
each other according to a score graph that models some physical or social
interaction. The goal is to design a distributed protocol, run by the agents,
allowing them to learn their unknown state among a finite set of possible
values. We propose a Bayesian framework in which scores and states are
associated to probabilistic events with unknown parameters and hyperparameters
respectively. We prove that each agent can learn its state by means of a local
Bayesian classifier and a (centralized) Maximum-Likelihood (ML) estimator of
the parameter-hyperparameter that combines plain ML and Empirical Bayes
approaches. By using tools from graphical models, which allow us to gain
insight on conditional dependences of scores and states, we provide two relaxed
probabilistic models that ultimately lead to ML parameter-hyperparameter
estimators amenable to distributed computation. In order to highlight the
appropriateness of the proposed relaxations, we demonstrate the distributed
estimators on a machine-to-machine testing set-up for anomaly detection and on
a social interaction set-up for user profiling.
Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation
Authors: Jinzhuo Wang, Wenmin Wang, Ronggang Wang, Wen Gao
Date: 2017-06-13 00:00:00

Monte Carlo tree search (MCTS) is extremely popular in computer Go which
determines each action by enormous simulations in a broad and deep search tree.
However, human experts select most actions by pattern analysis and careful
evaluation rather than brute search of millions of future nteractions. In this
paper, we propose a computer Go system that follows experts way of thinking and
playing. Our system consists of two parts. The first part is a novel deep
alternative neural network (DANN) used to generate candidates of next move.
Compared with existing deep convolutional neural network (DCNN), DANN inserts
recurrent layer after each convolutional layer and stacks them in an
alternative manner. We show such setting can preserve more contexts of local
features and its evolutions which are beneficial for move prediction. The
second part is a long-term evaluation (LTE) module used to provide a reliable
evaluation of candidates rather than a single probability from move predictor.
This is consistent with human experts nature of playing since they can foresee
tens of steps to give an accurate estimation of candidates. In our system, for
each candidate, LTE calculates a cumulative reward after several future
interactions when local variations are settled. Combining criteria from the two
parts, our system determines the optimal choice of next move. For more
comprehensive experiments, we introduce a new professional Go dataset (PGD),
consisting of 253233 professional records. Experiments on GoGoD and PGD
datasets show the DANN can substantially improve performance of move prediction
over pure DCNN. When combining LTE, our system outperforms most relevant
approaches and open engines based on MCTS.
Bayesian optimisation for fast approximate inference in state-space models with intractable likelihoods
Authors: Johan Dahlin, Mattias Villani, Thomas B. Schön
Date: 2015-06-23 00:00:00
Tasks: Bayesian Optimisation, State Space Models

We consider the problem of approximate Bayesian parameter inference in
non-linear state-space models with intractable likelihoods. Sequential Monte
Carlo with approximate Bayesian computations (SMC-ABC) is one approach to
approximate the likelihood in this type of models. However, such approximations
can be noisy and computationally costly which hinders efficient implementations
using standard methods based on optimisation and Monte Carlo methods. We
propose a computationally efficient novel method based on the combination of
Gaussian process optimisation and SMC-ABC to create a Laplace approximation of
the intractable posterior. We exemplify the proposed algorithm for inference in
stochastic volatility models with both synthetic and real-world data as well as
for estimating the Value-at-Risk for two portfolios using a copula model. We
document speed-ups of between one and two orders of magnitude compared to
state-of-the-art algorithms for posterior inference.
On Natural Language Generation of Formal Argumentation
Authors: Federico Cerutti, Alice Toniolo, Timothy J. Norman
Date: 2017-06-13 00:00:00
Tasks: Text Generation

In this paper we provide a first analysis of the research questions that
arise when dealing with the problem of communicating pieces of formal
argumentation through natural language interfaces. It is a generally held
opinion that formal models of argumentation naturally capture human argument,
and some preliminary studies have focused on justifying this view.
Unfortunately, the results are not only inconclusive, but seem to suggest that
explaining formal argumentation to humans is a rather articulated task.
Graphical models for expressing argumentation-based reasoning are appealing,
but often humans require significant training to use these tools effectively.
We claim that natural language interfaces to formal argumentation systems offer
a real alternative, and may be the way forward for systems that capture human
argument.
Recurrent Latent Variable Networks for Session-Based Recommendation
Authors: Sotirios Chatzis, Panayiotis Christodoulou, Andreas S. Andreou
Date: 2017-06-13 00:00:00
Tasks: Session-Based Recommendations, Variational Inference

In this work, we attempt to ameliorate the impact of data sparsity in the
context of session-based recommendation. Specifically, we seek to devise a
machine learning mechanism capable of extracting subtle and complex underlying
temporal dynamics in the observed session data, so as to inform the
recommendation algorithm. To this end, we improve upon systems that utilize
deep learning techniques with recurrently connected units; we do so by adopting
concepts from the field of Bayesian statistics, namely variational inference.
Our proposed approach consists in treating the network recurrent units as
stochastic latent variables with a prior distribution imposed over them. On
this basis, we proceed to infer corresponding posteriors; these can be used for
prediction and recommendation generation, in a way that accounts for the
uncertainty in the available sparse training data. To allow for our approach to
easily scale to large real-world datasets, we perform inference under an
approximate amortized variational inference (AVI) setup, whereby the learned
posteriors are parameterized via (conventional) neural networks. We perform an
extensive experimental evaluation of our approach using challenging benchmark
datasets, and illustrate its superiority over existing state-of-the-art
techniques.
Orthogonal Echo State Networks and stochastic evaluations of likelihoods
Authors: Norbert Michael Mayer, Ying-Hao Yu
Date: 2016-01-22 00:00:00
Tasks: Time Series, Time Series Analysis

We report about probabilistic likelihood estimates that are performed on time
series using an echo state network with orthogonal recurrent connectivity. The
results from tests using synthetic stochastic input time series with temporal
inference indicate that the capability of the network to infer depends on the
balance between input strength and recurrent activity. This balance has an
influence on the network with regard to the quality of inference from the short
term input history versus inference that accounts for influences that date back
a long time. Sensitivity of such networks against noise and the finite accuracy
of network states in the recurrent layer are investigated. In addition, a
measure based on mutual information between the output time series and the
reservoir is introduced. Finally, different types of recurrent connectivity are
evaluated. Orthogonal matrices show the best results of all investigated
connectivity types overall, but also in the way how the network performance
scales with the size of the recurrent layer.
Recurrent Inference Machines for Solving Inverse Problems
Authors: Patrick Putzky, Max Welling
Date: 2017-06-13 00:00:00
Tasks: Denoising, Image Denoising, Image Restoration, Super-Resolution

Much of the recent research on solving iterative inference problems focuses
on moving away from hand-chosen inference algorithms and towards learned
inference. In the latter, the inference process is unrolled in time and
interpreted as a recurrent neural network (RNN) which allows for joint learning
of model and inference parameters with back-propagation through time. In this
framework, the RNN architecture is directly derived from a hand-chosen
inference algorithm, effectively limiting its capabilities. We propose a
learning framework, called Recurrent Inference Machines (RIM), in which we turn
algorithm construction the other way round: Given data and a task, train an RNN
to learn an inference algorithm. Because RNNs are Turing complete [1, 2] they
are capable to implement any inference algorithm. The framework allows for an
abstraction which removes the need for domain knowledge. We demonstrate in
several image restoration experiments that this abstraction is effective,
allowing us to achieve state-of-the-art performance on image denoising and
super-resolution tasks and superior across-task generalization.
Variational Dropout Sparsifies Deep Neural Networks
Authors: Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov
Date: 2017-01-19 00:00:00
Proceeding: ICML 2017 8
Tasks: Sparse Learning

We explore a recently proposed Variational Dropout technique that provided an
elegant Bayesian interpretation to Gaussian Dropout. We extend Variational
Dropout to the case when dropout rates are unbounded, propose a way to reduce
the variance of the gradient estimator and report first experimental results
with individual dropout rates per weight. Interestingly, it leads to extremely
sparse solutions both in fully-connected and convolutional layers. This effect
is similar to automatic relevance determination effect in empirical Bayes but
has a number of advantages. We reduce the number of parameters up to 280 times
on LeNet architectures and up to 68 times on VGG-like networks with a
negligible decrease of accuracy.