pwc-archive/papers-with-abstracts huggingface dataset:
Blind system identification using kernel-based methods
Authors: Giulio Bottegal, Riccardo S. Risuleo, Håkan Hjalmarsson
Date: 2014-12-12 00:00:00

We propose a new method for blind system identification. Resorting to a
Gaussian regression framework, we model the impulse response of the unknown
linear system as a realization of a Gaussian process. The structure of the
covariance matrix (or kernel) of such a process is given by the stable spline
kernel, which has been recently introduced for system identification purposes
and depends on an unknown hyperparameter. We assume that the input can be
linearly described by few parameters. We estimate these parameters, together
with the kernel hyperparameter and the noise variance, using an empirical Bayes
approach. The related optimization problem is efficiently solved with a novel
iterative scheme based on the Expectation-Maximization method. In particular,
we show that each iteration consists of a set of simple update rules. We show,
through some numerical experiments, very promising performance of the proposed
method.
On the estimation of initial conditions in kernel-based system identification
Authors: Riccardo Sven Risuleo, Giulio Bottegal, Håkan Hjalmarsson
Date: 2015-04-30 00:00:00

Recent developments in system identification have brought attention to
regularized kernel-based methods, where, adopting the recently introduced
stable spline kernel, prior information on the unknown process is enforced.
This reduces the variance of the estimates and thus makes kernel-based methods
particularly attractive when few input-output data samples are available. In
such cases however, the influence of the system initial conditions may have a
significant impact on the output dynamics. In this paper, we specifically
address this point. We propose three methods that deal with the estimation of
initial conditions using different types of information. The methods consist in
various mixed maximum likelihood--a posteriori estimators which estimate the
initial conditions and tune the hyperparameters characterizing the stable
spline kernel. To solve the related optimization problems, we resort to the
expectation-maximization method, showing that the solutions can be attained by
iterating among simple update steps. Numerical experiments show the advantages,
in terms of accuracy in reconstructing the system impulse response, of the
proposed strategies, compared to other kernel-based schemes not accounting for
the effect initial conditions.
Siamese Instance Search for Tracking
Authors: Ran Tao, Efstratios Gavves, Arnold W. M. Smeulders
Date: 2016-05-19 00:00:00
Proceeding: CVPR 2016 6
Tasks: Geometric Matching, Instance Search

In this paper we present a tracker, which is radically different from
state-of-the-art trackers: we apply no model updating, no occlusion detection,
no combination of trackers, no geometric matching, and still deliver
state-of-the-art tracking performance, as demonstrated on the popular online
tracking benchmark (OTB) and six very challenging YouTube videos. The presented
tracker simply matches the initial patch of the target in the first frame with
candidates in a new frame and returns the most similar patch by a learned
matching function. The strength of the matching function comes from being
extensively trained generically, i.e., without any data of the target, using a
Siamese deep neural network, which we design for tracking. Once learned, the
matching function is used as is, without any adapting, to track previously
unseen targets. It turns out that the learned matching function is so powerful
that a simple tracker built upon it, coined Siamese INstance search Tracker,
SINT, which only uses the original observation of the target from the first
frame, suffices to reach state-of-the-art performance. Further, we show the
proposed tracker even allows for target re-identification after the target was
absent for a complete video shot.
Syntactically Guided Neural Machine Translation
Authors: Felix Stahlberg, Eva Hasler, Aurelien Waite, Bill Byrne
Date: 2016-05-15 00:00:00
Proceeding: ACL 2016 8
Tasks: Decoder, Language Modeling, Language Modelling, Machine Translation, NMT, Translation

We investigate the use of hierarchical phrase-based SMT lattices in
end-to-end neural machine translation (NMT). Weight pushing transforms the
Hiero scores for complete translation hypotheses, with the full translation
grammar score and full n-gram language model score, into posteriors compatible
with NMT predictive probabilities. With a slightly modified NMT beam-search
decoder we find gains over both Hiero and NMT decoding alone, with practical
advantages in extending NMT to very large input and output vocabularies.
Texture Synthesis Through Convolutional Neural Networks and Spectrum Constraints
Authors: Gang Liu, Yann Gousseau, Gui-Song Xia
Date: 2016-05-04 00:00:00
Tasks: Texture Synthesis

This paper presents a significant improvement for the synthesis of texture
images using convolutional neural networks (CNNs), making use of constraints on
the Fourier spectrum of the results. More precisely, the texture synthesis is
regarded as a constrained optimization problem, with constraints conditioning
both the Fourier spectrum and statistical features learned by CNNs. In contrast
with existing methods, the presented method inherits from previous CNN
approaches the ability to depict local structures and fine scale details, and
at the same time yields coherent large scale structures, even in the case of
quasi-periodic images. This is done at no extra computational cost. Synthesis
experiments on various images show a clear improvement compared to a recent
state-of-the art method relying on CNN constraints only.
On the Sampling Strategy for Evaluation of Spectral-spatial Methods in Hyperspectral Image Classification
Authors: Jie Liang, Jun Zhou, Yuntao Qian, Lian Wen, Xiao Bai, Yongsheng Gao
Date: 2016-05-19 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification, Learning Theory

Spectral-spatial processing has been increasingly explored in remote sensing
hyperspectral image classification. While extensive studies have focused on
developing methods to improve the classification accuracy, experimental setting
and design for method evaluation have drawn little attention. In the scope of
supervised classification, we find that traditional experimental designs for
spectral processing are often improperly used in the spectral-spatial
processing context, leading to unfair or biased performance evaluation. This is
especially the case when training and testing samples are randomly drawn from
the same image - a practice that has been commonly adopted in the experiments.
Under such setting, the dependence caused by overlap between the training and
testing samples may be artificially enhanced by some spatial information
processing methods such as spatial filtering and morphological operation. Such
interaction between training and testing sets has violated data independence
assumption that is abided by supervised learning theory and performance
evaluation mechanism. Therefore, the widely adopted pixel-based random sampling
strategy is not always suitable to evaluate spectral-spatial classification
algorithms because it is difficult to determine whether the improvement of
classification accuracy is caused by incorporating spatial information into
classifier or by increasing the overlap between training and testing samples.
To partially solve this problem, we propose a novel controlled random sampling
strategy for spectral-spatial methods. It can greatly reduce the overlap
between training and testing samples and provides more objective and accurate
evaluation.
Declarative Machine Learning - A Classification of Basic Properties and Types
Authors: Matthias Boehm, Alexandre V. Evfimievski, Niketan Pansare, Berthold Reinwald
Date: 2016-05-19 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Declarative machine learning (ML) aims at the high-level specification of ML
tasks or algorithms, and automatic generation of optimized execution plans from
these specifications. The fundamental goal is to simplify the usage and/or
development of ML algorithms, which is especially important in the context of
large-scale computations. However, ML systems at different abstraction levels
have emerged over time and accordingly there has been a controversy about the
meaning of this general definition of declarative ML. Specification
alternatives range from ML algorithms expressed in domain-specific languages
(DSLs) with optimization for performance, to ML task (learning problem)
specifications with optimization for performance and accuracy. We argue that
these different types of declarative ML complement each other as they address
different users (data scientists and end users). This paper makes an attempt to
create a taxonomy for declarative ML, including a definition of essential basic
properties and types of declarative ML. Along the way, we provide insights into
implications of these properties. We also use this taxonomy to classify
existing systems. Finally, we draw conclusions on defining appropriate
benchmarks and specification languages for declarative ML.
Clustering from Sparse Pairwise Measurements
Authors: Alaa Saade, Marc Lelarge, Florent Krzakala, Lenka Zdeborová
Date: 2016-01-25 00:00:00
Tasks: Clustering

We consider the problem of grouping items into clusters based on few random
pairwise comparisons between the items. We introduce three closely related
algorithms for this task: a belief propagation algorithm approximating the
Bayes optimal solution, and two spectral algorithms based on the
non-backtracking and Bethe Hessian operators. For the case of two symmetric
clusters, we conjecture that these algorithms are asymptotically optimal in
that they detect the clusters as soon as it is information theoretically
possible to do so. We substantiate this claim for one of the spectral
approaches we introduce.
Bacterial foraging optimization based brain magnetic resonance image segmentation
Authors: Abdul kayom Md Khairuzzaman
Date: 2016-05-19 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Segmentation partitions an image into its constituent parts. It is
essentially the pre-processing stage of image analysis and computer vision. In
this work, T1 and T2 weighted brain magnetic resonance images are segmented
using multilevel thresholding and bacterial foraging optimization (BFO)
algorithm. The thresholds are obtained by maximizing the between class variance
(multilevel Otsu method) of the image. The BFO algorithm is used to optimize
the threshold searching process. The edges are then obtained from the
thresholded image by comparing the intensity of each pixel with its eight
connected neighbourhood. Post processing is performed to remove spurious
responses in the segmented image. The proposed segmentation technique is
evaluated using edge detector evaluation parameters such as figure of merit,
Rand Index and variation of information. The proposed brain MR image
segmentation technique outperforms the traditional edge detectors such as canny
and sobel.
Recurrent Exponential-Family Harmoniums without Backprop-Through-Time
Authors: Joseph G. Makin, Benjamin K. Dichter, Philip N. Sabes
Date: 2016-05-19 00:00:00

Exponential-family harmoniums (EFHs), which extend restricted Boltzmann
machines (RBMs) from Bernoulli random variables to other exponential families
(Welling et al., 2005), are generative models that can be trained with
unsupervised-learning techniques, like contrastive divergence (Hinton et al.
2006; Hinton, 2002), as density estimators for static data. Methods for
extending RBMs--and likewise EFHs--to data with temporal dependencies have been
proposed previously (Sutskever and Hinton, 2007; Sutskever et al., 2009), the
learning procedure being validated by qualitative assessment of the generative
model. Here we propose and justify, from a very different perspective, an
alternative training procedure, proving sufficient conditions for optimal
inference under that procedure. The resulting algorithm can be learned with
only forward passes through the data--backprop-through-time is not required, as
in previous approaches. The proof exploits a recent result about information
retention in density estimators (Makin and Sabes, 2015), and applies it to a
"recurrent EFH" (rEFH) by induction. Finally, we demonstrate optimality by
simulation, testing the rEFH: (1) as a filter on training data generated with a
linear dynamical system, the position of which is noisily reported by a
population of "neurons" with Poisson-distributed spike counts; and (2) with the
qualitative experiments proposed by Sutskever et al. (2009).
A Generic Framework for Assessing the Performance Bounds of Image Feature Detectors
Authors: Shoaib Ehsan, Adrian F. Clark, Ales Leonardis, Naveed Ur Rehman, Klaus D. McDonald-Maier
Date: 2016-05-19 00:00:00

Since local feature detection has been one of the most active research areas
in computer vision during the last decade, a large number of detectors have
been proposed. The interest in feature-based applications continues to grow and
has thus rendered the task of characterizing the performance of various feature
detection methods an important issue in vision research. Inspired by the good
practices of electronic system design, a generic framework based on the
repeatability measure is presented in this paper that allows assessment of the
upper and lower bounds of detector performance and finds statistically
significant performance differences between detectors as a function of image
transformation amount by introducing a new variant of McNemars test in an
effort to design more reliable and effective vision systems. The proposed
framework is then employed to establish operating and guarantee regions for
several state-of-the-art detectors and to identify their statistical
performance differences for three specific image transformations: JPEG
compression, uniform light changes and blurring. The results are obtained using
a newly acquired, large image database (20482) images with 539 different
scenes. These results provide new insights into the behaviour of detectors and
are also useful from the vision systems design perspective.
A comparison of semi-deterministic and stochastic search techniques
Authors: Andy M. Connor, Kristina Shea
Date: 2016-05-18 00:00:00

This paper presents an investigation of two search techniques, tabu search
(TS) and simulated annealing (SA), to assess their relative merits when applied
to engineering design optimisation. Design optimisation problems are generally
characterised as having multi-modal search spaces and discontinuities making
global optimisation techniques beneficial. Both techniques claim to be capable
of locating globally optimum solutions on a range of problems but this
capability is derived from different underlying philosophies. While tabu search
uses a semi-deterministic approach to escape local optima, simulated annealing
uses a complete stochastic approach. The performance of each technique is
investigated using a structural optimisation problem. These performances are
then compared to each other as and to a steepest descent (SD) method.
Beamforming through regularized inverse problems in ultrasound medical imaging
Authors: Teodora Szasz, Adrian Basarab, Denis Kouamé
Date: 2015-07-29 00:00:00

Beamforming in ultrasound imaging has significant impact on the quality of
the final image, controlling its resolution and contrast. Despite its low
spatial resolution and contrast, delay-and-sum is still extensively used
nowadays in clinical applications, due to its real-time capabilities. The most
common alternatives are minimum variance method and its variants, which
overcome the drawbacks of delay-and-sum, at the cost of higher computational
complexity that limits its utilization in real-time applications.
  In this paper, we propose to perform beamforming in ultrasound imaging
through a regularized inverse problem based on a linear model relating the
reflected echoes to the signal to be recovered. Our approach presents two major
advantages: i) its flexibility in the choice of statistical assumptions on the
signal to be beamformed (Laplacian and Gaussian statistics are tested herein)
and ii) its robustness to a reduced number of pulse emissions. The proposed
framework is flexible and allows for choosing the right trade-off between noise
suppression and sharpness of the resulted image. We illustrate the performance
of our approach on both simulated and experimental data, with \textit{in vivo}
examples of carotid and thyroid. Compared to delay-and-sum, minimimum variance
and two other recently published beamforming techniques, our method offers
better spatial resolution, respectively contrast, when using Laplacian and
Gaussian priors.
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
Authors: Jure Žbontar, Yann Lecun
Date: 2015-10-20 00:00:00
Tasks: Binary Classification, Stereo Matching, Stereo Matching Hand

We present a method for extracting depth information from a rectified image
pair. Our approach focuses on the first stage of many stereo algorithms: the
matching cost computation. We approach the problem by learning a similarity
measure on small image patches using a convolutional neural network. Training
is carried out in a supervised manner by constructing a binary classification
data set with examples of similar and dissimilar pairs of patches. We examine
two network architectures for this task: one tuned for speed, the other for
accuracy. The output of the convolutional neural network is used to initialize
the stereo matching cost. A series of post-processing steps follow: cross-based
cost aggregation, semiglobal matching, a left-right consistency check, subpixel
enhancement, a median filter, and a bilateral filter. We evaluate our method on
the KITTI 2012, KITTI 2015, and Middlebury stereo data sets and show that it
outperforms other approaches on all three data sets.
The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response
Authors: Lina Al-Kanj, Warren B. Powell, Belgacem Bouzaiene-Ayari
Date: 2016-05-18 00:00:00
Tasks: Stochastic Optimization

Utilities face the challenge of responding to power outages due to storms and
ice damage, but most power grids are not equipped with sensors to pinpoint the
precise location of the faults causing the outage. Instead, utilities have to
depend primarily on phone calls (trouble calls) from customers who have lost
power to guide the dispatching of utility trucks. In this paper, we develop a
policy that routes a utility truck to restore outages in the power grid as
quickly as possible, using phone calls to create beliefs about outages, but
also using utility trucks as a mechanism for collecting additional information.
This means that routing decisions change not only the physical state of the
truck (as it moves from one location to another) and the grid (as the truck
performs repairs), but also our belief about the network, creating the first
stochastic vehicle routing problem that explicitly models information
collection and belief modeling. We address the problem of managing a single
utility truck, which we start by formulating as a sequential stochastic
optimization model which captures our belief about the state of the grid. We
propose a stochastic lookahead policy, and use Monte Carlo tree search (MCTS)
to produce a practical policy that is asymptotically optimal. Simulation
results show that the developed policy restores the power grid much faster
compared to standard industry heuristics.
Active Learning On Weighted Graphs Using Adaptive And Non-adaptive Approaches
Authors: Eyal En Gad, Akshay Gadde, A. Salman Avestimehr, Antonio Ortega
Date: 2016-05-18 00:00:00
Tasks: Active Learning

This paper studies graph-based active learning, where the goal is to
reconstruct a binary signal defined on the nodes of a weighted graph, by
sampling it on a small subset of the nodes. A new sampling algorithm is
proposed, which sequentially selects the graph nodes to be sampled, based on an
aggressive search for the boundary of the signal over the graph. The algorithm
generalizes a recent method for sampling nodes in unweighted graphs. The
generalization improves the sampling performance using the information gained
from the available graph weights. An analysis of the number of samples required
by the proposed algorithm is provided, and the gain over the unweighted method
is further demonstrated in simulations. Additionally, the proposed method is
compared with an alternative state of-the-art method, which is based on the
graph's spectral properties. It is shown that the proposed method significantly
outperforms the spectral sampling method, if the signal needs to be predicted
with high accuracy. On the other hand, if a higher level of inaccuracy is
tolerable, then the spectral method outperforms the proposed aggressive search
method. Consequently, we propose a hybrid method, which is shown to combine the
advantages of both approaches.
Online Algorithms For Parameter Mean And Variance Estimation In Dynamic Regression Models
Authors: Carlos Alberto Gomez-Uribe
Date: 2016-05-18 00:00:00
Tasks: parameter estimation, regression, Thompson Sampling

We study the problem of estimating the parameters of a regression model from
a set of observations, each consisting of a response and a predictor. The
response is assumed to be related to the predictor via a regression model of
unknown parameters. Often, in such models the parameters to be estimated are
assumed to be constant. Here we consider the more general scenario where the
parameters are allowed to evolve over time, a more natural assumption for many
applications. We model these dynamics via a linear update equation with
additive noise that is often used in a wide range of engineering applications,
particularly in the well-known and widely used Kalman filter (where the system
state it seeks to estimate maps to the parameter values here). We derive an
approximate algorithm to estimate both the mean and the variance of the
parameter estimates in an online fashion for a generic regression model. This
algorithm turns out to be equivalent to the extended Kalman filter. We
specialize our algorithm to the multivariate exponential family distribution to
obtain a generalization of the generalized linear model (GLM). Because the
common regression models encountered in practice such as logistic, exponential
and multinomial all have observations modeled through an exponential family
distribution, our results are used to easily obtain algorithms for online mean
and variance parameter estimation for all these regression models in the
context of time-dependent parameters. Lastly, we propose to use these
algorithms in the contextual multi-armed bandit scenario, where so far model
parameters are assumed static and observations univariate and Gaussian or
Bernoulli. Both of these restrictions can be relaxed using the algorithms
described here, which we combine with Thompson sampling to show the resulting
performance on a simulation.
Towards information based spatiotemporal patterns as a foundation for agent representation in dynamical systems
Authors: Martin Biehl, Takashi Ikegami, Daniel Polani
Date: 2016-05-18 00:00:00
Tasks: Artificial Life, counterfactual

We present some arguments why existing methods for representing agents fall
short in applications crucial to artificial life. Using a thought experiment
involving a fictitious dynamical systems model of the biosphere we argue that
the metabolism, motility, and the concept of counterfactual variation should be
compatible with any agent representation in dynamical systems. We then propose
an information-theoretic notion of \emph{integrated spatiotemporal patterns}
which we believe can serve as the basic building block of an agent definition.
We argue that these patterns are capable of solving the problems mentioned
before. We also test this in some preliminary experiments.