pwc-archive/papers-with-abstracts huggingface dataset:
Quantifying Emergent Behavior of Autonomous Robots
Authors: Georg Martius, Eckehard Olbrich
Date: 2015-10-06 00:00:00
Tasks: Time Series, Time Series Analysis

Quantifying behaviors of robots which were generated autonomously from
task-independent objective functions is an important prerequisite for objective
comparisons of algorithms and movements of animals. The temporal sequence of
such a behavior can be considered as a time series and hence complexity
measures developed for time series are natural candidates for its
quantification. The predictive information and the excess entropy are such
complexity measures. They measure the amount of information the past contains
about the future and thus quantify the nonrandom structure in the temporal
sequence. However, when using these measures for systems with continuous states
one has to deal with the fact that their values will depend on the resolution
with which the systems states are observed. For deterministic systems both
measures will diverge with increasing resolution. We therefore propose a new
decomposition of the excess entropy in resolution dependent and resolution
independent parts and discuss how they depend on the dimensionality of the
dynamics, correlations and the noise level. For the practical estimation we
propose to use estimates based on the correlation integral instead of the
direct estimation of the mutual information using the algorithm by Kraskov et
al. (2004) which is based on next neighbor statistics because the latter allows
less control of the scale dependencies. Using our algorithm we are able to show
how autonomous learning generates behavior of increasing complexity with
increasing learning duration.
Directional Global Three-part Image Decomposition
Authors: Duy Hoang Thai, Carsten Gottschlich
Date: 2015-10-06 00:00:00
Tasks: Image Compression, Optical Character Recognition, Optical Character Recognition (OCR)

We consider the task of image decomposition and we introduce a new model
coined directional global three-part decomposition (DG3PD) for solving it. As
key ingredients of the DG3PD model, we introduce a discrete multi-directional
total variation norm and a discrete multi-directional G-norm. Using these novel
norms, the proposed discrete DG3PD model can decompose an image into two parts
or into three parts. Existing models for image decomposition by Vese and Osher,
by Aujol and Chambolle, by Starck et al., and by Thai and Gottschlich are
included as special cases in the new model. Decomposition of an image by DG3PD
results in a cartoon image, a texture image and a residual image. Advantages of
the DG3PD model over existing ones lie in the properties enforced on the
cartoon and texture images. The geometric objects in the cartoon image have a
very smooth surface and sharp edges. The texture image yields oscillating
patterns on a defined scale which is both smooth and sparse. Moreover, the
DG3PD method achieves the goal of perfect reconstruction by summation of all
components better than the other considered methods. Relevant applications of
DG3PD are a novel way of image compression as well as feature extraction for
applications such as latent fingerprint processing and optical character
recognition.
Bayesian Markov Blanket Estimation
Authors: Dinu Kaufmann, Sonali Parbhoo, Aleksander Wieczorek, Sebastian Keller, David Adametz, Volker Roth
Date: 2015-10-06 00:00:00

This paper considers a Bayesian view for estimating a sub-network in a Markov
random field. The sub-network corresponds to the Markov blanket of a set of
query variables, where the set of potential neighbours here is big. We
factorize the posterior such that the Markov blanket is conditionally
independent of the network of the potential neighbours. By exploiting this
blockwise decoupling, we derive analytic expressions for posterior
conditionals. Subsequently, we develop an inference scheme which makes use of
the factorization. As a result, estimation of a sub-network is possible without
inferring an entire network. Since the resulting Gibbs sampler scales linearly
with the number of variables, it can handle relatively large neighbourhoods.
The proposed scheme results in faster convergence and superior mixing of the
Markov chain than existing Bayesian network estimation techniques.
Local Rademacher Complexity Bounds based on Covering Numbers
Authors: Yunwen Lei, Lixin Ding, Yingzhou Bi
Date: 2015-10-06 00:00:00

This paper provides a general result on controlling local Rademacher
complexities, which captures in an elegant form to relate the complexities with
constraint on the expected norm to the corresponding ones with constraint on
the empirical norm. This result is convenient to apply in real applications and
could yield refined local Rademacher complexity bounds for function classes
satisfying general entropy conditions. We demonstrate the power of our
complexity bounds by applying them to derive effective generalization error
bounds.
A Waveform Representation Framework for High-quality Statistical Parametric Speech Synthesis
Authors: Bo Fan, Siu Wa Lee, Xiaohai Tian, Lei Xie, Minghui Dong
Date: 2015-10-06 00:00:00
Tasks: Speech Synthesis, Vocal Bursts Intensity Prediction

State-of-the-art statistical parametric speech synthesis (SPSS) generally
uses a vocoder to represent speech signals and parameterize them into features
for subsequent modeling. Magnitude spectrum has been a dominant feature over
the years. Although perceptual studies have shown that phase spectrum is
essential to the quality of synthesized speech, it is often ignored by using a
minimum phase filter during synthesis and the speech quality suffers. To bypass
this bottleneck in vocoded speech, this paper proposes a phase-embedded
waveform representation framework and establishes a magnitude-phase joint
modeling platform for high-quality SPSS. Our experiments on waveform
reconstruction show that the performance is better than that of the widely-used
STRAIGHT. Furthermore, the proposed modeling and synthesis platform outperforms
a leading-edge, vocoded, deep bidirectional long short-term memory recurrent
neural network (DBLSTM-RNN)-based baseline system in various objective
evaluation metrics conducted.
Bayesian Masking: Sparse Bayesian Estimation with Weaker Shrinkage Bias
Authors: Yohei Kondo, Kohei Hayashi, Shin-ichi Maeda
Date: 2015-09-03 00:00:00
Tasks: Bayesian Inference, feature selection

A common strategy for sparse linear regression is to introduce
regularization, which eliminates irrelevant features by letting the
corresponding weights be zeros. However, regularization often shrinks the
estimator for relevant features, which leads to incorrect feature selection.
Motivated by the above-mentioned issue, we propose Bayesian masking (BM), a
sparse estimation method which imposes no regularization on the weights. The
key concept of BM is to introduce binary latent variables that randomly mask
features. Estimating the masking rates determines the relevance of the features
automatically. We derive a variational Bayesian inference algorithm that
maximizes the lower bound of the factorized information criterion (FIC), which
is a recently developed asymptotic criterion for evaluating the marginal
log-likelihood. In addition, we propose reparametrization to accelerate the
convergence of the derived algorithm. Finally, we show that BM outperforms
Lasso and automatic relevance determination (ARD) in terms of the
sparsity-shrinkage trade-off.
Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders
Authors: Huan Yang, Baoyuan Wang, Stephen Lin, David Wipf, Minyi Guo, Baining Guo
Date: 2015-10-06 00:00:00
Proceeding: ICCV 2015 12

With the growing popularity of short-form video sharing platforms such as
\em{Instagram} and \em{Vine}, there has been an increasing need for techniques
that automatically extract highlights from video. Whereas prior works have
approached this problem with heuristic rules or supervised learning, we present
an unsupervised learning approach that takes advantage of the abundance of
user-edited videos on social media websites such as YouTube. Based on the idea
that the most significant sub-events within a video class are commonly present
among edited videos while less interesting ones appear less frequently, we
identify the significant sub-events via a robust recurrent auto-encoder trained
on a collection of user-edited videos queried for each particular class of
interest. The auto-encoder is trained using a proposed shrinking exponential
loss function that makes it robust to noise in the web-crawled training data,
and is configured with bidirectional long short term memory
(LSTM)~\cite{LSTM:97} cells to better model the temporal structure of highlight
segments. Different from supervised techniques, our method can infer highlights
using only a set of downloaded edited videos, without also needing their
pre-edited counterparts which are rarely available online. Extensive
experiments indicate the promise of our proposed solution in this challenging
unsupervised settin
Harvesting Discriminative Meta Objects with Deep CNN Features for Scene Classification
Authors: Ruobing Wu, Baoyuan Wang, Wenping Wang, Yizhou Yu
Date: 2015-10-06 00:00:00
Proceeding: ICCV 2015 12
Tasks: Clustering, General Classification, Region Proposal, Scene Classification, Weakly-supervised Learning

Recent work on scene classification still makes use of generic CNN features
in a rudimentary manner. In this ICCV 2015 paper, we present a novel pipeline
built upon deep CNN features to harvest discriminative visual objects and parts
for scene classification. We first use a region proposal technique to generate
a set of high-quality patches potentially containing objects, and apply a
pre-trained CNN to extract generic deep features from these patches. Then we
perform both unsupervised and weakly supervised learning to screen these
patches and discover discriminative ones representing category-specific objects
and parts. We further apply discriminative clustering enhanced with local CNN
fine-tuning to aggregate similar objects and parts into groups, called meta
objects. A scene image representation is constructed by pooling the feature
response maps of all the learned meta objects at multiple spatial scales. We
have confirmed that the scene image representation obtained using this new
pipeline is capable of delivering state-of-the-art performance on two popular
scene benchmark datasets, MIT Indoor 67~\cite{MITIndoor67} and
Sun397~\cite{Sun397}
Improved Estimation of Class Prior Probabilities through Unlabeled Data
Authors: Norman Matloff
Date: 2015-10-06 00:00:00
Tasks: Classification, General Classification

Work in the classification literature has shown that in computing a
classification function, one need not know the class membership of all
observations in the training set; the unlabeled observations still provide
information on the marginal distribution of the feature set, and can thus
contribute to increased classification accuracy for future observations. The
present paper will show that this scheme can also be used for the estimation of
class prior probabilities, which would be very useful in applications in which
it is difficult or expensive to determine class membership. Both parametric and
nonparametric estimators are developed. Asymptotic distributions of the
estimators are derived, and it is proven that the use of the unlabeled
observations does reduce asymptotic variance. This methodology is also extended
to the estimation of subclass probabilities.
Fuzzy Adaptive Resonance Theory, Diffusion Maps and their applications to Clustering and Biclustering
Authors: S. B. Damelin, Y. Gu, D. C. Wunsch II, R. Xu
Date: 2014-11-21 00:00:00
Tasks: Clustering

In this paper, we describe an algorithm FARDiff (Fuzzy Adaptive Resonance
Dif- fusion) which combines Diffusion Maps and Fuzzy Adaptive Resonance Theory
to do clustering on high dimensional data. We describe some applications of
this method and some problems for future research.
On the Existence of Epipolar Matrices
Authors: Sameer Agarwal, Hon-Leung Lee, Bernd Sturmfels, Rekha R. Thomas
Date: 2015-10-06 00:00:00

This paper considers the foundational question of the existence of a
fundamental (resp. essential) matrix given $m$ point correspondences in two
views. We present a complete answer for the existence of fundamental matrices
for any value of $m$. Using examples we disprove the widely held beliefs that
fundamental matrices always exist whenever $m \leq 7$. At the same time, we
prove that they exist unconditionally when $m \leq 5$. Under a mild genericity
condition, we show that an essential matrix always exists when $m \leq 4$. We
also characterize the six and seven point configurations in two views for which
all matrices satisfying the epipolar constraint have rank at most one.
Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
Authors: George Papandreou, Liang-Chieh Chen, Kevin Murphy, Alan L. Yuille
Date: 2015-02-09 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Semi-Supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Deep convolutional neural networks (DCNNs) trained on a large number of
images with strong pixel-level annotations have recently significantly pushed
the state-of-art in semantic image segmentation. We study the more challenging
problem of learning DCNNs for semantic image segmentation from either (1)
weakly annotated training data such as bounding boxes or image-level labels or
(2) a combination of few strongly labeled and many weakly labeled images,
sourced from one or multiple datasets. We develop Expectation-Maximization (EM)
methods for semantic image segmentation model training under these weakly
supervised and semi-supervised settings. Extensive experimental evaluation
shows that the proposed techniques can learn models delivering competitive
results on the challenging PASCAL VOC 2012 image segmentation benchmark, while
requiring significantly less annotation effort. We share source code
implementing the proposed system at
https://bitbucket.org/deeplab/deeplab-public.
Batch Normalized Recurrent Neural Networks
Authors: César Laurent, Gabriel Pereyra, Philémon Brakel, Ying Zhang, Yoshua Bengio
Date: 2015-10-05 00:00:00
Tasks: Language Modelling, speech-recognition, Speech Recognition

Recurrent Neural Networks (RNNs) are powerful models for sequential data that
have the potential to learn long-term dependencies. However, they are
computationally expensive to train and difficult to parallelize. Recent work
has shown that normalizing intermediate representations of neural networks can
significantly improve convergence rates in feedforward neural networks . In
particular, batch normalization, which uses mini-batch statistics to
standardize features, was shown to significantly reduce training time. In this
paper, we show that applying batch normalization to the hidden-to-hidden
transitions of our RNNs doesn't help the training procedure. We also show that
when applied to the input-to-hidden transitions, batch normalization can lead
to a faster convergence of the training criterion but doesn't seem to improve
the generalization performance on both our language modelling and speech
recognition tasks. All in all, applying batch normalization to RNNs turns out
to be more challenging than applying it to feedforward networks, but certain
variants of it can still be beneficial.
Within-Brain Classification for Brain Tumor Segmentation
Authors: Mohammad Havaei, Hugo Larochelle, Philippe Poulin, Pierre-Marc Jodoin
Date: 2015-10-05 00:00:00
Tasks: BIG-bench Machine Learning, Brain Tumor Segmentation, Classification, General Classification, Tumor Segmentation

Purpose: In this paper, we investigate a framework for interactive brain
tumor segmentation which, at its core, treats the problem of interactive brain
tumor segmentation as a machine learning problem.
  Methods: This method has an advantage over typical machine learning methods
for this task where generalization is made across brains. The problem with
these methods is that they need to deal with intensity bias correction and
other MRI-specific noise. In this paper, we avoid these issues by approaching
the problem as one of within brain generalization. Specifically, we propose a
semi-automatic method that segments a brain tumor by training and generalizing
within that brain only, based on some minimum user interaction.
  Conclusion: We investigate how adding spatial feature coordinates (i.e. $i$,
$j$, $k$) to the intensity features can significantly improve the performance
of different classification methods such as SVM, kNN and random forests. This
would only be possible within an interactive framework. We also investigate the
use of a more appropriate kernel and the adaptation of hyper-parameters
specifically for each brain.
  Results: As a result of these experiments, we obtain an interactive method
whose results reported on the MICCAI-BRATS 2013 dataset are the second most
accurate compared to published methods, while using significantly less memory
and processing power than most state-of-the-art methods.
Tight Variational Bounds via Random Projections and I-Projections
Authors: Lun-Kai Hsu, Tudor Achim, Stefano Ermon
Date: 2015-10-05 00:00:00
Tasks: Variational Inference

Information projections are the key building block of variational inference
algorithms and are used to approximate a target probabilistic model by
projecting it onto a family of tractable distributions. In general, there is no
guarantee on the quality of the approximation obtained. To overcome this issue,
we introduce a new class of random projections to reduce the dimensionality and
hence the complexity of the original model. In the spirit of random
projections, the projection preserves (with high probability) key properties of
the target distribution. We show that information projections can be combined
with random projections to obtain provable guarantees on the quality of the
approximation obtained, regardless of the complexity of the original model. We
demonstrate empirically that augmenting mean field with a random projection
step dramatically improves partition function and marginal probability
estimates, both on synthetic and real world data.
Learning in Unlabeled Networks - An Active Learning and Inference Approach
Authors: Tomasz Kajdanowicz, Radosław Michalski, Katarzyna Musiał, Przemysław Kazienko
Date: 2015-10-05 00:00:00
Tasks: Active Learning, Classification, Clustering, General Classification

The task of determining labels of all network nodes based on the knowledge
about network structure and labels of some training subset of nodes is called
the within-network classification. It may happen that none of the labels of the
nodes is known and additionally there is no information about number of classes
to which nodes can be assigned. In such a case a subset of nodes has to be
selected for initial label acquisition. The question that arises is: "labels of
which nodes should be collected and used for learning in order to provide the
best classification accuracy for the whole network?". Active learning and
inference is a practical framework to study this problem.
  A set of methods for active learning and inference for within network
classification is proposed and validated. The utility score calculation for
each node based on network structure is the first step in the process. The
scores enable to rank the nodes. Based on the ranking, a set of nodes, for
which the labels are acquired, is selected (e.g. by taking top or bottom N from
the ranking). The new measure-neighbour methods proposed in the paper suggest
not obtaining labels of nodes from the ranking but rather acquiring labels of
their neighbours. The paper examines 29 distinct formulations of utility score
and selection methods reporting their impact on the results of two collective
classification algorithms: Iterative Classification Algorithm and Loopy Belief
Propagation.
  We advocate that the accuracy of presented methods depends on the structural
properties of the examined network. We claim that measure-neighbour methods
will work better than the regular methods for networks with higher clustering
coefficient and worse than regular methods for networks with low clustering
coefficient. According to our hypothesis, based on clustering coefficient we
are able to recommend appropriate active learning and inference method.
Efficient Object Detection for High Resolution Images
Authors: Yongxi Lu, Tara Javidi
Date: 2015-10-05 00:00:00
Tasks: Object, object-detection, Object Detection, Vocal Bursts Intensity Prediction

Efficient generation of high-quality object proposals is an essential step in
state-of-the-art object detection systems based on deep convolutional neural
networks (DCNN) features. Current object proposal algorithms are
computationally inefficient in processing high resolution images containing
small objects, which makes them the bottleneck in object detection systems. In
this paper we present effective methods to detect objects for high resolution
images. We combine two complementary strategies. The first approach is to
predict bounding boxes based on adjacent visual features. The second approach
uses high level image features to guide a two-step search process that
adaptively focuses on regions that are likely to contain small objects. We
extract features required for the two strategies by utilizing a pre-trained
DCNN model known as AlexNet. We demonstrate the effectiveness of our algorithm
by showing its performance on a high-resolution image subset of the SUN 2012
object detection dataset.
Bayesian Inference via Approximation of Log-likelihood for Priors in Exponential Family
Authors: Tohid Ardeshiri, Umut Orguner, Fredrik Gustafsson
Date: 2015-10-05 00:00:00
Tasks: Bayesian Inference

In this paper, a Bayesian inference technique based on Taylor series
approximation of the logarithm of the likelihood function is presented. The
proposed approximation is devised for the case, where the prior distribution
belongs to the exponential family of distributions. The logarithm of the
likelihood function is linearized with respect to the sufficient statistic of
the prior distribution in exponential family such that the posterior obtains
the same exponential family form as the prior. Similarities between the
proposed method and the extended Kalman filter for nonlinear filtering are
illustrated. Furthermore, an extended target measurement update for target
models where the target extent is represented by a random matrix having an
inverse Wishart distribution is derived. The approximate update covers the
important case where the spread of measurement is due to the target extent as
well as the measurement noise in the sensor.
Cross-Device Tracking: Matching Devices and Cookies
Authors: Roberto Díaz-Morales
Date: 2015-10-05 00:00:00
Tasks: BIG-bench Machine Learning

The number of computers, tablets and smartphones is increasing rapidly, which
entails the ownership and use of multiple devices to perform online tasks. As
people move across devices to complete these tasks, their identities becomes
fragmented. Understanding the usage and transition between those devices is
essential to develop efficient applications in a multi-device world. In this
paper we present a solution to deal with the cross-device identification of
users based on semi-supervised machine learning methods to identify which
cookies belong to an individual using a device. The method proposed in this
paper scored third in the ICDM 2015 Drawbridge Cross-Device Connections
challenge proving its good performance.