pwc-archive/papers-with-abstracts huggingface dataset:
Learning from Synthetic Data Using a Stacked Multichannel Autoencoder
Authors: Xi Zhang, Yanwei Fu, Shanshan Jiang, Leonid Sigal, Gady Agam
Date: 2015-09-17 00:00:00
Tasks: Sketch Recognition

Learning from synthetic data has many important and practical applications.
An example of application is photo-sketch recognition. Using synthetic data is
challenging due to the differences in feature distributions between synthetic
and real data, a phenomenon we term synthetic gap. In this paper, we
investigate and formalize a general framework-Stacked Multichannel Autoencoder
(SMCAE) that enables bridging the synthetic gap and learning from synthetic
data more efficiently. In particular, we show that our SMCAE can not only
transform and use synthetic data on the challenging face-sketch recognition
task, but that it can also help simulate real images, which can be used for
training classifiers for recognition. Preliminary experiments validate the
effectiveness of the framework.
High-for-Low and Low-for-High: Efficient Boundary Detection from Deep Object Features and its Applications to High-Level Vision
Authors: Gedas Bertasius, Jianbo Shi, Lorenzo Torresani
Date: 2015-04-23 00:00:00
Proceeding: ICCV 2015 12
Tasks: Boundary Detection, Object, Object Proposal Generation, Semantic Segmentation, Vocal Bursts Intensity Prediction

Most of the current boundary detection systems rely exclusively on low-level
features, such as color and texture. However, perception studies suggest that
humans employ object-level reasoning when judging if a particular pixel is a
boundary. Inspired by this observation, in this work we show how to predict
boundaries by exploiting object-level features from a pretrained
object-classification network. Our method can be viewed as a "High-for-Low"
approach where high-level object features inform the low-level boundary
detection process. Our model achieves state-of-the-art performance on an
established boundary detection benchmark and it is efficient to run.
  Additionally, we show that due to the semantic nature of our boundaries we
can use them to aid a number of high-level vision tasks. We demonstrate that
using our boundaries we improve the performance of state-of-the-art methods on
the problems of semantic boundary labeling, semantic segmentation and object
proposal generation. We can view this process as a "Low-for-High" scheme, where
low-level boundaries aid high-level vision tasks.
  Thus, our contributions include a boundary detection system that is accurate,
efficient, generalizes well to multiple datasets, and is also shown to improve
existing state-of-the-art high-level vision methods on three distinct tasks.
Evaluating the visualization of what a Deep Neural Network has learned
Authors: Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Bach, Klaus-Robert Müller
Date: 2015-09-21 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, speech-recognition, Speech Recognition

Deep Neural Networks (DNNs) have demonstrated impressive performance in
complex machine learning tasks such as image classification or speech
recognition. However, due to their multi-layer nonlinear structure, they are
not transparent, i.e., it is hard to grasp what makes them arrive at a
particular classification or recognition decision given a new unseen data
sample. Recently, several approaches have been proposed enabling one to
understand and interpret the reasoning embodied in a DNN for a single test
image. These methods quantify the ''importance'' of individual pixels wrt the
classification decision and allow a visualization in terms of a heatmap in
pixel/input space. While the usefulness of heatmaps can be judged subjectively
by a human, an objective quality measure is missing. In this paper we present a
general methodology based on region perturbation for evaluating ordered
collections of pixels such as heatmaps. We compare heatmaps computed by three
different methods on the SUN397, ILSVRC2012 and MIT Places data sets. Our main
result is that the recently proposed Layer-wise Relevance Propagation (LRP)
algorithm qualitatively and quantitatively provides a better explanation of
what made a DNN arrive at a particular classification decision than the
sensitivity-based approach or the deconvolution method. We provide theoretical
arguments to explain this result and discuss its practical implications.
Finally, we investigate the use of heatmaps for unsupervised assessment of
neural network performance.
A Bayesian Compressed Sensing Kalman Filter for Direction of Arrival Estimation
Authors: Matthew Hawes, Lyudmila Mihaylova, Francois Septier, Simon Godsill
Date: 2015-09-21 00:00:00
Tasks: compressed sensing, Compressive Sensing, Direction of Arrival Estimation

In this paper, we look to address the problem of estimating the dynamic
direction of arrival (DOA) of a narrowband signal impinging on a sensor array
from the far field. The initial estimate is made using a Bayesian compressive
sensing (BCS) framework and then tracked using a Bayesian compressed sensing
Kalman filter (BCSKF). The BCS framework splits the angular region into N
potential DOAs and enforces a belief that only a few of the DOAs will have a
non-zero valued signal present. A BCSKF can then be used to track the change in
the DOA using the same framework. There can be an issue when the DOA approaches
the endfire of the array. In this angular region current methods can struggle
to accurately estimate and track changes in the DOAs. To tackle this problem,
we propose changing the traditional sparse belief associated with BCS to a
belief that the estimated signals will match the predicted signals given a
known DOA change. This is done by modelling the difference between the expected
sparse received signals and the estimated sparse received signals as a Gaussian
distribution. Example test scenarios are provided and comparisons made with the
traditional BCS based estimation method. They show that an improvement in
estimation accuracy is possible without a significant increase in computational
complexity.
Artificial Neural Networks Applied to Taxi Destination Prediction
Authors: Alexandre de Brébisson, Étienne Simon, Alex Auvolat, Pascal Vincent, Yoshua Bengio
Date: 2015-07-31 00:00:00
Tasks: Prediction

We describe our first-place solution to the ECML/PKDD discovery challenge on
taxi destination prediction. The task consisted in predicting the destination
of a taxi based on the beginning of its trajectory, represented as a
variable-length sequence of GPS points, and diverse associated
meta-information, such as the departure time, the driver id and client
information. Contrary to most published competitor approaches, we used an
almost fully automated approach based on neural networks and we ranked first
out of 381 teams. The architectures we tried use multi-layer perceptrons,
bidirectional recurrent neural networks and models inspired from recently
introduced memory networks. Our approach could easily be adapted to other
applications in which the goal is to predict a fixed-length output from a
variable-length sequence.
Hybrid Optimization Algorithm for Large-Scale QoS-Aware Service Composition
Authors: Pablo Rodriguez-Mier, Manuel Mucientes, Manuel Lama
Date: 2015-09-21 00:00:00
Tasks: global-optimization, Service Composition

In this paper we present a hybrid approach for automatic composition of Web
services that generates semantic input-output based compositions with optimal
end-to-end QoS, minimizing the number of services of the resulting composition.
The proposed approach has four main steps: 1) generation of the composition
graph for a request; 2) computation of the optimal composition that minimizes a
single objective QoS function; 3) multi-step optimizations to reduce the search
space by identifying equivalent and dominated services; and 4) hybrid
local-global search to extract the optimal QoS with the minimum number of
services. An extensive validation with the datasets of the Web Service
Challenge 2009-2010 and randomly generated datasets shows that: 1) the
combination of local and global optimization is a general and powerful
technique to extract optimal compositions in diverse scenarios; and 2) the
hybrid strategy performs better than the state-of-the-art, obtaining solutions
with less services and optimal QoS.
LEWIS: Latent Embeddings for Word Images and their Semantics
Authors: Albert Gordo, Jon Almazan, Naila Murray, Florent Perronnin
Date: 2015-09-21 00:00:00
Proceeding: ICCV 2015 12
Tasks: Retrieval

The goal of this work is to bring semantics into the tasks of text
recognition and retrieval in natural images. Although text recognition and
retrieval have received a lot of attention in recent years, previous works have
focused on recognizing or retrieving exactly the same word used as a query,
without taking the semantics into consideration.
  In this paper, we ask the following question: \emph{can we predict semantic
concepts directly from a word image, without explicitly trying to transcribe
the word image or its characters at any point?} For this goal we propose a
convolutional neural network (CNN) with a weighted ranking loss objective that
ensures that the concepts relevant to the query image are ranked ahead of those
that are not relevant. This can also be interpreted as learning a Euclidean
space where word images and concepts are jointly embedded. This model is
learned in an end-to-end manner, from image pixels to semantic concepts, using
a dataset of synthetically generated word images and concepts mined from a
lexical database (WordNet). Our results show that, despite the complexity of
the task, word images and concepts can indeed be associated with a high degree
of accuracy
Denoising Autoencoders for fast Combinatorial Black Box Optimization
Authors: Malte Probst
Date: 2015-03-06 00:00:00
Tasks: Bayesian Optimization, Combinatorial Optimization, CPU, Denoising

Estimation of Distribution Algorithms (EDAs) require flexible probability
models that can be efficiently learned and sampled. Autoencoders (AE) are
generative stochastic networks with these desired properties. We integrate a
special type of AE, the Denoising Autoencoder (DAE), into an EDA and evaluate
the performance of DAE-EDA on several combinatorial optimization problems with
a single objective. We asses the number of fitness evaluations as well as the
required CPU times. We compare the results to the performance to the Bayesian
Optimization Algorithm (BOA) and RBM-EDA, another EDA which is based on a
generative neural network which has proven competitive with BOA. For the
considered problem instances, DAE-EDA is considerably faster than BOA and
RBM-EDA, sometimes by orders of magnitude. The number of fitness evaluations is
higher than for BOA, but competitive with RBM-EDA. These results show that DAEs
can be useful tools for problems with low but non-negligible fitness evaluation
costs.
(Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics
Authors: Sebastian J. Vollmer, Konstantinos C. Zygalakis, and Yee Whye Teh
Date: 2015-01-02 00:00:00

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data
sets is computationally infeasible. The recently proposed stochastic gradient
Langevin dynamics (SGLD) method circumvents this problem in three ways: it
generates proposed moves using only a subset of the data, it skips the
Metropolis-Hastings accept-reject step, and it uses sequences of decreasing
step sizes. In \cite{TehThierryVollmerSGLD2014}, we provided the mathematical
foundations for the decreasing step size SGLD, including consistency and a
central limit theorem. However, in practice the SGLD is run for a relatively
small number of iterations, and its step size is not decreased to zero. The
present article investigates the behaviour of the SGLD with fixed step size. In
particular we characterise the asymptotic bias explicitly, along with its
dependence on the step size and the variance of the stochastic gradient. On
that basis a modified SGLD which removes the asymptotic bias due to the
variance of the stochastic gradients up to first order in the step size is
derived. Moreover, we are able to obtain bounds on the finite-time bias,
variance and mean squared error (MSE). The theory is illustrated with a
Gaussian toy model for which the bias and the MSE for the estimation of moments
can be obtained explicitly. For this toy model we study the gain of the SGLD
over the standard Euler method in the limit of large data sets.
The Utility of Clustering in Prediction Tasks
Authors: Shubhendu Trivedi, Zachary A. Pardos, Neil T. Heffernan
Date: 2015-09-21 00:00:00
Tasks: Clustering, Data Compression, Prediction

We explore the utility of clustering in reducing error in various prediction
tasks. Previous work has hinted at the improvement in prediction accuracy
attributed to clustering algorithms if used to pre-process the data. In this
work we more deeply investigate the direct utility of using clustering to
improve prediction accuracy and provide explanations for why this may be so. We
look at a number of datasets, run k-means at different scales and for each
scale we train predictors. This produces k sets of predictions. These
predictions are then combined by a na\"ive ensemble. We observed that this use
of a predictor in conjunction with clustering improved the prediction accuracy
in most datasets. We believe this indicates the predictive utility of
exploiting structure in the data and the data compression handed over by
clustering. We also found that using this method improves upon the prediction
of even a Random Forests predictor which suggests this method is providing a
novel, and useful source of variance in the prediction process.
Noise Robust IOA/CAS Speech Separation and Recognition System For The Third 'CHIME' Challenge
Authors: Xiaofei Wang, Chao Wu, Pengyuan Zhang, Ziteng Wang, Yong liu, Xu Li, Qiang Fu, Yonghong Yan
Date: 2015-09-21 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition, Speech Separation

This paper presents the contribution to the third 'CHiME' speech separation
and recognition challenge including both front-end signal processing and
back-end speech recognition. In the front-end, Multi-channel Wiener filter
(MWF) is designed to achieve background noise reduction. Different from
traditional MWF, optimized parameter for the tradeoff between noise reduction
and target signal distortion is built according to the desired noise reduction
level. In the back-end, several techniques are taken advantage to improve the
noisy Automatic Speech Recognition (ASR) performance including Deep Neural
Network (DNN), Convolutional Neural Network (CNN) and Long short-term memory
(LSTM) using medium vocabulary, Lattice rescoring with a big vocabulary
language model finite state transducer, and ROVER scheme. Experimental results
show the proposed system combining front-end and back-end is effective to
improve the ASR performance.
Multilayer bootstrap network for unsupervised speaker recognition
Authors: Xiao-Lei Zhang
Date: 2015-09-21 00:00:00
Tasks: Clustering, Speaker Recognition

We apply multilayer bootstrap network (MBN), a recent proposed unsupervised
learning method, to unsupervised speaker recognition. The proposed method first
extracts supervectors from an unsupervised universal background model, then
reduces the dimension of the high-dimensional supervectors by multilayer
bootstrap network, and finally conducts unsupervised speaker recognition by
clustering the low-dimensional data. The comparison results with 2 unsupervised
and 1 supervised speaker recognition techniques demonstrate the effectiveness
and robustness of the proposed method.
Significance Analysis of High-Dimensional, Low-Sample Size Partially Labeled Data
Authors: Qiyi Lu, Xingye Qiao
Date: 2015-09-21 00:00:00
Tasks: Clustering, Vocal Bursts Intensity Prediction

Classification and clustering are both important topics in statistical
learning. A natural question herein is whether predefined classes are really
different from one another, or whether clusters are really there. Specifically,
we may be interested in knowing whether the two classes defined by some class
labels (when they are provided), or the two clusters tagged by a clustering
algorithm (where class labels are not provided), are from the same underlying
distribution. Although both are challenging questions for the high-dimensional,
low-sample size data, there has been some recent development for both. However,
when it is costly to manually place labels on observations, it is often that
only a small portion of the class labels is available. In this article, we
propose a significance analysis approach for such type of data, namely
partially labeled data. Our method makes use of the whole data and tries to
test the class difference as if all the labels were observed. Compared to a
testing method that ignores the label information, our method provides a
greater power, meanwhile, maintaining the size, illustrated by a comprehensive
simulation study. Theoretical properties of the proposed method are studied
with emphasis on the high-dimensional, low-sample size setting. Our simulated
examples help to understand when and how the information extracted from the
labeled data can be effective. A real data example further illustrates the
usefulness of the proposed method.
On Large-Scale Retrieval: Binary or n-ary Coding?
Authors: Mahyar Najibi, Mohammad Rastegari, Larry S. Davis
Date: 2015-09-20 00:00:00
Tasks: Image Retrieval, Quantization, Retrieval

The growing amount of data available in modern-day datasets makes the need to
efficiently search and retrieve information. To make large-scale search
feasible, Distance Estimation and Subset Indexing are the main approaches.
Although binary coding has been popular for implementing both techniques, n-ary
coding (known as Product Quantization) is also very effective for Distance
Estimation. However, their relative performance has not been studied for Subset
Indexing. We investigate whether binary or n-ary coding works better under
different retrieval strategies. This leads to the design of a new n-ary coding
method, "Linear Subspace Quantization (LSQ)" which, unlike other n-ary
encoders, can be used as a similarity-preserving embedding. Experiments on
image retrieval show that when Distance Estimation is used, n-ary LSQ
outperforms other methods. However, when Subset Indexing is applied,
interestingly, binary codings are more effective and binary LSQ achieves the
best accuracy.
A Statistical Theory of Deep Learning via Proximal Splitting
Authors: Nicholas G. Polson, Brandon T. Willard, Massoud Heidari
Date: 2015-09-20 00:00:00
Tasks: Deep Learning, Model Selection

In this paper we develop a statistical theory and an implementation of deep
learning models. We show that an elegant variable splitting scheme for the
alternating direction method of multipliers optimises a deep learning
objective. We allow for non-smooth non-convex regularisation penalties to
induce sparsity in parameter weights. We provide a link between traditional
shallow layer statistical models such as principal component and sliced inverse
regression and deep layer models. We also define the degrees of freedom of a
deep learning predictor and a predictive MSE criteria to perform model
selection for comparing architecture designs. We focus on deep multiclass
logistic learning although our methods apply more generally. Our results
suggest an interesting and previously under-exploited relationship between deep
learning and proximal splitting techniques. To illustrate our methodology, we
provide a multi-class logit classification analysis of Fisher's Iris data where
we illustrate the convergence of our algorithm. Finally, we conclude with
directions for future research.
Impact of noise on a dynamical system: prediction and uncertainties from a swarm-optimized neural network
Authors: C. H. López-Caraballo, J. A. Lazzús, I. Salfate, P. Rojas, M. Rivera, L. Palma-Chilla
Date: 2015-09-20 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Prediction

In this study, an artificial neural network (ANN) based on particle swarm
optimization (PSO) was developed for the time series prediction. The hybrid
ANN+PSO algorithm was applied on Mackey--Glass chaotic time series in the
short-term $x(t+6)$. The performance prediction was evaluated and compared with
another studies available in the literature. Also, we presented properties of
the dynamical system via the study of chaotic behaviour obtained from the
predicted time series. Next, the hybrid ANN+PSO algorithm was complemented with
a Gaussian stochastic procedure (called {\it stochastic} hybrid ANN+PSO) in
order to obtain a new estimator of the predictions, which also allowed us to
compute uncertainties of predictions for noisy Mackey--Glass chaotic time
series. Thus, we studied the impact of noise for several cases with a white
noise level ($\sigma_{N}$) from 0.01 to 0.1.
Early text classification: a Naive solution
Authors: Hugo Jair Escalante, Manuel Montes-y-Gómez, Luis Villaseñor-Pineda, Marcelo Luis Errecalde
Date: 2015-09-20 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

Text classification is a widely studied problem, and it can be considered
solved for some domains and under certain circumstances. There are scenarios,
however, that have received little or no attention at all, despite its
relevance and applicability. One of such scenarios is early text
classification, where one needs to know the category of a document by using
partial information only. A document is processed as a sequence of terms, and
the goal is to devise a method that can make predictions as fast as possible.
The importance of this variant of the text classification problem is evident in
domains like sexual predator detection, where one wants to identify an offender
as early as possible. This paper analyzes the suitability of the standard naive
Bayes classifier for approaching this problem. Specifically, we assess its
performance when classifying documents after seeing an increasingly number of
terms. A simple modification to the standard naive Bayes implementation allows
us to make predictions with partial information. To the best of our knowledge
naive Bayes has not been used for this purpose before. Throughout an extensive
experimental evaluation we show the effectiveness of the classifier for early
text classification. What is more, we show that this simple solution is very
competitive when compared with state of the art methodologies that are more
elaborated. We foresee our work will pave the way for the development of more
effective early text classification techniques based in the naive Bayes
formulation.