pwc-archive/papers-with-abstracts huggingface dataset:
Understanding Minimum Probability Flow for RBMs Under Various Kinds of Dynamics
Authors: Daniel Jiwoong Im, Ethan Buchman, Graham W. Taylor
Date: 2014-12-20 00:00:00

Energy-based models are popular in machine learning due to the elegance of
their formulation and their relationship to statistical physics. Among these,
the Restricted Boltzmann Machine (RBM), and its staple training algorithm
contrastive divergence (CD), have been the prototype for some recent
advancements in the unsupervised training of deep neural networks. However, CD
has limited theoretical motivation, and can in some cases produce undesirable
behavior. Here, we investigate the performance of Minimum Probability Flow
(MPF) learning for training RBMs. Unlike CD, with its focus on approximating an
intractable partition function via Gibbs sampling, MPF proposes a tractable,
consistent, objective function defined in terms of a Taylor expansion of the KL
divergence with respect to sampling dynamics. Here we propose a more general
form for the sampling dynamics in MPF, and explore the consequences of
different choices for these dynamics for training RBMs. Experimental results
show MPF outperforming CD for various RBM configurations.
Tensor machines for learning target-specific polynomial features
Authors: Jiyan Yang, Alex Gittens
Date: 2015-04-07 00:00:00

Recent years have demonstrated that using random feature maps can
significantly decrease the training and testing times of kernel-based
algorithms without significantly lowering their accuracy. Regrettably, because
random features are target-agnostic, typically thousands of such features are
necessary to achieve acceptable accuracies. In this work, we consider the
problem of learning a small number of explicit polynomial features. Our
approach, named Tensor Machines, finds a parsimonious set of features by
optimizing over the hypothesis class introduced by Kar and Karnick for random
feature maps in a target-specific manner. Exploiting a natural connection
between polynomials and tensors, we provide bounds on the generalization error
of Tensor Machines. Empirically, Tensor Machines behave favorably on several
real-world datasets compared to other state-of-the-art techniques for learning
polynomial features, and deliver significantly more parsimonious models.
Large Margin Nearest Neighbor Embedding for Knowledge Representation
Authors: Miao Fan, Qiang Zhou, Thomas Fang Zheng, Ralph Grishman
Date: 2015-04-07 00:00:00
Tasks: Link Prediction, Triplet

Traditional way of storing facts in triplets ({\it head\_entity, relation,
tail\_entity}), abbreviated as ({\it h, r, t}), makes the knowledge intuitively
displayed and easily acquired by mankind, but hardly computed or even reasoned
by AI machines. Inspired by the success in applying {\it Distributed
Representations} to AI-related fields, recent studies expect to represent each
entity and relation with a unique low-dimensional embedding, which is different
from the symbolic and atomic framework of displaying knowledge in triplets. In
this way, the knowledge computing and reasoning can be essentially facilitated
by means of a simple {\it vector calculation}, i.e. ${\bf h} + {\bf r} \approx
{\bf t}$. We thus contribute an effective model to learn better embeddings
satisfying the formula by pulling the positive tail entities ${\bf t^{+}}$ to
get together and close to {\bf h} + {\bf r} ({\it Nearest Neighbor}), and
simultaneously pushing the negatives ${\bf t^{-}}$ away from the positives
${\bf t^{+}}$ via keeping a {\it Large Margin}. We also design a corresponding
learning algorithm to efficiently find the optimal solution based on {\it
Stochastic Gradient Descent} in iterative fashion. Quantitative experiments
illustrate that our approach can achieve the state-of-the-art performance,
compared with several latest methods on some benchmark datasets for two
classical applications, i.e. {\it Link prediction} and {\it Triplet
classification}. Moreover, we analyze the parameter complexities among all the
evaluated models, and analytical results indicate that our model needs fewer
computational resources on outperforming the other methods.
From Averaging to Acceleration, There is Only a Step-size
Authors: Nicolas Flammarion, Francis Bach
Date: 2015-04-07 00:00:00

We show that accelerated gradient descent, averaged gradient descent and the
heavy-ball method for non-strongly-convex problems may be reformulated as
constant parameter second-order difference equation algorithms, where stability
of the system is equivalent to convergence at rate O(1/n 2), where n is the
number of iterations. We provide a detailed analysis of the eigenvalues of the
corresponding linear dynamical system , showing various oscillatory and
non-oscillatory behaviors, together with a sharp stability result with explicit
constants. We also consider the situation where noisy gradients are available,
where we extend our general convergence result, which suggests an alternative
algorithm (i.e., with different step sizes) that exhibits the good aspects of
both averaging and acceleration.
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Authors: Zuxuan Wu, Xi Wang, Yu-Gang Jiang, Hao Ye, xiangyang xue
Date: 2015-04-07 00:00:00
Tasks: Classification, General Classification, Video Classification

Classifying videos according to content semantics is an important problem
with a wide range of applications. In this paper, we propose a hybrid deep
learning framework for video classification, which is able to model static
spatial information, short-term motion, as well as long-term temporal clues in
the videos. Specifically, the spatial and the short-term motion features are
extracted separately by two Convolutional Neural Networks (CNN). These two
types of CNN-based features are then combined in a regularized feature fusion
network for classification, which is able to learn and utilize feature
relationships for improved performance. In addition, Long Short Term Memory
(LSTM) networks are applied on top of the two features to further model
longer-term temporal clues. The main contribution of this work is the hybrid
learning framework that can model several important aspects of the video data.
We also show that (1) combining the spatial and the short-term motion features
in the regularized fusion network is better than direct classification and
fusion using the CNN with a softmax layer, and (2) the sequence-based LSTM is
highly complementary to the traditional classification strategy without
considering the temporal frame orders. Extensive experiments are conducted on
two popular and challenging benchmarks, the UCF-101 Human Actions and the
Columbia Consumer Videos (CCV). On both benchmarks, our framework achieves
to-date the best reported performance: $91.3\%$ on the UCF-101 and $83.5\%$ on
the CCV.
Separable time-causal and time-recursive spatio-temporal receptive fields
Authors: Tony Lindeberg
Date: 2015-04-07 00:00:00

We present an improved model and theory for time-causal and time-recursive
spatio-temporal receptive fields, obtained by a combination of Gaussian
receptive fields over the spatial domain and first-order integrators or
equivalently truncated exponential filters coupled in cascade over the temporal
domain. Compared to previous spatio-temporal scale-space formulations in terms
of non-enhancement of local extrema or scale invariance, these receptive fields
are based on different scale-space axiomatics over time by ensuring
non-creation of new local extrema or zero-crossings with increasing temporal
scale. Specifically, extensions are presented about parameterizing the
intermediate temporal scale levels, analysing the resulting temporal dynamics
and transferring the theory to a discrete implementation in terms of recursive
filters over time.
Fractional differentiation based image processing
Authors: Amelia Carolina Sparavigna
Date: 2009-10-13 00:00:00
Tasks: Edge Detection, Image Restoration

There are many resources useful for processing images, most of them freely
available and quite friendly to use. In spite of this abundance of tools, a
study of the processing methods is still worthy of efforts. Here, we want to
discuss the possibilities arising from the use of fractional differential
calculus. This calculus evolved in the research field of pure mathematics until
1920, when applied science started to use it. Only recently, fractional
calculus was involved in image processing methods. As we shall see, the
fractional calculation is able to enhance the quality of images, with
interesting possibilities in edge detection and image restoration. We suggest
also the fractional differentiation as a tool to reveal faint objects in
astronomical images.
Voice based self help System: User Experience Vs Accuracy
Authors: Sunil Kumar Kopparapu
Date: 2015-04-07 00:00:00
Tasks: speech-recognition, Speech Recognition, Speech-to-Text

In general, self help systems are being increasingly deployed by service
based industries because they are capable of delivering better customer service
and increasingly the switch is to voice based self help systems because they
provide a natural interface for a human to interact with a machine. A speech
based self help system ideally needs a speech recognition engine to convert
spoken speech to text and in addition a language processing engine to take care
of any misrecognitions by the speech recognition engine. Any off-the-shelf
speech recognition engine is generally a combination of acoustic processing and
speech grammar. While this is the norm, we believe that ideally a speech
recognition application should have in addition to a speech recognition engine
a separate language processing engine to give the system better performance. In
this paper, we discuss ways in which the speech recognition engine and the
language processing engine can be combined to give a better user experience.
Efficient SDP Inference for Fully-connected CRFs Based on Low-rank Decomposition
Authors: Peng Wang, Chunhua Shen, Anton Van Den Hengel
Date: 2015-04-07 00:00:00
Proceeding: CVPR 2015 6

Conditional Random Fields (CRF) have been widely used in a variety of
computer vision tasks. Conventional CRFs typically define edges on neighboring
image pixels, resulting in a sparse graph such that efficient inference can be
performed. However, these CRFs fail to model long-range contextual
relationships. Fully-connected CRFs have thus been proposed. While there are
efficient approximate inference methods for such CRFs, usually they are
sensitive to initialization and make strong assumptions. In this work, we
develop an efficient, yet general algorithm for inference on fully-connected
CRFs. The algorithm is based on a scalable SDP algorithm and the low- rank
approximation of the similarity/kernel matrix. The core of the proposed
algorithm is a tailored quasi-Newton method that takes advantage of the
low-rank matrix approximation when solving the specialized SDP dual problem.
Experiments demonstrate that our method can be applied on fully-connected CRFs
that cannot be solved previously, such as pixel-level image co-segmentation.
On-line Handwritten Devanagari Character Recognition using Fuzzy Directional Features
Authors: Sunil Kumar Kopparapu, Lajish VL
Date: 2015-04-07 00:00:00

This paper describes a new feature set for use in the recognition of on-line
handwritten Devanagari script based on Fuzzy Directional Features. Experiments
are conducted for the automatic recognition of isolated handwritten character
primitives (sub-character units). Initially we describe the proposed feature
set, called the Fuzzy Directional Features (FDF) and then show how these
features can be effectively utilized for writer independent character
recognition. Experimental results show that FDF set perform well for writer
independent data set at stroke level recognition. The main contribution of this
paper is the introduction of a novel feature set and establish experimentally
its ability in recognition of handwritten Devanagari script.
Transferring Knowledge from a RNN to a DNN
Authors: William Chan, Nan Rosemary Ke, Ian Lane
Date: 2015-04-07 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Deep Neural Network (DNN) acoustic models have yielded many state-of-the-art
results in Automatic Speech Recognition (ASR) tasks. More recently, Recurrent
Neural Network (RNN) models have been shown to outperform DNNs counterparts.
However, state-of-the-art DNN and RNN models tend to be impractical to deploy
on embedded systems with limited computational capacity. Traditionally, the
approach for embedded platforms is to either train a small DNN directly, or to
train a small DNN that learns the output distribution of a large DNN. In this
paper, we utilize a state-of-the-art RNN to transfer knowledge to small DNN. We
use the RNN model to generate soft alignments and minimize the Kullback-Leibler
divergence against the small DNN. The small DNN trained on the soft RNN
alignments achieved a 3.93 WER on the Wall Street Journal (WSJ) eval92 task
compared to a baseline 4.54 WER or more than 13% relative improvement.
Deep Recurrent Neural Networks for Acoustic Modelling
Authors: William Chan, Ian Lane
Date: 2015-04-07 00:00:00
Tasks: Acoustic Modelling, Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

We present a novel deep Recurrent Neural Network (RNN) model for acoustic
modelling in Automatic Speech Recognition (ASR). We term our contribution as a
TC-DNN-BLSTM-DNN model, the model combines a Deep Neural Network (DNN) with
Time Convolution (TC), followed by a Bidirectional Long Short-Term Memory
(BLSTM), and a final DNN. The first DNN acts as a feature processor to our
model, the BLSTM then generates a context from the sequence acoustic signal,
and the final DNN takes the context and models the posterior probabilities of
the acoustic states. We achieve a 3.47 WER on the Wall Street Journal (WSJ)
eval92 task or more than 8% relative improvement over the baseline DNN models.
Mobile Phone Based Vehicle License Plate Recognition for Road Policing
Authors: Lajish V. L., Sunil Kumar Kopparapu
Date: 2015-04-07 00:00:00
Tasks: License Plate Recognition, Management

Identity of a vehicle is done through the vehicle license plate by traffic
police in general. Au- tomatic vehicle license plate recognition has several
applications in intelligent traffic management systems. The security situation
across the globe and particularly in India demands a need to equip the traffic
police with a system that enables them to get instant details of a vehicle. The
system should be easy to use, should be mobile, and work 24 x 7. In this paper,
we describe a mobile phone based, client-server architected, license plate
recognition system. While we use the state of the art image processing and
pattern recognition algorithms tuned for Indian conditions to automatically
recognize non-uniform license plates, the main contribution is in creating an
end to end usable solution. The client application runs on a mobile device and
a server application, with access to vehicle information database, is hosted
centrally. The solution enables capture of license plate image captured by the
phone camera and passes to the server; on the server the license plate number
is recognized; the data associated with the number plate is then sent back to
the mobile device, instantaneously. We describe the end to end system
architecture in detail. A working prototype of the proposed system has been
implemented in the lab environment.
The geometry of kernelized spectral clustering
Authors: Geoffrey Schiebinger, Martin J. Wainwright, Bin Yu
Date: 2014-04-29 00:00:00
Tasks: Clustering

Clustering of data sets is a standard problem in many areas of science and
engineering. The method of spectral clustering is based on embedding the data
set using a kernel function, and using the top eigenvectors of the normalized
Laplacian to recover the connected components. We study the performance of
spectral clustering in recovering the latent labels of i.i.d. samples from a
finite mixture of nonparametric distributions. The difficulty of this label
recovery problem depends on the overlap between mixture components and how
easily a mixture component is divided into two nonoverlapping components. When
the overlap is small compared to the indivisibility of the mixture components,
the principal eigenspace of the population-level normalized Laplacian operator
is approximately spanned by the square-root kernelized component densities. In
the finite sample setting, and under the same assumption, embedded samples from
different components are approximately orthogonal with high probability when
the sample size is large. As a corollary we control the fraction of samples
mislabeled by spectral clustering under finite mixtures with nonparametric
components.
A comparative study between proposed Hyper Kurtosis based Modified Duo-Histogram Equalization (HKMDHE) and Contrast Limited Adaptive Histogram Equalization (CLAHE) for Contrast Enhancement Purpose of Low Contrast Human Brain CT scan images
Authors: Sabyasachi Mukhopadhyay, Soham Mandal, Sawon Pratiher, Satyasaran Changdar, Ritwik Burman, Nirmalya Ghosh, Prasanta K. Panigrahi
Date: 2015-04-07 00:00:00

In this paper, a comparative study between proposed hyper kurtosis based
modified duo-histogram equalization (HKMDHE) algorithm and contrast limited
adaptive histogram enhancement (CLAHE) has been presented for the
implementation of contrast enhancement and brightness preservation of low
contrast human brain CT scan images. In HKMDHE algorithm, contrast enhancement
is done on the hyper-kurtosis based application. The results are very promising
of proposed HKMDHE technique with improved PSNR values and lesser AMMBE values
than CLAHE technique.
Totally Corrective Boosting with Cardinality Penalization
Authors: Vasil S. Denchev, Nan Ding, Shin Matsushima, S. V. N. Vishwanathan, Hartmut Neven
Date: 2015-04-07 00:00:00
Tasks: Benchmarking, Combinatorial Optimization

We propose a totally corrective boosting algorithm with explicit cardinality
regularization. The resulting combinatorial optimization problems are not known
to be efficiently solvable with existing classical methods, but emerging
quantum optimization technology gives hope for achieving sparser models in
practice. In order to demonstrate the utility of our algorithm, we use a
distributed classical heuristic optimizer as a stand-in for quantum hardware.
Even though this evaluation methodology incurs large time and resource costs on
classical computing machinery, it allows us to gauge the potential gains in
generalization performance and sparsity of the resulting boosted ensembles. Our
experimental results on public data sets commonly used for benchmarking of
boosting algorithms decidedly demonstrate the existence of such advantages. If
actual quantum optimization were to be used with this algorithm in the future,
we would expect equivalent or superior results at much smaller time and energy
costs during training. Moreover, studying cardinality-penalized boosting also
sheds light on why unregularized boosting algorithms with early stopping often
yield better results than their counterparts with explicit convex
regularization: Early stopping performs suboptimal cardinality regularization.
The results that we present here indicate it is beneficial to explicitly solve
the combinatorial problem still left open at early termination.
A Metric to Classify Style of Spoken Speech
Authors: Sunil Kopparapu, Saurabh Bhatnagar, K. Sahana, Sathyanarayana, Akhilesh Srivastava, P. V. S. Rao
Date: 2015-04-06 00:00:00

The ability to classify spoken speech based on the style of speaking is an
important problem. With the advent of BPO's in recent times, specifically those
that cater to a population other than the local population, it has become
necessary for BPO's to identify people with certain style of speaking
(American, British etc). Today BPO's employ accent analysts to identify people
having the required style of speaking. This process while involving human bias,
it is becoming increasingly infeasible because of the high attrition rate in
the BPO industry. In this paper, we propose a new metric, which robustly and
accurately helps classify spoken speech based on the style of speaking. The
role of the proposed metric is substantiated by using it to classify real
speech data collected from over seventy different people working in a BPO. We
compare the performance of the metric against human experts who independently
carried out the classification process. Experimental results show that the
performance of the system using the novel metric performs better than two
different human expert.
Explorations on high dimensional landscapes
Authors: Levent Sagun, V. Ugur Guney, Gerard Ben Arous, Yann Lecun
Date: 2014-12-20 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Finding minima of a real valued non-convex function over a high dimensional
space is a major challenge in science. We provide evidence that some such
functions that are defined on high dimensional domains have a narrow band of
values whose pre-image contains the bulk of its critical points. This is in
contrast with the low dimensional picture in which this band is wide. Our
simulations agree with the previous theoretical work on spin glasses that
proves the existence of such a band when the dimension of the domain tends to
infinity. Furthermore our experiments on teacher-student networks with the
MNIST dataset establish a similar phenomenon in deep networks. We finally
observe that both the gradient descent and the stochastic gradient descent
methods can reach this level within the same number of steps.
Knowledge driven Offline to Online Script Conversion
Authors: Sunil Kopparapu, Devanuj, Akhilesh Srivastava, P. V. S. Rao
Date: 2015-04-06 00:00:00

The problem of offline to online script conversion is a challenging and an
ill-posed problem. The interest in offline to online conversion exists because
there are a plethora of robust algorithms in online script literature which can
not be used on offline scripts. In this paper, we propose a method, based on
heuristics, to extract online script information from offline bitmap image. We
show the performance of the proposed method on a real sample signature offline
image, whose online information is known.
Theano-based Large-Scale Visual Recognition with Multiple GPUs
Authors: Weiguang Ding, Ruoyan Wang, Fei Mao, Graham Taylor
Date: 2014-12-07 00:00:00
Tasks: GPU, Object Recognition

In this report, we describe a Theano-based AlexNet (Krizhevsky et al., 2012)
implementation and its naive data parallelism on multiple GPUs. Our performance
on 2 GPUs is comparable with the state-of-art Caffe library (Jia et al., 2014)
run on 1 GPU. To the best of our knowledge, this is the first open-source
Python-based AlexNet implementation to-date.