pwc-archive/papers-with-abstracts huggingface dataset:
Automatic Prosody Prediction for Chinese Speech Synthesis using BLSTM-RNN and Embedding Features
Authors: Chuang Ding, Lei Xie, Jie Yan, Weini Zhang, Yang Liu
Date: 2015-11-02 00:00:00
Tasks: Feature Engineering, Prosody Prediction, Speech Synthesis

Prosody affects the naturalness and intelligibility of speech. However,
automatic prosody prediction from text for Chinese speech synthesis is still a
great challenge and the traditional conditional random fields (CRF) based
method always heavily relies on feature engineering. In this paper, we propose
to use neural networks to predict prosodic boundary labels directly from
Chinese characters without any feature engineering. Experimental results show
that stacking feed-forward and bidirectional long short-term memory (BLSTM)
recurrent network layers achieves superior performance over the CRF-based
method. The embedding features learned from raw text further enhance the
performance.
Recursion in RDF Data Shape Languages
Authors: Arthur Ryman
Date: 2015-05-19 00:00:00

An RDF data shape is a description of the expected contents of an RDF
document (aka graph) or dataset. A major part of this description is the set of
constraints that the document or dataset is required to satisfy. W3C recently
(2014) chartered the RDF Data Shapes Working Group to define SHACL, a standard
RDF data shape language. We refer to the ability to name and reference shape
language elements as recursion. This article provides a precise definition of
the meaning of recursion as used in Resource Shape 2.0. The definition of
recursion presented in this article is largely independent of language-specific
details. We speculate that it also applies to ShEx and to all three of the
current proposals for SHACL. In particular, recursion is not permitted in the
SHACL-SPARQL proposal, but we conjecture that recursion could be added by using
the definition proposed here as a top-level control structure.
Stochastic Top-k ListNet
Authors: Tianyi Luo, Dong Wang, Rong Liu, Yiqiao Pan
Date: 2015-11-01 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Learning-To-Rank

ListNet is a well-known listwise learning to rank model and has gained much
attention in recent years. A particular problem of ListNet, however, is the
high computation complexity in model training, mainly due to the large number
of object permutations involved in computing the gradients. This paper proposes
a stochastic ListNet approach which computes the gradient within a bounded
permutation subset. It significantly reduces the computation complexity of
model training and allows extension to Top-k models, which is impossible with
the conventional implementation based on full-set permutations. Meanwhile, the
new approach utilizes partial ranking information of human labels, which helps
improve model quality. Our experiments demonstrated that the stochastic ListNet
method indeed leads to better ranking performance and speeds up the model
training remarkably.
Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
Authors: Rie Johnson, Tong Zhang
Date: 2015-04-06 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Classification, General Classification, Sentiment Analysis, Sentiment Classification, Text Categorization, Text Classification, Topic Classification, Word Embeddings

This paper presents a new semi-supervised framework with convolutional neural
networks (CNNs) for text categorization. Unlike the previous approaches that
rely on word embeddings, our method learns embeddings of small text regions
from unlabeled data for integration into a supervised CNN. The proposed scheme
for embedding learning is based on the idea of two-view semi-supervised
learning, which is intended to be useful for the task of interest even though
the training is done on unlabeled data. Our models achieve better results than
previous approaches on sentiment classification and topic classification tasks.
Computing the Ramsey Number R(4,3,3) using Abstraction and Symmetry breaking
Authors: Michael Codish, Michael Frank, Avraham Itzhakov, Alice Miller
Date: 2015-10-28 00:00:00

The number $R(4,3,3)$ is often presented as the unknown Ramsey number with
the best chances of being found "soon". Yet, its precise value has remained
unknown for almost 50 years. This paper presents a methodology based on
\emph{abstraction} and \emph{symmetry breaking} that applies to solve hard
graph edge-coloring problems. The utility of this methodology is demonstrated
by using it to compute the value $R(4,3,3)=30$. Along the way it is required to
first compute the previously unknown set ${\cal R}(3,3,3;13)$ consisting of
78{,}892 Ramsey colorings.
LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization
Authors: Ilya Loshchilov
Date: 2015-11-01 00:00:00

The limited memory BFGS method (L-BFGS) of Liu and Nocedal (1989) is often
considered to be the method of choice for continuous optimization when first-
and/or second- order information is available. However, the use of L-BFGS can
be complicated in a black-box scenario where gradient information is not
available and therefore should be numerically estimated. The accuracy of this
estimation, obtained by finite difference methods, is often problem-dependent
that may lead to premature convergence of the algorithm.
  In this paper, we demonstrate an alternative to L-BFGS, the limited memory
Covariance Matrix Adaptation Evolution Strategy (LM-CMA) proposed by Loshchilov
(2014). The LM-CMA is a stochastic derivative-free algorithm for numerical
optimization of non-linear, non-convex optimization problems. Inspired by the
L-BFGS, the LM-CMA samples candidate solutions according to a covariance matrix
reproduced from $m$ direction vectors selected during the optimization process.
The decomposition of the covariance matrix into Cholesky factors allows to
reduce the memory complexity to $O(mn)$, where $n$ is the number of decision
variables. The time complexity of sampling one candidate solution is also
$O(mn)$, but scales as only about 25 scalar-vector multiplications in practice.
The algorithm has an important property of invariance w.r.t. strictly
increasing transformations of the objective function, such transformations do
not compromise its ability to approach the optimum. The LM-CMA outperforms the
original CMA-ES and its large scale versions on non-separable ill-conditioned
problems with a factor increasing with problem dimension. Invariance properties
of the algorithm do not prevent it from demonstrating a comparable performance
to L-BFGS on non-trivial large scale smooth and nonsmooth optimization
problems.
End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
Authors: Jianshu Chen, Ji He, Yelong Shen, Lin Xiao, Xiaodong He, Jianfeng Gao, Xinying Song, Li Deng
Date: 2015-08-14 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: General Classification, parameter estimation, Topic Models

We develop a fully discriminative learning approach for supervised Latent
Dirichlet Allocation (LDA) model using Back Propagation (i.e., BP-sLDA), which
maximizes the posterior probability of the prediction variable given the input
document. Different from traditional variational learning or Gibbs sampling
approaches, the proposed learning method applies (i) the mirror descent
algorithm for maximum a posterior inference and (ii) back propagation over a
deep architecture together with stochastic gradient/mirror descent for model
parameter estimation, leading to scalable and end-to-end discriminative
learning of the model. As a byproduct, we also apply this technique to develop
a new learning method for the traditional unsupervised LDA model (i.e.,
BP-LDA). Experimental results on three real-world regression and classification
tasks show that the proposed methods significantly outperform the previous
supervised topic models, neural networks, and is on par with deep neural
networks.
A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
Authors: Peilu Wang, Yao Qian, Frank K. Soong, Lei He, Hai Zhao
Date: 2015-11-01 00:00:00
Tasks: Chunking, Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging

Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has
been shown to be very effective for modeling and predicting sequential data,
e.g. speech utterances or handwritten documents. In this study, we propose to
use BLSTM-RNN for a unified tagging solution that can be applied to various
tagging tasks including part-of-speech tagging, chunking and named entity
recognition. Instead of exploiting specific features carefully optimized for
each task, our solution only uses one set of task-independent features and
internal representations learnt from unlabeled text for all tasks.Requiring no
task specific knowledge or sophisticated feature engineering, our approach gets
nearly state-of-the-art performance in all these three tagging tasks.
On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units
Authors: Zhibin Liao, Gustavo Carneiro
Date: 2015-08-03 00:00:00
Tasks: General Classification, Image Classification

Deep feedforward neural networks with piecewise linear activations are
currently producing the state-of-the-art results in several public datasets.
The combination of deep learning models and piecewise linear activation
functions allows for the estimation of exponentially complex functions with the
use of a large number of subnetworks specialized in the classification of
similar input examples. During the training process, these subnetworks avoid
overfitting with an implicit regularization scheme based on the fact that they
must share their parameters with other subnetworks. Using this framework, we
have made an empirical observation that can improve even more the performance
of such models. We notice that these models assume a balanced initial
distribution of data points with respect to the domain of the piecewise linear
activation function. If that assumption is violated, then the piecewise linear
activation units can degenerate into purely linear activation units, which can
result in a significant reduction of their capacity to learn complex functions.
Furthermore, as the number of model layers increases, this unbalanced initial
distribution makes the model ill-conditioned. Therefore, we propose the
introduction of batch normalisation units into deep feedforward neural networks
with piecewise linear activations, which drives a more balanced use of these
activation units, where each region of the activation function is trained with
a relatively large proportion of training samples. Also, this batch
normalisation promotes the pre-conditioning of very deep learning models. We
show that by introducing maxout and batch normalisation units to the network in
network model results in a model that produces classification results that are
better than or comparable to the current state of the art in CIFAR-10,
CIFAR-100, MNIST, and SVHN datasets.
ADASECANT: Robust Adaptive Secant Method for Stochastic Gradient
Authors: Caglar Gulcehre, Marcin Moczulski, Yoshua Bengio
Date: 2014-12-23 00:00:00

Stochastic gradient algorithms have been the main focus of large-scale
learning problems and they led to important successes in machine learning. The
convergence of SGD depends on the careful choice of learning rate and the
amount of the noise in stochastic estimates of the gradients. In this paper, we
propose a new adaptive learning rate algorithm, which utilizes curvature
information for automatically tuning the learning rates. The information about
the element-wise curvature of the loss function is estimated from the local
statistics of the stochastic first order gradients. We further propose a new
variance reduction technique to speed up the convergence. In our preliminary
experiments with deep neural networks, we obtained better performance compared
to the popular stochastic gradient algorithms.
A Complete Recipe for Stochastic Gradient MCMC
Authors: Yi-An Ma, Tianqi Chen, Emily B. Fox
Date: 2015-06-15 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Physical Intuition

Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous
dynamics to define a transition kernel that efficiently explores a target
distribution. In tandem, a focus has been on devising scalable variants that
subsample the data and use stochastic gradients in place of full-data gradients
in the dynamic simulations. However, such stochastic gradient MCMC samplers
have lagged behind their full-data counterparts in terms of the complexity of
dynamics considered since proving convergence in the presence of the stochastic
gradient noise is non-trivial. Even with simple dynamics, significant physical
intuition is often required to modify the dynamical system to account for the
stochastic gradient noise. In this paper, we provide a general recipe for
constructing MCMC samplers--including stochastic gradient versions--based on
continuous Markov processes specified via two matrices. We constructively prove
that the framework is complete. That is, any continuous Markov process that
provides samples from the target distribution can be written in our framework.
We show how previous continuous-dynamic samplers can be trivially "reinvented"
in our framework, avoiding the complicated sampler-specific proofs. We likewise
use our recipe to straightforwardly propose a new state-adaptive sampler:
stochastic gradient Riemann Hamiltonian Monte Carlo (SGRHMC). Our experiments
on simulated data and a streaming Wikipedia analysis demonstrate that the
proposed SGRHMC sampler inherits the benefits of Riemann HMC, with the
scalability of stochastic gradient methods.
Blitzkriging: Kronecker-structured Stochastic Gaussian Processes
Authors: Thomas Nickson, Tom Gunter, Chris Lloyd, Michael A. Osborne, Stephen Roberts
Date: 2015-10-27 00:00:00
Tasks: Gaussian Processes, General Classification, regression, Time Series, Time Series Analysis, Time Series Prediction

We present Blitzkriging, a new approach to fast inference for Gaussian
processes, applicable to regression, optimisation and classification.
State-of-the-art (stochastic) inference for Gaussian processes on very large
datasets scales cubically in the number of 'inducing inputs', variables
introduced to factorise the model. Blitzkriging shares state-of-the-art scaling
with data, but reduces the scaling in the number of inducing points to
approximately linear. Further, in contrast to other methods, Blitzkriging: does
not force the data to conform to any particular structure (including
grid-like); reduces reliance on error-prone optimisation of inducing point
locations; and is able to learn rich (covariance) structure from the data. We
demonstrate the benefits of our approach on real data in regression,
time-series prediction and signal-interpolation experiments.
Efficient Face Alignment via Locality-constrained Representation for Robust Recognition
Authors: Yandong Wen, Weiyang Liu, Meng Yang, Zhifeng Li
Date: 2015-07-25 00:00:00
Tasks: Face Alignment, Face Recognition

Practical face recognition has been studied in the past decades, but still
remains an open challenge. Current prevailing approaches have already achieved
substantial breakthroughs in recognition accuracy. However, their performance
usually drops dramatically if face samples are severely misaligned. To address
this problem, we propose a highly efficient misalignment-robust
locality-constrained representation (MRLR) algorithm for practical real-time
face recognition. Specifically, the locality constraint that activates the most
correlated atoms and suppresses the uncorrelated ones, is applied to construct
the dictionary for face alignment. Then we simultaneously align the warped face
and update the locality-constrained dictionary, eventually obtaining the final
alignment. Moreover, we make use of the block structure to accelerate the
derived analytical solution. Experimental results on public data sets show that
MRLR significantly outperforms several state-of-the-art approaches in terms of
efficiency and scalability with even better performance.
Regional Active Contours based on Variational level sets and Machine Learning for Image Segmentation
Authors: M. Abdelsamea
Date: 2015-10-31 00:00:00
Tasks: BIG-bench Machine Learning, Image Segmentation, Object, Object Recognition, Object Tracking, Segmentation, Semantic Segmentation

Image segmentation is the problem of partitioning an image into different
subsets, where each subset may have a different characterization in terms of
color, intensity, texture, and/or other features. Segmentation is a fundamental
component of image processing, and plays a significant role in computer vision,
object recognition, and object tracking. Active Contour Models (ACMs)
constitute a powerful energy-based minimization framework for image
segmentation, which relies on the concept of contour evolution. Starting from
an initial guess, the contour is evolved with the aim of approximating better
and better the actual object boundary. Handling complex images in an efficient,
effective, and robust way is a real challenge, especially in the presence of
intensity inhomogeneity, overlap between the foreground/background intensity
distributions, objects characterized by many different intensities, and/or
additive noise. In this thesis, to deal with these challenges, we propose a
number of image segmentation models relying on variational level set methods
and specific kinds of neural networks, to handle complex images in both
supervised and unsupervised ways. Experimental results demonstrate the high
accuracy of the segmentation results, obtained by the proposed models on
various benchmark synthetic and real images compared with state-of-the-art
active contour models.
Fast Neuromimetic Object Recognition using FPGA Outperforms GPU Implementations
Authors: Garrick Orchard, Jacob G. Martin, R. Jacob Vogelstein, Ralph Etienne-Cummings
Date: 2015-10-31 00:00:00
Tasks: GPU, Object, Object Recognition

Recognition of objects in still images has traditionally been regarded as a
difficult computational problem. Although modern automated methods for visual
object recognition have achieved steadily increasing recognition accuracy, even
the most advanced computational vision approaches are unable to obtain
performance equal to that of humans. This has led to the creation of many
biologically-inspired models of visual object recognition, among them the HMAX
model. HMAX is traditionally known to achieve high accuracy in visual object
recognition tasks at the expense of significant computational complexity.
Increasing complexity, in turn, increases computation time, reducing the number
of images that can be processed per unit time. In this paper we describe how
the computationally intensive, biologically inspired HMAX model for visual
object recognition can be modified for implementation on a commercial Field
Programmable Gate Array, specifically the Xilinx Virtex 6 ML605 evaluation
board with XC6VLX240T FPGA. We show that with minor modifications to the
traditional HMAX model we can perform recognition on images of size 128x128
pixels at a rate of 190 images per second with a less than 1% loss in
recognition accuracy in both binary and multi-class visual object recognition
tasks.
Sketch-based Image Retrieval from Millions of Images under Rotation, Translation and Scale Variations
Authors: Sarthak Parui, Anurag Mittal
Date: 2015-10-31 00:00:00
Tasks: Image Retrieval, Object, object-detection, Object Detection, Retrieval, Sketch-Based Image Retrieval, Translation

Proliferation of touch-based devices has made sketch-based image retrieval
practical. While many methods exist for sketch-based object detection/image
retrieval on small datasets, relatively less work has been done on large
(web)-scale image retrieval. In this paper, we present an efficient approach
for image retrieval from millions of images based on user-drawn sketches.
Unlike existing methods for this problem which are sensitive to even
translation or scale variations, our method handles rotation, translation,
scale (i.e. a similarity transformation) and small deformations. The object
boundaries are represented as chains of connected segments and the database
images are pre-processed to obtain such chains that have a high chance of
containing the object. This is accomplished using two approaches in this work:
a) extracting long chains in contour segment networks and b) extracting
boundaries of segmented object proposals. These chains are then represented by
similarity-invariant variable length descriptors. Descriptor similarities are
computed by a fast Dynamic Programming-based partial matching algorithm. This
matching mechanism is used to generate a hierarchical k-medoids based indexing
structure for the extracted chains of all database images in an offline process
which is used to efficiently retrieve a small set of possible matched images
for query chains. Finally, a geometric verification step is employed to test
geometric consistency of multiple chain matches to improve results. Qualitative
and quantitative results clearly demonstrate superiority of the approach over
existing methods.
Semantic Cross-View Matching
Authors: Francesco Castaldo, Amir Zamir, Roland Angst, Francesco Palmieri, Silvio Savarese
Date: 2015-10-31 00:00:00

Matching cross-view images is challenging because the appearance and
viewpoints are significantly different. While low-level features based on
gradient orientations or filter responses can drastically vary with such
changes in viewpoint, semantic information of images however shows an invariant
characteristic in this respect. Consequently, semantically labeled regions can
be used for performing cross-view matching. In this paper, we therefore explore
this idea and propose an automatic method for detecting and representing the
semantic information of an RGB image with the goal of performing cross-view
matching with a (non-RGB) geographic information system (GIS). A segmented
image forms the input to our system with segments assigned to semantic concepts
such as traffic signs, lakes, roads, foliage, etc. We design a descriptor to
robustly capture both, the presence of semantic concepts and the spatial layout
of those segments. Pairwise distances between the descriptors extracted from
the GIS map and the query image are then used to generate a shortlist of the
most promising locations with similar semantic concepts in a consistent spatial
layout. An experimental evaluation with challenging query images and a large
urban area shows promising results.
Bioinspired Visual Motion Estimation
Authors: Garrick Orchard, Ralph Etienne-Cummings
Date: 2015-10-31 00:00:00
Tasks: Motion Estimation

Visual motion estimation is a computationally intensive, but important task
for sighted animals. Replicating the robustness and efficiency of biological
visual motion estimation in artificial systems would significantly enhance the
capabilities of future robotic agents. 25 years ago, in this very journal,
Carver Mead outlined his argument for replicating biological processing in
silicon circuits. His vision served as the foundation for the field of
neuromorphic engineering, which has experienced a rapid growth in interest over
recent years as the ideas and technologies mature. Replicating biological
visual sensing was one of the first tasks attempted in the neuromorphic field.
In this paper we focus specifically on the task of visual motion estimation. We
describe the task itself, present the progression of works from the early first
attempts through to the modern day state-of-the-art, and provide an outlook for
future directions in the field.