pwc-archive/papers-with-abstracts huggingface dataset:
Sequence to Sequence Learning with Neural Networks
Authors: Ilya Sutskever, Oriol Vinyals, Quoc V. Le
Date: 2014-09-10 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Machine Translation, Sentence, Time Series Forecasting, Traffic Prediction

Deep Neural Networks (DNNs) are powerful models that have achieved excellent
performance on difficult learning tasks. Although DNNs work well whenever large
labeled training sets are available, they cannot be used to map sequences to
sequences. In this paper, we present a general end-to-end approach to sequence
learning that makes minimal assumptions on the sequence structure. Our method
uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to
a vector of a fixed dimensionality, and then another deep LSTM to decode the
target sequence from the vector. Our main result is that on an English to
French translation task from the WMT'14 dataset, the translations produced by
the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's
BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did
not have difficulty on long sentences. For comparison, a phrase-based SMT
system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM
to rerank the 1000 hypotheses produced by the aforementioned SMT system, its
BLEU score increases to 36.5, which is close to the previous best result on
this task. The LSTM also learned sensible phrase and sentence representations
that are sensitive to word order and are relatively invariant to the active and
the passive voice. Finally, we found that reversing the order of the words in
all source sentences (but not target sentences) improved the LSTM's performance
markedly, because doing so introduced many short term dependencies between the
source and the target sentence which made the optimization problem easier.
Tools for Terminology Processing
Authors: C. Enguehard, B. Daille, E. Morin
Date: 2014-12-14 00:00:00
Tasks: Information Retrieval, Retrieval

Automatic terminology processing appeared 10 years ago when electronic
corpora became widely available. Such processing may be statistically or
linguistically based and produces terminology resources that can be used in a
number of applications : indexing, information retrieval, technology watch,
etc. We present the tools that have been developed in the IRIN Institute. They
all take as input texts (or collection of texts) and reflect different states
of terminology processing: term acquisition, term recognition and term
structuring.
Bach in 2014: Music Composition with Recurrent Neural Network
Authors: I-Ting Liu, Bhiksha Ramakrishnan
Date: 2014-12-10 00:00:00

We propose a framework for computer music composition that uses resilient
propagation (RProp) and long short term memory (LSTM) recurrent neural network.
In this paper, we show that LSTM network learns the structure and
characteristics of music pieces properly by demonstrating its ability to
recreate music. We also show that predicting existing music using RProp
outperforms Back propagation through time (BPTT).
First order algorithms in variational image processing
Authors: Martin Burger, Alex Sawatzky, Gabriele Steidl
Date: 2014-12-13 00:00:00
Tasks: Deblurring, Denoising, Optical Flow Estimation, Super-Resolution

Variational methods in imaging are nowadays developing towards a quite
universal and flexible tool, allowing for highly successful approaches on tasks
like denoising, deblurring, inpainting, segmentation, super-resolution,
disparity, and optical flow estimation. The overall structure of such
approaches is of the form ${\cal D}(Ku) + \alpha {\cal R} (u) \rightarrow
\min_u$ ; where the functional ${\cal D}$ is a data fidelity term also
depending on some input data $f$ and measuring the deviation of $Ku$ from such
and ${\cal R}$ is a regularization functional. Moreover $K$ is a (often linear)
forward operator modeling the dependence of data on an underlying image, and
$\alpha$ is a positive regularization parameter. While ${\cal D}$ is often
smooth and (strictly) convex, the current practice almost exclusively uses
nonsmooth regularization functionals. The majority of successful techniques is
using nonsmooth and convex functionals like the total variation and
generalizations thereof or $\ell_1$-norms of coefficients arising from scalar
products with some frame system. The efficient solution of such variational
problems in imaging demands for appropriate algorithms. Taking into account the
specific structure as a sum of two very different terms to be minimized,
splitting algorithms are a quite canonical choice. Consequently this field has
revived the interest in techniques like operator splittings or augmented
Lagrangians. Here we shall provide an overview of methods currently developed
and recent results as well as some computational studies providing a comparison
of different methods and also illustrating their success in applications.
Optimization of Reliability of Network of Given Connectivity using Genetic Algorithm
Authors: Ho Tat Lam, Kwok Yip Szeto
Date: 2014-12-13 00:00:00

Reliability is one of the important measures of how well the system meets its
design objective, and mathematically is the probability that a system will
perform satisfactorily for at least a given period of time. When the system is
described by a connected network of N components (nodes) and their L connection
(links), the reliability of the system becomes a difficult network design
problem which solutions are of great practical interest in science and
engineering. This paper discusses the numerical method of finding the most
reliable network for a given N and L using genetic algorithm. For a given
topology of the network, the reliability is numerically computed using
adjacency matrix. For a search in the space of all possible topologies of the
connected network with N nodes and L links, genetic operators such as mutation
and crossover are applied to the adjacency matrix through a string
representation. In the context of graphs, the mutation of strings in genetic
algorithm corresponds to the rewiring of graphs, while crossover corresponds to
the interchange of the sub-graphs. For small networks where the most reliable
network can be found by exhaustive search, genetic algorithm is very efficient.
For larger networks, our results not only demonstrate the efficiency of our
algorithm, but also suggest that the most reliable network will have high
symmetry.
A Study of Sindhi Related and Arabic Script Adapted languages Recognition
Authors: Dil Nawaz Hakro, A. Z. Talib, Zeeshan Bhatti, G. N. Moja
Date: 2014-12-13 00:00:00
Tasks: Articles, Optical Character Recognition, Optical Character Recognition (OCR)

A large number of publications are available for the Optical Character
Recognition (OCR). Significant researches, as well as articles are present for
the Latin, Chinese and Japanese scripts. Arabic script is also one of mature
script from OCR perspective. The adaptive languages which share Arabic script
or its extended characters; still lacking the OCRs for their language. In this
paper we present the efforts of researchers on Arabic and its related and
adapted languages. This survey is organized in different sections, in which
introduction is followed by properties of Sindhi Language. OCR process
techniques and methods used by various researchers are presented. The last
section is dedicated for future work and conclusion is also discussed.
The application of the Bayes Ying Yang harmony based GMMs in on-line signature verification
Authors: Xiaosha Zhao, Mandan Liu
Date: 2014-12-13 00:00:00
Tasks: Model Selection

In this contribution, a Bayes Ying Yang(BYY) harmony based approach for
on-line signature verification is presented. In the proposed method, a simple
but effective Gaussian Mixture Models(GMMs) is used to represent for each
user's signature model based on the prior information collected. Different from
the early works, in this paper, we use the Bayes Ying Yang machine combined
with the harmony function to achieve Automatic Model Selection(AMS) during the
parameter learning for the GMMs, so that a better approximation of the user
model is assured. Experiments on a database from the First International
Signature Verification Competition(SVC 2004) confirm that this combined
algorithm yields quite satisfactory results.
Descriptor Ensemble: An Unsupervised Approach to Descriptor Fusion in the Homography Space
Authors: Yuan-Ting Hu, Yen-Yu Lin, Hsin-Yi Chen, Kuang-Jui Hsu, Bing-Yu Chen
Date: 2014-12-13 00:00:00

With the aim to improve the performance of feature matching, we present an
unsupervised approach to fuse various local descriptors in the space of
homographies. Inspired by the observation that the homographies of correct
feature correspondences vary smoothly along the spatial domain, our approach
stands on the unsupervised nature of feature matching, and can select a good
descriptor for matching each feature point. Specifically, the homography space
serves as the common domain, in which a correspondence obtained by any
descriptor is considered as a point, for integrating various heterogeneous
descriptors. Both geometric coherence and spatial continuity among
correspondences are considered via computing their geodesic distances in the
space. In this way, mutual verification across different descriptors is
allowed, and correct correspondences will be highlighted with a high degree of
consistency (i.e., short geodesic distances here). It follows that one-class
SVM can be applied to identifying these correct correspondences, and boosts the
performance of feature matching. The proposed approach is comprehensively
compared with the state-of-the-art approaches, and evaluated on four benchmarks
of image matching. The promising results manifest its effectiveness.
An Evaluation of Support Vector Machines as a Pattern Recognition Tool
Authors: Eugene Borovikov
Date: 2014-12-13 00:00:00
Tasks: General Classification

The purpose of this report is in examining the generalization performance of
Support Vector Machines (SVM) as a tool for pattern recognition and object
classification. The work is motivated by the growing popularity of the method
that is claimed to guarantee a good generalization performance for the task in
hand. The method is implemented in MATLAB. SVMs based on various kernels are
tested for classifying data from various domains.
A survey of modern optical character recognition techniques
Authors: Eugene Borovikov
Date: 2014-12-13 00:00:00
Tasks: Image Enhancement, Optical Character Recognition, Optical Character Recognition (OCR), Survey

This report explores the latest advances in the field of digital document
recognition. With the focus on printed document imagery, we discuss the major
developments in optical character recognition (OCR) and document image
enhancement/restoration in application to Latin and non-Latin scripts. In
addition, we review and discuss the available technologies for hand-written
document recognition. In this report, we also provide some company-accumulated
benchmark results on available OCR engines.
The Statistics of Streaming Sparse Regression
Authors: Jacob Steinhardt, Stefan Wager, Percy Liang
Date: 2014-12-13 00:00:00
Tasks: regression

We present a sparse analogue to stochastic gradient descent that is
guaranteed to perform well under similar conditions to the lasso. In the linear
regression setup with irrepresentable noise features, our algorithm recovers
the support set of the optimal parameter vector with high probability, and
achieves a statistically quasi-optimal rate of convergence of Op(k log(d)/T),
where k is the sparsity of the solution, d is the number of features, and T is
the number of training examples. Meanwhile, our algorithm does not require any
more computational resources than stochastic gradient descent. In our
experiments, we find that our method substantially out-performs existing
streaming algorithms on both real and simulated data.
Optimizing Over Radial Kernels on Compact Manifolds
Authors: Sadeep Jayasumana, Richard Hartley, Mathieu Salzmann, Hongdong Li, Mehrtash Harandi
Date: 2014-12-13 00:00:00
Proceeding: CVPR 2014 6
Tasks: General Classification

We tackle the problem of optimizing over all possible positive definite
radial kernels on Riemannian manifolds for classification. Kernel methods on
Riemannian manifolds have recently become increasingly popular in computer
vision. However, the number of known positive definite kernels on manifolds
remain very limited. Furthermore, most kernels typically depend on at least one
parameter that needs to be tuned for the problem at hand. A poor choice of
kernel, or of parameter value, may yield significant performance drop-off.
Here, we show that positive definite radial kernels on the unit n-sphere, the
Grassmann manifold and Kendall's shape manifold can be expressed in a simple
form whose parameters can be automatically optimized within a support vector
machine framework. We demonstrate the benefits of our kernel learning algorithm
on object, face, action and shape recognition.
Feature Weight Tuning for Recursive Neural Networks
Authors: Jiwei Li
Date: 2014-12-11 00:00:00

This paper addresses how a recursive neural network model can automatically
leave out useless information and emphasize important evidence, in other words,
to perform "weight tuning" for higher-level representation acquisition. We
propose two models, Weighted Neural Network (WNN) and Binary-Expectation Neural
Network (BENN), which automatically control how much one specific unit
contributes to the higher-level representation. The proposed model can be
viewed as incorporating a more powerful compositional function for embedding
acquisition in recursive neural networks. Experimental results demonstrate the
significant improvement over standard neural models.
A Framework for Shape Analysis via Hilbert Space Embedding
Authors: Sadeep Jayasumana, Mathieu Salzmann, Hongdong Li, Mehrtash Harandi
Date: 2014-12-13 00:00:00
Tasks: Clustering, General Classification, Retrieval

We propose a framework for 2D shape analysis using positive definite kernels
defined on Kendall's shape manifold. Different representations of 2D shapes are
known to generate different nonlinear spaces. Due to the nonlinearity of these
spaces, most existing shape classification algorithms resort to nearest
neighbor methods and to learning distances on shape spaces. Here, we propose to
map shapes on Kendall's shape manifold to a high dimensional Hilbert space
where Euclidean geometry applies. To this end, we introduce a kernel on this
manifold that permits such a mapping, and prove its positive definiteness. This
kernel lets us extend kernel-based algorithms developed for Euclidean spaces,
such as SVM, MKL and kernel PCA, to the shape manifold. We demonstrate the
benefits of our approach over the state-of-the-art methods on shape
classification, clustering and retrieval.
Combinatorial Structure of the Deterministic Seriation Method with Multiple Subset Solutions
Authors: Mark E. Madsen, Carl P. Lipo
Date: 2014-12-13 00:00:00

Seriation methods order a set of descriptions given some criterion (e.g.,
unimodality or minimum distance between similarity scores). Seriation is thus
inherently a problem of finding the optimal solution among a set of
permutations of objects. In this short technical note, we review the
combinatorial structure of the classical seriation problem, which seeks a
single solution out of a set of objects. We then extend those results to the
iterative frequency seriation approach introduced by Lipo (1997), which finds
optimal subsets of objects which each satisfy the unimodality criterion within
each subset. The number of possible solutions across multiple solution subsets
is larger than $n!$, which underscores the need to find new algorithms and
heuristics to assist in the deterministic frequency seriation problem.
Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices
Authors: Sadeep Jayasumana, Richard Hartley, Mathieu Salzmann, Hongdong Li, Mehrtash Harandi
Date: 2014-12-13 00:00:00
Proceeding: CVPR 2013 6
Tasks: Motion Segmentation, Pedestrian Detection, Segmentation, Texture Classification

Symmetric Positive Definite (SPD) matrices have become popular to encode
image information. Accounting for the geometry of the Riemannian manifold of
SPD matrices has proven key to the success of many algorithms. However, most
existing methods only approximate the true shape of the manifold locally by its
tangent plane. In this paper, inspired by kernel methods, we propose to map SPD
matrices to a high dimensional Hilbert space where Euclidean geometry applies.
To encode the geometry of the manifold in the mapping, we introduce a family of
provably positive definite kernels on the Riemannian manifold of SPD matrices.
These kernels are derived from the Gaussian ker- nel, but exploit different
metrics on the manifold. This lets us extend kernel-based algorithms developed
for Euclidean spaces, such as SVM and kernel PCA, to the Riemannian manifold of
SPD matrices. We demonstrate the benefits of our approach on the problems of
pedestrian detection, ob- ject categorization, texture analysis, 2D motion
segmentation and Diffusion Tensor Imaging (DTI) segmentation.
Expanded Alternating Optimization of Nonconvex Functions with Applications to Matrix Factorization and Penalized Regression
Authors: W. James Murdoch, Mu Zhu
Date: 2014-12-12 00:00:00
Tasks: Recommendation Systems, regression

We propose a general technique for improving alternating optimization (AO) of
nonconvex functions. Starting from the solution given by AO, we conduct another
sequence of searches over subspaces that are both meaningful to the
optimization problem at hand and different from those used by AO. To
demonstrate the utility of our approach, we apply it to the matrix
factorization (MF) algorithm for recommender systems and the coordinate descent
algorithm for penalized regression (PR), and show meaningful improvements using
both real-world (for MF) and simulated (for PR) data sets. Moreover, we
demonstrate for MF that, by constructing search spaces customized to the given
data set, we can significantly increase the convergence rate of our technique.
Representing Data by a Mixture of Activated Simplices
Authors: Chunyu Wang, John Flynn, Yizhou Wang, Alan L. Yuille
Date: 2014-12-12 00:00:00

We present a new model which represents data as a mixture of simplices.
Simplices are geometric structures that generalize triangles. We give a simple
geometric understanding that allows us to learn a simplicial structure
efficiently. Our method requires that the data are unit normalized (and thus
lie on the unit sphere). We show that under this restriction, building a model
with simplices amounts to constructing a convex hull inside the sphere whose
boundary facets is close to the data. We call the boundary facets of the convex
hull that are close to the data Activated Simplices. While the total number of
bases used to build the simplices is a parameter of the model, the dimensions
of the individual activated simplices are learned from the data. Simplices can
have different dimensions, which facilitates modeling of inhomogeneous data
sources. The simplicial structure is bounded --- this is appropriate for
modeling data with constraints, such as human elbows can not bend more than 180
degrees. The simplices are easy to interpret and extremes within the data can
be discovered among the vertices. The method provides good reconstruction and
regularization. It supports good nearest neighbor classification and it allows
realistic generative models to be constructed. It achieves state-of-the-art
results on benchmark datasets, including 3D poses and digits.
Multiscale Fields of Patterns
Authors: Pedro F. Felzenszwalb, John G. Oberlin
Date: 2014-06-04 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Contour Detection

We describe a framework for defining high-order image models that can be used
in a variety of applications. The approach involves modeling local patterns in
a multiscale representation of an image. Local properties of a coarsened image
reflect non-local properties of the original image. In the case of binary
images local properties are defined by the binary patterns observed over small
neighborhoods around each pixel. With the multiscale representation we capture
the frequency of patterns observed at different scales of resolution. This
framework leads to expressive priors that depend on a relatively small number
of parameters. For inference and learning we use an MCMC method for block
sampling with very large blocks. We evaluate the approach with two example
applications. One involves contour detection. The other involves binary
segmentation.
Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso
Authors: Antoine Bonnefoy, Valentin Emiya, Liva Ralaivola, Rémi Gribonval
Date: 2014-12-12 00:00:00
Tasks: regression

Recent computational strategies based on screening tests have been proposed
to accelerate algorithms addressing penalized sparse regression problems such
as the Lasso. Such approaches build upon the idea that it is worth dedicating
some small computational effort to locate inactive atoms and remove them from
the dictionary in a preprocessing stage so that the regression algorithm
working with a smaller dictionary will then converge faster to the solution of
the initial problem. We believe that there is an even more efficient way to
screen the dictionary and obtain a greater acceleration: inside each iteration
of the regression algorithm, one may take advantage of the algorithm
computations to obtain a new screening test for free with increasing screening
effects along the iterations. The dictionary is henceforth dynamically screened
instead of being screened statically, once and for all, before the first
iteration. We formalize this dynamic screening principle in a general
algorithmic scheme and apply it by embedding inside a number of first-order
algorithms adapted existing screening tests to solve the Lasso or new screening
tests to solve the Group-Lasso. Computational gains are assessed in a large set
of experiments on synthetic data as well as real-world sounds and images. They
show both the screening efficiency and the gain in terms running times.