pwc-archive/papers-with-abstracts huggingface dataset:
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
Authors: Roy Frostig, Rong Ge, Sham M. Kakade, Aaron Sidford
Date: 2015-06-24 00:00:00

We develop a family of accelerated stochastic algorithms that minimize sums
of convex functions. Our algorithms improve upon the fastest running time for
empirical risk minimization (ERM), and in particular linear least-squares
regression, across a wide range of problem settings. To achieve this, we
establish a framework based on the classical proximal point algorithm. Namely,
we provide several algorithms that reduce the minimization of a strongly convex
function to approximate minimizations of regularizations of the function. Using
these results, we accelerate recent fast stochastic algorithms in a black-box
fashion. Empirically, we demonstrate that the resulting algorithms exhibit
notions of stability that are advantageous in practice. Both in theory and in
practice, the provided algorithms reap the computational benefits of adding a
large strongly convex regularization term, without incurring a corresponding
bias to the original problem.
Objective Variables for Probabilistic Revenue Maximization in Second-Price Auctions with Reserve
Authors: Maja R. Rudolph, Joseph G. Ellis, David M. Blei
Date: 2015-06-24 00:00:00
Tasks: Decision Making, regression

Many online companies sell advertisement space in second-price auctions with
reserve. In this paper, we develop a probabilistic method to learn a profitable
strategy to set the reserve price. We use historical auction data with features
to fit a predictor of the best reserve price. This problem is delicate - the
structure of the auction is such that a reserve price set too high is much
worse than a reserve price set too low. To address this we develop objective
variables, a new framework for combining probabilistic modeling with optimal
decision-making. Objective variables are "hallucinated observations" that
transform the revenue maximization task into a regularized maximum likelihood
estimation problem, which we solve with an EM algorithm. This framework enables
a variety of prediction mechanisms to set the reserve price. As examples, we
study objective variable methods with regression, kernelized regression, and
neural networks on simulated and real data. Our methods outperform previous
approaches both in terms of scalability and profit.
Attention-Based Models for Speech Recognition
Authors: Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, Yoshua Bengio
Date: 2015-06-24 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Machine Translation, Phoneme Recognition, Speech Recognition, Translation

Recurrent sequence generators conditioned on input data through an attention
mechanism have recently shown very good performance on a range of tasks in-
cluding machine translation, handwriting synthesis and image caption gen-
eration. We extend the attention-mechanism with features needed for speech
recognition. We show that while an adaptation of the model used for machine
translation in reaches a competitive 18.7% phoneme error rate (PER) on the
TIMIT phoneme recognition task, it can only be applied to utterances which are
roughly as long as the ones it was trained on. We offer a qualitative
explanation of this failure and propose a novel and generic method of adding
location-awareness to the attention mechanism to alleviate this issue. The new
method yields a model that is robust to long inputs and achieves 18% PER in
single utterances and 20% in 10-times longer (repeated) utterances. Finally, we
propose a change to the at- tention mechanism that prevents it from
concentrating too much on single frames, which further reduces PER to 17.6%
level.
Efficient Learning for Undirected Topic Models
Authors: Jiatao Gu, Victor O. K. Li
Date: 2015-06-24 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: General Classification, Retrieval, Topic Models

Replicated Softmax model, a well-known undirected topic model, is powerful in
extracting semantic representations of documents. Traditional learning
strategies such as Contrastive Divergence are very inefficient. This paper
provides a novel estimator to speed up the learning based on Noise Contrastive
Estimate, extended for documents of variant lengths and weighted inputs.
Experiments on two benchmarks show that the new estimator achieves great
learning efficiency and high accuracy on document retrieval and classification.
Nonnegative Matrix Factorization applied to reordered pixels of single images based on patches to achieve structured nonnegative dictionaries
Authors: Richard M. Charles, Kye M. Taylor, James H. Curry
Date: 2015-06-24 00:00:00

Recent improvements in computing allow for the processing and analysis of
very large datasets in a variety of fields. Often the analysis requires the
creation of low-rank approximations to the datasets leading to efficient
storage. This article presents and analyzes a novel approach for creating
nonnegative, structured dictionaries using NMF applied to reordered pixels of
single, natural images. We reorder the pixels based on patches and present our
approach in general. We investigate our approach when using the Singular Value
Decomposition (SVD) and Nonnegative Matrix Factorizations (NMF) as low-rank
approximations. Peak Signal-to-Noise Ratio (PSNR) and Mean Structural
Similarity Index (MSSIM) are used to evaluate the algorithm. We report that
while the SVD provides the best reconstructions, its dictionary of vectors lose
both the sign structure of the original image and details of localized image
content. In contrast, the dictionaries produced using NMF preserves the sign
structure of the original image matrix and offer a nonnegative, parts-based
dictionary.
Highly Accurate Multispectral Palmprint Recognition Using Statistical and Wavelet Features
Authors: Shervin Minaee, Amirali Abdolrashidi
Date: 2014-08-16 00:00:00
Tasks: General Classification

Palmprint is one of the most useful physiological biometrics that can be used
as a powerful means in personal recognition systems. The major features of the
palmprints are palm lines, wrinkles and ridges, and many approaches use them in
different ways towards solving the palmprint recognition problem. Here we have
proposed to use a set of statistical and wavelet-based features; statistical to
capture the general characteristics of palmprints; and wavelet-based to find
those information not evident in the spatial domain. Also we use two different
classification approaches, minimum distance classifier scheme and weighted
majority voting algorithm, to perform palmprint matching. The proposed method
is tested on a well-known palmprint dataset of 6000 samples and has shown an
impressive accuracy rate of 99.65\%-100\% for most scenarios.
Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
Authors: Marijn F. Stollenga, Wonmin Byeon, Marcus Liwicki, Juergen Schmidhuber
Date: 2015-06-24 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Brain Image Segmentation, Image Segmentation, Semantic Segmentation

Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D
videos to segment them. They have a fixed input size and typically perceive
only small local contexts of the pixels to be classified as foreground or
background. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive
the entire spatio-temporal context of each pixel in a few sweeps through all
pixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite
these theoretical advantages, however, unlike CNNs, previous MD-LSTM variants
were hard to parallelize on GPUs. Here we re-arrange the traditional cuboid
order of computations in MD-LSTM in pyramidal fashion. The resulting
PyraMiD-LSTM is easy to parallelize, especially for 3D data such as stacks of
brain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image
segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).
Unshredding of Shredded Documents: Computational Framework and Implementation
Authors: Lei Kristoffer R. Lactuan, Jaderick P. Pabico
Date: 2015-06-24 00:00:00

A shredded document $D$ is a document whose pages have been cut into strips
for the purpose of destroying private, confidential, or sensitive information
$I$ contained in $D$. Shredding has become a standard means of government
organizations, businesses, and private individuals to destroy archival records
that have been officially classified for disposal. It can also be used to
destroy documentary evidence of wrongdoings by entities who are trying to hide
$I$.
  In this paper, we present an optimal $O((n\times m)^2)$ algorithm $A$ that
reconstructs an $n$-page $D$, where each page $p$ is shredded into $m$ strips.
We also present the efficacy of $A$ in reconstructing three document types:
hand-written, machine typed-set, and images.
Salient Object Detection via Objectness Measure
Authors: Sai Srivatsa R, R. Venkatesh Babu
Date: 2015-06-24 00:00:00
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Salient object detection has become an important task in many image
processing applications. The existing approaches exploit background prior and
contrast prior to attain state of the art results. In this paper, instead of
using background cues, we estimate the foreground regions in an image using
objectness proposals and utilize it to obtain smooth and accurate saliency
maps. We propose a novel saliency measure called `foreground connectivity'
which determines how tightly a pixel or a region is connected to the estimated
foreground. We use the values assigned by this measure as foreground weights
and integrate these in an optimization framework to obtain the final saliency
maps. We extensively evaluate the proposed approach on two benchmark databases
and demonstrate that the results obtained are better than the existing state of
the art approaches.
Sequential Extensions of Causal and Evidential Decision Theory
Authors: Tom Everitt, Jan Leike, Marcus Hutter
Date: 2015-06-24 00:00:00
Tasks: Decision Making

Moving beyond the dualistic view in AI where agent and environment are
separated incurs new challenges for decision making, as calculation of expected
utility is no longer straightforward. The non-dualistic decision theory
literature is split between causal decision theory and evidential decision
theory. We extend these decision algorithms to the sequential setting where the
agent alternates between taking actions and observing their consequences. We
find that evidential decision theory has two natural extensions while causal
decision theory only has one.
Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
Authors: Philipp Fischer, Alexey Dosovitskiy, Thomas Brox
Date: 2014-05-22 00:00:00
Tasks: General Classification

Latest results indicate that features learned via convolutional neural
networks outperform previous descriptors on classification tasks by a large
margin. It has been shown that these networks still work well when they are
applied to datasets or recognition tasks different from those they were trained
on. However, descriptors like SIFT are not only used in recognition but also
for many correspondence problems that rely on descriptor matching. In this
paper we compare features from various layers of convolutional neural nets to
standard SIFT descriptors. We consider a network that was trained on ImageNet
and another one that was trained without supervision. Surprisingly,
convolutional neural networks clearly outperform SIFT on descriptor matching.
This paper has been merged with arXiv:1406.6909
Natural Scene Recognition Based on Superpixels and Deep Boltzmann Machines
Authors: Jinfu Yang, Jingyu Gao, Guanghui Wang, Shanshan Zhang
Date: 2015-06-24 00:00:00
Tasks: Clustering, Handwritten Digit Recognition, Object Recognition, Scene Recognition, Superpixels

The Deep Boltzmann Machines (DBM) is a state-of-the-art unsupervised learning
model, which has been successfully applied to handwritten digit recognition
and, as well as object recognition. However, the DBM is limited in scene
recognition due to the fact that natural scene images are usually very large.
In this paper, an efficient scene recognition approach is proposed based on
superpixels and the DBMs. First, a simple linear iterative clustering (SLIC)
algorithm is employed to generate superpixels of input images, where each
superpixel is regarded as an input of a learning model. Then, a two-layer DBM
model is constructed by stacking two restricted Boltzmann machines (RBMs), and
a greedy layer-wise algorithm is applied to train the DBM model. Finally, a
softmax regression is utilized to categorize scene images. The proposed
technique can effectively reduce the computational complexity and enhance the
performance for large natural image recognition. The approach is verified and
evaluated by extensive experiments, including the fifteen-scene categories
dataset the UIUC eight-sports dataset, and the SIFT flow dataset, are used to
evaluate the proposed method. The experimental results show that the proposed
approach outperforms other state-of-the-art methods in terms of recognition
rate.
A Novel Feature Extraction Method for Scene Recognition Based on Centered Convolutional Restricted Boltzmann Machines
Authors: Jingyu Gao, Jinfu Yang, Guanghui Wang, Mingai Li
Date: 2015-06-24 00:00:00
Tasks: Object Recognition, Scene Recognition

Scene recognition is an important research topic in computer vision, while
feature extraction is a key step of object recognition. Although classical
Restricted Boltzmann machines (RBM) can efficiently represent complicated data,
it is hard to handle large images due to its complexity in computation. In this
paper, a novel feature extraction method, named Centered Convolutional
Restricted Boltzmann Machines (CCRBM), is proposed for scene recognition. The
proposed model is an improved Convolutional Restricted Boltzmann Machines
(CRBM) by introducing centered factors in its learning strategy to reduce the
source of instabilities. First, the visible units of the network are redefined
using centered factors. Then, the hidden units are learned with a modified
energy function by utilizing a distribution function, and the visible units are
reconstructed using the learned hidden units. In order to achieve better
generative ability, the Centered Convolutional Deep Belief Networks (CCDBN) is
trained in a greedy layer-wise way. Finally, a softmax regression is
incorporated for scene recognition. Extensive experimental evaluations using
natural scenes, MIT-indoor scenes, and Caltech 101 datasets show that the
proposed approach performs better than other counterparts in terms of
stability, generalization, and discrimination. The CCDBN model is more suitable
for natural scene image recognition by virtue of convolutional property.
Unconfused ultraconservative multiclass algorithms
Authors: Ugo Louche, Liva Ralaivola
Date: 2015-06-24 00:00:00

We tackle the problem of learning linear classifiers from noisy datasets in a
multiclass setting. The two-class version of this problem was studied a few
years ago where the proposed approaches to combat the noise revolve around a
Per-ceptron learning scheme fed with peculiar examples computed through a
weighted average of points from the noisy training set. We propose to build
upon these approaches and we introduce a new algorithm called UMA (for
Unconfused Multiclass additive Algorithm) which may be seen as a generalization
to the multiclass setting of the previous approaches. In order to characterize
the noise we use the confusion matrix as a multiclass extension of the
classification noise studied in the aforemen-tioned literature. Theoretically
well-founded, UMA furthermore displays very good empirical noise robustness, as
evidenced by numerical simulations conducted on both synthetic and real data.
Benchmark of structured machine learning methods for microbial identification from mass-spectrometry data
Authors: Kévin Vervier, Pierre Mahé, Jean-Baptiste Veyrieras, Jean-Philippe Vert
Date: 2015-06-24 00:00:00
Tasks: BIG-bench Machine Learning

Microbial identification is a central issue in microbiology, in particular in
the fields of infectious diseases diagnosis and industrial quality control. The
concept of species is tightly linked to the concept of biological and clinical
classification where the proximity between species is generally measured in
terms of evolutionary distances and/or clinical phenotypes. Surprisingly, the
information provided by this well-known hierarchical structure is rarely used
by machine learning-based automatic microbial identification systems.
Structured machine learning methods were recently proposed for taking into
account the structure embedded in a hierarchy and using it as additional a
priori information, and could therefore allow to improve microbial
identification systems. We test and compare several state-of-the-art machine
learning methods for microbial identification on a new Matrix-Assisted Laser
Desorption/Ionization Time-of-Flight mass spectrometry (MALDI-TOF MS) dataset.
We include in the benchmark standard and structured methods, that leverage the
knowledge of the underlying hierarchical structure in the learning process. Our
results show that although some methods perform better than others, structured
methods do not consistently perform better than their "flat" counterparts. We
postulate that this is partly due to the fact that standard methods already
reach a high level of accuracy in this context, and that they mainly confuse
species close to each other in the tree, a case where using the known hierarchy
is not helpful.
Deep CNN Ensemble with Data Augmentation for Object Detection
Authors: Jian Guo, Stephen Gould
Date: 2015-06-24 00:00:00
Tasks: Data Augmentation, Object, object-detection, Object Detection

We report on the methods used in our recent DeepEnsembleCoco submission to
the PASCAL VOC 2012 challenge, which achieves state-of-the-art performance on
the object detection task. Our method is a variant of the R-CNN model proposed
Girshick:CVPR14 with two key improvements to training and evaluation. First,
our method constructs an ensemble of deep CNN models with different
architectures that are complementary to each other. Second, we augment the
PASCAL VOC training set with images from the Microsoft COCO dataset to
significantly enlarge the amount training data. Importantly, we select a subset
of the Microsoft COCO images to be consistent with the PASCAL VOC task. Results
on the PASCAL VOC evaluation server show that our proposed method outperform
all previous methods on the PASCAL VOC 2012 detection task at time of
submission.
Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks
Authors: Yangtuo Peng, Hui Jiang
Date: 2015-06-24 00:00:00
Proceeding: NAACL 2016 6
Tasks: Stock Prediction, Word Embeddings

Financial news contains useful information on public companies and the
market. In this paper we apply the popular word embedding methods and deep
neural networks to leverage financial news to predict stock price movements in
the market. Experimental results have shown that our proposed methods are
simple but very effective, which can significantly improve the stock prediction
accuracy on a standard financial database over the baseline system using only
the historical price information.
deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
Authors: Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, Bill Dolan
Date: 2015-06-23 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Sentence

We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic
evaluation of generated text in tasks that admit a diverse range of possible
outputs. Reference strings are scored for quality by human raters on a scale of
[-1, +1] to weight multi-reference BLEU. In tasks involving generation of
conversational responses, deltaBLEU correlates reasonably with human judgments
and outperforms sentence-level and IBM BLEU in terms of both Spearman's rho and
Kendall's tau.
Context-Dependent Translation Selection Using Convolutional Neural Network
Authors: Zhaopeng Tu, Baotian Hu, Zhengdong Lu, Hang Li
Date: 2015-03-09 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Machine Translation, Semantic Similarity, Semantic Textual Similarity, Sentence, Translation

We propose a novel method for translation selection in statistical machine
translation, in which a convolutional neural network is employed to judge the
similarity between a phrase pair in two languages. The specifically designed
convolutional architecture encodes not only the semantic similarity of the
translation pair, but also the context containing the phrase in the source
language. Therefore, our approach is able to capture context-dependent semantic
similarities of translation pairs. We adopt a curriculum learning strategy to
train the model: we classify the training examples into easy, medium, and
difficult categories, and gradually build the ability of representing phrase
and sentence level context by using training examples from easy to difficult.
Experimental results show that our approach significantly outperforms the
baseline system by up to 1.4 BLEU points.
Elicitation Complexity of Statistical Properties
Authors: Rafael Frongillo, Ian A. Kash
Date: 2015-06-23 00:00:00

A property, or statistical functional, is said to be elicitable if it minimizes expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limitations of point estimation and empirical risk minimization. While recent work asks which properties are elicitable, we instead advocate for a more nuanced question: how many dimensions are required to indirectly elicit a given property? This number is called the elicitation complexity of the property. We lay the foundation for a general theory of elicitation complexity, including several basic results about how elicitation complexity behaves, and the complexity of standard properties of interest. Building on this foundation, our main result gives tight complexity bounds for the broad class of Bayes risks. We apply these results to several properties of interest, including variance, entropy, norms, and several classes of financial risk measures. We conclude with discussion and open directions.
Clustering, Classification, Discriminant Analysis, and Dimension Reduction via Generalized Hyperbolic Mixtures
Authors: Katherine Morris, Paul D. McNicholas
Date: 2013-08-28 00:00:00
Tasks: Clustering, Dimensionality Reduction, General Classification

A method for dimension reduction with clustering, classification, or
discriminant analysis is introduced. This mixture model-based approach is based
on fitting generalized hyperbolic mixtures on a reduced subspace within the
paradigm of model-based clustering, classification, or discriminant analysis. A
reduced subspace of the data is derived by considering the extent to which
group means and group covariances vary. The members of the subspace arise
through linear combinations of the original data, and are ordered by importance
via the associated eigenvalues. The observations can be projected onto the
subspace, resulting in a set of variables that captures most of the clustering
information available. The use of generalized hyperbolic mixtures gives a
robust framework capable of dealing with skewed clusters. Although dimension
reduction is increasingly in demand across many application areas, the authors
are most familiar with biological applications and so two of the five real data
examples are within that sphere. Simulated data are also used for illustration.
The approach introduced herein can be considered the most general such approach
available, and so we compare results to three special and limiting cases.
Comparisons with several well established techniques illustrate its promising
performance.