pwc-archive/papers-with-abstracts huggingface dataset:
Of Quantiles and Expectiles: Consistent Scoring Functions, Choquet Representations, and Forecast Rankings
Authors: Werner Ehm, Tilmann Gneiting, Alexander Jordan, Fabian Krüger
Date: 2015-03-27 00:00:00
Tasks: Decision Making

In the practice of point prediction, it is desirable that forecasters receive
a directive in the form of a statistical functional, such as the mean or a
quantile of the predictive distribution. When evaluating and comparing
competing forecasts, it is then critical that the scoring function used for
these purposes be consistent for the functional at hand, in the sense that the
expected score is minimized when following the directive.
  We show that any scoring function that is consistent for a quantile or an
expectile functional, respectively, can be represented as a mixture of extremal
scoring functions that form a linearly parameterized family. Scoring functions
for the mean value and probability forecasts of binary events constitute
important examples. The quantile and expectile functionals along with the
respective extremal scoring functions admit appealing economic interpretations
in terms of thresholds in decision making.
  The Choquet type mixture representations give rise to simple checks of
whether a forecast dominates another in the sense that it is preferable under
any consistent scoring function. In empirical settings it suffices to compare
the average scores for only a finite number of extremal elements. Plots of the
average scores with respect to the extremal scoring functions, which we call
Murphy diagrams, permit detailed comparisons of the relative merits of
competing forecasts.
Asymptotic Accuracy of Bayesian Estimation for a Single Latent Variable
Authors: Keisuke Yamazaki
Date: 2014-08-25 00:00:00

In data science and machine learning, hierarchical parametric models, such as
mixture models, are often used. They contain two kinds of variables: observable
variables, which represent the parts of the data that can be directly measured,
and latent variables, which represent the underlying processes that generate
the data. Although there has been an increase in research on the estimation
accuracy for observable variables, the theoretical analysis of estimating
latent variables has not been thoroughly investigated. In a previous study, we
determined the accuracy of a Bayes estimation for the joint probability of the
latent variables in a dataset, and we proved that the Bayes method is
asymptotically more accurate than the maximum-likelihood method. However, the
accuracy of the Bayes estimation for a single latent variable remains unknown.
In the present paper, we derive the asymptotic expansions of the error
functions, which are defined by the Kullback-Leibler divergence, for two types
of single-variable estimations when the statistical regularity is satisfied.
Our results indicate that the accuracies of the Bayes and maximum-likelihood
methods are asymptotically equivalent and clarify that the Bayes method is only
advantageous for multivariable estimations.
VIP: Finding Important People in Images
Authors: Clint Solomon Mathialagan, Andrew C. Gallagher, Dhruv Batra
Date: 2015-02-19 00:00:00
Proceeding: CVPR 2015 6

People preserve memories of events such as birthdays, weddings, or vacations
by capturing photos, often depicting groups of people. Invariably, some
individuals in the image are more important than others given the context of
the event. This paper analyzes the concept of the importance of individuals in
group photographs. We address two specific questions -- Given an image, who are
the most important individuals in it? Given multiple images of a person, which
image depicts the person in the most important role? We introduce a measure of
importance of people in images and investigate the correlation between
importance and visual saliency. We find that not only can we automatically
predict the importance of people from purely visual cues, incorporating this
predicted importance results in significant improvement in applications such as
im2text (generating sentences that describe images of groups of people).
Feasibility Preserving Constraint-Handling Strategies for Real Parameter Evolutionary Optimization
Authors: Nikhil Padhye, Pulkit Mittal, Kalyanmoy Deb
Date: 2015-04-17 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary Algorithms (EAs) are being routinely applied for a variety of
optimization tasks, and real-parameter optimization in the presence of
constraints is one such important area. During constrained optimization EAs
often create solutions that fall outside the feasible region; hence a viable
constraint- handling strategy is needed. This paper focuses on the class of
constraint-handling strategies that repair infeasible solutions by bringing
them back into the search space and explicitly preserve feasibility of the
solutions. Several existing constraint-handling strategies are studied, and two
new single parameter constraint-handling methodologies based on parent-centric
and inverse parabolic probability (IP) distribution are proposed. The existing
and newly proposed constraint-handling methods are first studied with PSO, DE,
GAs, and simulation results on four scalable test-problems under different
location settings of the optimum are presented. The newly proposed
constraint-handling methods exhibit robustness in terms of performance and also
succeed on search spaces comprising up-to 500 variables while locating the
optimum within an error of 10$^{-10}$. The working principle of the IP based
methods is also demonstrated on (i) some generic constrained optimization
problems, and (ii) a classic `Weld' problem from structural design and
mechanics. The successful performance of the proposed methods clearly exhibits
their efficacy as a generic constrained-handling strategy for a wide range of
applications.
An Empirical Evaluation of Deep Learning on Highway Driving
Authors: Brody Huval, Tao Wang, Sameep Tandon, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng-Yue, Fernando Mujica, Adam Coates, Andrew Y. Ng
Date: 2015-04-07 00:00:00
Tasks: Autonomous Driving, Deep Learning, Lane Detection, vehicle detection

Numerous groups have applied a variety of deep learning techniques to
computer vision problems in highway perception scenarios. In this paper, we
presented a number of empirical evaluations of recent deep learning advances.
Computer vision, combined with deep learning, has the potential to bring about
a relatively inexpensive, robust solution to autonomous driving. To prepare
deep learning for industry uptake and practical applications, neural networks
will require large data sets that represent all possible driving environments
and scenarios. We collect a large data set of highway data and apply deep
learning and computer vision algorithms to problems such as car and lane
detection. We show how existing convolutional neural networks (CNNs) can be
used to perform lane and vehicle detection while running at frame rates
required for a real-time system. Our results lend credence to the hypothesis
that deep learning holds promise for autonomous driving.
Random Forests Can Hash
Authors: Qiang Qiu, Guillermo Sapiro, Alex Bronstein
Date: 2014-12-16 00:00:00
Tasks: Retrieval

Hash codes are a very efficient data representation needed to be able to cope
with the ever growing amounts of data. We introduce a random forest semantic
hashing scheme with information-theoretic code aggregation, showing for the
first time how random forest, a technique that together with deep learning have
shown spectacular results in classification, can also be extended to
large-scale retrieval. Traditional random forest fails to enforce the
consistency of hashes generated from each tree for the same class data, i.e.,
to preserve the underlying similarity, and it also lacks a principled way for
code aggregation across trees. We start with a simple hashing scheme, where
independently trained random trees in a forest are acting as hashing functions.
We the propose a subspace model as the splitting function, and show that it
enforces the hash consistency in a tree for data from the same class. We also
introduce an information-theoretic approach for aggregating codes of individual
trees into a single hash code, producing a near-optimal unique hash for each
class. Experiments on large-scale public datasets are presented, showing that
the proposed approach significantly outperforms state-of-the-art hashing
methods for retrieval tasks.
Learning Longer Memory in Recurrent Neural Networks
Authors: Tomas Mikolov, Armand Joulin, Sumit Chopra, Michael Mathieu, Marc'Aurelio Ranzato
Date: 2014-12-24 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent neural network is a powerful model that learns temporal patterns in
sequential data. For a long time, it was believed that recurrent networks are
difficult to train using simple optimizers, such as stochastic gradient
descent, due to the so-called vanishing gradient problem. In this paper, we
show that learning longer term patterns in real data, such as in natural
language, is perfectly possible using gradient descent. This is achieved by
using a slight structural modification of the simple recurrent neural network
architecture. We encourage some of the hidden units to change their state
slowly by making part of the recurrent weight matrix close to identity, thus
forming kind of a longer term memory. We evaluate our model in language
modeling experiments, where we obtain similar performance to the much more
complex Long Short Term Memory (LSTM) networks (Hochreiter & Schmidhuber,
1997).
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
Authors: Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar
Date: 2015-04-16 00:00:00

We apply stochastic average gradient (SAG) algorithms for training
conditional random fields (CRFs). We describe a practical implementation that
uses structure in the CRF gradient to reduce the memory requirement of this
linearly-convergent stochastic gradient method, propose a non-uniform sampling
scheme that substantially improves practical performance, and analyze the rate
of convergence of the SAGA variant under non-uniform sampling. Our experimental
results reveal that our method often significantly outperforms existing methods
in terms of the training objective, and performs as well or better than
optimally-tuned stochastic gradient methods in terms of test error.
Breaking the News: First Impressions Matter on Online News
Authors: Julio Reis, Fabricio Benevenuto, Pedro O. S. Vaz de Melo, Raquel Prates, Haewoon Kwak, Jisun An
Date: 2015-03-26 00:00:00

A growing number of people are changing the way they consume news, replacing
the traditional physical newspapers and magazines by their virtual online
versions or/and weblogs. The interactivity and immediacy present in online news
are changing the way news are being produced and exposed by media corporations.
News websites have to create effective strategies to catch people's attention
and attract their clicks. In this paper we investigate possible strategies used
by online news corporations in the design of their news headlines. We analyze
the content of 69,907 headlines produced by four major global media
corporations during a minimum of eight consecutive months in 2014. In order to
discover strategies that could be used to attract clicks, we extracted features
from the text of the news headlines related to the sentiment polarity of the
headline. We discovered that the sentiment of the headline is strongly related
to the popularity of the news and also with the dynamics of the posted comments
on that particular news.
On Learning Vector Representations in Hierarchical Label Spaces
Authors: Jinseok Nam, Johannes Fürnkranz
Date: 2014-12-22 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

An important problem in multi-label classification is to capture label
patterns or underlying structures that have an impact on such patterns. This
paper addresses one such problem, namely how to exploit hierarchical structures
over labels. We present a novel method to learn vector representations of a
label space given a hierarchy of labels and label co-occurrence patterns. Our
experimental results demonstrate qualitatively that the proposed method is able
to learn regularities among labels by exploiting a label hierarchy as well as
label co-occurrences. It highlights the importance of the hierarchical
information in order to obtain regularities which facilitate analogical
reasoning over a label space. We also experimentally illustrate the dependency
of the learned representations on the label hierarchy.
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
Authors: Behnam Neyshabur, Ryota Tomioka, Nathan Srebro
Date: 2014-12-20 00:00:00
Tasks: Inductive Bias

We present experiments demonstrating that some other form of capacity
control, different from network size, plays a central role in learning
multilayer feed-forward networks. We argue, partially through analogy to matrix
factorization, that this is an inductive bias that can help shed light on deep
learning.
High-performance Kernel Machines with Implicit Distributed Optimization and Randomization
Authors: Vikas Sindhwani, Haim Avron
Date: 2014-09-03 00:00:00
Tasks: Distributed Optimization, Vocal Bursts Intensity Prediction

In order to fully utilize "big data", it is often required to use "big
models". Such models tend to grow with the complexity and size of the training
data, and do not make strong parametric assumptions upfront on the nature of
the underlying statistical dependencies. Kernel methods fit this need well, as
they constitute a versatile and principled statistical methodology for solving
a wide range of non-parametric modelling problems. However, their high
computational costs (in storage and time) pose a significant barrier to their
widespread adoption in big data applications.
  We propose an algorithmic framework and high-performance implementation for
massive-scale training of kernel-based statistical models, based on combining
two key technical ingredients: (i) distributed general purpose convex
optimization, and (ii) the use of randomization to improve the scalability of
kernel methods. Our approach is based on a block-splitting variant of the
Alternating Directions Method of Multipliers, carefully reconfigured to handle
very large random feature matrices, while exploiting hybrid parallelism
typically found in modern clusters of multicore machines. Our implementation
supports a variety of statistical learning tasks by enabling several loss
functions, regularization schemes, kernels, and layers of randomized
approximations for both dense and sparse datasets, in a highly extensible
framework. We evaluate the ability of our framework to learn models on data
from applications, and provide a comparison against existing sequential and
parallel libraries.
Towards a relation extraction framework for cyber-security concepts
Authors: Corinne L. Jones, Robert A. Bridges, Kelly Huffer, John Goodall
Date: 2015-04-16 00:00:00
Tasks: Active Learning, Information Retrieval, Relation, Relation Extraction, Retrieval

In order to assist security analysts in obtaining information pertaining to
their network, such as novel vulnerabilities, exploits, or patches, information
retrieval methods tailored to the security domain are needed. As labeled text
data is scarce and expensive, we follow developments in semi-supervised Natural
Language Processing and implement a bootstrapping algorithm for extracting
security entities and their relationships from text. The algorithm requires
little input data, specifically, a few relations or patterns (heuristics for
identifying relations), and incorporates an active learning component which
queries the user on the most important decisions to prevent drifting from the
desired relations. Preliminary testing on a small corpus shows promising
results, obtaining precision of .82.
Reweighted Wake-Sleep
Authors: Jörg Bornschein, Yoshua Bengio
Date: 2014-06-11 00:00:00

Training deep directed graphical models with many hidden variables and
performing inference remains a major challenge. Helmholtz machines and deep
belief networks are such models, and the wake-sleep algorithm has been proposed
to train them. The wake-sleep algorithm relies on training not just the
directed generative model but also a conditional generative model (the
inference network) that runs backward from visible to latent, estimating the
posterior distribution of latent given visible. We propose a novel
interpretation of the wake-sleep algorithm which suggests that better
estimators of the gradient can be obtained by sampling latent variables
multiple times from the inference network. This view is based on importance
sampling as an estimator of the likelihood, with the approximate inference
network as a proposal distribution. This interpretation is confirmed
experimentally, showing that better likelihood can be achieved with this
reweighted wake-sleep procedure. Based on this interpretation, we propose that
a sigmoidal belief network is not sufficiently powerful for the layers of the
inference network in order to recover a good estimator of the posterior
distribution of latent variables. Our experiments show that using a more
powerful layer model, such as NADE, yields substantially better generative
models.
Metric Localization using Google Street View
Authors: Pratik Agarwal, Wolfram Burgard, Luciano Spinello
Date: 2015-03-14 00:00:00

Accurate metrical localization is one of the central challenges in mobile
robotics. Many existing methods aim at localizing after building a map with the
robot. In this paper, we present a novel approach that instead uses geotagged
panoramas from the Google Street View as a source of global positioning. We
model the problem of localization as a non-linear least squares estimation in
two phases. The first estimates the 3D position of tracked feature points from
short monocular camera sequences. The second computes the rigid body
transformation between the Street View panoramas and the estimated points. The
only input of this approach is a stream of monocular camera images and odometry
estimates. We quantified the accuracy of the method by running the approach on
a robotic platform in a parking lot by using visual fiducials as ground truth.
Additionally, we applied the approach in the context of personal localization
in a real urban scenario by using data from a Google Tango tablet.
Predictive Encoding of Contextual Relationships for Perceptual Inference, Interpolation and Prediction
Authors: Ming-Min Zhao, Chengxu Zhuang, Yizhou Wang, Tai Sing Lee
Date: 2014-11-14 00:00:00

We propose a new neurally-inspired model that can learn to encode the global
relationship context of visual events across time and space and to use the
contextual information to modulate the analysis by synthesis process in a
predictive coding framework. The model learns latent contextual representations
by maximizing the predictability of visual events based on local and global
contextual information through both top-down and bottom-up processes. In
contrast to standard predictive coding models, the prediction error in this
model is used to update the contextual representation but does not alter the
feedforward input for the next layer, and is thus more consistent with
neurophysiological observations. We establish the computational feasibility of
this model by demonstrating its ability in several aspects. We show that our
model can outperform state-of-art performances of gated Boltzmann machines
(GBM) in estimation of contextual information. Our model can also interpolate
missing events or predict future events in image sequences while simultaneously
estimating contextual information. We show it achieves state-of-art
performances in terms of prediction accuracy in a variety of tasks and
possesses the ability to interpolate missing frames, a function that is lacking
in GBM.
Image Specificity
Authors: Mainak Jas, Devi Parikh
Date: 2015-02-16 00:00:00
Proceeding: CVPR 2015 6
Tasks: Image Retrieval, Retrieval, Specificity

For some images, descriptions written by multiple people are consistent with
each other. But for other images, descriptions across people vary considerably.
In other words, some images are specific $-$ they elicit consistent
descriptions from different people $-$ while other images are ambiguous.
Applications involving images and text can benefit from an understanding of
which images are specific and which ones are ambiguous. For instance, consider
text-based image retrieval. If a query description is moderately similar to the
caption (or reference description) of an ambiguous image, that query may be
considered a decent match to the image. But if the image is very specific, a
moderate similarity between the query and the reference description may not be
sufficient to retrieve the image.
  In this paper, we introduce the notion of image specificity. We present two
mechanisms to measure specificity given multiple descriptions of an image: an
automated measure and a measure that relies on human judgement. We analyze
image specificity with respect to image content and properties to better
understand what makes an image specific. We then train models to automatically
predict the specificity of an image from image features alone without requiring
textual descriptions of the image. Finally, we show that modeling image
specificity leads to improvements in a text-based image retrieval application.
Purine: A bi-graph based deep learning framework
Authors: Min Lin, Shuo Li, Xuan Luo, Shuicheng Yan
Date: 2014-12-19 00:00:00
Tasks: Deep Learning

In this paper, we introduce a novel deep learning framework, termed Purine.
In Purine, a deep network is expressed as a bipartite graph (bi-graph), which
is composed of interconnected operators and data tensors. With the bi-graph
abstraction, networks are easily solvable with event-driven task dispatcher. We
then demonstrate that different parallelism schemes over GPUs and/or CPUs on
single or multiple PCs can be universally implemented by graph composition.
This eases researchers from coding for various parallelization schemes, and the
same dispatcher can be used for solving variant graphs. Scheduled by the task
dispatcher, memory transfers are fully overlapped with other computations,
which greatly reduce the communication overhead and help us achieve approximate
linear acceleration.
Genetic algorithm implementation for effective document subject search
Authors: V. K. Ivanov, P. I. Meskin
Date: 2015-04-16 00:00:00
Tasks: General Classification

This paper describes the software implementation of genetic algorithm for
identifying and selecting most relevant results received during sequentially
executed subject search operations. Simulated evolutionary process generates
sustainable and effective population of search queries, forms search pattern of
documents or semantic core, creates relevant sets of required documents, allows
automatic classification of search results. The paper discusses the features of
subject search, justifies the use of a genetic algorithm, describes arguments
of the fitness function and describes basic steps and parameters of the
algorithm.