pwc-archive/papers-with-abstracts huggingface dataset:
Feature ranking for multi-label classification using Markov Networks
Authors: Paweł Teisseyre
Date: 2016-02-24 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We propose a simple and efficient method for ranking features in multi-label
classification. The method produces a ranking of features showing their
relevance in predicting labels, which in turn allows to choose a final subset
of features. The procedure is based on Markov Networks and allows to model the
dependencies between labels and features in a direct way. In the first step we
build a simple network using only labels and then we test how much adding a
single feature affects the initial network. More specifically, in the first
step we use the Ising model whereas the second step is based on the score
statistic, which allows to test a significance of added features very quickly.
The proposed approach does not require transformation of label space, gives
interpretable results and allows for attractive visualization of dependency
structure. We give a theoretical justification of the procedure by discussing
some theoretical properties of the Ising model and the score statistic. We also
discuss feature ranking procedure based on fitting Ising model using $l_1$
regularized logistic regressions. Numerical experiments show that the proposed
methods outperform the conventional approaches on the considered artificial and
real datasets.
Hand-Object Interaction and Precise Localization in Transitive Action Recognition
Authors: Amir Rosenfeld, Shimon Ullman
Date: 2015-11-12 00:00:00
Tasks: Action Recognition, Action Recognition In Still Images, Object, Object Recognition, Pose Estimation, Semantic Segmentation, Temporal Action Localization

Action recognition in still images has seen major improvement in recent years
due to advances in human pose estimation, object recognition and stronger
feature representations produced by deep neural networks. However, there are
still many cases in which performance remains far from that of humans. A major
difficulty arises in distinguishing between transitive actions in which the
overall actor pose is similar, and recognition therefore depends on details of
the grasp and the object, which may be largely occluded. In this paper we
demonstrate how recognition is improved by obtaining precise localization of
the action-object and consequently extracting details of the object shape
together with the actor-object interaction. To obtain exact localization of the
action object and its interaction with the actor, we employ a coarse-to-fine
approach which combines semantic segmentation and contextual features, in
successive stages. We focus on (but are not limited) to face-related actions, a
set of actions that includes several currently challenging categories. We
present an average relative improvement of 35% over state-of-the art and
validate through experimentation the effectiveness of our approach.
Parametric Prediction from Parametric Agents
Authors: Yuan Luo, Nihar B. Shah, Jianwei Huang, Jean Walrand
Date: 2016-02-24 00:00:00
Tasks: Learning Theory, Prediction

We consider a problem of prediction based on opinions elicited from
heterogeneous rational agents with private information. Making an accurate
prediction with a minimal cost requires a joint design of the incentive
mechanism and the prediction algorithm. Such a problem lies at the nexus of
statistical learning theory and game theory, and arises in many domains such as
consumer surveys and mobile crowdsourcing. In order to elicit heterogeneous
agents' private information and incentivize agents with different capabilities
to act in the principal's best interest, we design an optimal joint incentive
mechanism and prediction algorithm called COPE (COst and Prediction
Elicitation), the analysis of which offers several valuable engineering
insights. First, when the costs incurred by the agents are linear in the
exerted effort, COPE corresponds to a "crowd contending" mechanism, where the
principal only employs the agent with the highest capability. Second, when the
costs are quadratic, COPE corresponds to a "crowd-sourcing" mechanism that
employs multiple agents with different capabilities at the same time. Numerical
simulations show that COPE improves the principal's profit and the network
profit significantly (larger than 30% in our simulations), comparing to those
mechanisms that assume all agents have equal capabilities.
Max-Margin Nonparametric Latent Feature Models for Link Prediction
Authors: Jun Zhu, Jiaming Song, Bei Chen
Date: 2016-02-24 00:00:00
Tasks: Link Prediction, Prediction, Variational Inference

Link prediction is a fundamental task in statistical network analysis. Recent
advances have been made on learning flexible nonparametric Bayesian latent
feature models for link prediction. In this paper, we present a max-margin
learning method for such nonparametric latent feature relational models. Our
approach attempts to unite the ideas of max-margin learning and Bayesian
nonparametrics to discover discriminative latent features for link prediction.
It inherits the advances of nonparametric Bayesian methods to infer the unknown
latent social dimension, while for discriminative link prediction, it adopts
the max-margin learning principle by minimizing a hinge-loss using the linear
expectation operator, without dealing with a highly nonlinear link likelihood
function. For posterior inference, we develop an efficient stochastic
variational inference algorithm under a truncated mean-field assumption. Our
methods can scale up to large-scale real networks with millions of entities and
tens of millions of positive links. We also provide a full Bayesian
formulation, which can avoid tuning regularization hyper-parameters.
Experimental results on a diverse range of real datasets demonstrate the
benefits inherited from max-margin learning and Bayesian nonparametric
inference.
Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression
Authors: Aymeric Dieuleveut, Nicolas Flammarion, Francis Bach
Date: 2016-02-17 00:00:00
Tasks: regression

We consider the optimization of a quadratic objective function whose
gradients are only accessible through a stochastic oracle that returns the
gradient at any given point plus a zero-mean finite variance random error. We
present the first algorithm that achieves jointly the optimal prediction error
rates for least-squares regression, both in terms of forgetting of initial
conditions in O(1/n 2), and in terms of dependence on the noise and dimension d
of the problem, as O(d/n). Our new algorithm is based on averaged accelerated
regularized gradient descent, and may also be analyzed through finer
assumptions on initial conditions and the Hessian matrix, leading to
dimension-free quantities that may still be small while the " optimal " terms
above are large. In order to characterize the tightness of these new bounds, we
consider an application to non-parametric regression and use the known lower
bounds on the statistical performance (without computational limits), which
happen to match our bounds obtained from a single pass on the data and thus
show optimality of our algorithm in a wide variety of particular trade-offs
between bias and variance.
Improved Accent Classification Combining Phonetic Vowels with Acoustic Features
Authors: Zhenhao Ge
Date: 2016-02-24 00:00:00
Tasks: Classification, General Classification

Researches have shown accent classification can be improved by integrating
semantic information into pure acoustic approach. In this work, we combine
phonetic knowledge, such as vowels, with enhanced acoustic features to build an
improved accent classification system. The classifier is based on Gaussian
Mixture Model-Universal Background Model (GMM-UBM), with normalized Perceptual
Linear Predictive (PLP) features. The features are further optimized by
Principle Component Analysis (PCA) and Hetroscedastic Linear Discriminant
Analysis (HLDA). Using 7 major types of accented speech from the Foreign
Accented English (FAE) corpus, the system achieves classification accuracy 54%
with input test data as short as 20 seconds, which is competitive to the state
of the art in this field.
Domain Specific Author Attribution Based on Feedforward Neural Network Language Models
Authors: Zhenhao Ge, Yufang Sun
Date: 2016-02-24 00:00:00
Tasks: Author Attribution, Authorship Attribution, Language Modeling, Language Modelling

Authorship attribution refers to the task of automatically determining the
author based on a given sample of text. It is a problem with a long history and
has a wide range of application. Building author profiles using language models
is one of the most successful methods to automate this task. New language
modeling methods based on neural networks alleviate the curse of dimensionality
and usually outperform conventional N-gram methods. However, there have not
been much research applying them to authorship attribution. In this paper, we
present a novel setup of a Neural Network Language Model (NNLM) and apply it to
a database of text samples from different authors. We investigate how the NNLM
performs on a task with moderate author set size and relatively limited
training and test data, and how the topics of the text samples affect the
accuracy. NNLM achieves nearly 2.5% reduction in perplexity, a measurement of
fitness of a trained language model to the test data. Given 5 random test
sentences, it also increases the author classification accuracy by 3.43% on
average, compared with the N-gram methods using SRILM tools. An open source
implementation of our methodology is freely available at
https://github.com/zge/authorship-attribution/.
The Myopia of Crowds: A Study of Collective Evaluation on Stack Exchange
Authors: Keith Burghardt, Emanuel F. Alsina, Michelle Girvan, William Rand, Kristina Lerman
Date: 2016-02-24 00:00:00
Tasks: Question Answering

Crowds can often make better decisions than individuals or small groups of
experts by leveraging their ability to aggregate diverse information. Question
answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect
to identify the best answers to questions asked by users. We analyze data from
250 communities on the Stack Exchange network to pinpoint factors affecting
which answers are chosen as the best answers. Our results suggest that, rather
than evaluate all available answers to a question, users rely on simple
cognitive heuristics to choose an answer to vote for or accept. These cognitive
heuristics are linked to an answer's salience, such as the order in which it is
listed and how much screen space it occupies. While askers appear to depend
more on heuristics, compared to voting users, when choosing an answer to accept
as the most helpful one, voters use acceptance itself as a heuristic: they are
more likely to choose the answer after it is accepted than before that very
same answer was accepted. These heuristics become more important in explaining
and predicting behavior as the number of available answers increases. Our
findings suggest that crowd judgments may become less reliable as the number of
answers grow.
Automatic Moth Detection from Trap Images for Pest Management
Authors: Weiguang Ding, Graham Taylor
Date: 2016-02-24 00:00:00
Tasks: Management

Monitoring the number of insect pests is a crucial component in
pheromone-based pest management systems. In this paper, we propose an automatic
detection pipeline based on deep learning for identifying and counting pests in
images taken inside field traps. Applied to a commercial codling moth dataset,
our method shows promising performance both qualitatively and quantitatively.
Compared to previous attempts at pest detection, our approach uses no
pest-specific engineering which enables it to adapt to other species and
environments with minimal human effort. It is amenable to implementation on
parallel hardware and therefore capable of deployment in settings where
real-time performance is required.
Accent Classification with Phonetic Vowel Representation
Authors: Zhenhao Ge, Yingyi Tan, Aravind Ganapathiraju
Date: 2016-02-24 00:00:00
Tasks: Classification, General Classification

Previous accent classification research focused mainly on detecting accents
with pure acoustic information without recognizing accented speech. This work
combines phonetic knowledge such as vowels with acoustic information to build
Guassian Mixture Model (GMM) classifier with Perceptual Linear Predictive (PLP)
features, optimized by Hetroscedastic Linear Discriminant Analysis (HLDA). With
input about 20-second accented speech, this system achieves classification rate
of 51% on a 7-way classification system focusing on the major types of accents
in English, which is competitive to the state-of-the-art results in this field.
On Study of the Binarized Deep Neural Network for Image Classification
Authors: Song Wang, Dongchun Ren, Li Chen, Wei Fan, Jun Sun, Satoshi Naoi
Date: 2016-02-24 00:00:00
Tasks: General Classification, image-classification, Image Classification

Recently, the deep neural network (derived from the artificial neural
network) has attracted many researchers' attention by its outstanding
performance. However, since this network requires high-performance GPUs and
large storage, it is very hard to use it on individual devices. In order to
improve the deep neural network, many trials have been made by refining the
network structure or training strategy. Unlike those trials, in this paper, we
focused on the basic propagation function of the artificial neural network and
proposed the binarized deep neural network. This network is a pure binary
system, in which all the values and calculations are binarized. As a result,
our network can save a lot of computational resource and storage. Therefore, it
is possible to use it on various devices. Moreover, the experimental results
proved the feasibility of the proposed network.
The Possibilities and Limitations of Private Prediction Markets
Authors: Rachel Cummings, David M. Pennock, Jennifer Wortman Vaughan
Date: 2016-02-24 00:00:00
Tasks: Prediction

We consider the design of private prediction markets, financial markets
designed to elicit predictions about uncertain events without revealing too
much information about market participants' actions or beliefs. Our goal is to
design market mechanisms in which participants' trades or wagers influence the
market's behavior in a way that leads to accurate predictions, yet no single
participant has too much influence over what others are able to observe. We
study the possibilities and limitations of such mechanisms using tools from
differential privacy. We begin by designing a private one-shot wagering
mechanism in which bettors specify a belief about the likelihood of a future
event and a corresponding monetary wager. Wagers are redistributed among
bettors in a way that more highly rewards those with accurate predictions. We
provide a class of wagering mechanisms that are guaranteed to satisfy
truthfulness, budget balance in expectation, and other desirable properties
while additionally guaranteeing epsilon-joint differential privacy in the
bettors' reported beliefs, and analyze the trade-off between the achievable
level of privacy and the sensitivity of a bettor's payment to her own report.
We then ask whether it is possible to obtain privacy in dynamic prediction
markets, focusing our attention on the popular cost-function framework in which
securities with payments linked to future events are bought and sold by an
automated market maker. We show that under general conditions, it is impossible
for such a market maker to simultaneously achieve bounded worst-case loss and
epsilon-differential privacy without allowing the privacy guarantee to degrade
extremely quickly as the number of trades grows, making such markets
impractical in settings in which privacy is valued. We conclude by suggesting
several avenues for potentially circumventing this lower bound.
New Results on Equilibria in Strategic Candidacy
Authors: Jérôme Lang, Nicolas Maudet, Maria Polukarov, Alice Cohen-Hadria
Date: 2013-06-07 00:00:00

We consider a voting setting where candidates have preferences about the
outcome of the election and are free to join or leave the election. The
corresponding candidacy game, where candidates choose strategically to
participate or not, has been studied %initially by Dutta et al., who showed
that no non-dictatorial voting procedure satisfying unanimity is
candidacy-strategyproof, that is, is such that the joint action where all
candidates enter the election is always a pure strategy Nash equilibrium. Dutta
et al. also showed that for some voting tree procedures, there are candidacy
games with no pure Nash equilibria, and that for the rule that outputs the
sophisticated winner of voting by successive elimination, all games have a pure
Nash equilibrium. No results were known about other voting rules. Here we prove
several such results. For four candidates, the message is, roughly, that most
scoring rules (with the exception of Borda) do not guarantee the existence of a
pure Nash equilibrium but that Condorcet-consistent rules, for an odd number of
voters, do. For five candidates, most rules we study no longer have this
guarantee. Finally, we identify one prominent rule that guarantees the
existence of a pure Nash equilibrium for any number of candidates (and for an
odd number of voters): the Copeland rule. We also show that under mild
assumptions on the voting rule, the existence of strong equilibria cannot be
guaranteed.
adaQN: An Adaptive Quasi-Newton Algorithm for Training RNNs
Authors: Nitish Shirish Keskar, Albert S. Berahas
Date: 2015-11-04 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent Neural Networks (RNNs) are powerful models that achieve exceptional
performance on several pattern recognition problems. However, the training of
RNNs is a computationally difficult task owing to the well-known
"vanishing/exploding" gradient problem. Algorithms proposed for training RNNs
either exploit no (or limited) curvature information and have cheap
per-iteration complexity, or attempt to gain significant curvature information
at the cost of increased per-iteration cost. The former set includes
diagonally-scaled first-order methods such as ADAGRAD and ADAM, while the
latter consists of second-order algorithms like Hessian-Free Newton and K-FAC.
In this paper, we present adaQN, a stochastic quasi-Newton algorithm for
training RNNs. Our approach retains a low per-iteration cost while allowing for
non-diagonal scaling through a stochastic L-BFGS updating scheme. The method
uses a novel L-BFGS scaling initialization scheme and is judicious in storing
and retaining L-BFGS curvature pairs. We present numerical experiments on two
language modeling tasks and show that adaQN is competitive with popular RNN
training algorithms.
Order Matters: Sequence to sequence for sets
Authors: Oriol Vinyals, Samy Bengio, Manjunath Kudlur
Date: 2015-11-19 00:00:00
Tasks: Language Modeling

Sequences have become first class citizens in supervised learning thanks to
the resurgence of recurrent neural networks. Many complex tasks that require
mapping from or to a sequence of observations can now be formulated with the
sequence-to-sequence (seq2seq) framework which employs the chain rule to
efficiently represent the joint probability of sequences. In many cases,
however, variable sized inputs and/or outputs might not be naturally expressed
as sequences. For instance, it is not clear how to input a set of numbers into
a model where the task is to sort them; similarly, we do not know how to
organize outputs when they correspond to random variables and the task is to
model their unknown joint probability. In this paper, we first show using
various examples that the order in which we organize input and/or output data
matters significantly when learning an underlying model. We then discuss an
extension of the seq2seq framework that goes beyond sequences and handles input
sets in a principled way. In addition, we propose a loss which, by searching
over possible orders during training, deals with the lack of structure of
output sets. We show empirical evidence of our claims regarding ordering, and
on the modifications to the seq2seq framework on benchmark language modeling
and parsing tasks, as well as two artificial tasks -- sorting numbers and
estimating the joint probability of unknown graphical models.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Authors: Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, Fei-Fei Li
Date: 2016-02-23 00:00:00
Tasks: image-classification, Image Classification, Image Description, Question Answering

Despite progress in perceptual tasks such as image classification, computers
still perform poorly on cognitive tasks such as image description and question
answering. Cognition is core to tasks that involve not just recognizing, but
reasoning about our visual world. However, models used to tackle the rich
content in images for cognitive tasks are still being trained using the same
datasets designed for perceptual tasks. To achieve success at cognitive tasks,
models need to understand the interactions and relationships between objects in
an image. When asked "What vehicle is the person riding?", computers will need
to identify the objects in an image as well as the relationships riding(man,
carriage) and pulling(horse, carriage) in order to answer correctly that "the
person is riding a horse-drawn carriage".
  In this paper, we present the Visual Genome dataset to enable the modeling of
such relationships. We collect dense annotations of objects, attributes, and
relationships within each image to learn these models. Specifically, our
dataset contains over 100K images where each image has an average of 21
objects, 18 attributes, and 18 pairwise relationships between objects. We
canonicalize the objects, attributes, relationships, and noun phrases in region
descriptions and questions answer pairs to WordNet synsets. Together, these
annotations represent the densest and largest dataset of image descriptions,
objects, attributes, relationships, and question answers.
Stuck in a What? Adventures in Weight Space
Authors: Zachary C. Lipton
Date: 2016-02-23 00:00:00

Deep learning researchers commonly suggest that converged models are stuck in
local minima. More recently, some researchers observed that under reasonable
assumptions, the vast majority of critical points are saddle points, not true
minima. Both descriptions suggest that weights converge around a point in
weight space, be it a local optima or merely a critical point. However, it's
possible that neither interpretation is accurate. As neural networks are
typically over-complete, it's easy to show the existence of vast continuous
regions through weight space with equal loss. In this paper, we build on recent
work empirically characterizing the error surfaces of neural networks. We
analyze training paths through weight space, presenting evidence that apparent
convergence of loss does not correspond to weights arriving at critical points,
but instead to large movements through flat regions of weight space. While it's
trivial to show that neural network error surfaces are globally non-convex, we
show that error surfaces are also locally non-convex, even after breaking
symmetry with a random initialization and also after partial training.