pwc-archive/papers-with-abstracts huggingface dataset:
Structured Prediction: From Gaussian Perturbations to Linear-Time Principled Algorithms
Authors: Jean Honorio, Tommi Jaakkola
Date: 2015-08-05 00:00:00
Tasks: Decoder, Structured Prediction

Margin-based structured prediction commonly uses a maximum loss over all
possible structured outputs \cite{Altun03,Collins04b,Taskar03}. In natural
language processing, recent work \cite{Zhang14,Zhang15} has proposed the use of
the maximum loss over random structured outputs sampled independently from some
proposal distribution. This method is linear-time in the number of random
structured outputs and trivially parallelizable. We study this family of loss
functions in the PAC-Bayes framework under Gaussian perturbations
\cite{McAllester07}. Under some technical conditions and up to statistical
accuracy, we show that this family of loss functions produces a tighter upper
bound of the Gibbs decoder distortion than commonly used methods. Thus, using
the maximum loss over random structured outputs is a principled way of learning
the parameter of structured prediction models. Besides explaining the
experimental success of \cite{Zhang14,Zhang15}, our theoretical results show
that more general techniques are possible.
Scene Grammars, Factor Graphs, and Belief Propagation
Authors: Jeroen Chua, Pedro F. Felzenszwalb
Date: 2016-06-03 00:00:00

We describe a general framework for probabilistic modeling of complex scenes and inference from ambiguous observations. The approach is motivated by applications in image analysis and is based on the use of priors defined by stochastic grammars. We define a class of grammars that capture relationships between the objects in a scene and provide important contextual cues for statistical inference. The distribution over scenes defined by a probabilistic scene grammar can be represented by a graphical model and this construction can be used for efficient inference with loopy belief propagation. We show experimental results with two different applications. One application involves the reconstruction of binary contour maps. Another application involves detecting and localizing faces in images. In both applications the same framework leads to robust inference algorithms that can effectively combine local information to reason about a scene.
An Attentional Neural Conversation Model with Improved Specificity
Authors: Kaisheng Yao, Baolin Peng, Geoffrey Zweig, Kam-Fai Wong
Date: 2016-06-03 00:00:00
Tasks: Retrieval, Specificity

In this paper we propose a neural conversation model for conducting
dialogues. We demonstrate the use of this model to generate help desk
responses, where users are asking questions about PC applications. Our model is
distinguished by two characteristics. First, it models intention across turns
with a recurrent network, and incorporates an attention model that is
conditioned on the representation of intention. Secondly, it avoids generating
non-specific responses by incorporating an IDF term in the objective function.
The model is evaluated both as a pure generation model in which a help-desk
response is generated from scratch, and as a retrieval model with performance
measured using recall rates of the correct response. Experimental results
indicate that the model outperforms previously proposed neural conversation
architectures, and that using specificity in the objective function
significantly improves performances for both generation and retrieval.
Statistical Pattern Recognition for Driving Styles Based on Bayesian Probability and Kernel Density Estimation
Authors: Wenshuo Wang, Junqiang Xi, Xiaohan Li
Date: 2016-06-03 00:00:00
Tasks: Density Estimation

Driving styles have a great influence on vehicle fuel economy, active safety,
and drivability. To recognize driving styles of path-tracking behaviors for
different divers, a statistical pattern-recognition method is developed to deal
with the uncertainty of driving styles or characteristics based on probability
density estimation. First, to describe driver path-tracking styles, vehicle
speed and throttle opening are selected as the discriminative parameters, and a
conditional kernel density function of vehicle speed and throttle opening is
built, respectively, to describe the uncertainty and probability of two
representative driving styles, e.g., aggressive and normal. Meanwhile, a
posterior probability of each element in feature vector is obtained using full
Bayesian theory. Second, a Euclidean distance method is involved to decide to
which class the driver should be subject instead of calculating the complex
covariance between every two elements of feature vectors. By comparing the
Euclidean distance between every elements in feature vector, driving styles are
classified into seven levels ranging from low normal to high aggressive.
Subsequently, to show benefits of the proposed pattern-recognition method, a
cross-validated method is used, compared with a fuzzy logic-based
pattern-recognition method. The experiment results show that the proposed
statistical pattern-recognition method for driving styles based on kernel
density estimation is more efficient and stable than the fuzzy logic-based
method.
Enhancing the LexVec Distributed Word Representation Model Using Positional Contexts and External Memory
Authors: Alexandre Salle, Marco Idiart, Aline Villavicencio
Date: 2016-06-03 00:00:00
Tasks: Word Similarity

In this paper we take a state-of-the-art model for distributed word
representation that explicitly factorizes the positive pointwise mutual
information (PPMI) matrix using window sampling and negative sampling and
address two of its shortcomings. We improve syntactic performance by using
positional contexts, and solve the need to store the PPMI matrix in memory by
working on aggregate data in external memory. The effectiveness of both
modifications is shown using word similarity and analogy tasks.
Differences between Industrial Models of Autonomy and Systemic Models of Autonomy
Authors: Aleksander Lodwich
Date: 2016-05-24 00:00:00

This paper discusses the idea of levels of autonomy of systems - be this
technical or organic - and compares the insights with models employed by
industries used to describe maturity and capability of their products.
Characterizing Diseases from Unstructured Text: A Vocabulary Driven Word2vec Approach
Authors: Saurav Ghosh, Prithwish Chakraborty, Emily Cohn, John S. Brownstein, Naren Ramakrishnan
Date: 2016-03-01 00:00:00
Tasks: Word Embeddings

Traditional disease surveillance can be augmented with a wide variety of
real-time sources such as, news and social media. However, these sources are in
general unstructured and, construction of surveillance tools such as
taxonomical correlations and trace mapping involves considerable human
supervision. In this paper, we motivate a disease vocabulary driven word2vec
model (Dis2Vec) to model diseases and constituent attributes as word embeddings
from the HealthMap news corpus. We use these word embeddings to automatically
create disease taxonomies and evaluate our model against corresponding human
annotated taxonomies. We compare our model accuracies against several
state-of-the art word2vec methods. Our results demonstrate that Dis2Vec
outperforms traditional distributed vector representations in its ability to
faithfully capture taxonomical attributes across different class of diseases
such as endemic, emerging and rare.
End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning
Authors: Jason D. Williams, Geoffrey Zweig
Date: 2016-06-03 00:00:00
Tasks: Feature Engineering, Reinforcement Learning, Reinforcement Learning (RL)

This paper presents a model for end-to-end learning of task-oriented dialog
systems. The main component of the model is a recurrent neural network (an
LSTM), which maps from raw dialog history directly to a distribution over
system actions. The LSTM automatically infers a representation of dialog
history, which relieves the system developer of much of the manual feature
engineering of dialog state. In addition, the developer can provide software
that expresses business rules and provides access to programmatic APIs,
enabling the LSTM to take actions in the real world on behalf of the user. The
LSTM can be optimized using supervised learning (SL), where a domain expert
provides example dialogs which the LSTM should imitate; or using reinforcement
learning (RL), where the system improves by interacting directly with end
users. Experiments show that SL and RL are complementary: SL alone can derive a
reasonable initial policy from a small number of training dialogs; and starting
RL optimization with a policy trained with SL substantially accelerates the
learning rate of RL.
Minimizing Regret on Reflexive Banach Spaces and Learning Nash Equilibria in Continuous Zero-Sum Games
Authors: Maximilian Balandat, Walid Krichene, Claire Tomlin, Alexandre Bayen
Date: 2016-06-03 00:00:00

We study a general version of the adversarial online learning problem. We are
given a decision set $\mathcal{X}$ in a reflexive Banach space $X$ and a
sequence of reward vectors in the dual space of $X$. At each iteration, we
choose an action from $\mathcal{X}$, based on the observed sequence of previous
rewards. Our goal is to minimize regret, defined as the gap between the
realized reward and the reward of the best fixed action in hindsight. Using
results from infinite dimensional convex analysis, we generalize the method of
Dual Averaging (or Follow the Regularized Leader) to our setting and obtain
general upper bounds on the worst-case regret that subsume a wide range of
results from the literature. Under the assumption of uniformly continuous
rewards, we obtain explicit anytime regret bounds in a setting where the
decision set is the set of probability distributions on a compact metric space
$S$ whose Radon-Nikodym derivatives are elements of $L^p(S)$ for some $p > 1$.
Importantly, we make no convexity assumptions on either the set $S$ or the
reward functions. We also prove a general lower bound on the worst-case regret
for any online algorithm. We then apply these results to the problem of
learning in repeated continuous two-player zero-sum games, in which players'
strategy sets are compact metric spaces. In doing so, we first prove that if
both players play a Hannan-consistent strategy, then with probability 1 the
empirical distributions of play weakly converge to the set of Nash equilibria
of the game. We then show that, under mild assumptions, Dual Averaging on the
(infinite-dimensional) space of probability distributions indeed achieves
Hannan-consistency. Finally, we illustrate our results through numerical
examples.
Grid-like structure is optimal for path integration
Authors: Reza Moazzezi
Date: 2016-06-03 00:00:00

Grid cells in medial entorhinal cortex are believed to play a key role in
path integration. However, the relation between path integration and the
grid-like arrangement of their firing field remains unclear. We provide
theoretical evidence that grid-like structure and path integration are closely
related. In one dimension, the grid-like structure provides the optimal
solution for path integration assuming that the noise correlation structure is
Gaussian. In two dimensions, assuming that the noise is Gaussian, rectangular
grid-like structure is the optimal solution provided that 1- both noise
correlation and receptive field structures of the neurons can be
multiplicatively decomposed into orthogonal components and 2- the eigenvalues
of the decomposed correlation matrices decrease faster than the square of the
frequency of the corresponding eigenvectors. We will also address the decoding
mechanism and show that the problem of decoding reduces to the problem of
extracting task relevant information in the presence of task irrelevant
information. Change-based Population Coding provides the optimal solution for
this problem.
Learning Stylometric Representations for Authorship Analysis
Authors: Steven H. H. Ding, Benjamin C. M. Fung, Farkhund Iqbal, William K. Cheung
Date: 2016-06-03 00:00:00
Tasks: Authorship Verification, Feature Engineering, Sentence

Authorship analysis (AA) is the study of unveiling the hidden properties of
authors from a body of exponentially exploding textual data. It extracts an
author's identity and sociolinguistic characteristics based on the reflected
writing styles in the text. It is an essential process for various areas, such
as cybercrime investigation, psycholinguistics, political socialization, etc.
However, most of the previous techniques critically depend on the manual
feature engineering process. Consequently, the choice of feature set has been
shown to be scenario- or dataset-dependent. In this paper, to mimic the human
sentence composition process using a neural network approach, we propose to
incorporate different categories of linguistic features into distributed
representation of words in order to learn simultaneously the writing style
representations based on unlabeled texts for authorship analysis. In
particular, the proposed models allow topical, lexical, syntactical, and
character-level feature vectors of each document to be extracted as
stylometrics. We evaluate the performance of our approach on the problems of
authorship characterization and authorship verification with the Twitter,
novel, and essay datasets. The experiments suggest that our proposed text
representation outperforms the bag-of-lexical-n-grams, Latent Dirichlet
Allocation, Latent Semantic Analysis, PVDM, PVDBOW, and word2vec
representations.
Distributed stochastic optimization via matrix exponential learning
Authors: Panayotis Mertikopoulos, E. Veronica Belmega, Romain Negrel, Luca Sanguinetti
Date: 2016-06-03 00:00:00
Tasks: Stochastic Optimization, valid

In this paper, we investigate a distributed learning scheme for a broad class
of stochastic optimization problems and games that arise in signal processing
and wireless communications. The proposed algorithm relies on the method of
matrix exponential learning (MXL) and only requires locally computable gradient
observations that are possibly imperfect and/or obsolete. To analyze it, we
introduce the notion of a stable Nash equilibrium and we show that the
algorithm is globally convergent to such equilibria - or locally convergent
when an equilibrium is only locally stable. We also derive an explicit linear
bound for the algorithm's convergence speed, which remains valid under
measurement errors and uncertainty of arbitrarily high variance. To validate
our theoretical analysis, we test the algorithm in realistic
multi-carrier/multiple-antenna wireless scenarios where several users seek to
maximize their energy efficiency. Our results show that learning allows users
to attain a net increase between 100% and 500% in energy efficiency, even under
very high uncertainty.
Reinforcement Learning for Semantic Segmentation in Indoor Scenes
Authors: Md. Alimoor Reza, Jana Kosecka
Date: 2016-06-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Segmentation, Semantic Segmentation

Future advancements in robot autonomy and sophistication of robotics tasks
rest on robust, efficient, and task-dependent semantic understanding of the
environment. Semantic segmentation is the problem of simultaneous segmentation
and categorization of a partition of sensory data. The majority of current
approaches tackle this using multi-class segmentation and labeling in a
Conditional Random Field (CRF) framework or by generating multiple object
hypotheses and combining them sequentially. In practical settings, the subset
of semantic labels that are needed depend on the task and particular scene and
labelling every single pixel is not always necessary. We pursue these
observations in developing a more modular and flexible approach to multi-class
parsing of RGBD data based on learning strategies for combining independent
binary object-vs-background segmentations in place of the usual monolithic
multi-label CRF approach. Parameters for the independent binary segmentation
models can be learned very efficiently, and the combination strategy---learned
using reinforcement learning---can be set independently and can vary over
different tasks and environments. Accuracy is comparable to state-of-art
methods on a subset of the NYU-V2 dataset of indoor scenes, while providing
additional flexibility and modularity.
How Deep is the Feature Analysis underlying Rapid Visual Categorization?
Authors: Sven Eberhardt, Jonah Cader, Thomas Serre
Date: 2016-06-03 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Object Recognition

Rapid categorization paradigms have a long history in experimental
psychology: Characterized by short presentation times and speedy behavioral
responses, these tasks highlight the efficiency with which our visual system
processes natural object categories. Previous studies have shown that
feed-forward hierarchical models of the visual cortex provide a good fit to
human visual decisions. At the same time, recent work in computer vision has
demonstrated significant gains in object recognition accuracy with increasingly
deep hierarchical architectures. But it is unclear how well these models
account for human visual decisions and what they may reveal about the
underlying brain processes.
  We have conducted a large-scale psychophysics study to assess the correlation
between computational models and human participants on a rapid animal vs.
non-animal categorization task. We considered visual representations of varying
complexity by analyzing the output of different stages of processing in three
state-of-the-art deep networks. We found that recognition accuracy increases
with higher stages of visual processing (higher level stages indeed
outperforming human participants on the same task) but that human decisions
agree best with predictions from intermediate stages.
  Overall, these results suggest that human participants may rely on visual
features of intermediate complexity and that the complexity of visual
representations afforded by modern deep network models may exceed those used by
human participants during rapid categorization.
Density Evolution in the Degree-correlated Stochastic Block Model
Authors: Elchanan Mossel, Jiaming Xu
Date: 2015-09-10 00:00:00
Tasks: Stochastic Block Model

There is a recent surge of interest in identifying the sharp recovery
thresholds for cluster recovery under the stochastic block model. In this
paper, we address the more refined question of how many vertices that will be
misclassified on average. We consider the binary form of the stochastic block
model, where $n$ vertices are partitioned into two clusters with edge
probability $a/n$ within the first cluster, $c/n$ within the second cluster,
and $b/n$ across clusters. Suppose that as $n \to \infty$, $a= b+ \mu \sqrt{ b}
$, $c=b+ \nu \sqrt{ b} $ for two fixed constants $\mu, \nu$, and $b \to \infty$
with $b=n^{o(1)}$. When the cluster sizes are balanced and $\mu \neq \nu$, we
show that the minimum fraction of misclassified vertices on average is given by
$Q(\sqrt{v^*})$, where $Q(x)$ is the Q-function for standard normal, $v^*$ is
the unique fixed point of $v= \frac{(\mu-\nu)^2}{16} + \frac{ (\mu+\nu)^2 }{16}
\mathbb{E}[ \tanh(v+ \sqrt{v} Z)],$ and $Z$ is standard normal. Moreover, the
minimum misclassified fraction on average is attained by a local algorithm,
namely belief propagation, in time linear in the number of edges. Our proof
techniques are based on connecting the cluster recovery problem to tree
reconstruction problems, and analyzing the density evolution of belief
propagation on trees with Gaussian approximations.
Combining Multiple Clusterings via Crowd Agreement Estimation and Multi-Granularity Link Analysis
Authors: Dong Huang, Jian-Huang Lai, Chang-Dong Wang
Date: 2014-05-06 00:00:00
Tasks: Clustering, Clustering Ensemble, graph partitioning

The clustering ensemble technique aims to combine multiple clusterings into a
probably better and more robust clustering and has been receiving an increasing
attention in recent years. There are mainly two aspects of limitations in the
existing clustering ensemble approaches. Firstly, many approaches lack the
ability to weight the base clusterings without access to the original data and
can be affected significantly by the low-quality, or even ill clusterings.
Secondly, they generally focus on the instance level or cluster level in the
ensemble system and fail to integrate multi-granularity cues into a unified
model. To address these two limitations, this paper proposes to solve the
clustering ensemble problem via crowd agreement estimation and
multi-granularity link analysis. We present the normalized crowd agreement
index (NCAI) to evaluate the quality of base clusterings in an unsupervised
manner and thus weight the base clusterings in accordance with their clustering
validity. To explore the relationship between clusters, the source aware
connected triple (SACT) similarity is introduced with regard to their common
neighbors and the source reliability. Based on NCAI and multi-granularity
information collected among base clusterings, clusters, and data instances, we
further propose two novel consensus functions, termed weighted evidence
accumulation clustering (WEAC) and graph partitioning with multi-granularity
link analysis (GP-MGLA) respectively. The experiments are conducted on eight
real-world datasets. The experimental results demonstrate the effectiveness and
robustness of the proposed methods.
Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning
Authors: Jiang Guo, Wanxiang Che, Haifeng Wang, Ting Liu
Date: 2016-06-03 00:00:00
Tasks: Dependency Parsing, Multi-Task Learning

Various treebanks have been released for dependency parsing. Despite that
treebanks may belong to different languages or have different annotation
schemes, they contain syntactic knowledge that is potential to benefit each
other. This paper presents an universal framework for exploiting these
multi-typed treebanks to improve parsing with deep multi-task learning. We
consider two kinds of treebanks as source: the multilingual universal treebanks
and the monolingual heterogeneous treebanks. Multiple treebanks are trained
jointly and interacted with multi-level parameter sharing. Experiments on
several benchmark datasets in various languages demonstrate that our approach
can make effective use of arbitrary source treebanks to improve target parsing
models.
Robust Ensemble Clustering Using Probability Trajectories
Authors: Dong Huang, Jian-Huang Lai, Chang-Dong Wang
Date: 2016-06-03 00:00:00
Tasks: Clustering

Although many successful ensemble clustering approaches have been developed
in recent years, there are still two limitations to most of the existing
approaches. First, they mostly overlook the issue of uncertain links, which may
mislead the overall consensus process. Second, they generally lack the ability
to incorporate global information to refine the local links. To address these
two limitations, in this paper, we propose a novel ensemble clustering approach
based on sparse graph representation and probability trajectory analysis. In
particular, we present the elite neighbor selection strategy to identify the
uncertain links by locally adaptive thresholds and build a sparse graph with a
small number of probably reliable links. We argue that a small number of
probably reliable links can lead to significantly better consensus results than
using all graph links regardless of their reliability. The random walk process
driven by a new transition probability matrix is utilized to explore the global
information in the graph. We derive a novel and dense similarity measure from
the sparse graph by analyzing the probability trajectories of the random
walkers, based on which two consensus functions are further proposed.
Experimental results on multiple real-world datasets demonstrate the
effectiveness and efficiency of our approach.