pwc-archive/papers-with-abstracts huggingface dataset:
A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution
Authors: Shaohua Li, Jun Zhu, Chunyan Miao
Date: 2015-08-16 00:00:00
Proceeding: EMNLP 2015 9

Most existing word embedding methods can be categorized into Neural Embedding
Models and Matrix Factorization (MF)-based methods. However some models are
opaque to probabilistic interpretation, and MF-based methods, typically solved
using Singular Value Decomposition (SVD), may incur loss of corpus information.
In addition, it is desirable to incorporate global latent factors, such as
topics, sentiments or writing styles, into the word embedding model. Since
generative models provide a principled way to incorporate latent factors, we
propose a generative word embedding model, which is easy to interpret, and can
serve as a basis of more sophisticated latent factor models. The model
inference reduces to a low rank weighted positive semidefinite approximation
problem. Its optimization is approached by eigendecomposition on a submatrix,
followed by online blockwise regression, which is scalable and avoids the
information loss in SVD. In experiments on 7 common benchmark datasets, our
vectors are competitive to word2vec, and better than other MF-based methods.
From Observational Studies to Causal Rule Mining
Authors: Jiuyong Li, Thuc Duy Le, Lin Liu, Jixue Liu, Zhou Jin, Bingyu Sun, Saisai Ma
Date: 2015-08-16 00:00:00
Tasks: Causal Discovery

Randomised controlled trials (RCTs) are the most effective approach to causal
discovery, but in many circumstances it is impossible to conduct RCTs.
Therefore observational studies based on passively observed data are widely
accepted as an alternative to RCTs. However, in observational studies, prior
knowledge is required to generate the hypotheses about the cause-effect
relationships to be tested, hence they can only be applied to problems with
available domain knowledge and a handful of variables. In practice, many data
sets are of high dimensionality, which leaves observational studies out of the
opportunities for causal discovery from such a wealth of data sources. In
another direction, many efficient data mining methods have been developed to
identify associations among variables in large data sets. The problem is,
causal relationships imply associations, but the reverse is not always true.
However we can see the synergy between the two paradigms here. Specifically,
association rule mining can be used to deal with the high-dimensionality
problem while observational studies can be utilised to eliminate non-causal
associations. In this paper we propose the concept of causal rules (CRs) and
develop an algorithm for mining CRs in large data sets. We use the idea of
retrospective cohort studies to detect CRs based on the results of association
rule mining. Experiments with both synthetic and real world data sets have
demonstrated the effectiveness and efficiency of CR mining. In comparison with
the commonly used causal discovery methods, the proposed approach in general is
faster and has better or competitive performance in finding correct or sensible
causes. It is also capable of finding a cause consisting of multiple variables,
a feature that other causal discovery methods do not possess.
Causal Decision Trees
Authors: Jiuyong Li, Saisai Ma, Thuc Duy Le, Lin Liu, Jixue Liu
Date: 2015-08-16 00:00:00
Tasks: Causal Discovery, Causal Inference, Classification, General Classification

Uncovering causal relationships in data is a major objective of data
analytics. Causal relationships are normally discovered with designed
experiments, e.g. randomised controlled trials, which, however are expensive or
infeasible to be conducted in many cases. Causal relationships can also be
found using some well designed observational studies, but they require domain
experts' knowledge and the process is normally time consuming. Hence there is a
need for scalable and automated methods for causal relationship exploration in
data. Classification methods are fast and they could be practical substitutes
for finding causal signals in data. However, classification methods are not
designed for causal discovery and a classification method may find false causal
signals and miss the true ones. In this paper, we develop a causal decision
tree where nodes have causal interpretations. Our method follows a well
established causal inference framework and makes use of a classic statistical
test. The method is practical for finding causal signals in large data sets.
Beat-Event Detection in Action Movie Franchises
Authors: Danila Potapov, Matthijs Douze, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Date: 2015-08-15 00:00:00
Tasks: Classification, Event Detection, General Classification

While important advances were recently made towards temporally localizing and
recognizing specific human actions or activities in videos, efficient detection
and classification of long video chunks belonging to semantically defined
categories such as "pursuit" or "romance" remains challenging.We introduce a
new dataset, Action Movie Franchises, consisting of a collection of Hollywood
action movie franchises. We define 11 non-exclusive semantic categories -
called beat-categories - that are broad enough to cover most of the movie
footage. The corresponding beat-events are annotated as groups of video shots,
possibly overlapping.We propose an approach for localizing beat-events based on
classifying shots into beat-categories and learning the temporal constraints
between shots. We show that temporal constraints significantly improve the
classification performance. We set up an evaluation protocol for beat-event
localization as well as for shot classification, depending on whether movies
from the same franchise are present or not in the training data.
Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network
Authors: Mark D. McDonnell, Tony Vladusich
Date: 2015-03-16 00:00:00
Tasks: General Classification, image-classification, Image Classification

We present a neural network architecture and training method designed to
enable very rapid training and low implementation complexity. Due to its
training speed and very few tunable parameters, the method has strong potential
for applications requiring frequent retraining or online training. The approach
is characterized by (a) convolutional filters based on biologically inspired
visual processing filters, (b) randomly-valued classifier-stage input weights,
(c) use of least squares regression to train the classifier output weights in a
single batch, and (d) linear classifier-stage output units. We demonstrate the
efficacy of the method by applying it to image classification. Our results
match existing state-of-the-art results on the MNIST (0.37% error) and
NORB-small (2.2% error) image classification databases, but with very fast
training times compared to standard deep network approaches. The network's
performance on the Google Street View House Number (SVHN) (4% error) database
is also competitive with state-of-the art methods.
Diversity in Spectral Learning for Natural Language Parsing
Authors: Shashi Narayan, Shay B. Cohen
Date: 2015-05-31 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Diversity

We describe an approach to create a diverse set of predictions with spectral
learning of latent-variable PCFGs (L-PCFGs). Our approach works by creating
multiple spectral models where noise is added to the underlying features in the
training set before the estimation of each model. We describe three ways to
decode with multiple models. In addition, we describe a simple variant of the
spectral algorithm for L-PCFGs that is fast and leads to compact models. Our
experiments for natural language parsing, for English and German, show that we
get a significant improvement over baselines comparable to state of the art.
For English, we achieve the $F_1$ score of 90.18, and for German we achieve the
$F_1$ score of 83.38.
A Comparative Study on Regularization Strategies for Embedding-based Neural Networks
Authors: Hao Peng, Lili Mou, Ge Li, Yunchuan Chen, Yangyang Lu, Zhi Jin
Date: 2015-08-15 00:00:00
Proceeding: EMNLP 2015 9

This paper aims to compare different regularization strategies to address a
common phenomenon, severe overfitting, in embedding-based neural networks for
NLP. We chose two widely studied neural models and tasks as our testbed. We
tried several frequently applied or newly proposed regularization strategies,
including penalizing weights (embeddings excluded), penalizing embeddings,
re-embedding words, and dropout. We also emphasized on incremental
hyperparameter tuning, and combining different regularizations. The results
provide a picture on tuning hyperparameters for neural NLP models.
Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path
Authors: Xu Yan, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, Zhi Jin
Date: 2015-08-15 00:00:00
Tasks: Classification, General Classification, Relation, Relation Classification, Sentence

Relation classification is an important research arena in the field of
natural language processing (NLP). In this paper, we present SDP-LSTM, a novel
neural network to classify the relation of two entities in a sentence. Our
neural architecture leverages the shortest dependency path (SDP) between two
entities; multichannel recurrent neural networks, with long short term memory
(LSTM) units, pick up heterogeneous information along the SDP. Our proposed
model has several distinct features: (1) The shortest dependency paths retain
most relevant information (to relation classification), while eliminating
irrelevant words in the sentence. (2) The multichannel LSTM networks allow
effective information integration from heterogeneous sources over the
dependency paths. (3) A customized dropout strategy regularizes the neural
network to alleviate overfitting. We test our model on the SemEval 2010
relation classification task, and achieve an $F_1$-score of 83.7\%, higher than
competing methods in the literature.
Modeling Relation Paths for Representation Learning of Knowledge Bases
Authors: Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, Song Liu
Date: 2015-06-01 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Knowledge Base Completion, Relation, Relation Extraction, Representation Learning, Semantic Composition

Representation learning of knowledge bases (KBs) aims to embed both entities
and relations into a low-dimensional space. Most existing methods only consider
direct relations in representation learning. We argue that multiple-step
relation paths also contain rich inference patterns between entities, and
propose a path-based representation learning model. This model considers
relation paths as translations between entities for representation learning,
and addresses two key challenges: (1) Since not all relation paths are
reliable, we design a path-constraint resource allocation algorithm to measure
the reliability of relation paths. (2) We represent relation paths via semantic
composition of relation embeddings. Experimental results on real-world datasets
show that, as compared with baselines, our model achieves significant and
consistent improvements on knowledge base completion and relation extraction
from text.
Towards an Axiomatic Approach to Hierarchical Clustering of Measures
Authors: Philipp Thomann, Ingo Steinwart, Nico Schmid
Date: 2015-08-15 00:00:00
Tasks: Clustering

We propose some axioms for hierarchical clustering of probability measures
and investigate their ramifications. The basic idea is to let the user
stipulate the clusters for some elementary measures. This is done without the
need of any notion of metric, similarity or dissimilarity. Our main results
then show that for each suitable choice of user-defined clustering on
elementary measures we obtain a unique notion of clustering on a large set of
distributions satisfying a set of additivity and continuity axioms. We
illustrate the developed theory by numerous examples including some with and
some without a density.
A Novel Approach For Finger Vein Verification Based on Self-Taught Learning
Authors: Mohsen Fayyaz, Masoud Pourreza, Mohammad Hajizadeh Saffar, Mohammad Sabokrou, Mahmood Fathy
Date: 2015-08-15 00:00:00

In this paper, we propose a method for user Finger Vein Authentication (FVA)
as a biometric system. Using the discriminative features for classifying theses
finger veins is one of the main tips that make difference in related works,
Thus we propose to learn a set of representative features, based on
autoencoders. We model the user finger vein using a Gaussian distribution.
Experimental results show that our algorithm perform like a state-of-the-art on
SDUMLA-HMT benchmark.
Neyman-Pearson Classification under High-Dimensional Settings
Authors: Anqi Zhao, Yang Feng, Lie Wang, Xin Tong
Date: 2015-08-13 00:00:00
Tasks: Binary Classification, Classification, General Classification, LEMMA, Vocal Bursts Intensity Prediction, Vocal Bursts Type Prediction

Most existing binary classification methods target on the optimization of the
overall classification risk and may fail to serve some real-world applications
such as cancer diagnosis, where users are more concerned with the risk of
misclassifying one specific class than the other. Neyman-Pearson (NP) paradigm
was introduced in this context as a novel statistical framework for handling
asymmetric type I/II error priorities. It seeks classifiers with a minimal type
II error and a constrained type I error under a user specified level. This
article is the first attempt to construct classifiers with guaranteed
theoretical performance under the NP paradigm in high-dimensional settings.
Based on the fundamental Neyman-Pearson Lemma, we used a plug-in approach to
construct NP-type classifiers for Naive Bayes models. The proposed classifiers
satisfy the NP oracle inequalities, which are natural NP paradigm counterparts
of the oracle inequalities in classical binary classification. Besides their
desirable theoretical properties, we also demonstrated their numerical
advantages in prioritized error control via both simulation and real data
studies.
Beyond-Quantum Modeling of Question Order Effects and Response Replicability in Psychological Measurements
Authors: Diederik Aerts, Massimiliano Sassoli de Bianchi
Date: 2015-08-15 00:00:00

A general tension-reduction (GTR) model was recently considered to derive
quantum probabilities as (universal) averages over all possible forms of
non-uniform fluctuations, and explain their considerable success in describing
experimental situations also outside of the domain of physics, for instance in
the ambit of quantum models of cognition and decision. Yet, this result also
highlighted the possibility of observing violations of the predictions of the
Born rule, in those situations where the averaging would not be large enough,
or would be altered because of the combination of multiple measurements. In
this article we show that this is indeed the case in typical psychological
measurements exhibiting question order effects, by showing that their
statistics of outcomes are inherently non-Hilbertian, and require the larger
framework of the GTR-model to receive an exact mathematical description. We
also consider another unsolved problem of quantum cognition: response
replicability. It is has been observed that when question order effects and
response replicability occur together, the situation cannot be handled anymore
by quantum theory. However, we show that it can be easily and naturally
described in the GTR-model. Based on these findings, we motivate the adoption
in cognitive science of a hidden-measurements interpretation of the quantum
formalism, and of its GTR-model generalization, as the natural interpretational
framework explaining the data of psychological measurements on conceptual
entities.
Unbounded Bayesian Optimization via Regularization
Authors: Bobak Shahriari, Alexandre Bouchard-Côté, Nando de Freitas
Date: 2015-08-14 00:00:00
Tasks: Bayesian Optimization, Benchmarking, global-optimization

Bayesian optimization has recently emerged as a popular and efficient tool
for global optimization and hyperparameter tuning. Currently, the established
Bayesian optimization practice requires a user-defined bounding box which is
assumed to contain the optimizer. However, when little is known about the
probed objective function, it can be difficult to prescribe such bounds. In
this work we modify the standard Bayesian optimization framework in a
principled way to allow automatic resizing of the search space. We introduce
two alternative methods and compare them on two common synthetic benchmarking
test functions as well as the tasks of tuning the stochastic gradient descent
optimizer of a multi-layered perceptron and a convolutional neural network on
MNIST.
Is Stack Overflow Overflowing With Questions and Tags
Authors: Ranjitha R. K., Sanjay Singh
Date: 2015-08-14 00:00:00

Programming question and answer (Q & A) websites, such as Quora, Stack
Overflow, and Yahoo! Answer etc. helps us to understand the programming
concepts easily and quickly in a way that has been tested and applied by many
software developers. Stack Overflow is one of the most frequently used
programming Q\&A website where the questions and answers posted are presently
analyzed manually, which requires a huge amount of time and resource. To save
the effort, we present a topic modeling based technique to analyze the words of
the original texts to discover the themes that run through them. We also
propose a method to automate the process of reviewing the quality of questions
on Stack Overflow dataset in order to avoid ballooning the stack overflow with
insignificant questions. The proposed method also recommends the appropriate
tags for the new post, which averts the creation of unnecessary tags on Stack
Overflow.
Faster Rates for the Frank-Wolfe Method over Strongly-Convex Sets
Authors: Dan Garber, Elad Hazan
Date: 2014-06-05 00:00:00

The Frank-Wolfe method (a.k.a. conditional gradient algorithm) for smooth
optimization has regained much interest in recent years in the context of large
scale optimization and machine learning. A key advantage of the method is that
it avoids projections - the computational bottleneck in many applications -
replacing it by a linear optimization step. Despite this advantage, the known
convergence rates of the FW method fall behind standard first order methods for
most settings of interest. It is an active line of research to derive faster
linear optimization-based algorithms for various settings of convex
optimization.
  In this paper we consider the special case of optimization over strongly
convex sets, for which we prove that the vanila FW method converges at a rate
of $\frac{1}{t^2}$. This gives a quadratic improvement in convergence rate
compared to the general case, in which convergence is of the order
$\frac{1}{t}$, and known to be tight. We show that various balls induced by
$\ell_p$ norms, Schatten norms and group norms are strongly convex on one hand
and on the other hand, linear optimization over these sets is straightforward
and admits a closed-form solution. We further show how several previous
fast-rate results for the FW method follow easily from our analysis.
A Linearly Convergent Conditional Gradient Algorithm with Applications to Online and Stochastic Optimization
Authors: Dan Garber, Elad Hazan
Date: 2013-01-20 00:00:00
Tasks: Open-Ended Question Answering, Stochastic Optimization

Linear optimization is many times algorithmically simpler than non-linear
convex optimization. Linear optimization over matroid polytopes, matching
polytopes and path polytopes are example of problems for which we have simple
and efficient combinatorial algorithms, but whose non-linear convex counterpart
is harder and admits significantly less efficient algorithms. This motivates
the computational model of convex optimization, including the offline, online
and stochastic settings, using a linear optimization oracle. In this
computational model we give several new results that improve over the previous
state-of-the-art. Our main result is a novel conditional gradient algorithm for
smooth and strongly convex optimization over polyhedral sets that performs only
a single linear optimization step over the domain on each iteration and enjoys
a linear convergence rate. This gives an exponential improvement in convergence
rate over previous results.
  Based on this new conditional gradient algorithm we give the first algorithms
for online convex optimization over polyhedral sets that perform only a single
linear optimization step over the domain while having optimal regret
guarantees, answering an open question of Kalai and Vempala, and Hazan and
Kale. Our online algorithms also imply conditional gradient algorithms for
non-smooth and stochastic convex optimization with the same convergence rates
as projected (sub)gradient methods.
Sparsity Based Methods for Overparameterized Variational Problems
Authors: Raja Giryes, Michael Elad, Alfred M. Bruckstein
Date: 2014-05-20 00:00:00
Tasks: Denoising, Optical Flow Estimation

Two complementary approaches have been extensively used in signal and image
processing leading to novel results, the sparse representation methodology and
the variational strategy. Recently, a new sparsity based model has been
proposed, the cosparse analysis framework, which may potentially help in
bridging sparse approximation based methods to the traditional total-variation
minimization. Based on this, we introduce a sparsity based framework for
solving overparameterized variational problems. The latter has been used to
improve the estimation of optical flow and also for general denoising of
signals and images. However, the recovery of the space varying parameters
involved was not adequately addressed by traditional variational methods. We
first demonstrate the efficiency of the new framework for one dimensional
signals in recovering a piecewise linear and polynomial function. Then, we
illustrate how the new technique can be used for denoising and segmentation of
images.
Sufficient and necessary conditions for Dynamic Programming in Valuation-Based Systems
Authors: Jordi Roca-Lacostena, JesUs Cerquides, Marc Pouly
Date: 2015-08-14 00:00:00

Valuation algebras abstract a large number of formalisms for automated
reasoning and enable the definition of generic inference procedures. Many of
these formalisms provide some notion of solution. Typical examples are
satisfying assignments in constraint systems, models in logics or solutions to
linear equation systems.
  Many widely used dynamic programming algorithms for optimization problems
rely on low treewidth decompositions and can be understood as particular cases
of a single algorithmic scheme for finding solutions in a valuation algebra.
The most encompassing description of this algorithmic scheme to date has been
proposed by Pouly and Kohlas together with sufficient conditions for its
correctness. Unfortunately, the formalization relies on a theorem for which we
provide counterexamples. In spite of that, the mainline of Pouly and Kohlas'
theory is correct, although some of the necessary conditions have to be
revised. In this paper we analyze the impact that the counter-examples have on
the theory, and rebuild the theory providing correct sufficient conditions for
the algorithms. Furthermore, we also provide necessary conditions for the
algorithms, allowing for a sharper characterization of when the algorithmic
scheme can be applied.