pwc-archive/papers-with-abstracts huggingface dataset:
Deep Boosting: Joint Feature Selection and Analysis Dictionary Learning in Hierarchy
Authors: Zhanglin Peng, Ya Li, Zhaoquan Cai, Liang Lin
Date: 2015-08-08 00:00:00
Tasks: Classification, Dictionary Learning, feature selection, General Classification, image-classification, Image Classification, Image Generation

This work investigates how the traditional image classification pipelines can
be extended into a deep architecture, inspired by recent successes of deep
neural networks. We propose a deep boosting framework based on layer-by-layer
joint feature boosting and dictionary learning. In each layer, we construct a
dictionary of filters by combining the filters from the lower layer, and
iteratively optimize the image representation with a joint
discriminative-generative formulation, i.e. minimization of empirical
classification error plus regularization of analysis image generation over
training images. For optimization, we perform two iterating steps: i) to
minimize the classification error, select the most discriminative features
using the gentle adaboost algorithm; ii) according to the feature selection,
update the filters to minimize the regularization on analysis image
representation using the gradient descent method. Once the optimization is
converged, we learn the higher layer representation in the same way. Our model
delivers several distinct advantages. First, our layer-wise optimization
provides the potential to build very deep architectures. Second, the generated
image representation is compact and meaningful. In several visual recognition
tasks, our framework outperforms existing state-of-the-art approaches.
Simulating Brain Reaction to Methamphetamine Regarding Consumer Personality
Authors: Maryam Keyvanara, Seyed Amirhassan Monadjemi
Date: 2015-08-11 00:00:00

Addiction, as a nervous disease, can be analysed using mathematical modelling
and computer simulations. In this paper, we use an existing mathematical model
to predict and simulate human brain response to the consumption of a single
dose of methamphetamine. The model is implemented and coded in Matlab. Three
types of personalities including introverts, ambiverts and extroverts are
studied. The parameters of the mathematical model are calibrated and optimized,
according to psychological theories, using a real coded genetic algorithm. The
simulations show significant correlation between people response to
methamphetamine abuse and their personality. They also show that one of the
causes of tendency to stimulants roots in consumers personality traits. The
results can be used as a tool for reducing attitude towards addiction.
Crowd Access Path Optimization: Diversity Matters
Authors: Besmira Nushi, Adish Singla, Anja Gruenheid, Erfan Zamanian, Andreas Krause, Donald Kossmann
Date: 2015-08-08 00:00:00
Tasks: Diversity

Quality assurance is one the most important challenges in crowdsourcing.
Assigning tasks to several workers to increase quality through redundant
answers can be expensive if asking homogeneous sources. This limitation has
been overlooked by current crowdsourcing platforms resulting therefore in
costly solutions. In order to achieve desirable cost-quality tradeoffs it is
essential to apply efficient crowd access optimization techniques. Our work
argues that optimization needs to be aware of diversity and correlation of
information within groups of individuals so that crowdsourcing redundancy can
be adequately planned beforehand. Based on this intuitive idea, we introduce
the Access Path Model (APM), a novel crowd model that leverages the notion of
access paths as an alternative way of retrieving information. APM aggregates
answers ensuring high quality and meaningful confidence. Moreover, we devise a
greedy optimization algorithm for this model that finds a provably good
approximate plan to access the crowd. We evaluate our approach on three
crowdsourced datasets that illustrate various aspects of the problem. Our
results show that the Access Path Model combined with greedy optimization is
cost-efficient and practical to overcome common difficulties in large-scale
crowdsourcing like data sparsity and anonymity.
Removing Biases from Trainable MT Metrics by Using Self-Training
Authors: Miloš Stanojević
Date: 2015-08-10 00:00:00
Tasks: Domain Adaptation, Machine Translation, Translation

Most trainable machine translation (MT) metrics train their weights on human
judgments of state-of-the-art MT systems outputs. This makes trainable metrics
biases in many ways. One of them is preferring longer translations. These
biased metrics when used for tuning are evaluating different types of
translations -- n-best lists of translations with very diverse quality. Systems
tuned with these metrics tend to produce overly long translations that are
preferred by the metric but not by humans. This is usually solved by manually
tweaking metric's weights to equally value recall and precision. Our solution
is more general: (1) it does not address only the recall bias but also all
other biases that might be present in the data and (2) it does not require any
knowledge of the types of features used which is useful in cases when manual
tuning of metric's weights is not possible. This is accomplished by
self-training on unlabeled n-best lists by using metric that was initially
trained on standard human judgments. One way of looking at this is as domain
adaptation from the domain of state-of-the-art MT translations to diverse
n-best list translations.
FactorBase: SQL for Learning A Multi-Relational Graphical Model
Authors: Oliver Schulte, Zhensong Qian
Date: 2015-08-10 00:00:00
Tasks: Management, Model Discovery, Philosophy, Relational Reasoning

We describe FactorBase, a new SQL-based framework that leverages a relational
database management system to support multi-relational model discovery. A
multi-relational statistical model provides an integrated analysis of the
heterogeneous and interdependent data resources in the database. We adopt the
BayesStore design philosophy: statistical models are stored and managed as
first-class citizens inside a database. Whereas previous systems like
BayesStore support multi-relational inference, FactorBase supports
multi-relational learning. A case study on six benchmark databases evaluates
how our system supports a challenging machine learning application, namely
learning a first-order Bayesian network model for an entire database. Model
learning in this setting has to examine a large number of potential statistical
associations across data tables. Our implementation shows how the SQL
constructs in FactorBase facilitate the fast, modular, and reliable development
of highly scalable model learning systems.
Kernel Methods for Linear Discrete-Time Equations
Authors: Fritz Colonius, Boumediene Hamzi
Date: 2015-07-11 00:00:00
Tasks: Learning Theory

Methods from learning theory are used in the state space of linear dynamical
and control systems in order to estimate the system matrices. An application to
stabilization via algebraic Riccati equations is included. The approach is
illustrated via a series of numerical examples.
Gait Assessment for Multiple Sclerosis Patients Using Microsoft Kinect
Authors: Farnood Gholami, Daria A. Trojan, Jozsef Kovecses, Wassim M. Haddad, Behnood Gholami
Date: 2015-08-10 00:00:00
Tasks: Dynamic Time Warping

Gait analysis of patients with neurological disorders, including multiple
sclerosis (MS), is important for rehabilitation and treatment. The Mircrosoft
Kinect sensor, which was developed for motion recognition in gaming
applications, is an ideal candidate for an inexpensive system providing the
capability for human gait analysis. In this research, we develop a framework to
quantify the gait abnormality of MS patients using a Kinect for Windows camera.
In addition to the previously introduced gait indices, a novel set of MS gait
indices based on the concept of dynamic time warping is introduced. The newly
introduced indices can characterize a patient's gait pattern as a whole and
quantify a subject's gait distance from the healthy population. We will
investigate the correlation of gait indices with the multiple sclerosis walking
scale (MSWS) and the clinical ambulation score. This work establishes the
feasibility of using the Kinect sensor for clinical gait assessment for MS
patients.
Approximation-Aware Dependency Parsing by Belief Propagation
Authors: Matthew R. Gormley, Mark Dredze, Jason Eisner
Date: 2015-08-10 00:00:00
Proceeding: TACL 2015 1
Tasks: Dependency Parsing

We show how to train the fast dependency parser of Smith and Eisner (2008)
for improved accuracy. This parser can consider higher-order interactions among
edges while retaining O(n^3) runtime. It outputs the parse with maximum
expected recall -- but for speed, this expectation is taken under a posterior
distribution that is constructed only approximately, using loopy belief
propagation through structured factors. We show how to adjust the model
parameters to compensate for the errors introduced by this approximation, by
following the gradient of the actual loss on training data. We find this
gradient by back-propagation. That is, we treat the entire parser
(approximations and all) as a differentiable circuit, as Stoyanov et al. (2011)
and Domke (2010) did for loopy CRFs. The resulting trained parser obtains
higher accuracy with fewer iterations of belief propagation than one trained by
conditional log-likelihood.
Training Conditional Random Fields with Natural Gradient Descent
Authors: Yuan Cao
Date: 2015-08-10 00:00:00
Tasks: parameter estimation

We propose a novel parameter estimation procedure that works efficiently for
conditional random fields (CRF). This algorithm is an extension to the maximum
likelihood estimation (MLE), using loss functions defined by Bregman
divergences which measure the proximity between the model expectation and the
empirical mean of the feature vectors. This leads to a flexible training
framework from which multiple update strategies can be derived using natural
gradient descent (NGD). We carefully choose the convex function inducing the
Bregman divergence so that the types of updates are reduced, while making the
optimization procedure more effective by transforming the gradients of the
log-likelihood loss function. The derived algorithms are very simple and can be
easily implemented on top of the existing stochastic gradient descent (SGD)
optimization procedure, yet it is very effective as illustrated by experimental
results.
Local Algorithms for Block Models with Side Information
Authors: Elchanan Mossel, Jiaming Xu
Date: 2015-08-10 00:00:00
Tasks: Community Detection, Stochastic Block Model

There has been a recent interest in understanding the power of local
algorithms for optimization and inference problems on sparse graphs. Gamarnik
and Sudan (2014) showed that local algorithms are weaker than global algorithms
for finding large independent sets in sparse random regular graphs. Montanari
(2015) showed that local algorithms are suboptimal for finding a community with
high connectivity in the sparse Erd\H{o}s-R\'enyi random graphs. For the
symmetric planted partition problem (also named community detection for the
block models) on sparse graphs, a simple observation is that local algorithms
cannot have non-trivial performance.
  In this work we consider the effect of side information on local algorithms
for community detection under the binary symmetric stochastic block model. In
the block model with side information each of the $n$ vertices is labeled $+$
or $-$ independently and uniformly at random; each pair of vertices is
connected independently with probability $a/n$ if both of them have the same
label or $b/n$ otherwise. The goal is to estimate the underlying vertex
labeling given 1) the graph structure and 2) side information in the form of a
vertex labeling positively correlated with the true one. Assuming that the
ratio between in and out degree $a/b$ is $\Theta(1)$ and the average degree $
(a+b) / 2 = n^{o(1)}$, we characterize three different regimes under which a
local algorithm, namely, belief propagation run on the local neighborhoods,
maximizes the expected fraction of vertices labeled correctly. Thus, in
contrast to the case of symmetric block models without side information, we
show that local algorithms can achieve optimal performance for the block model
with side information.
Measuring Word Significance using Distributed Representations of Words
Authors: Adriaan M. J. Schakel, Benjamin J. Wilson
Date: 2015-08-10 00:00:00

Distributed representations of words as real-valued vectors in a relatively
low-dimensional space aim at extracting syntactic and semantic features from
large text corpora. A recently introduced neural network, named word2vec
(Mikolov et al., 2013a; Mikolov et al., 2013b), was shown to encode semantic
information in the direction of the word vectors. In this brief report, it is
proposed to use the length of the vectors, together with the term frequency, as
measure of word significance in a corpus. Experimental evidence using a
domain-specific corpus of abstracts is presented to support this proposal. A
useful visualization technique for text corpora emerges, where words are mapped
onto a two-dimensional plane and automatically ranked by significance.
Adapting Phrase-based Machine Translation to Normalise Medical Terms in Social Media Messages
Authors: Nut Limsopatham, Nigel Collier
Date: 2015-08-10 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Machine Translation, Translation

Previous studies have shown that health reports in social media, such as
DailyStrength and Twitter, have potential for monitoring health conditions
(e.g. adverse drug reactions, infectious diseases) in particular communities.
However, in order for a machine to understand and make inferences on these
health conditions, the ability to recognise when laymen's terms refer to a
particular medical concept (i.e.\ text normalisation) is required. To achieve
this, we propose to adapt an existing phrase-based machine translation (MT)
technique and a vector representation of words to map between a social media
phrase and a medical concept. We evaluate our proposed approach using a
collection of phrases from tweets related to adverse drug reactions. Our
experimental results show that the combination of a phrase-based MT technique
and the similarity between word vector representations outperforms the
baselines that apply only either of them by up to 55%.
Dropout Training for SVMs with Data Augmentation
Authors: Ning Chen, Jun Zhu, Jianfei Chen, Ting Chen
Date: 2015-08-10 00:00:00
Tasks: Data Augmentation, Representation Learning

Dropout and other feature noising schemes have shown promising results in
controlling over-fitting by artificially corrupting the training data. Though
extensive theoretical and empirical studies have been performed for generalized
linear models, little work has been done for support vector machines (SVMs),
one of the most successful approaches for supervised learning. This paper
presents dropout training for both linear SVMs and the nonlinear extension with
latent representation learning. For linear SVMs, to deal with the intractable
expectation of the non-smooth hinge loss under corrupting distributions, we
develop an iteratively re-weighted least square (IRLS) algorithm by exploring
data augmentation techniques. Our algorithm iteratively minimizes the
expectation of a re-weighted least square problem, where the re-weights are
analytically updated. For nonlinear latent SVMs, we consider learning one layer
of latent representations in SVMs and extend the data augmentation technique in
conjunction with first-order Taylor-expansion to deal with the intractable
expected non-smooth hinge loss and the nonlinearity of latent representations.
Finally, we apply the similar data augmentation ideas to develop a new IRLS
algorithm for the expected logistic loss under corrupting distributions, and we
further develop a non-linear extension of logistic regression by incorporating
one layer of latent representations. Our algorithms offer insights on the
connection and difference between the hinge loss and logistic loss in dropout
training. Empirical results on several real datasets demonstrate the
effectiveness of dropout training on significantly boosting the classification
accuracy of both linear and nonlinear SVMs. In addition, the nonlinear SVMs
further improve the prediction performance on several image datasets.
Feature Learning for Interaction Activity Recognition in RGBD Videos
Authors: Ngu Nguyen
Date: 2015-08-10 00:00:00
Tasks: Activity Recognition, Human Activity Recognition

This paper proposes a human activity recognition method which is based on
features learned from 3D video data without incorporating domain knowledge. The
experiments on data collected by RGBD cameras produce results outperforming
other techniques. Our feature encoding method follows the bag-of-visual-word
model, then we use a SVM classifier to recognise the activities. We do not use
skeleton or tracking information and the same technique is applied on color and
depth data.
Model-based SIR for dimension reduction
Authors: Luca Scrucca
Date: 2015-08-10 00:00:00
Tasks: Dimensionality Reduction, General Classification, model, regression

A new dimension reduction method based on Gaussian finite mixtures is
proposed as an extension to sliced inverse regression (SIR). The model-based
SIR (MSIR) approach allows the main limitation of SIR to be overcome, i.e.,
failure in the presence of regression symmetric relationships, without the need
to impose further assumptions. Extensive numerical studies are presented to
compare the new method with some of most popular dimension reduction methods,
such as SIR, sliced average variance estimation, principal Hessian direction,
and directional regression. MSIR appears sufficiently flexible to accommodate
various regression functions, and its performance is comparable with or better,
particularly as sample size grows, than other available methods. Lastly, MSIR
is illustrated with two real data examples about ozone concentration
regression, and hand-written digit classification.
Automatic Extraction of the Passing Strategies of Soccer Teams
Authors: Laszlo Gyarmati, Xavier Anguera
Date: 2015-08-10 00:00:00
Tasks: Dynamic Time Warping

Technology offers new ways to measure the locations of the players and of the
ball in sports. This translates to the trajectories the ball takes on the field
as a result of the tactics the team applies. The challenge professionals in
soccer are facing is to take the reverse path: given the trajectories of the
ball is it possible to infer the underlying strategy/tactic of a team? We
propose a method based on Dynamic Time Warping to reveal the tactics of a team
through the analysis of repeating series of events. Based on the analysis of an
entire season, we derive insights such as passing strategies for maintaining
ball possession or counter attacks, and passing styles with a focus on the team
or on the capabilities of the individual players.
Optimal estimates for short horizon travel time prediction in urban areas
Authors: Indre Zliobaite, Mikhail Khokhlov
Date: 2015-07-30 00:00:00
Tasks: Prediction, Travel Time Estimation

Increasing popularity of mobile route planning applications based on GPS
technology provides opportunities for collecting traffic data in urban
environments. One of the main challenges for travel time estimation and
prediction in such a setting is how to aggregate data from vehicles that have
followed different routes, and predict travel time for other routes of
interest. One approach is to predict travel times for route segments, and sum
those estimates to obtain a prediction for the whole route. We study how to
obtain optimal predictions in this scenario. It appears that the optimal
estimate, minimizing the expected mean absolute error, is a combination of the
mean and the median travel times on each segment, where the combination
function depends on the number of segments in the route of interest. We present
a methodology for obtaining such predictions, and demonstrate its effectiveness
with a case study using travel time data from a district of St. Petersburg
collected over one year. The proposed methodology can be applied for real-time
prediction of expected travel times in an urban road network.
Feature-based Decipherment for Large Vocabulary Machine Translation
Authors: Iftekhar Naim, Daniel Gildea
Date: 2015-08-10 00:00:00
Tasks: Decipherment, Machine Translation, Translation

Orthographic similarities across languages provide a strong signal for
probabilistic decipherment, especially for closely related language pairs. The
existing decipherment models, however, are not well-suited for exploiting these
orthographic similarities. We propose a log-linear model with latent variables
that incorporates orthographic similarity features. Maximum likelihood training
is computationally expensive for the proposed log-linear model. To address this
challenge, we perform approximate inference via MCMC sampling and contrastive
divergence. Our results show that the proposed log-linear model with
contrastive divergence scales to large vocabularies and outperforms the
existing generative decipherment models by exploiting the orthographic
features.
Learning Structural Kernels for Natural Language Processing
Authors: Daniel Beck, Trevor Cohn, Christian Hardmeier, Lucia Specia
Date: 2015-08-10 00:00:00
Proceeding: TACL 2015 1
Tasks: Gaussian Processes, Hyperparameter Optimization, Model Selection

Structural kernels are a flexible learning paradigm that has been widely used
in Natural Language Processing. However, the problem of model selection in
kernel-based methods is usually overlooked. Previous approaches mostly rely on
setting default values for kernel hyperparameters or using grid search, which
is slow and coarse-grained. In contrast, Bayesian methods allow efficient model
selection by maximizing the evidence on the training data through
gradient-based methods. In this paper we show how to perform this in the
context of structural kernels by using Gaussian Processes. Experimental results
on tree kernels show that this procedure results in better prediction
performance compared to hyperparameter optimization via grid search. The
framework proposed in this paper can be adapted to other structures besides
trees, e.g., strings and graphs, thereby extending the utility of kernel-based
methods.
Learning the nonlinear geometry of high-dimensional data: Models and algorithms
Authors: Tong Wu, Waheed U. Bajwa
Date: 2014-12-21 00:00:00
Tasks: Clustering, Vocal Bursts Intensity Prediction

Modern information processing relies on the axiom that high-dimensional data
lie near low-dimensional geometric structures. This paper revisits the problem
of data-driven learning of these geometric structures and puts forth two new
nonlinear geometric models for data describing "related" objects/phenomena. The
first one of these models straddles the two extremes of the subspace model and
the union-of-subspaces model, and is termed the metric-constrained
union-of-subspaces (MC-UoS) model. The second one of these models---suited for
data drawn from a mixture of nonlinear manifolds---generalizes the kernel
subspace model, and is termed the metric-constrained kernel union-of-subspaces
(MC-KUoS) model. The main contributions of this paper in this regard include
the following. First, it motivates and formalizes the problems of MC-UoS and
MC-KUoS learning. Second, it presents algorithms that efficiently learn an
MC-UoS or an MC-KUoS underlying data of interest. Third, it extends these
algorithms to the case when parts of the data are missing. Last, but not least,
it reports the outcomes of a series of numerical experiments involving both
synthetic and real data that demonstrate the superiority of the proposed
geometric models and learning algorithms over existing approaches in the
literature. These experiments also help clarify the connections between this
work and the literature on (subspace and kernel k-means) clustering.