pwc-archive/papers-with-abstracts huggingface dataset:
Learning Articulated Motion Models from Visual and Lingual Signals
Authors: Zhengyang Wu, Mohit Bansal, Matthew R. Walter
Date: 2015-11-17 00:00:00
Tasks: Language Modeling, Language Modelling, Word Embeddings

In order for robots to operate effectively in homes and workplaces, they must
be able to manipulate the articulated objects common within environments built
for and by humans. Previous work learns kinematic models that prescribe this
manipulation from visual demonstrations. Lingual signals, such as natural
language descriptions and instructions, offer a complementary means of
conveying knowledge of such manipulation models and are suitable to a wide
range of interactions (e.g., remote manipulation). In this paper, we present a
multimodal learning framework that incorporates both visual and lingual
information to estimate the structure and parameters that define kinematic
models of articulated objects. The visual signal takes the form of an RGB-D
image stream that opportunistically captures object motion in an unprepared
scene. Accompanying natural language descriptions of the motion constitute the
lingual signal. We present a probabilistic language model that uses word
embeddings to associate lingual verbs with their corresponding kinematic
structures. By exploiting the complementary nature of the visual and lingual
input, our method infers correct kinematic structures for various multiple-part
objects on which the previous state-of-the-art, visual-only system fails. We
evaluate our multimodal learning framework on a dataset comprised of a variety
of household objects, and demonstrate a 36% improvement in model accuracy over
the vision-only baseline.
Automated 5-year Mortality Prediction using Deep Learning and Radiomics Features from Chest Computed Tomography
Authors: Gustavo Carneiro, Luke Oakden-Rayner, Andrew P. Bradley, Jacinto Nascimento, Lyle Palmer
Date: 2016-07-01 00:00:00
Tasks: Computed Tomography (CT), feature selection, Mortality Prediction, Prediction

We propose new methods for the prediction of 5-year mortality in elderly
individuals using chest computed tomography (CT). The methods consist of a
classifier that performs this prediction using a set of features extracted from
the CT image and segmentation maps of multiple anatomic structures. We explore
two approaches: 1) a unified framework based on deep learning, where features
and classifier are automatically learned in a single optimisation process; and
2) a multi-stage framework based on the design and selection/extraction of
hand-crafted radiomics features, followed by the classifier learning process.
Experimental results, based on a dataset of 48 annotated chest CTs, show that
the deep learning model produces a mean 5-year mortality prediction accuracy of
68.5%, while radiomics produces a mean accuracy that varies between 56% to 66%
(depending on the feature selection/extraction method and classifier). The
successful development of the proposed models has the potential to make a
profound impact in preventive and personalised healthcare.
F-measure Maximization in Multi-Label Classification with Conditionally Independent Label Subsets
Authors: Maxime Gasse, Alex Aussem
Date: 2016-04-26 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We discuss a method to improve the exact F-measure maximization algorithm
called GFM, proposed in (Dembczynski et al. 2011) for multi-label
classification, assuming the label set can be can partitioned into
conditionally independent subsets given the input features. If the labels were
all independent, the estimation of only $m$ parameters ($m$ denoting the number
of labels) would suffice to derive Bayes-optimal predictions in $O(m^2)$
operations. In the general case, $m^2+1$ parameters are required by GFM, to
solve the problem in $O(m^3)$ operations. In this work, we show that the number
of parameters can be reduced further to $m^2/n$, in the best case, assuming the
label set can be partitioned into $n$ conditionally independent subsets. As
this label partition needs to be estimated from the data beforehand, we use
first the procedure proposed in (Gasse et al. 2015) that finds such partition
and then infer the required parameters locally in each label subset. The latter
are aggregated and serve as input to GFM to form the Bayes-optimal prediction.
We show on a synthetic experiment that the reduction in the number of
parameters brings about significant benefits in terms of performance.
Evaluating Unsupervised Dutch Word Embeddings as a Linguistic Resource
Authors: Stéphan Tulkens, Chris Emmery, Walter Daelemans
Date: 2016-07-01 00:00:00
Proceeding: LREC 2016 5
Tasks: Dialect Identification, Relation, Word Embeddings

Word embeddings have recently seen a strong increase in interest as a result
of strong performance gains on a variety of tasks. However, most of this
research also underlined the importance of benchmark datasets, and the
difficulty of constructing these for a variety of language-specific tasks.
Still, many of the datasets used in these tasks could prove to be fruitful
linguistic resources, allowing for unique observations into language use and
variability. In this paper we demonstrate the performance of multiple types of
embeddings, created with both count and prediction-based architectures on a
variety of corpora, in two language-specific tasks: relation evaluation, and
dialect identification. For the latter, we compare unsupervised methods with a
traditional, hand-crafted dictionary. With this research, we provide the
embeddings themselves, the relation evaluation task benchmark for use in
further research, and demonstrate how the benchmarked embeddings prove a useful
unsupervised linguistic resource, effectively used in a downstream task.
Gender Inference using Statistical Name Characteristics in Twitter
Authors: Juergen Mueller, Gerd Stumme
Date: 2016-06-17 00:00:00

Much attention has been given to the task of gender inference of Twitter
users. Although names are strong gender indicators, the names of Twitter users
are rarely used as a feature; probably due to the high number of ill-formed
names, which cannot be found in any name dictionary. Instead of relying solely
on a name database, we propose a novel name classifier. Our approach extracts
characteristics from the user names and uses those in order to assign the names
to a gender. This enables us to classify international first names as well as
ill-formed names.
Guarantees in Wasserstein Distance for the Langevin Monte Carlo Algorithm
Authors: Thomas Bonis
Date: 2016-02-08 00:00:00

We study the problem of sampling from a distribution $\target$ using the
Langevin Monte Carlo algorithm and provide rate of convergences for this
algorithm in terms of Wasserstein distance of order $2$. Our result holds as
long as the continuous diffusion process associated with the algorithm
converges exponentially fast to the target distribution along with some
technical assumptions. While such an exponential convergence holds for example
in the log-concave measure case, it also holds for the more general case of
asymptoticaly log-concave measures. Our results thus extends the known rates of
convergence in total variation and Wasserstein distances which have only been
obtained in the log-concave case. Moreover, using a sharper approximation bound
of the continuous process, we obtain better asymptotic rates than traditional
results. We also look into variations of the Langevin Monte Carlo algorithm
using other discretization schemes. In a first time, we look into the use of
the Ozaki's discretization but are unable to obtain any significative
improvement in terms of convergence rates compared to the Euler's scheme. We
then provide a (sub-optimal) way to study more general schemes, however our
approach only holds for the log-concave case.
Sharing Network Parameters for Crosslingual Named Entity Recognition
Authors: Rudra Murthy V, Mitesh Khapra, Pushpak Bhattacharyya
Date: 2016-07-01 00:00:00
Tasks: Decoder, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Most state of the art approaches for Named Entity Recognition rely on hand
crafted features and annotated corpora. Recently Neural network based models
have been proposed which do not require handcrafted features but still require
annotated corpora. However, such annotated corpora may not be available for
many languages. In this paper, we propose a neural network based model which
allows sharing the decoder as well as word and character level parameters
between two languages thereby allowing a resource fortunate language to aid a
resource deprived language. Specifically, we focus on the case when limited
annotated corpora is available in one language ($L_1$) and abundant annotated
corpora is available in another language ($L_2$). Sharing the network
architecture and parameters between $L_1$ and $L_2$ leads to improved
performance in $L_1$. Further, our approach does not require any hand crafted
features but instead directly learns meaningful feature representations from
the training data itself. We experiment with 4 language pairs and show that
indeed in a resource constrained setup (lesser annotated corpora), a model
jointly trained with data from another language performs better than a model
trained only on the limited corpora in one language.
Throwing fuel on the embers: Probability or Dichotomy, Cognitive or Linguistic?
Authors: David M. W. Powers
Date: 2016-07-01 00:00:00
Tasks: Position

Prof. Robert Berwick's abstract for his forthcoming invited talk at the
ACL2016 workshop on Cognitive Aspects of Computational Language Learning
revives an ancient debate. Entitled "Why take a chance?", Berwick seems to
refer implicitly to Chomsky's critique of the statistical approach of Harris as
well as the currently dominant paradigms in CoNLL.
  Berwick avoids Chomsky's use of "innate" but states that "the debate over the
existence of sophisticated mental grammars was settled with Chomsky's Logical
Structure of Linguistic Theory (1957/1975)", acknowledging that "this debate
has often been revived".
  This paper agrees with the view that this debate has long since been settled,
but with the opposite outcome! Given the embers have not yet died away, and the
questions remain fundamental, perhaps it is appropriate to refuel the debate,
so I would like to join Bob in throwing fuel on this fire by reviewing the
evidence against the Chomskian position!
Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning
Authors: Janarthanan Rajendran, Mitesh M. Khapra, Sarath Chandar, Balaraman Ravindran
Date: 2015-10-13 00:00:00
Proceeding: NAACL 2016 6
Tasks: Document Classification, Representation Learning, Retrieval, Transfer Learning

Recently there has been a lot of interest in learning common representations
for multiple views of data. Typically, such common representations are learned
using a parallel corpus between the two views (say, 1M images and their English
captions). In this work, we address a real-world scenario where no direct
parallel data is available between two views of interest (say, $V_1$ and $V_2$)
but parallel data is available between each of these views and a pivot view
($V_3$). We propose a model for learning a common representation for $V_1$,
$V_2$ and $V_3$ using only the parallel data available between $V_1V_3$ and
$V_2V_3$. The proposed model is generic and even works when there are $n$ views
of interest and only one pivot view which acts as a bridge between them. There
are two specific downstream applications that we focus on (i) transfer learning
between languages $L_1$,$L_2$,...,$L_n$ using a pivot language $L$ and (ii)
cross modal access between images and a language $L_1$ using a pivot language
$L_2$. Our model achieves state-of-the-art performance in multilingual document
classification on the publicly available multilingual TED corpus and promising
results in multilingual multimodal retrieval on a new dataset created and
released as a part of this work.
Visualizing the Effects of a Changing Distance on Data Using Continuous Embeddings
Authors: Gina Gruenhage, Manfred Opper, Simon Barthelme
Date: 2013-11-08 00:00:00
Tasks: Clustering, Dimensionality Reduction, Topological Data Analysis

Most Machine Learning (ML) methods, from clustering to classification, rely
on a distance function to describe relationships between datapoints. For
complex datasets it is hard to avoid making some arbitrary choices when
defining a distance function. To compare images, one must choose a spatial
scale, for signals, a temporal scale. The right scale is hard to pin down and
it is preferable when results do not depend too tightly on the exact value one
picked. Topological data analysis seeks to address this issue by focusing on
the notion of neighbourhood instead of distance. It is shown that in some cases
a simpler solution is available. It can be checked how strongly distance
relationships depend on a hyperparameter using dimensionality reduction. A
variant of dynamical multi-dimensional scaling (MDS) is formulated, which
embeds datapoints as curves. The resulting algorithm is based on the
Concave-Convex Procedure (CCCP) and provides a simple and efficient way of
visualizing changes and invariances in distance patterns as a hyperparameter is
varied. A variant to analyze the dependence on multiple hyperparameters is also
presented. A cMDS algorithm that is straightforward to implement, use and
extend is provided. To illustrate the possibilities of cMDS, cMDS is applied to
several real-world data sets.
Efficient and Consistent Robust Time Series Analysis
Authors: Kush Bhatia, Prateek Jain, Parameswaran Kamalaruban, Purushottam Kar
Date: 2016-07-01 00:00:00
Tasks: regression, Time Series, Time Series Analysis

We study the problem of robust time series analysis under the standard
auto-regressive (AR) time series model in the presence of arbitrary outliers.
We devise an efficient hard thresholding based algorithm which can obtain a
consistent estimate of the optimal AR model despite a large fraction of the
time series points being corrupted. Our algorithm alternately estimates the
corrupted set of points and the model parameters, and is inspired by recent
advances in robust regression and hard-thresholding methods. However, a direct
application of existing techniques is hindered by a critical difference in the
time-series domain: each point is correlated with all previous points rendering
existing tools inapplicable directly. We show how to overcome this hurdle using
novel proof techniques. Using our techniques, we are also able to provide the
first efficient and provably consistent estimator for the robust regression
problem where a standard linear observation model with white additive noise is
corrupted arbitrarily. We illustrate our methods on synthetic datasets and show
that our methods indeed are able to consistently recover the optimal parameters
despite a large fraction of points being corrupted.
Sparse Graphical Representation based Discriminant Analysis for Heterogeneous Face Recognition
Authors: Chunlei Peng, Xinbo Gao, Nannan Wang, Jie Li
Date: 2016-07-01 00:00:00
Tasks: Face Recognition, Heterogeneous Face Recognition

Face images captured in heterogeneous environments, e.g., sketches generated
by the artists or composite-generation software, photos taken by common cameras
and infrared images captured by corresponding infrared imaging devices, usually
subject to large texture (i.e., style) differences. This results in heavily
degraded performance of conventional face recognition methods in comparison
with the performance on images captured in homogeneous environments. In this
paper, we propose a novel sparse graphical representation based discriminant
analysis (SGR-DA) approach to address aforementioned face recognition in
heterogeneous scenarios. An adaptive sparse graphical representation scheme is
designed to represent heterogeneous face images, where a Markov networks model
is constructed to generate adaptive sparse vectors. To handle the complex
facial structure and further improve the discriminability, a spatial
partition-based discriminant analysis framework is presented to refine the
adaptive sparse vectors for face matching. We conducted experiments on six
commonly used heterogeneous face datasets and experimental results illustrate
that our proposed SGR-DA approach achieves superior performance in comparison
with state-of-the-art methods.
Missing Data Estimation in High-Dimensional Datasets: A Swarm Intelligence-Deep Neural Network Approach
Authors: Collins Leke, Tshilidzi Marwala
Date: 2016-07-01 00:00:00
Tasks: Deep Learning

In this paper, we examine the problem of missing data in high-dimensional
datasets by taking into consideration the Missing Completely at Random and
Missing at Random mechanisms, as well as theArbitrary missing pattern.
Additionally, this paper employs a methodology based on Deep Learning and Swarm
Intelligence algorithms in order to provide reliable estimates for missing
data. The deep learning technique is used to extract features from the input
data via an unsupervised learning approach by modeling the data distribution
based on the input. This deep learning technique is then used as part of the
objective function for the swarm intelligence technique in order to estimate
the missing data after a supervised fine-tuning phase by minimizing an error
function based on the interrelationship and correlation between features in the
dataset. The investigated methodology in this paper therefore has longer
running times, however, the promising potential outcomes justify the trade-off.
Also, basic knowledge of statistics is presumed.
Less-forgetting Learning in Deep Neural Networks
Authors: Heechul Jung, Jeongwoo Ju, Minju Jung, Junmo Kim
Date: 2016-07-01 00:00:00

A catastrophic forgetting problem makes deep neural networks forget the
previously learned information, when learning data collected in new
environments, such as by different sensors or in different light conditions.
This paper presents a new method for alleviating the catastrophic forgetting
problem. Unlike previous research, our method does not use any information from
the source domain. Surprisingly, our method is very effective to forget less of
the information in the source domain, and we show the effectiveness of our
method using several experiments. Furthermore, we observed that the forgetting
problem occurs between mini-batches when performing general training processes
using stochastic gradient descent methods, and this problem is one of the
factors that degrades generalization performance of the network. We also try to
solve this problem using the proposed method. Finally, we show our
less-forgetting learning method is also helpful to improve the performance of
deep neural networks in terms of recognition rates.
Combining Gradient Boosting Machines with Collective Inference to Predict Continuous Values
Authors: Iman Alodah, Jennifer Neville
Date: 2016-07-01 00:00:00
Tasks: regression

Gradient boosting of regression trees is a competitive procedure for learning
predictive models of continuous data that fits the data with an additive
non-parametric model. The classic version of gradient boosting assumes that the
data is independent and identically distributed. However, relational data with
interdependent, linked instances is now common and the dependencies in such
data can be exploited to improve predictive performance. Collective inference
is one approach to exploit relational correlation patterns and significantly
reduce classification error. However, much of the work on collective learning
and inference has focused on discrete prediction tasks rather than continuous.
%target values has not got that attention in terms of collective inference. In
this work, we investigate how to combine these two paradigms together to
improve regression in relational domains. Specifically, we propose a boosting
algorithm for learning a collective inference model that predicts a continuous
target variable. In the algorithm, we learn a basic relational model,
collectively infer the target values, and then iteratively learn relational
models to predict the residuals. We evaluate our proposed algorithm on a real
network dataset and show that it outperforms alternative boosting methods.
However, our investigation also revealed that the relational features interact
together to produce better predictions.
Randomized block proximal damped Newton method for composite self-concordant minimization
Authors: Zhaosong Lu
Date: 2016-07-01 00:00:00

In this paper we consider the composite self-concordant (CSC) minimization
problem, which minimizes the sum of a self-concordant function $f$ and a
(possibly nonsmooth) proper closed convex function $g$. The CSC minimization is
the cornerstone of the path-following interior point methods for solving a
broad class of convex optimization problems. It has also found numerous
applications in machine learning. The proximal damped Newton (PDN) methods have
been well studied in the literature for solving this problem that enjoy a nice
iteration complexity. Given that at each iteration these methods typically
require evaluating or accessing the Hessian of $f$ and also need to solve a
proximal Newton subproblem, the cost per iteration can be prohibitively high
when applied to large-scale problems. Inspired by the recent success of block
coordinate descent methods, we propose a randomized block proximal damped
Newton (RBPDN) method for solving the CSC minimization. Compared to the PDN
methods, the computational cost per iteration of RBPDN is usually significantly
lower. The computational experiment on a class of regularized logistic
regression problems demonstrate that RBPDN is indeed promising in solving
large-scale CSC minimization problems. The convergence of RBPDN is also
analyzed in the paper. In particular, we show that RBPDN is globally convergent
when $g$ is Lipschitz continuous. It is also shown that RBPDN enjoys a local
linear convergence. Moreover, we show that for a class of $g$ including the
case where $g$ is Lipschitz differentiable, RBPDN enjoys a global linear
convergence. As a striking consequence, it shows that the classical damped
Newton methods [22,40] and the PDN [31] for such $g$ are globally linearly
convergent, which was previously unknown in the literature. Moreover, this
result can be used to sharpen the existing iteration complexity of these
methods.
Neural Summarization by Extracting Sentences and Words
Authors: Jianpeng Cheng, Mirella Lapata
Date: 2016-03-23 00:00:00
Proceeding: ACL 2016 8
Tasks: Document Summarization, Extractive Summarization, Sentence

Traditional approaches to extractive summarization rely heavily on
human-engineered features. In this work we propose a data-driven approach based
on neural networks and continuous sentence features. We develop a general
framework for single-document summarization composed of a hierarchical document
encoder and an attention-based extractor. This architecture allows us to
develop different classes of summarization models which can extract sentences
or words. We train our models on large scale corpora containing hundreds of
thousands of document-summary pairs. Experimental results on two summarization
datasets demonstrate that our models obtain results comparable to the state of
the art without any access to linguistic annotation.