pwc-archive/papers-with-abstracts huggingface dataset:
Reinforcement Learning Neural Turing Machines - Revised
Authors: Wojciech Zaremba, Ilya Sutskever
Date: 2015-05-04 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The Neural Turing Machine (NTM) is more expressive than all previously
considered models because of its external memory. It can be viewed as a broader
effort to use abstract external Interfaces and to learn a parametric model that
interacts with them.
  The capabilities of a model can be extended by providing it with proper
Interfaces that interact with the world. These external Interfaces include
memory, a database, a search engine, or a piece of software such as a theorem
verifier. Some of these Interfaces are provided by the developers of the model.
However, many important existing Interfaces, such as databases and search
engines, are discrete.
  We examine feasibility of learning models to interact with discrete
Interfaces. We investigate the following discrete Interfaces: a memory Tape, an
input Tape, and an output Tape. We use a Reinforcement Learning algorithm to
train a neural network that interacts with such Interfaces to solve simple
algorithmic tasks. Our Interfaces are expressive enough to make our model
Turing complete.
Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints
Authors: Ehsan Hosseini-Asl, Jacek M. Zurada, Olfa Nasraoui
Date: 2016-01-12 00:00:00

We demonstrate a new deep learning autoencoder network, trained by a
nonnegativity constraint algorithm (NCAE), that learns features which show
part-based representation of data. The learning algorithm is based on
constraining negative weights. The performance of the algorithm is assessed
based on decomposing data into parts and its prediction performance is tested
on three standard image data sets and one text dataset. The results indicate
that the nonnegativity constraint forces the autoencoder to learn features that
amount to a part-based representation of data, while improving sparsity and
reconstruction quality in comparison with the traditional sparse autoencoder
and Nonnegative Matrix Factorization. It is also shown that this newly acquired
representation improves the prediction performance of a deep neural network.
Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
Authors: Majid Janzamin, Hanie Sedghi, Anima Anandkumar
Date: 2015-06-28 00:00:00
Tasks: Tensor Decomposition

Training neural networks is a challenging non-convex optimization problem,
and backpropagation or gradient descent can get stuck in spurious local optima.
We propose a novel algorithm based on tensor decomposition for guaranteed
training of two-layer neural networks. We provide risk bounds for our proposed
method, with a polynomial sample complexity in the relevant parameters, such as
input dimension and number of neurons. While learning arbitrary target
functions is NP-hard, we provide transparent conditions on the function and the
input for learnability. Our training method is based on tensor decomposition,
which provably converges to the global optimum, under a set of mild
non-degeneracy conditions. It consists of simple embarrassingly parallel linear
and multi-linear operations, and is competitive with standard stochastic
gradient descent (SGD), in terms of computational complexity. Thus, we propose
a computationally efficient method with guaranteed risk bounds for training
neural networks with one hidden layer.
IRLS and Slime Mold: Equivalence and Convergence
Authors: Damian Straszak, Nisheeth K. Vishnoi
Date: 2016-01-12 00:00:00
Tasks: compressed sensing

In this paper we present a connection between two dynamical systems arising
in entirely different contexts: one in signal processing and the other in
biology. The first is the famous Iteratively Reweighted Least Squares (IRLS)
algorithm used in compressed sensing and sparse recovery while the second is
the dynamics of a slime mold (Physarum polycephalum). Both of these dynamics
are geared towards finding a minimum l1-norm solution in an affine subspace.
Despite its simplicity the convergence of the IRLS method has been shown only
for a certain regularization of it and remains an important open problem. Our
first result shows that the two dynamics are projections of the same dynamical
system in higher dimensions. As a consequence, and building on the recent work
on Physarum dynamics, we are able to prove convergence and obtain complexity
bounds for a damped version of the IRLS algorithm.
Personalized Course Sequence Recommendations
Authors: Jie Xu, Tianwei Xing, Mihaela van der Schaar
Date: 2015-12-30 00:00:00
Tasks: Multi-Armed Bandits

Given the variability in student learning it is becoming increasingly
important to tailor courses as well as course sequences to student needs. This
paper presents a systematic methodology for offering personalized course
sequence recommendations to students. First, a forward-search
backward-induction algorithm is developed that can optimally select course
sequences to decrease the time required for a student to graduate. The
algorithm accounts for prerequisite requirements (typically present in higher
level education) and course availability. Second, using the tools of
multi-armed bandits, an algorithm is developed that can optimally recommend a
course sequence that both reduces the time to graduate while also increasing
the overall GPA of the student. The algorithm dynamically learns how students
with different contextual backgrounds perform for given course sequences and
then recommends an optimal course sequence for new students. Using real-world
student data from the UCLA Mechanical and Aerospace Engineering department, we
illustrate how the proposed algorithms outperform other methods that do not
include student contextual information when making course sequence
recommendations.
Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding
Authors: Jaeyong Sung, Seok Hyun Jin, Ian Lenz, Ashutosh Saxena
Date: 2016-01-12 00:00:00
Tasks: Structured Prediction

There is a large variety of objects and appliances in human environments,
such as stoves, coffee dispensers, juice extractors, and so on. It is
challenging for a roboticist to program a robot for each of these object types
and for each of their instantiations. In this work, we present a novel approach
to manipulation planning based on the idea that many household objects share
similarly-operated object parts. We formulate the manipulation planning as a
structured prediction problem and learn to transfer manipulation strategy
across different objects by embedding point-cloud, natural language, and
manipulation trajectory data into a shared embedding space using a deep neural
network. In order to learn semantically meaningful spaces throughout our
network, we introduce a method for pre-training its lower layers for multimodal
feature embedding and a method for fine-tuning this embedding space using a
loss-based margin. In order to collect a large number of manipulation
demonstrations for different objects, we develop a new crowd-sourcing platform
called Robobarista. We test our model on our dataset consisting of 116 objects
and appliances with 249 parts along with 250 language instructions, for which
there are 1225 crowd-sourced manipulation demonstrations. We further show that
our robot with our model can even prepare a cup of a latte with appliances it
has never seen before.
Inference rules for RDF(S) and OWL in N3Logic
Authors: Dominik Tomaszuk
Date: 2016-01-11 00:00:00

This paper presents inference rules for Resource Description Framework (RDF),
RDF Schema (RDFS) and Web Ontology Language (OWL). Our formalization is based
on Notation 3 Logic, which extended RDF by logical symbols and created Semantic
Web logic for deductive RDF graph stores. We also propose OWL-P that is a
lightweight formalism of OWL and supports soft inferences by omitting complex
language constructs.
Evaluating the Performance of a Speech Recognition based System
Authors: Vinod Kumar Pandey, Sunil Kumar Kopparapu
Date: 2016-01-11 00:00:00
Tasks: speech-recognition, Speech Recognition

Speech based solutions have taken center stage with growth in the services
industry where there is a need to cater to a very large number of people from
all strata of the society. While natural language speech interfaces are the
talk in the research community, yet in practice, menu based speech solutions
thrive. Typically in a menu based speech solution the user is required to
respond by speaking from a closed set of words when prompted by the system. A
sequence of human speech response to the IVR prompts results in the completion
of a transaction. A transaction is deemed successful if the speech solution can
correctly recognize all the spoken utterances of the user whenever prompted by
the system. The usual mechanism to evaluate the performance of a speech
solution is to do an extensive test of the system by putting it to actual
people use and then evaluating the performance by analyzing the logs for
successful transactions. This kind of evaluation could lead to dissatisfied
test users especially if the performance of the system were to result in a poor
transaction completion rate. To negate this the Wizard of Oz approach is
adopted during evaluation of a speech system. Overall this kind of evaluations
is an expensive proposition both in terms of time and cost. In this paper, we
propose a method to evaluate the performance of a speech solution without
actually putting it to people use. We first describe the methodology and then
show experimentally that this can be used to identify the performance
bottlenecks of the speech solution even before the system is actually used thus
saving evaluation time and expenses.
Investigating gated recurrent neural networks for speech synthesis
Authors: Zhizheng Wu, Simon King
Date: 2016-01-11 00:00:00
Tasks: Speech Synthesis

Recently, recurrent neural networks (RNNs) as powerful sequence models have
re-emerged as a potential acoustic model for statistical parametric speech
synthesis (SPSS). The long short-term memory (LSTM) architecture is
particularly attractive because it addresses the vanishing gradient problem in
standard RNNs, making them easier to train. Although recent studies have
demonstrated that LSTMs can achieve significantly better performance on SPSS
than deep feed-forward neural networks, little is known about why. Here we
attempt to answer two questions: a) why do LSTMs work well as a sequence model
for SPSS; b) which component (e.g., input gate, output gate, forget gate) is
most important. We present a visual analysis alongside a series of experiments,
resulting in a proposal for a simplified architecture. The simplified
architecture has significantly fewer parameters than an LSTM, thus reducing
generation complexity considerably without degrading quality.
Optimal Copula Transport for Clustering Multivariate Time Series
Authors: Gautier Marti, Frank Nielsen, Philippe Donnat
Date: 2015-09-27 00:00:00
Tasks: Clustering, Clustering Multivariate Time Series, Time Series, Time Series Analysis

This paper presents a new methodology for clustering multivariate time series
leveraging optimal transport between copulas. Copulas are used to encode both
(i) intra-dependence of a multivariate time series, and (ii) inter-dependence
between two time series. Then, optimal copula transport allows us to define two
distances between multivariate time series: (i) one for measuring
intra-dependence dissimilarity, (ii) another one for measuring inter-dependence
dissimilarity based on a new multivariate dependence coefficient which is
robust to noise, deterministic, and which can target specified dependencies.
Client Profiling for an Anti-Money Laundering System
Authors: Claudio Alexandre, João Balsa
Date: 2015-10-03 00:00:00
Tasks: Classification, General Classification

We present a data mining approach for profiling bank clients in order to
support the process of detection of anti-money laundering operations. We first
present the overall system architecture, and then focus on the relevant
component for this paper. We detail the experiments performed on real world
data from a financial institution, which allowed us to group clients in
clusters and then generate a set of classification rules. We discuss the
relevance of the founded client profiles and of the generated classification
rules. According to the defined overall agent-based architecture, these rules
will be incorporated in the knowledge base of the intelligent agents
responsible for the signaling of suspicious transactions.
How to learn a graph from smooth signals
Authors: Vassilis Kalofolias
Date: 2016-01-11 00:00:00
Tasks: Graph Learning

We propose a framework that learns the graph structure underlying a set of
smooth signals. Given $X\in\mathbb{R}^{m\times n}$ whose rows reside on the
vertices of an unknown graph, we learn the edge weights
$w\in\mathbb{R}_+^{m(m-1)/2}$ under the smoothness assumption that
$\text{tr}{X^\top LX}$ is small. We show that the problem is a weighted
$\ell$-1 minimization that leads to naturally sparse solutions. We point out
how known graph learning or construction techniques fall within our framework
and propose a new model that performs better than the state of the art in many
settings. We present efficient, scalable primal-dual based algorithms for both
our model and the previous state of the art, and evaluate their performance on
artificial and real data.
Trans-gram, Fast Cross-lingual Word-embeddings
Authors: Jocelyn Coulmance, Jean-Marc Marty, Guillaume Wenzek, Amine Benhalloum
Date: 2016-01-11 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Cross-Lingual Word Embeddings, General Classification, Sentence, text-classification, Text Classification, Translation, Word Embeddings, Word Translation

We introduce Trans-gram, a simple and computationally-efficient method to
simultaneously learn and align wordembeddings for a variety of languages, using
only monolingual data and a smaller set of sentence-aligned data. We use our
new method to compute aligned wordembeddings for twenty-one languages using
English as a pivot language. We show that some linguistic features are aligned
across languages for which we do not have aligned data, even though those
properties do not exist in the pivot language. We also achieve state of the art
results on standard cross-lingual text classification and word translation
tasks.
Facial Expression Recognition in the Wild using Rich Deep Features
Authors: Abubakrelsedik Karali, Ahmad Bassiouny, Motaz El-Saban
Date: 2016-01-11 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Facial Expression Recognition is an active area of research in computer
vision with a wide range of applications. Several approaches have been
developed to solve this problem for different benchmark datasets. However,
Facial Expression Recognition in the wild remains an area where much work is
still needed to serve real-world applications. To this end, in this paper we
present a novel approach towards facial expression recognition. We fuse rich
deep features with domain knowledge through encoding discriminant facial
patches. We conduct experiments on two of the most popular benchmark datasets;
CK and TFE. Moreover, we present a novel dataset that, unlike its precedents,
consists of natural - not acted - expression images. Experimental results show
that our approach achieves state-of-the-art results over standard benchmarks
and our own dataset
Git4Voc: Git-based Versioning for Collaborative Vocabulary Development
Authors: Lavdim Halilaj, Irlán Grangel-González, Gökhan Coskun, Sören Auer
Date: 2016-01-11 00:00:00
Tasks: Data Integration

Collaborative vocabulary development in the context of data integration is
the process of finding consensus between the experts of the different systems
and domains. The complexity of this process is increased with the number of
involved people, the variety of the systems to be integrated and the dynamics
of their domain. In this paper we advocate that the realization of a powerful
version control system is the heart of the problem. Driven by this idea and the
success of Git in the context of software development, we investigate the
applicability of Git for collaborative vocabulary development. Even though
vocabulary development and software development have much more similarities
than differences there are still important differences. These need to be
considered within the development of a successful versioning and collaboration
system for vocabulary development. Therefore, this paper starts by presenting
the challenges we were faced with during the creation of vocabularies
collaboratively and discusses its distinction to software development. Based on
these insights we propose Git4Voc which comprises guidelines how Git can be
adopted to vocabulary development. Finally, we demonstrate how Git hooks can be
implemented to go beyond the plain functionality of Git by realizing
vocabulary-specific features like syntactic validation and semantic diffs.
The Effects of Age, Gender and Region on Non-standard Linguistic Variation in Online Social Networks
Authors: Claudia Peersman, Walter Daelemans, Reinhild Vandekerckhove, Bram Vandekerckhove, Leona Van Vaerenbergh
Date: 2016-01-11 00:00:00

We present a corpus-based analysis of the effects of age, gender and region
of origin on the production of both "netspeak" or "chatspeak" features and
regional speech features in Flemish Dutch posts that were collected from a
Belgian online social network platform. The present study shows that combining
quantitative and qualitative approaches is essential for understanding
non-standard linguistic variation in a CMC corpus. It also presents a
methodology that enables the systematic study of this variation by including
all non-standard words in the corpus. The analyses resulted in a convincing
illustration of the Adolescent Peak Principle. In addition, our approach
revealed an intriguing correlation between the use of regional speech features
and chatspeak features.
A Parameter-free Affinity Based Clustering
Authors: Bhaskar Mukhoty, Ruchir Gupta, Y. N. Singh
Date: 2015-07-20 00:00:00
Tasks: Clustering

Several methods have been proposed to estimate the number of clusters in a
dataset; the basic ideal behind all of them has been to study an index that
measures inter-cluster separation and intra-cluster cohesion over a range of
cluster numbers and report the number which gives an optimum value of the
index. In this paper we propose a simple, parameter free approach that is like
human cognition to form clusters, where closely lying points are easily
identified to form a cluster and total number of clusters are revealed. To
identify closely lying points, affinity of two points is defined as a function
of distance and a threshold affinity is identified, above which two points in a
dataset are likely to be in the same cluster. Well separated clusters are
identified even in the presence of outliers, whereas for not so well separated
dataset, final number of clusters are estimated and the detected clusters are
merged to produce the final clusters. Experiments performed with several large
dimensional synthetic and real datasets show good results with robustness to
noise and density variation within dataset.
Implicit Look-alike Modelling in Display Ads: Transfer Collaborative Filtering to CTR Estimation
Authors: Wei-Nan Zhang, Lingxi Chen, Jun Wang
Date: 2016-01-11 00:00:00
Tasks: Collaborative Filtering, Transfer Learning

User behaviour targeting is essential in online advertising. Compared with
sponsored search keyword targeting and contextual advertising page content
targeting, user behaviour targeting builds users' interest profiles via
tracking their online behaviour and then delivers the relevant ads according to
each user's interest, which leads to higher targeting accuracy and thus more
improved advertising performance. The current user profiling methods include
building keywords and topic tags or mapping users onto a hierarchical taxonomy.
However, to our knowledge, there is no previous work that explicitly
investigates the user online visits similarity and incorporates such similarity
into their ad response prediction. In this work, we propose a general framework
which learns the user profiles based on their online browsing behaviour, and
transfers the learned knowledge onto prediction of their ad response.
Technically, we propose a transfer learning model based on the probabilistic
latent factor graphic models, where the users' ad response profiles are
generated from their online browsing profiles. The large-scale experiments
based on real-world data demonstrate significant improvement of our solution
over some strong baselines.
Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction
Authors: Wei-Nan Zhang, Tianming Du, Jun Wang
Date: 2016-01-11 00:00:00
Tasks: Click-Through Rate Prediction

Predicting user responses, such as click-through rate and conversion rate,
are critical in many web applications including web search, personalised
recommendation, and online advertising. Different from continuous raw features
that we usually found in the image and audio domains, the input features in web
space are always of multi-field and are mostly discrete and categorical while
their dependencies are little known. Major user response prediction models have
to either limit themselves to linear models or require manually building up
high-order combination features. The former loses the ability of exploring
feature interactions, while the latter results in a heavy computation in the
large feature space. To tackle the issue, we propose two novel models using
deep neural networks (DNNs) to automatically learn effective patterns from
categorical feature interactions and make predictions of users' ad clicks. To
get our DNNs efficiently work, we propose to leverage three feature
transformation methods, i.e., factorisation machines (FMs), restricted
Boltzmann machines (RBMs) and denoising auto-encoders (DAEs). This paper
presents the structure of our models and their efficient training algorithms.
The large-scale experiments with real-world data demonstrate that our methods
work better than major state-of-the-art models.
Highway Long Short-Term Memory RNNs for Distant Speech Recognition
Authors: Yu Zhang, Guoguo Chen, Dong Yu, Kaisheng Yao, Sanjeev Khudanpur, James Glass
Date: 2015-10-30 00:00:00
Tasks: Distant Speech Recognition, speech-recognition, Speech Recognition

In this paper, we extend the deep long short-term memory (DLSTM) recurrent
neural networks by introducing gated direct connections between memory cells in
adjacent layers. These direct links, called highway connections, enable
unimpeded information flow across different layers and thus alleviate the
gradient vanishing problem when building deeper LSTMs. We further introduce the
latency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole
history while keeping the latency under control. Efficient algorithms are
proposed to train these novel networks using both frame and sequence
discriminative criteria. Experiments on the AMI distant speech recognition
(DSR) task indicate that we can train deeper LSTMs and achieve better
improvement from sequence training with highway LSTMs (HLSTMs). Our novel model
obtains $43.9/47.7\%$ WER on AMI (SDM) dev and eval sets, outperforming all
previous works. It beats the strong DNN and DLSTM baselines with $15.7\%$ and
$5.3\%$ relative improvement respectively.