pwc-archive/papers-with-abstracts huggingface dataset:
TwiSE at SemEval-2016 Task 4: Twitter Sentiment Classification
Authors: Georgios Balikas, Massih-Reza Amini
Date: 2016-06-14 00:00:00
Proceeding: SEMEVAL 2016 6
Tasks: Classification, Ensemble Learning, General Classification, Sentiment Analysis, Sentiment Classification

This paper describes the participation of the team "TwiSE" in the SemEval
2016 challenge. Specifically, we participated in Task 4, namely "Sentiment
Analysis in Twitter" for which we implemented sentiment classification systems
for subtasks A, B, C and D. Our approach consists of two steps. In the first
step, we generate and validate diverse feature sets for twitter sentiment
evaluation, inspired by the work of participants of previous editions of such
challenges. In the second step, we focus on the optimization of the evaluation
measures of the different subtasks. To this end, we examine different learning
strategies by validating them on the data provided by the task organisers. For
our final submissions we used an ensemble learning approach (stacked
generalization) for Subtask A and single linear models for the rest of the
subtasks. In the official leaderboard we were ranked 9/35, 8/19, 1/11 and 2/14
for subtasks A, B, C and D respectively.\footnote{We make the code available
for research purposes at
\url{https://github.com/balikasg/SemEval2016-Twitter\_Sentiment\_Evaluation}.}
Digits that are not: Generating new types through deep neural nets
Authors: Akın Kazakçıand Mehdi Cherti, Balázs Kégl
Date: 2016-06-14 00:00:00

For an artificial creative agent, an essential driver of the search for
novelty is a value function which is often provided by the system designer or
users. We argue that an important barrier for progress in creativity research
is the inability of these systems to develop their own notion of value for
novelty. We propose a notion of knowledge-driven creativity that circumvent the
need for an externally imposed value function, allowing the system to explore
based on what it has learned from a set of referential objects. The concept is
illustrated by a specific knowledge model provided by a deep generative
autoencoder. Using the described system, we train a knowledge model on a set of
digit images and we use the same model to build coherent sets of new digits
that do not belong to known digit types.
Optimizing Spectral Learning for Parsing
Authors: Shashi Narayan, Shay B. Cohen
Date: 2016-06-07 00:00:00
Proceeding: ACL 2016 8

We describe a search algorithm for optimizing the number of latent states
when estimating latent-variable PCFGs with spectral methods. Our results show
that contrary to the common belief that the number of latent states for each
nonterminal in an L-PCFG can be decided in isolation with spectral methods,
parsing results significantly improve if the number of latent states for each
nonterminal is globally optimized, while taking into account interactions
between the different nonterminals. In addition, we contribute an empirical
analysis of spectral algorithms on eight morphologically rich languages:
Basque, French, German, Hebrew, Hungarian, Korean, Polish and Swedish. Our
results show that our estimation consistently performs better or close to
coarse-to-fine expectation-maximization techniques for these languages.
LLFR: A Lanczos-Based Latent Factor Recommender for Big Data Scenarios
Authors: Maria Kalantzi
Date: 2016-06-14 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

The purpose if this master's thesis is to study and develop a new algorithmic
framework for Collaborative Filtering to produce recommendations in the top-N
recommendation problem. Thus, we propose Lanczos Latent Factor Recommender
(LLFR); a novel "big data friendly" collaborative filtering algorithm for top-N
recommendation. Using a computationally efficient Lanczos-based procedure, LLFR
builds a low dimensional item similarity model, that can be readily exploited
to produce personalized ranking vectors over the item space. A number of
experiments on real datasets indicate that LLFR outperforms other
state-of-the-art top-N recommendation methods from a computational as well as a
qualitative perspective. Our experimental results also show that its relative
performance gains, compared to competing methods, increase as the data get
sparser, as in the Cold Start Problem. More specifically, this is true both
when the sparsity is generalized - as in the New Community Problem, a very
common problem faced by real recommender systems in their beginning stages,
when there is not sufficient number of ratings for the collaborative filtering
algorithms to uncover similarities between items or users - and in the very
interesting case where the sparsity is localized in a small fraction of the
dataset - as in the New Users Problem, where new users are introduced to the
system, they have not rated many items and thus, the CF algorithm can not make
reliable personalized recommendations yet.
Calibration of Phone Likelihoods in Automatic Speech Recognition
Authors: David A. van Leeuwen, Joost van Doremalen
Date: 2016-06-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), scoring rule, speech-recognition, Speech Recognition

In this paper we study the probabilistic properties of the posteriors in a
speech recognition system that uses a deep neural network (DNN) for acoustic
modeling. We do this by reducing Kaldi's DNN shared pdf-id posteriors to phone
likelihoods, and using test set forced alignments to evaluate these using a
calibration sensitive metric. Individual frame posteriors are in principle
well-calibrated, because the DNN is trained using cross entropy as the
objective function, which is a proper scoring rule. When entire phones are
assessed, we observe that it is best to average the log likelihoods over the
duration of the phone. Further scaling of the average log likelihoods by the
logarithm of the duration slightly improves the calibration, and this
improvement is retained when tested on independent test data.
Viral Search algorithm
Authors: Matteo Gardini
Date: 2016-06-14 00:00:00

The article, after a brief introduction on genetic algorithms and their
functioning, presents a kind of genetic algorithm called Viral Search. We
present the key concepts, we formally derive the algorithm and we perform
numerical tests designed to illustrate the potential and limits.
Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series
Authors: Maximilian Soelch, Justin Bayer, Marvin Ludersdorfer, Patrick van der Smagt
Date: 2016-02-23 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis, Variational Inference

Approximate variational inference has shown to be a powerful tool for
modeling unknown complex probability distributions. Recent advances in the
field allow us to learn probabilistic models of sequences that actively exploit
spatial and temporal structure. We apply a Stochastic Recurrent Network (STORN)
to learn robot time series data. Our evaluation demonstrates that we can
robustly detect anomalies both off- and on-line.
Cross-Lingual Morphological Tagging for Low-Resource Languages
Authors: Jan Buys, Jan A. Botha
Date: 2016-06-14 00:00:00
Proceeding: ACL 2016 8
Tasks: Morphological Tagging

Morphologically rich languages often lack the annotated linguistic resources
required to develop accurate natural language processing tools. We propose
models suitable for training morphological taggers with rich tagsets for
low-resource languages without using direct supervision. Our approach extends
existing approaches of projecting part-of-speech tags across languages, using
bitext to infer constraints on the possible tags for a given word type or
token. We propose a tagging model using Wsabie, a discriminative
embedding-based model with rank-based learning. In our evaluation on 11
languages, on average this model performs on par with a baseline
weakly-supervised HMM, while being more scalable. Multilingual experiments show
that the method performs best when projecting between related language pairs.
Despite the inherently lossy projection, we show that the morphological tags
predicted by our models improve the downstream performance of a parser by +0.6
LAS on average.
Exact and efficient top-K inference for multi-target prediction by querying separable linear relational models
Authors: Michiel Stock, Krzysztof Dembczynski, Bernard De Baets, Willem Waegeman
Date: 2016-06-14 00:00:00
Tasks: BIG-bench Machine Learning, Collaborative Filtering, Information Retrieval, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Prediction, Retrieval

Many complex multi-target prediction problems that concern large target
spaces are characterised by a need for efficient prediction strategies that
avoid the computation of predictions for all targets explicitly. Examples of
such problems emerge in several subfields of machine learning, such as
collaborative filtering, multi-label classification, dyadic prediction and
biological network inference. In this article we analyse efficient and exact
algorithms for computing the top-$K$ predictions in the above problem settings,
using a general class of models that we refer to as separable linear relational
models. We show how to use those inference algorithms, which are modifications
of well-known information retrieval methods, in a variety of machine learning
settings. Furthermore, we study the possibility of scoring items incompletely,
while still retaining an exact top-K retrieval. Experimental results in several
application domains reveal that the so-called threshold algorithm is very
scalable, performing often many orders of magnitude more efficiently than the
naive approach.
Efficient Pairwise Learning Using Kernel Ridge Regression: an Exact Two-Step Method
Authors: Michiel Stock, Tapio Pahikkala, Antti Airola, Bernard De Baets, Willem Waegeman
Date: 2016-06-14 00:00:00
Tasks: Collaborative Filtering, Matrix Completion, Multi-Task Learning, Prediction, regression, Transfer Learning, Zero-Shot Learning

Pairwise learning or dyadic prediction concerns the prediction of properties
for pairs of objects. It can be seen as an umbrella covering various machine
learning problems such as matrix completion, collaborative filtering,
multi-task learning, transfer learning, network prediction and zero-shot
learning. In this work we analyze kernel-based methods for pairwise learning,
with a particular focus on a recently-suggested two-step method. We show that
this method offers an appealing alternative for commonly-applied
Kronecker-based methods that model dyads by means of pairwise feature
representations and pairwise kernels. In a series of theoretical results, we
establish correspondences between the two types of methods in terms of linear
algebra and spectral filtering, and we analyze their statistical consistency.
In addition, the two-step method allows us to establish novel algorithmic
shortcuts for efficient training and validation on very large datasets. Putting
those properties together, we believe that this simple, yet powerful method can
become a standard tool for many problems. Extensive experimental results for a
range of practical settings are reported.
Context Trees: Augmenting Geospatial Trajectories with Context
Authors: Alasdair Thomason, Nathan Griffiths, Victor Sanchez
Date: 2016-06-14 00:00:00

Exposing latent knowledge in geospatial trajectories has the potential to
provide a better understanding of the movements of individuals and groups.
Motivated by such a desire, this work presents the context tree, a new
hierarchical data structure that summarises the context behind user actions in
a single model. We propose a method for context tree construction that augments
geospatial trajectories with land usage data to identify such contexts. Through
evaluation of the construction method and analysis of the properties of
generated context trees, we demonstrate the foundation for understanding and
modelling behaviour afforded. Summarising user contexts into a single data
structure gives easy access to information that would otherwise remain latent,
providing the basis for better understanding and predicting the actions and
behaviours of individuals and groups. Finally, we also present a method for
pruning context trees, for use in applications where it is desirable to reduce
the size of the tree while retaining useful information.
Local Canonical Correlation Analysis for Nonlinear Common Variables Discovery
Authors: Or Yair, Ronen Talmon
Date: 2016-06-14 00:00:00

In this paper, we address the problem of hidden common variables discovery
from multimodal data sets of nonlinear high-dimensional observations. We
present a metric based on local applications of canonical correlation analysis
(CCA) and incorporate it in a kernel-based manifold learning technique.We show
that this metric discovers the hidden common variables underlying the
multimodal observations by estimating the Euclidean distance between them. Our
approach can be viewed both as an extension of CCA to a nonlinear setting as
well as an extension of manifold learning to multiple data sets. Experimental
results show that our method indeed discovers the common variables underlying
high-dimensional nonlinear observations without assuming prior rigid model
assumptions.
Variational Inference with Normalizing Flows
Authors: Danilo Jimenez Rezende, Shakir Mohamed
Date: 2015-05-21 00:00:00
Tasks: Variational Inference

The choice of approximate posterior distribution is one of the core problems
in variational inference. Most applications of variational inference employ
simple families of posterior approximations in order to allow for efficient
inference, focusing on mean-field or other simple structured approximations.
This restriction has a significant impact on the quality of inferences made
using variational methods. We introduce a new approach for specifying flexible,
arbitrarily complex and scalable approximate posterior distributions. Our
approximations are distributions constructed through a normalizing flow,
whereby a simple initial density is transformed into a more complex one by
applying a sequence of invertible transformations until a desired level of
complexity is attained. We use this view of normalizing flows to develop
categories of finite and infinitesimal flows and provide a unified view of
approaches for constructing rich posterior approximations. We demonstrate that
the theoretical advantages of having posteriors that better match the true
posterior, combined with the scalability of amortized variational approaches,
provides a clear improvement in performance and applicability of variational
inference.
Average Drift Analysis and Population Scalability
Authors: Jun He, Xin Yao
Date: 2013-08-14 00:00:00
Tasks: Evolutionary Algorithms

This paper aims to study how the population size affects the computation time
of evolutionary algorithms in a rigorous way. The computation time of an
evolutionary algorithm can be measured by either the expected number of
generations (hitting time) or the expected number of fitness evaluations
(running time) to find an optimal solution. Population scalability is the ratio
of the expected hitting time between a benchmark algorithm and an algorithm
using a larger population size. Average drift analysis is presented for
comparing the expected hitting time of two algorithms and estimating lower and
upper bounds on population scalability. Several intuitive beliefs are
rigorously analysed. It is prove that (1) using a population sometimes
increases rather than decreases the expected hitting time; (2) using a
population cannot shorten the expected running time of any elitist evolutionary
algorithm on unimodal functions in terms of the time-fitness landscape, but
this is not true in terms of the distance-based fitness landscape; (3) using a
population cannot always reduce the expected running time on fully-deceptive
functions, which depends on the benchmark algorithm using elitist selection or
random selection.
Sorting out typicality with the inverse moment matrix SOS polynomial
Authors: Jean-Bernard Lasserre, Edouard Pauwels
Date: 2016-06-13 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Intrusion Detection, Network Intrusion Detection

We study a surprising phenomenon related to the representation of a cloud of
data points using polynomials. We start with the previously unnoticed empirical
observation that, given a collection (a cloud) of data points, the sublevel
sets of a certain distinguished polynomial capture the shape of the cloud very
accurately. This distinguished polynomial is a sum-of-squares (SOS) derived in
a simple manner from the inverse of the empirical moment matrix. In fact, this
SOS polynomial is directly related to orthogonal polynomials and the
Christoffel function. This allows to generalize and interpret extremality
properties of orthogonal polynomials and to provide a mathematical rationale
for the observed phenomenon. Among diverse potential applications, we
illustrate the relevance of our results on a network intrusion detection task
for which we obtain performances similar to existing dedicated methods reported
in the literature.
Neural Associative Memory for Dual-Sequence Modeling
Authors: Dirk Weissenborn
Date: 2016-06-13 00:00:00
Proceeding: WS 2016 8
Tasks: Natural Language Inference

Many important NLP problems can be posed as dual-sequence or
sequence-to-sequence modeling tasks. Recent advances in building end-to-end
neural architectures have been highly successful in solving such tasks. In this
work we propose a new architecture for dual-sequence modeling that is based on
associative memory. We derive AM-RNNs, a recurrent associative memory (AM)
which augments generic recurrent neural networks (RNN). This architecture is
extended to the Dual AM-RNN which operates on two AMs at once. Our models
achieve very competitive results on textual entailment. A qualitative analysis
demonstrates that long range dependencies between source and target-sequence
can be bridged effectively using Dual AM-RNNs. However, an initial experiment
on auto-encoding reveals that these benefits are not exploited by the system
when learning to solve sequence-to-sequence tasks which indicates that
additional supervision or regularization is needed.
DCNNs on a Diet: Sampling Strategies for Reducing the Training Set Size
Authors: Maya Kabkab, Azadeh Alavi, Rama Chellappa
Date: 2016-06-14 00:00:00
Tasks: General Classification

Large-scale supervised classification algorithms, especially those based on
deep convolutional neural networks (DCNNs), require vast amounts of training
data to achieve state-of-the-art performance. Decreasing this data requirement
would significantly speed up the training process and possibly improve
generalization. Motivated by this objective, we consider the task of adaptively
finding concise training subsets which will be iteratively presented to the
learner. We use convex optimization methods, based on an objective criterion
and feedback from the current performance of the classifier, to efficiently
identify informative samples to train on. We propose an algorithm to decompose
the optimization problem into smaller per-class problems, which can be solved
in parallel. We test our approach on standard classification tasks and
demonstrate its effectiveness in decreasing the training set size without
compromising performance. We also show that our approach can make the
classifier more robust in the presence of label noise and class imbalance.
Conditional Generative Moment-Matching Networks
Authors: Yong Ren, Jialian Li, Yucen Luo, Jun Zhu
Date: 2016-06-14 00:00:00
Proceeding: NeurIPS 2016 12

Maximum mean discrepancy (MMD) has been successfully applied to learn deep
generative models for characterizing a joint distribution of variables via
kernel mean embedding. In this paper, we present conditional generative moment-
matching networks (CGMMN), which learn a conditional distribution given some
input variables based on a conditional maximum mean discrepancy (CMMD)
criterion. The learning is performed by stochastic gradient descent with the
gradient calculated by back-propagation. We evaluate CGMMN on a wide range of
tasks, including predictive modeling, contextual generation, and Bayesian dark
knowledge, which distills knowledge from a Bayesian model by learning a
relatively small CGMMN student network. Our results demonstrate competitive
performance in all the tasks.
Word Representation Models for Morphologically Rich Languages in Neural Machine Translation
Authors: Ekaterina Vylomova, Trevor Cohn, Xuanli He, Gholamreza Haffari
Date: 2016-06-14 00:00:00
Proceeding: WS 2017 9
Tasks: Hard Attention, Machine Translation, Translation

Dealing with the complex word forms in morphologically rich languages is an
open problem in language processing, and is particularly important in
translation. In contrast to most modern neural systems of translation, which
discard the identity for rare words, in this paper we propose several
architectures for learning word representations from character and morpheme
level word decompositions. We incorporate these representations in a novel
machine translation model which jointly learns word alignments and translations
via a hard attention mechanism. Evaluating on translating from several
morphologically rich languages into English, we show consistent improvements
over strong baseline methods, of between 1 and 1.5 BLEU points.
Spreadsheet Probabilistic Programming
Authors: Mike Wu, Yura Perov, Frank Wood, Hongseok Yang
Date: 2016-06-14 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty, Probabilistic Programming, Variational Inference

Spreadsheet workbook contents are simple programs. Because of this,
probabilistic programming techniques can be used to perform Bayesian inversion
of spreadsheet computations. What is more, existing execution engines in
spreadsheet applications such as Microsoft Excel can be made to do this using
only built-in functionality. We demonstrate this by developing a native Excel
implementation of both a particle Markov Chain Monte Carlo variant and
black-box variational inference for spreadsheet probabilistic programming. The
resulting engine performs probabilistically coherent inference over spreadsheet
computations, notably including spreadsheets that include user-defined
black-box functions. Spreadsheet engines that choose to integrate the
functionality we describe in this paper will give their users the ability to
both easily develop probabilistic models and maintain them over time by
including actuals via a simple user-interface mechanism. For spreadsheet
end-users this would mean having access to efficient and probabilistically
coherent probabilistic modeling and inference for use in all kinds of decision
making under uncertainty.
A Systematic Approach to Blocking Convolutional Neural Networks
Authors: Xuan Yang, Jing Pu, Blaine Burton Rister, Nikhil Bhagdikar, Stephen Richardson, Shahar Kvatinsky, Jonathan Ragan-Kelley, Ardavan Pedram, Mark Horowitz
Date: 2016-06-14 00:00:00
Tasks: Blocking, CPU

Convolutional Neural Networks (CNNs) are the state of the art solution for
many computer vision problems, and many researchers have explored optimized
implementations. Most implementations heuristically block the computation to
deal with the large data sizes and high data reuse of CNNs. This paper explores
how to block CNN computations for memory locality by creating an analytical
model for CNN-like loop nests. Using this model we automatically derive
optimized blockings for common networks that improve the energy efficiency of
custom hardware implementations by up to an order of magnitude. Compared to
traditional CNN CPU implementations based on highly-tuned, hand-optimized BLAS
libraries,our x86 programs implementing the optimal blocking reduce the number
of memory accesses by up to 90%.