pwc-archive/papers-with-abstracts huggingface dataset:
A Comparison of Lauritzen-Spiegelhalter, Hugin, and Shenoy-Shafer Architectures for Computing Marginals of Probability Distributions
Authors: Vasilica Lepar, Prakash P. Shenoy
Date: 2013-01-30 00:00:00
Tasks: Computational Efficiency

In the last decade, several architectures have been proposed for exact
computation of marginals using local computation. In this paper, we compare
three architectures - Lauritzen-Spiegelhalter, Hugin, and Shenoy-Shafer - from
the perspective of graphical structure for message propagation, message-passing
scheme, computational efficiency, and storage efficiency.
ADADELTA: An Adaptive Learning Rate Method
Authors: Matthew D. Zeiler
Date: 2012-12-22 00:00:00
Tasks: General Classification

We present a novel per-dimension learning rate method for gradient descent
called ADADELTA. The method dynamically adapts over time using only first order
information and has minimal computational overhead beyond vanilla stochastic
gradient descent. The method requires no manual tuning of a learning rate and
appears robust to noisy gradient information, different model architecture
choices, various data modalities and selection of hyperparameters. We show
promising results compared to other methods on the MNIST digit classification
task using a single machine and on a large scale voice dataset in a distributed
cluster environment.
Paper2vec: Citation-Context Based Document Distributed Representation for Scholar Recommendation
Authors: Han Tian, Hankz Hankui Zhuo
Date: 2017-03-20 00:00:00

Due to the availability of references of research papers and the rich
information contained in papers, various citation analysis approaches have been
proposed to identify similar documents for scholar recommendation. Despite of
the success of previous approaches, they are, however, based on co-occurrence
of items. Once there are no co-occurrence items available in documents, they
will not work well. Inspired by distributed representations of words in the
literature of natural language processing, we propose a novel approach to
measuring the similarity of papers based on distributed representations learned
from the citation context of papers. We view the set of papers as the
vocabulary, define the weighted citation context of papers, and convert it to
weight matrix similar to the word-word cooccurrence matrix in natural language
processing. After that we explore a variant of matrix factorization approach to
train distributed representations of papers on the matrix, and leverage the
distributed representations to measure similarities of papers. In the
experiment, we exhibit that our approach outperforms state-of-theart
citation-based approaches by 25%, and better than other distributed
representation based methods.
Related Pins at Pinterest: The Evolution of a Real-World Recommender System
Authors: David C. Liu, Stephanie Rogers, Raymond Shiau, Dmitry Kislyuk, Kevin C. Ma, Zhigang Zhong, Jenny Liu, Yushi Jing
Date: 2017-02-26 00:00:00
Tasks: Recommendation Systems

Related Pins is the Web-scale recommender system that powers over 40% of user
engagement on Pinterest. This paper is a longitudinal study of three years of
its development, exploring the evolution of the system and its components from
prototypes to present state. Each component was originally built with many
constraints on engineering effort and computational resources, so we
prioritized the simplest and highest-leverage solutions. We show how organic
growth led to a complex system and how we managed this complexity. Many
challenges arose while building this system, such as avoiding feedback loops,
evaluating performance, activating content, and eliminating legacy heuristics.
Finally, we offer suggestions for tackling these challenges when engineering
Web-scale recommender systems.
Online Robust Subspace Tracking from Partial Information
Authors: Jun He, Laura Balzano, John C. S. Lui
Date: 2011-09-18 00:00:00
Tasks: Matrix Completion

This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking
Algorithm), an efficient and robust online algorithm for tracking subspaces
from highly incomplete information. The algorithm uses a robust $l^1$-norm cost
function in order to estimate and track non-stationary subspaces when the
streaming data vectors are corrupted with outliers. We apply GRASTA to the
problems of robust matrix completion and real-time separation of background
from foreground in video. In this second application, we show that GRASTA
performs high-quality separation of moving objects from background at
exceptional speeds: In one popular benchmark video example, GRASTA achieves a
rate of 57 frames per second, even when run in MATLAB on a personal laptop.
BPR: Bayesian Personalized Ranking from Implicit Feedback
Authors: Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme
Date: 2012-05-09 00:00:00

Item recommendation is the task of predicting a personalized ranking on a set
of items (e.g. websites, movies, products). In this paper, we investigate the
most common scenario with implicit feedback (e.g. clicks, purchases). There are
many methods for item recommendation from implicit feedback like matrix
factorization (MF) or adaptive knearest-neighbor (kNN). Even though these
methods are designed for the item prediction task of personalized ranking, none
of them is directly optimized for ranking. In this paper we present a generic
optimization criterion BPR-Opt for personalized ranking that is the maximum
posterior estimator derived from a Bayesian analysis of the problem. We also
provide a generic learning algorithm for optimizing models with respect to
BPR-Opt. The learning method is based on stochastic gradient descent with
bootstrap sampling. We show how to apply our method to two state-of-the-art
recommender models: matrix factorization and adaptive kNN. Our experiments
indicate that for the task of personalized ranking our optimization method
outperforms the standard learning techniques for MF and kNN. The results show
the importance of optimizing models for the right criterion.
A Dataset for StarCraft AI \& an Example of Armies Clustering
Authors: Gabriel Synnaeve, Pierre Bessiere
Date: 2012-11-19 00:00:00
Tasks: Clustering, Real-Time Strategy Games, Starcraft

This paper advocates the exploration of the full state of recorded real-time
strategy (RTS) games, by human or robotic players, to discover how to reason
about tactics and strategy. We present a dataset of StarCraft games
encompassing the most of the games' state (not only player's orders). We
explain one of the possible usages of this dataset by clustering armies on
their compositions. This reduction of armies compositions to mixtures of
Gaussian allow for strategic reasoning at the level of the components. We
evaluated this clustering method by predicting the outcomes of battles based on
armies compositions' mixtures components
XML Representation of Constraint Networks: Format XCSP 2.1
Authors: Olivier Roussel, Christophe Lecoutre
Date: 2009-02-13 00:00:00

We propose a new extended format to represent constraint networks using XML.
This format allows us to represent constraints defined either in extension or
in intension. It also allows us to reference global constraints. Any instance
of the problems CSP (Constraint Satisfaction Problem), QCSP (Quantified CSP)
and WCSP (Weighted CSP) can be represented using this format.
Natural Language Processing (almost) from Scratch
Authors: Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa
Date: 2011-03-02 00:00:00
Tasks: Chunking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, Semantic Role Labeling

We propose a unified neural network architecture and learning algorithm that
can be applied to various natural language processing tasks including:
part-of-speech tagging, chunking, named entity recognition, and semantic role
labeling. This versatility is achieved by trying to avoid task-specific
engineering and therefore disregarding a lot of prior knowledge. Instead of
exploiting man-made input features carefully optimized for each task, our
system learns internal representations on the basis of vast amounts of mostly
unlabeled training data. This work is then used as a basis for building a
freely available tagging system with good performance and minimal computational
requirements.
On the difficulty of training Recurrent Neural Networks
Authors: Razvan Pascanu, Tomas Mikolov, Yoshua Bengio
Date: 2012-11-21 00:00:00

There are two widely known issues with properly training Recurrent Neural
Networks, the vanishing and the exploding gradient problems detailed in Bengio
et al. (1994). In this paper we attempt to improve the understanding of the
underlying issues by exploring these problems from an analytical, a geometric
and a dynamical systems perspective. Our analysis is used to justify a simple
yet effective solution. We propose a gradient norm clipping strategy to deal
with exploding gradients and a soft constraint for the vanishing gradients
problem. We validate empirically our hypothesis and proposed solutions in the
experimental section.
Cross-Lingual Adaptation using Structural Correspondence Learning
Authors: Peter Prettenhofer, Benno Stein
Date: 2010-08-04 00:00:00
Tasks: Classification, Domain Adaptation, Machine Translation, Sentiment Analysis, Sentiment Classification, Specificity, Topic Classification, Translation, Word Translation

Cross-lingual adaptation, a special case of domain adaptation, refers to the
transfer of classification knowledge between two languages. In this article we
describe an extension of Structural Correspondence Learning (SCL), a recently
proposed algorithm for domain adaptation, for cross-lingual adaptation. The
proposed method uses unlabeled documents from both languages, along with a word
translation oracle, to induce cross-lingual feature correspondences. From these
correspondences a cross-lingual representation is created that enables the
transfer of classification knowledge from the source to the target language.
The main advantages of this approach over other approaches are its resource
efficiency and task specificity.
  We conduct experiments in the area of cross-language topic and sentiment
classification involving English as source language and German, French, and
Japanese as target languages. The results show a significant improvement of the
proposed method over a machine translation baseline, reducing the relative
error due to cross-lingual adaptation by an average of 30% (topic
classification) and 59% (sentiment classification). We further report on
empirical analyses that reveal insights into the use of unlabeled data, the
sensitivity with respect to important hyperparameters, and the nature of the
induced cross-lingual correspondences.
Detecting Real-World Influence Through Twitter
Authors: Jean-Val{è}re Cossu, Nicolas Dugu{é}, Vincent Labatut
Date: 2015-06-19 00:00:00

In this paper, we investigate the issue of detecting the real-life influence of people based on their Twitter account. We propose an overview of common Twitter features used to characterize such accounts and their activity, and show that these are inefficient in this context. In particular, retweets and followers numbers, and Klout score are not relevant to our analysis. We thus propose several Machine Learning approaches based on Natural Language Processing and Social Network Analysis to label Twitter users as Influencers or not. We also rank them according to a predicted influence level. Our proposals are evaluated over the CLEF RepLab 2014 dataset, and outmatch state-of-the-art ranking methods.
Conscious Machines and Consciousness Oriented Programming
Authors: Norbert Bátfai
Date: 2011-08-14 00:00:00

In this paper, we investigate the following question: how could you write
such computer programs that can work like conscious beings? The motivation
behind this question is that we want to create such applications that can see
the future. The aim of this paper is to provide an overall conceptual framework
for this new approach to machine consciousness. So we introduce a new
programming paradigm called Consciousness Oriented Programming (COP).
Multi-column Deep Neural Networks for Image Classification
Authors: Dan Cireşan, Ueli Meier, Juergen Schmidhuber
Date: 2012-02-13 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Traffic Sign Recognition

Traditional methods of computer vision and machine learning cannot match
human performance on tasks such as the recognition of handwritten digits or
traffic signs. Our biologically plausible deep artificial neural network
architectures can. Small (often minimal) receptive fields of convolutional
winner-take-all neurons yield large network depth, resulting in roughly as many
sparsely connected neural layers as found in mammals between retina and visual
cortex. Only winner neurons are trained. Several deep neural columns become
experts on inputs preprocessed in different ways; their predictions are
averaged. Graphics cards allow for fast training. On the very competitive MNIST
handwriting benchmark, our method is the first to achieve near-human
performance. On a traffic sign recognition benchmark it outperforms humans by a
factor of two. We also improve the state-of-the-art on a plethora of common
image classification benchmarks.
Building high-level features using large scale unsupervised learning
Authors: Quoc V. Le, Marc'Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado, Jeff Dean, Andrew Y. Ng
Date: 2011-12-29 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We consider the problem of building high-level, class-specific feature
detectors from only unlabeled data. For example, is it possible to learn a face
detector using only unlabeled images? To answer this, we train a 9-layered
locally connected sparse autoencoder with pooling and local contrast
normalization on a large dataset of images (the model has 1 billion
connections, the dataset has 10 million 200x200 pixel images downloaded from
the Internet). We train this network using model parallelism and asynchronous
SGD on a cluster with 1,000 machines (16,000 cores) for three days. Contrary to
what appears to be a widely-held intuition, our experimental results reveal
that it is possible to train a face detector without having to label images as
containing a face or not. Control experiments show that this feature detector
is robust not only to translation but also to scaling and out-of-plane
rotation. We also find that the same network is sensitive to other high-level
concepts such as cat faces and human bodies. Starting with these learned
features, we trained our network to obtain 15.8% accuracy in recognizing 20,000
object categories from ImageNet, a leap of 70% relative improvement over the
previous state-of-the-art.
SpeedRead: A Fast Named Entity Recognition Pipeline
Authors: Rami Al-Rfou', Steven Skiena
Date: 2013-01-14 00:00:00
Proceeding: COLING 2012 12
Tasks: BIG-bench Machine Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), POS

Online content analysis employs algorithmic methods to identify entities in
unstructured text. Both machine learning and knowledge-base approaches lie at
the foundation of contemporary named entities extraction systems. However, the
progress in deploying these approaches on web-scale has been been hampered by
the computational cost of NLP over massive text corpora. We present SpeedRead
(SR), a named entity recognition pipeline that runs at least 10 times faster
than Stanford NLP pipeline. This pipeline consists of a high performance Penn
Treebank- compliant tokenizer, close to state-of-art part-of-speech (POS)
tagger and knowledge-based named entity recognizer.
Multi-Dimensional Recurrent Neural Networks
Authors: Alex Graves, Santiago Fernandez, Juergen Schmidhuber
Date: 2007-05-14 00:00:00
Tasks: Handwriting Recognition, Image Segmentation, Semantic Segmentation

Recurrent neural networks (RNNs) have proved effective at one dimensional
sequence learning tasks, such as speech and online handwriting recognition.
Some of the properties that make RNNs suitable for such tasks, for example
robustness to input warping, and the ability to access contextual information,
are also desirable in multidimensional domains. However, there has so far been
no direct way of applying RNNs to data with more than one spatio-temporal
dimension. This paper introduces multi-dimensional recurrent neural networks
(MDRNNs), thereby extending the potential applicability of RNNs to vision,
video processing, medical imaging and many other areas, while avoiding the
scaling problems that have plagued other multi-dimensional models. Experimental
results are provided for two image segmentation tasks.
Path Integral Policy Improvement with Covariance Matrix Adaptation
Authors: Freek Stulp, Olivier Sigaud
Date: 2012-06-18 00:00:00
Tasks: Reinforcement Learning

There has been a recent focus in reinforcement learning on addressing
continuous state and action problems by optimizing parameterized policies. PI2
is a recent example of this approach. It combines a derivation from first
principles of stochastic optimal control with tools from statistical estimation
theory. In this paper, we consider PI2 as a member of the wider family of
methods which share the concept of probability-weighted averaging to
iteratively update parameters to optimize a cost function. We compare PI2 to
other members of the same family - Cross-Entropy Methods and CMAES - at the
conceptual level and in terms of performance. The comparison suggests the
derivation of a novel algorithm which we call PI2-CMA for "Path Integral Policy
Improvement with Covariance Matrix Adaptation". PI2-CMA's main advantage is
that it determines the magnitude of the exploration noise automatically.
Efficient Probabilistic Inference in Generic Neural Networks Trained with Non-Probabilistic Feedback
Authors: A. Emin Orhan, Wei Ji Ma
Date: 2016-01-12 00:00:00

Animals perform near-optimal probabilistic inference in a wide range of
psychophysical tasks. Probabilistic inference requires trial-to-trial
representation of the uncertainties associated with task variables and
subsequent use of this representation. Previous work has implemented such
computations using neural networks with hand-crafted and task-dependent
operations. We show that generic neural networks trained with a simple
error-based learning rule perform near-optimal probabilistic inference in nine
common psychophysical tasks. In a probabilistic categorization task,
error-based learning in a generic network simultaneously explains a monkey's
learning curve and the evolution of qualitative aspects of its choice behavior.
In all tasks, the number of neurons required for a given level of performance
grows sub-linearly with the input population size, a substantial improvement on
previous implementations of probabilistic inference. The trained networks
develop a novel sparsity-based probabilistic population code. Our results
suggest that probabilistic inference emerges naturally in generic neural
networks trained with error-based learning rules.
Advances in Optimizing Recurrent Networks
Authors: Yoshua Bengio, Nicolas Boulanger-Lewandowski, Razvan Pascanu
Date: 2012-12-04 00:00:00

After a more than decade-long period of relatively little research activity
in the area of recurrent neural networks, several new developments will be
reviewed here that have allowed substantial progress both in understanding and
in technical solutions towards more efficient training of recurrent networks.
These advances have been motivated by and related to the optimization issues
surrounding deep learning. Although recurrent networks are extremely powerful
in what they can in principle represent in terms of modelling sequences,their
training is plagued by two aspects of the same issue regarding the learning of
long-term dependencies. Experiments reported here evaluate the use of clipping
gradients, spanning longer time ranges with leaky integration, advanced
momentum techniques, using more powerful output probability models, and
encouraging sparser gradients to help symmetry breaking and credit assignment.
The experiments are performed on text and music data and show off the combined
effects of these techniques in generally improving both training and test
error.
Generative Adversarial Networks for Multimodal Representation Learning in Video Hyperlinking
Authors: Vedran Vukotic, Christian Raymond, Guillaume Gravier
Date: 2017-05-15 00:00:00
Tasks: Information Retrieval, Representation Learning, Retrieval

Continuous multimodal representations suitable for multimodal information
retrieval are usually obtained with methods that heavily rely on multimodal
autoencoders. In video hyperlinking, a task that aims at retrieving video
segments, the state of the art is a variation of two interlocked networks
working in opposing directions. These systems provide good multimodal
embeddings and are also capable of translating from one representation space to
the other. Operating on representation spaces, these networks lack the ability
to operate in the original spaces (text or image), which makes it difficult to
visualize the crossmodal function, and do not generalize well to unseen data.
Recently, generative adversarial networks have gained popularity and have been
used for generating realistic synthetic data and for obtaining high-level,
single-modal latent representation spaces. In this work, we evaluate the
feasibility of using GANs to obtain multimodal representations. We show that
GANs can be used for multimodal representation learning and that they provide
multimodal representations that are superior to representations obtained with
multimodal autoencoders. Additionally, we illustrate the ability of visualizing
crossmodal translations that can provide human-interpretable insights on
learned GAN-based video hyperlinking models.
An Ensemble-based Approach to Click-Through Rate Prediction for Promoted Listings at Etsy
Authors: Kamelia Aryafar, Devin Guillory, Liangjie Hong
Date: 2017-11-04 00:00:00
Tasks: Click-Through Rate Prediction, Ensemble Learning

Etsy is a global marketplace where people across the world connect to make,
buy and sell unique goods. Sellers at Etsy can promote their product listings
via advertising campaigns similar to traditional sponsored search ads.
Click-Through Rate (CTR) prediction is an integral part of online search
advertising systems where it is utilized as an input to auctions which
determine the final ranking of promoted listings to a particular user for each
query. In this paper, we provide a holistic view of Etsy's promoted listings'
CTR prediction system and propose an ensemble learning approach which is based
on historical or behavioral signals for older listings as well as content-based
features for new listings. We obtain representations from texts and images by
utilizing state-of-the-art deep learning techniques and employ multimodal
learning to combine these different signals. We compare the system to
non-trivial baselines on a large-scale real world dataset from Etsy,
demonstrating the effectiveness of the model and strong correlations between
offline experiments and online performance. The paper is also the first
technical overview to this kind of product in e-commerce context.