pwc-archive/papers-with-abstracts huggingface dataset:
Kernel Alignment for Unsupervised Transfer Learning
Authors: Ievgen Redko, Younès Bennani
Date: 2016-10-20 00:00:00
Tasks: Transfer Learning

The ability of a human being to extrapolate previously gained knowledge to
other domains inspired a new family of methods in machine learning called
transfer learning. Transfer learning is often based on the assumption that
objects in both target and source domains share some common feature and/or data
space. In this paper, we propose a simple and intuitive approach that minimizes
iteratively the distance between source and target task distributions by
optimizing the kernel target alignment (KTA). We show that this procedure is
suitable for transfer learning by relating it to Hilbert-Schmidt Independence
Criterion (HSIC) and Quadratic Mutual Information (QMI) maximization. We run
our method on benchmark computer vision data sets and show that it can
outperform some state-of-art methods.
A Growing Long-term Episodic & Semantic Memory
Authors: Marc Pickett, Rami Al-Rfou, Louis Shao, Chris Tar
Date: 2016-10-20 00:00:00
Tasks: Lifelong learning, Transfer Learning

The long-term memory of most connectionist systems lies entirely in the
weights of the system. Since the number of weights is typically fixed, this
bounds the total amount of knowledge that can be learned and stored. Though
this is not normally a problem for a neural network designed for a specific
task, such a bound is undesirable for a system that continually learns over an
open range of domains. To address this, we describe a lifelong learning system
that leverages a fast, though non-differentiable, content-addressable memory
which can be exploited to encode both a long history of sequential episodic
knowledge and semantic knowledge over many episodes for an unbounded number of
domains. This opens the door for investigation into transfer learning, and
leveraging prior knowledge that has been learned over a lifetime of experiences
to new domains.
Clinical Text Prediction with Numerically Grounded Conditional Language Models
Authors: Georgios P. Spithourakis, Steffen E. Petersen, Sebastian Riedel
Date: 2016-10-20 00:00:00
Proceeding: WS 2016 11

Assisted text input techniques can save time and effort and improve text
quality. In this paper, we investigate how grounded and conditional extensions
to standard neural language models can bring improvements in the tasks of word
prediction and completion. These extensions incorporate a structured knowledge
base and numerical values from the text into the context used to predict the
next word. Our automated evaluation on a clinical dataset shows extended models
significantly outperform standard models. Our best system uses both
conditioning and grounding, because of their orthogonal benefits. For word
prediction with a list of 5 suggestions, it improves recall from 25.03% to
71.28% and for word completion it improves keystroke savings from 34.35% to
44.81%, where theoretical bound for this dataset is 58.78%. We also perform a
qualitative investigation of how models with lower perplexity occasionally fare
better at the tasks. We found that at test time numbers have more influence on
the document level than on individual word probabilities.
Retrieving challenging vessel connections in retinal images by line co-occurrence statistics
Authors: Samaneh Abbasi-Sureshjani, Jiong Zhang, Remco Duits, Bart ter Haar Romeny
Date: 2016-10-20 00:00:00
Tasks: Clustering

Natural images contain often curvilinear structures, which might be
disconnected, or partly occluded. Recovering the missing connection of
disconnected structures is an open issue and needs appropriate geometric
reasoning. We propose to find line co-occurrence statistics from the
centerlines of blood vessels in retinal images and show its remarkable
similarity to a well-known probabilistic model for the connectivity pattern in
the primary visual cortex. Furthermore, the probabilistic model is trained from
the data via statistics and used for automated grouping of interrupted vessels
in a spectral clustering based approach. Several challenging image patches are
investigated around junction points, where successful results indicate the
perfect match of the trained model to the profiles of blood vessels in retinal
images. Also, comparisons among several statistical models obtained from
different datasets reveals their high similarity i.e., they are independent of
the dataset. On top of that, the best approximation of the statistical model
with the symmetrized extension of the probabilistic model on the projective
line bundle is found with a least square error smaller than 2%. Apparently, the
direction process on the projective line bundle is a good continuation model
for vessels in retinal images.
Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation
Authors: Qi Ye, Shanxin Yuan, Tae-Kyun Kim
Date: 2016-04-12 00:00:00
Tasks: Hand Pose Estimation, Pose Estimation

Discriminative methods often generate hand poses kinematically implausible,
then generative methods are used to correct (or verify) these results in a
hybrid method. Estimating 3D hand pose in a hierarchy, where the
high-dimensional output space is decomposed into smaller ones, has been shown
effective. Existing hierarchical methods mainly focus on the decomposition of
the output space while the input space remains almost the same along the
hierarchy. In this paper, a hybrid hand pose estimation method is proposed by
applying the kinematic hierarchy strategy to the input space (as well as the
output space) of the discriminative method by a spatial attention mechanism and
to the optimization of the generative method by hierarchical Particle Swarm
Optimization (PSO). The spatial attention mechanism integrates cascaded and
hierarchical regression into a CNN framework by transforming both the input(and
feature space) and the output space, which greatly reduces the viewpoint and
articulation variations. Between the levels in the hierarchy, the hierarchical
PSO forces the kinematic constraints to the results of the CNNs. The
experimental results show that our method significantly outperforms four
state-of-the-art methods and three baselines on three public benchmarks.
Maximizing positive opinion influence using an evidential approach
Authors: Siwar Jendoubi, Arnaud Martin, Ludovic Liétard, Hend Hadji, Boutheina Yaghlane
Date: 2016-10-20 00:00:00

In this paper, we propose a new data based model for influence maximization
in online social networks. We use the theory of belief functions to overcome
the data imperfection problem. Besides, the proposed model searches to detect
influencer users that adopt a positive opinion about the product, the idea,
etc, to be propagated. Moreover, we present some experiments to show the
performance of our model.
Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalisation
Authors: Piotr Antonik, François Duport, Michiel Hermans, Anteo Smerieri, Marc Haelterman, Serge Massar
Date: 2016-10-20 00:00:00
Tasks: speech-recognition, Speech Recognition, Time Series, Time Series Analysis, Time Series Prediction

Reservoir Computing is a bio-inspired computing paradigm for processing time
dependent signals. The performance of its analogue implementation are
comparable to other state of the art algorithms for tasks such as speech
recognition or chaotic time series prediction, but these are often constrained
by the offline training methods commonly employed. Here we investigated the
online learning approach by training an opto-electronic reservoir computer
using a simple gradient descent algorithm, programmed on an FPGA chip. Our
system was applied to wireless communications, a quickly growing domain with an
increasing demand for fast analogue devices to equalise the nonlinear distorted
channels. We report error rates up to two orders of magnitude lower than
previous implementations on this task. We show that our system is particularly
well-suited for realistic channel equalisation by testing it on a drifting and
a switching channels and obtaining good performances
Adaptive Substring Extraction and Modified Local NBNN Scoring for Binary Feature-based Local Mobile Visual Search without False Positives
Authors: Yusuke Uchida, Shigeyuki Sakazawa, Shin'ichi Satoh
Date: 2016-10-20 00:00:00
Tasks: Image Retrieval, Retrieval

In this paper, we propose a stand-alone mobile visual search system based on
binary features and the bag-of-visual words framework. The contribution of this
study is three-fold: (1) We propose an adaptive substring extraction method
that adaptively extracts informative bits from the original binary vector and
stores them in the inverted index. These substrings are used to refine visual
word-based matching. (2) A modified local NBNN scoring method is proposed in
the context of image retrieval, which considers the density of binary features
in scoring each feature matching. (3) In order to suppress false positives, we
introduce a convexity check step that imposes a convexity constraint on the
configuration of a transformed reference image. The proposed system improves
retrieval accuracy by 11% compared with a conventional method without
increasing the database size. Furthermore, our system with the convexity check
does not lead to false positive results.
DeepGraph: Graph Structure Predicts Network Growth
Authors: Cheng Li, Xiaoxiao Guo, Qiaozhu Mei
Date: 2016-10-20 00:00:00

The topological (or graph) structures of real-world networks are known to be
predictive of multiple dynamic properties of the networks. Conventionally, a
graph structure is represented using an adjacency matrix or a set of
hand-crafted structural features. These representations either fail to
highlight local and global properties of the graph or suffer from a severe loss
of structural information. There lacks an effective graph representation, which
hinges the realization of the predictive power of network structures.
  In this study, we propose to learn the represention of a graph, or the
topological structure of a network, through a deep learning model. This
end-to-end prediction model, named DeepGraph, takes the input of the raw
adjacency matrix of a real-world network and outputs a prediction of the growth
of the network. The adjacency matrix is first represented using a graph
descriptor based on the heat kernel signature, which is then passed through a
multi-column, multi-resolution convolutional neural network. Extensive
experiments on five large collections of real-world networks demonstrate that
the proposed prediction model significantly improves the effectiveness of
existing methods, including linear or nonlinear regressors that use
hand-crafted features, graph kernels, and competing deep learning methods.
Multilevel Anomaly Detection for Mixed Data
Authors: Kien Do, Truyen Tran, Svetha Venkatesh
Date: 2016-10-20 00:00:00
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

Anomalies are those deviating from the norm. Unsupervised anomaly detection
often translates to identifying low density regions. Major problems arise when
data is high-dimensional and mixed of discrete and continuous attributes. We
propose MIXMAD, which stands for MIXed data Multilevel Anomaly Detection, an
ensemble method that estimates the sparse regions across multiple levels of
abstraction of mixed data. The hypothesis is for domains where multiple data
abstractions exist, a data point may be anomalous with respect to the raw
representation or more abstract representations. To this end, our method
sequentially constructs an ensemble of Deep Belief Nets (DBNs) with varying
depths. Each DBN is an energy-based detector at a predefined abstraction level.
At the bottom level of each DBN, there is a Mixed-variate Restricted Boltzmann
Machine that models the density of mixed data. Predictions across the ensemble
are finally combined via rank aggregation. The proposed MIXMAD is evaluated on
high-dimensional realworld datasets of different characteristics. The results
demonstrate that for anomaly detection, (a) multilevel abstraction of
high-dimensional and mixed data is a sensible strategy, and (b) empirically,
MIXMAD is superior to popular unsupervised detection methods for both
homogeneous and mixed data.
Enhancing ICA Performance by Exploiting Sparsity: Application to FMRI Analysis
Authors: Zois Boukouvalas, Yuri Levin-Schwartz, Tulay Adali
Date: 2016-10-19 00:00:00
Tasks: blind source separation, Diversity

Independent component analysis (ICA) is a powerful method for blind source
separation based on the assumption that sources are statistically independent.
Though ICA has proven useful and has been employed in many applications,
complete statistical independence can be too restrictive an assumption in
practice. Additionally, important prior information about the data, such as
sparsity, is usually available. Sparsity is a natural property of the data, a
form of diversity, which, if incorporated into the ICA model, can relax the
independence assumption, resulting in an improvement in the overall separation
performance. In this work, we propose a new variant of ICA by entropy bound
minimization (ICA-EBM)-a flexible, yet parameter-free algorithm-through the
direct exploitation of sparsity. Using this new SparseICA-EBM algorithm, we
study the synergy of independence and sparsity through simulations on synthetic
as well as functional magnetic resonance imaging (fMRI)-like data.
A Theme-Rewriting Approach for Generating Algebra Word Problems
Authors: Rik Koncel-Kedziorski, Ioannis Konstas, Luke Zettlemoyer, Hannaneh Hajishirzi
Date: 2016-10-19 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Math, Text Generation

Texts present coherent stories that have a particular theme or overall
setting, for example science fiction or western. In this paper, we present a
text generation method called {\it rewriting} that edits existing
human-authored narratives to change their theme without changing the underlying
story. We apply the approach to math word problems, where it might help
students stay more engaged by quickly transforming all of their homework
assignments to the theme of their favorite movie without changing the math
concepts that are being taught. Our rewriting method uses a two-stage decoding
process, which proposes new words from the target theme and scores the
resulting stories according to a number of factors defining aspects of
syntactic, semantic, and thematic coherence. Experiments demonstrate that the
final stories typically represent the new theme well while still testing the
original math concepts, outperforming a number of baselines. We also release a
new dataset of human-authored rewrites of math word problems in several themes.
Artistic style transfer for videos
Authors: Manuel Ruder, Alexey Dosovitskiy, Thomas Brox
Date: 2016-04-28 00:00:00
Tasks: Style Transfer

In the past, manually re-drawing an image in a certain artistic style
required a professional artist and a long time. Doing this for a video sequence
single-handed was beyond imagination. Nowadays computers provide new
possibilities. We present an approach that transfers the style from one image
(for example, a painting) to a whole video sequence. We make use of recent
advances in style transfer in still images and propose new initializations and
loss functions applicable to videos. This allows us to generate consistent and
stable stylized video sequences, even in cases with large motion and strong
occlusion. We show that the proposed method clearly outperforms simpler
baselines both qualitatively and quantitatively.
From Behavior to Sparse Graphical Games: Efficient Recovery of Equilibria
Authors: Asish Ghoshal, Jean Honorio
Date: 2016-07-11 00:00:00

In this paper we study the problem of exact recovery of the pure-strategy
Nash equilibria (PSNE) set of a graphical game from noisy observations of joint
actions of the players alone. We consider sparse linear influence games --- a
parametric class of graphical games with linear payoffs, and represented by
directed graphs of n nodes (players) and in-degree of at most k. We present an
$\ell_1$-regularized logistic regression based algorithm for recovering the
PSNE set exactly, that is both computationally efficient --- i.e. runs in
polynomial time --- and statistically efficient --- i.e. has logarithmic sample
complexity. Specifically, we show that the sufficient number of samples
required for exact PSNE recovery scales as $\mathcal{O}(\mathrm{poly}(k) \log
n)$. We also validate our theoretical results using synthetic experiments.
Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning
Authors: Qianli Liao, Kenji Kawaguchi, Tomaso Poggio
Date: 2016-10-19 00:00:00
Tasks: GPU

We systematically explored a spectrum of normalization algorithms related to
Batch Normalization (BN) and propose a generalized formulation that
simultaneously solves two major limitations of BN: (1) online learning and (2)
recurrent learning. Our proposal is simpler and more biologically-plausible.
Unlike previous approaches, our technique can be applied out of the box to all
learning scenarios (e.g., online learning, batch learning, fully-connected,
convolutional, feedforward, recurrent and mixed --- recurrent and
convolutional) and compare favorably with existing approaches. We also propose
Lp Normalization for normalizing by different orders of statistical moments. In
particular, L1 normalization is well-performing, simple to implement, fast to
compute, more biologically-plausible and thus ideal for GPU or hardware
implementations.
The Generalized Reparameterization Gradient
Authors: Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei
Date: 2016-10-07 00:00:00
Proceeding: NeurIPS 2016
Tasks: Variational Inference

The reparameterization gradient has become a widely used method to obtain
Monte Carlo gradients to optimize the variational objective. However, this
technique does not easily apply to commonly used distributions such as beta or
gamma without further approximations, and most practical applications of the
reparameterization gradient fit Gaussian distributions. In this paper, we
introduce the generalized reparameterization gradient, a method that extends
the reparameterization gradient to a wider class of variational distributions.
Generalized reparameterizations use invertible transformations of the latent
variables which lead to transformed distributions that weakly depend on the
variational parameters. This results in new Monte Carlo gradients that combine
reparameterization gradients and score function gradients. We demonstrate our
approach on variational inference for two complex probabilistic models. The
generalized reparameterization is effective: even a single sample from the
variational distribution is enough to obtain a low-variance gradient.
POI: Multiple Object Tracking with High Performance Detection and Appearance Feature
Authors: Fengwei Yu, Wenbo Li, Quanquan Li, Yu Liu, Xiaohua Shi, Junjie Yan
Date: 2016-10-19 00:00:00
Tasks: Multiple Object Tracking, Object Tracking, Vocal Bursts Intensity Prediction

Detection and learning based appearance feature play the central role in data
association based multiple object tracking (MOT), but most recent MOT works
usually ignore them and only focus on the hand-crafted feature and association
algorithms. In this paper, we explore the high-performance detection and deep
learning based appearance feature, and show that they lead to significantly
better MOT results in both online and offline setting. We make our detection
and appearance feature publicly available. In the following part, we first
summarize the detection and appearance feature, and then introduce our tracker
named Person of Interest (POI), which has both online and offline version.
Learning to learn with backpropagation of Hebbian plasticity
Authors: Thomas Miconi
Date: 2016-09-08 00:00:00
Tasks: Continual Learning, Lifelong learning, One-Shot Learning

Hebbian plasticity is a powerful principle that allows biological brains to
learn from their lifetime experience. By contrast, artificial neural networks
trained with backpropagation generally have fixed connection weights that do
not change once training is complete. While recent methods can endow neural
networks with long-term memories, Hebbian plasticity is currently not amenable
to gradient descent. Here we derive analytical expressions for activity
gradients in neural networks with Hebbian plastic connections. Using these
expressions, we can use backpropagation to train not just the baseline weights
of the connections, but also their plasticity. As a result, the networks "learn
how to learn" in order to solve the problem at hand: the trained networks
automatically perform fast learning of unpredictable environmental features
during their lifetime, expanding the range of solvable problems. We test the
algorithm on various on-line learning tasks, including pattern completion,
one-shot learning, and reversal learning. The algorithm successfully learns how
to learn the relevant associations from one-shot instruction, and fine-tunes
the temporal dynamics of plasticity to allow for continual learning in response
to changing environmental parameters. We conclude that backpropagation of
Hebbian plasticity offers a powerful model for lifelong learning.
Efficiency of active learning for the allocation of workers on crowdsourced classification tasks
Authors: Edoardo Manino, Long Tran-Thanh, Nicholas R. Jennings
Date: 2016-10-19 00:00:00
Tasks: Active Learning, General Classification

Crowdsourcing has been successfully employed in the past as an effective and
cheap way to execute classification tasks and has therefore attracted the
attention of the research community. However, we still lack a theoretical
understanding of how to collect the labels from the crowd in an optimal way. In
this paper we focus on the problem of worker allocation and compare two active
learning policies proposed in the empirical literature with a uniform
allocation of the available budget. To this end we make a thorough mathematical
analysis of the problem and derive a new bound on the performance of the
system. Furthermore we run extensive simulations in a more realistic scenario
and show that our theoretical results hold in practice.
An Information Theoretic Feature Selection Framework for Big Data under Apache Spark
Authors: Sergio Ramírez-Gallego, Héctor Mouriño-Talín, David Martínez-Rego, Verónica Bolón-Canedo, José Manuel Benítez, Amparo Alonso-Betanzos, Francisco Herrera
Date: 2016-10-13 00:00:00
Tasks: Dimensionality Reduction, feature selection

With the advent of extremely high dimensional datasets, dimensionality
reduction techniques are becoming mandatory. Among many techniques, feature
selection has been growing in interest as an important tool to identify
relevant features on huge datasets --both in number of instances and
features--. The purpose of this work is to demonstrate that standard feature
selection methods can be parallelized in Big Data platforms like Apache Spark,
boosting both performance and accuracy. We thus propose a distributed
implementation of a generic feature selection framework which includes a wide
group of well-known Information Theoretic methods. Experimental results on a
wide set of real-world datasets show that our distributed framework is capable
of dealing with ultra-high dimensional datasets as well as those with a huge
number of samples in a short period of time, outperforming the sequential
version in all the cases studied.
Learning to Learn Neural Networks
Authors: Tom Bosc
Date: 2016-10-19 00:00:00
Tasks: Meta-Learning

Meta-learning consists in learning learning algorithms. We use a Long Short
Term Memory (LSTM) based network to learn to compute on-line updates of the
parameters of another neural network. These parameters are stored in the cell
state of the LSTM. Our framework allows to compare learned algorithms to
hand-made algorithms within the traditional train and test methodology. In an
experiment, we learn a learning algorithm for a one-hidden layer Multi-Layer
Perceptron (MLP) on non-linearly separable datasets. The learned algorithm is
able to update parameters of both layers and generalise well on similar
datasets.