pwc-archive/papers-with-abstracts huggingface dataset:
A brief survey on deep belief networks and introducing a new object oriented toolbox (DeeBNet)
Authors: Mohammad Ali Keyvanrad, Mohammad Mehdi Homayounpour
Date: 2014-08-14 00:00:00
Tasks: General Classification, GPU

Nowadays, this is very popular to use the deep architectures in machine
learning. Deep Belief Networks (DBNs) are deep architectures that use stack of
Restricted Boltzmann Machines (RBM) to create a powerful generative model using
training data. DBNs have many ability like feature extraction and
classification that are used in many applications like image processing, speech
processing and etc. This paper introduces a new object oriented MATLAB toolbox
with most of abilities needed for the implementation of DBNs. In the new
version, the toolbox can be used in Octave. According to the results of the
experiments conducted on MNIST (image), ISOLET (speech), and 20 Newsgroups
(text) datasets, it was shown that the toolbox can learn automatically a good
representation of the input from unlabeled data with better discrimination
between different classes. Also on all datasets, the obtained classification
errors are comparable to those of state of the art classifiers. In addition,
the toolbox supports different sampling methods (e.g. Gibbs, CD, PCD and our
new FEPCD method), different sparsity methods (quadratic, rate distortion and
our new normal method), different RBM types (generative and discriminative),
using GPU, etc. The toolbox is a user-friendly open source software and is
freely available on the website
http://ceit.aut.ac.ir/~keyvanrad/DeeBNet%20Toolbox.html .
A simple technique for improving multi-class classification with neural networks
Authors: Thomas Kopinski, Alexander Gepperth, Uwe Handmann
Date: 2016-01-06 00:00:00
Tasks: Classification, General Classification, Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition, Multi-class Classification

We present a novel method to perform multi-class pattern classification with
neural networks and test it on a challenging 3D hand gesture recognition
problem. Our method consists of a standard one-against-all (OAA)
classification, followed by another network layer classifying the resulting
class scores, possibly augmented by the original raw input vector. This allows
the network to disambiguate hard-to-separate classes as the distribution of
class scores carries considerable information as well, and is in fact often
used for assessing the confidence of a decision. We show that by this approach
we are able to significantly boost our results, overall as well as for
particular difficult cases, on the hard 10-class gesture classification task.
Streaming Gibbs Sampling for LDA Model
Authors: Yang Gao, Jianfei Chen, Jun Zhu
Date: 2016-01-06 00:00:00
Tasks: model

Streaming variational Bayes (SVB) is successful in learning LDA models in an
online manner. However previous attempts toward developing online Monte-Carlo
methods for LDA have little success, often by having much worse perplexity than
their batch counterparts. We present a streaming Gibbs sampling (SGS) method,
an online extension of the collapsed Gibbs sampling (CGS). Our empirical study
shows that SGS can reach similar perplexity as CGS, much better than SVB. Our
distributed version of SGS, DSGS, is much more scalable than SVB mainly because
the updates' communication complexity is small.
A pragmatic approach to multi-class classification
Authors: Thomas Kopinski, Stéphane Magand, Uwe Handmann, Alexander Gepperth
Date: 2016-01-06 00:00:00
Tasks: Classification, General Classification, Gesture Recognition, Multi-class Classification

We present a novel hierarchical approach to multi-class classification which
is generic in that it can be applied to different classification models (e.g.,
support vector machines, perceptrons), and makes no explicit assumptions about
the probabilistic structure of the problem as it is usually done in multi-class
classification. By adding a cascade of additional classifiers, each of which
receives the previous classifier's output in addition to regular input data,
the approach harnesses unused information that manifests itself in the form of,
e.g., correlations between predicted classes. Using multilayer perceptrons as a
classification model, we demonstrate the validity of this approach by testing
it on a complex ten-class 3D gesture recognition task.
Low-Rank Representation over the Manifold of Curves
Authors: Stephen Tierney, Junbin Gao, Yi Guo, Zhengwu Zhang
Date: 2016-01-05 00:00:00

In machine learning it is common to interpret each data point as a vector in
Euclidean space. However the data may actually be functional i.e.\ each data
point is a function of some variable such as time and the function is
discretely sampled. The naive treatment of functional data as traditional
multivariate data can lead to poor performance since the algorithms are
ignoring the correlation in the curvature of each function. In this paper we
propose a method to analyse subspace structure of the functional data by using
the state of the art Low-Rank Representation (LRR). Experimental evaluation on
synthetic and real data reveals that this method massively outperforms
conventional LRR in tasks concerning functional data.
INRIASAC: Simple Hypernym Extraction Methods
Authors: Gregory Grefenstette
Date: 2015-02-04 00:00:00
Proceeding: SEMEVAL 2015 6
Tasks: Sentence

Given a set of terms from a given domain, how can we structure them into a
taxonomy without manual intervention? This is the task 17 of SemEval 2015. Here
we present our simple taxonomy structuring techniques which, despite their
simplicity, ranked first in this 2015 benchmark. We use large quantities of
text (English Wikipedia) and simple heuristics such as term overlap and
document and sentence co-occurrence to produce hypernym lists. We describe
these techniques and pre-sent an initial evaluation of results.
A Planning based Framework for Essay Generation
Authors: Bing Qin, Duyu Tang, Xinwei Geng, Dandan Ning, Jiahao Liu, Ting Liu
Date: 2015-12-18 00:00:00
Tasks: Sentence

Generating an article automatically with computer program is a challenging
task in artificial intelligence and natural language processing. In this paper,
we target at essay generation, which takes as input a topic word in mind and
generates an organized article under the theme of the topic. We follow the idea
of text planning \cite{Reiter1997} and develop an essay generation framework.
The framework consists of three components, including topic understanding,
sentence extraction and sentence reordering. For each component, we studied
several statistical algorithms and empirically compared between them in terms
of qualitative or quantitative analysis. Although we run experiments on Chinese
corpus, the method is language independent and can be easily adapted to other
language. We lay out the remaining challenges and suggest avenues for future
research.
Memory Matters: Convolutional Recurrent Neural Network for Scene Text Recognition
Authors: Guo Qiang, Tu Dan, Li Guohui, Lei Jun
Date: 2016-01-06 00:00:00
Tasks: Scene Text Recognition, Segmentation

Text recognition in natural scene is a challenging problem due to the many
factors affecting text appearance. In this paper, we presents a method that
directly transcribes scene text images to text without needing of sophisticated
character segmentation. We leverage recent advances of deep neural networks to
model the appearance of scene text images with temporal dynamics. Specifically,
we integrates convolutional neural network (CNN) and recurrent neural network
(RNN) which is motivated by observing the complementary modeling capabilities
of the two models. The main contribution of this work is investigating how
temporal memory helps in an segmentation free fashion for this specific
problem. By using long short-term memory (LSTM) blocks as hidden units, our
model can retain long-term memory compared with HMMs which only maintain
short-term state dependences. We conduct experiments on Street View House
Number dataset containing highly variable number images. The results
demonstrate the superiority of the proposed method over traditional HMM based
methods.
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Authors: Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun
Date: 2015-06-04 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: 2D Object Detection, GPU, Object, Object Detection, Real-Time Object Detection, Region Proposal, Robust Object Detection, Vessel Detection

State-of-the-art object detection networks depend on region proposal
algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN
have reduced the running time of these detection networks, exposing region
proposal computation as a bottleneck. In this work, we introduce a Region
Proposal Network (RPN) that shares full-image convolutional features with the
detection network, thus enabling nearly cost-free region proposals. An RPN is a
fully convolutional network that simultaneously predicts object bounds and
objectness scores at each position. The RPN is trained end-to-end to generate
high-quality region proposals, which are used by Fast R-CNN for detection. We
further merge RPN and Fast R-CNN into a single network by sharing their
convolutional features---using the recently popular terminology of neural
networks with 'attention' mechanisms, the RPN component tells the unified
network where to look. For the very deep VGG-16 model, our detection system has
a frame rate of 5fps (including all steps) on a GPU, while achieving
state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS
COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015
competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning
entries in several tracks. Code has been made publicly available.
Learning Multi-Domain Convolutional Neural Networks for Visual Tracking
Authors: Hyeonseob Nam, Bohyung Han
Date: 2015-10-27 00:00:00
Proceeding: CVPR 2016 6
Tasks: Binary Classification, General Classification, Visual Tracking

We propose a novel visual tracking algorithm based on the representations
from a discriminatively trained Convolutional Neural Network (CNN). Our
algorithm pretrains a CNN using a large set of videos with tracking
ground-truths to obtain a generic target representation. Our network is
composed of shared layers and multiple branches of domain-specific layers,
where domains correspond to individual training sequences and each branch is
responsible for binary classification to identify the target in each domain. We
train the network with respect to each domain iteratively to obtain generic
target representations in the shared layers. When tracking a target in a new
sequence, we construct a new network by combining the shared layers in the
pretrained CNN with a new binary classification layer, which is updated online.
Online tracking is performed by evaluating the candidate windows randomly
sampled around the previous target state. The proposed algorithm illustrates
outstanding performance compared with state-of-the-art methods in existing
tracking benchmarks.
Incorporating Structural Alignment Biases into an Attentional Neural Translation Model
Authors: Trevor Cohn, Cong Duy Vu Hoang, Ekaterina Vymolova, Kaisheng Yao, Chris Dyer, Gholamreza Haffari
Date: 2016-01-06 00:00:00
Proceeding: NAACL 2016 6
Tasks: Decoder, Machine Translation, Translation

Neural encoder-decoder models of machine translation have achieved impressive
results, rivalling traditional translation models. However their modelling
formulation is overly simplistic, and omits several key inductive biases built
into traditional models. In this paper we extend the attentional neural
translation model to include structural biases from word based alignment
models, including positional bias, Markov conditioning, fertility and agreement
over translation directions. We show improvements over a baseline attentional
model and standard phrase-based model over several language pairs, evaluating
on difficult languages in a low resource setting.
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Authors: Orhan Firat, Kyunghyun Cho, Yoshua Bengio
Date: 2016-01-06 00:00:00
Proceeding: NAACL 2016 6
Tasks: Machine Translation, Translation

We propose multi-way, multilingual neural machine translation. The proposed
approach enables a single neural translation model to translate between
multiple languages, with a number of parameters that grows only linearly with
the number of languages. This is made possible by having a single attention
mechanism that is shared across all language pairs. We train the proposed
multi-way, multilingual model on ten language pairs from WMT'15 simultaneously
and observe clear performance improvements over models trained on only one
language pair. In particular, we observe that the proposed model significantly
improves the translation quality of low-resource language pairs.
Low-rank Matrix Factorization under General Mixture Noise Distributions
Authors: Xiangyong Cao, Qian Zhao, Deyu Meng, Yang Chen, Zongben Xu
Date: 2016-01-06 00:00:00
Proceeding: ICCV 2015 12
Tasks: Image Restoration

Many computer vision problems can be posed as learning a low-dimensional
subspace from high dimensional data. The low rank matrix factorization (LRMF)
represents a commonly utilized subspace learning strategy. Most of the current
LRMF techniques are constructed on the optimization problems using L1-norm and
L2-norm losses, which mainly deal with Laplacian and Gaussian noises,
respectively. To make LRMF capable of adapting more complex noise, this paper
proposes a new LRMF model by assuming noise as Mixture of Exponential Power
(MoEP) distributions and proposes a penalized MoEP (PMoEP) model by combining
the penalized likelihood method with MoEP distributions. Such setting
facilitates the learned LRMF model capable of automatically fitting the real
noise through MoEP distributions. Each component in this mixture is adapted
from a series of preliminary super- or sub-Gaussian candidates. Moreover, by
facilitating the local continuity of noise components, we embed Markov random
field into the PMoEP model and further propose the advanced PMoEP-MRF model. An
Expectation Maximization (EM) algorithm and a variational EM (VEM) algorithm
are also designed to infer the parameters involved in the proposed PMoEP and
the PMoEP-MRF model, respectively. The superseniority of our methods is
demonstrated by extensive experiments on synthetic data, face modeling,
hyperspectral image restoration and background subtraction.
Achieving Exact Cluster Recovery Threshold via Semidefinite Programming
Authors: Bruce Hajek, Yihong Wu, Jiaming Xu
Date: 2014-11-24 00:00:00
Tasks: Stochastic Block Model

The binary symmetric stochastic block model deals with a random graph of $n$
vertices partitioned into two equal-sized clusters, such that each pair of
vertices is connected independently with probability $p$ within clusters and
$q$ across clusters. In the asymptotic regime of $p=a \log n/n$ and $q=b \log
n/n$ for fixed $a,b$ and $n \to \infty$, we show that the semidefinite
programming relaxation of the maximum likelihood estimator achieves the optimal
threshold for exactly recovering the partition from the graph with probability
tending to one, resolving a conjecture of Abbe et al. \cite{Abbe14}.
Furthermore, we show that the semidefinite programming relaxation also achieves
the optimal recovery threshold in the planted dense subgraph model containing a
single cluster of size proportional to $n$.
Forecasting Social Navigation in Crowded Complex Scenes
Authors: Alexandre Robicquet, Alexandre Alahi, Amir Sadeghian, Bryan Anenberg, John Doherty, Eli Wu, Silvio Savarese
Date: 2016-01-05 00:00:00
Tasks: Common Sense Reasoning, Navigate, Sensitivity, Social Navigation, Trajectory Forecasting, Trajectory Prediction

When humans navigate a crowed space such as a university campus or the
sidewalks of a busy street, they follow common sense rules based on social
etiquette. In this paper, we argue that in order to enable the design of new
algorithms that can take fully advantage of these rules to better solve tasks
such as target tracking or trajectory forecasting, we need to have access to
better data in the first place. To that end, we contribute the very first large
scale dataset (to the best of our knowledge) that collects images and videos of
various types of targets (not just pedestrians, but also bikers, skateboarders,
cars, buses, golf carts) that navigate in a real-world outdoor environment such
as a university campus. We present an extensive evaluation where different
methods for trajectory forecasting are evaluated and compared. Moreover, we
present a new algorithm for trajectory prediction that exploits the complexity
of our new dataset and allows to: i) incorporate inter-class interactions into
trajectory prediction models (e.g, pedestrian vs bike) as opposed to just
intra-class interactions (e.g., pedestrian vs pedestrian); ii) model the degree
to which the social forces are regulating an interaction. We call the latter
"social sensitivity"and it captures the sensitivity to which a target is
responding to a certain interaction. An extensive experimental evaluation
demonstrates the effectiveness of our novel approach.
Crater Detection via Convolutional Neural Networks
Authors: Joseph Paul Cohen, Henry Z. Lo, Ting-ting Lu, Wei Ding
Date: 2016-01-05 00:00:00
Tasks: General Classification

Craters are among the most studied geomorphic features in the Solar System
because they yield important information about the past and present geological
processes and provide information about the relative ages of observed geologic
formations. We present a method for automatic crater detection using advanced
machine learning to deal with the large amount of satellite imagery collected.
The challenge of automatically detecting craters comes from their is complex
surface because their shape erodes over time to blend into the surface.
Bandeira provided a seminal dataset that embodied this challenge that is still
an unsolved pattern recognition problem to this day. There has been work to
solve this challenge based on extracting shape and contrast features and then
applying classification models on those features. The limiting factor in this
existing work is the use of hand crafted filters on the image such as Gabor or
Sobel filters or Haar features. These hand crafted methods rely on domain
knowledge to construct. We would like to learn the optimal filters and features
based on training examples. In order to dynamically learn filters and features
we look to Convolutional Neural Networks (CNNs) which have shown their
dominance in computer vision. The power of CNNs is that they can learn image
filters which generate features for high accuracy classification.
Optimally Pruning Decision Tree Ensembles With Feature Cost
Authors: Feng Nan, Joseph Wang, Venkatesh Saligrama
Date: 2016-01-05 00:00:00
Tasks: Ensemble Pruning, Prediction

We consider the problem of learning decision rules for prediction with
feature budget constraint. In particular, we are interested in pruning an
ensemble of decision trees to reduce expected feature cost while maintaining
high prediction accuracy for any test example. We propose a novel 0-1 integer
program formulation for ensemble pruning. Our pruning formulation is general -
it takes any ensemble of decision trees as input. By explicitly accounting for
feature-sharing across trees together with accuracy/cost trade-off, our method
is able to significantly reduce feature cost by pruning subtrees that introduce
more loss in terms of feature cost than benefit in terms of prediction accuracy
gain. Theoretically, we prove that a linear programming relaxation produces the
exact solution of the original integer program. This allows us to use efficient
convex optimization tools to obtain an optimally pruned ensemble for any given
budget. Empirically, we see that our pruning algorithm significantly improves
the performance of the state of the art ensemble method BudgetRF.
Gradient Estimation Using Stochastic Computation Graphs
Authors: John Schulman, Nicolas Heess, Theophane Weber, Pieter Abbeel
Date: 2015-06-17 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Reinforcement Learning

In a variety of problems originating in supervised, unsupervised, and
reinforcement learning, the loss function is defined by an expectation over a
collection of random variables, which might be part of a probabilistic model or
the external world. Estimating the gradient of this loss function, using
samples, lies at the core of gradient-based learning algorithms for these
problems. We introduce the formalism of stochastic computation
graphs---directed acyclic graphs that include both deterministic functions and
conditional probability distributions---and describe how to easily and
automatically derive an unbiased estimator of the loss function's gradient. The
resulting algorithm for computing the gradient estimator is a simple
modification of the standard backpropagation algorithm. The generic scheme we
propose unifies estimators derived in variety of prior work, along with
variance-reduction techniques therein. It could assist researchers in
developing intricate models involving a combination of stochastic and
deterministic operations, enabling, for example, attention, memory, and control
actions.
The minimal hitting set generation problem: algorithms and computation
Authors: Andrew Gainer-Dewar, Paola Vera-Licona
Date: 2016-01-05 00:00:00
Tasks: General Classification

Finding inclusion-minimal "hitting sets" for a given collection of sets is a
fundamental combinatorial problem with applications in domains as diverse as
Boolean algebra, computational biology, and data mining. Much of the
algorithmic literature focuses on the problem of *recognizing* the collection
of minimal hitting sets; however, in many of the applications, it is more
important to *generate* these hitting sets. We survey twenty algorithms from
across a variety of domains, considering their history, classification, useful
features, and computational performance on a variety of synthetic and
real-world inputs. We also provide a suite of implementations of these
algorithms with a ready-to-use, platform-agnostic interface based on Docker
containers and the AlgoRun framework, so that interested computational
scientists can easily perform similar tests with inputs from their own research
areas on their own computers or through a convenient Web interface.
Complex Decomposition of the Negative Distance kernel
Authors: Tim vor der Brück, Steffen Eger, Alexander Mehler
Date: 2016-01-05 00:00:00
Tasks: Document Classification, General Classification, Text Categorization, text-classification, Text Classification

A Support Vector Machine (SVM) has become a very popular machine learning
method for text classification. One reason for this relates to the range of
existing kernels which allow for classifying data that is not linearly
separable. The linear, polynomial and RBF (Gaussian Radial Basis Function)
kernel are commonly used and serve as a basis of comparison in our study. We
show how to derive the primal form of the quadratic Power Kernel (PK) -- also
called the Negative Euclidean Distance Kernel (NDK) -- by means of complex
numbers. We exemplify the NDK in the framework of text categorization using the
Dewey Document Classification (DDC) as the target scheme. Our evaluation shows
that the power kernel produces F-scores that are comparable to the reference
kernels, but is -- except for the linear kernel -- faster to compute. Finally,
we show how to extend the NDK-approach by including the Mahalanobis distance.