pwc-archive/papers-with-abstracts huggingface dataset:
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
Authors: Rie Johnson, Tong Zhang
Date: 2016-02-07 00:00:00
Tasks: Sentiment Analysis, Text Categorization, Text Classification

One-hot CNN (convolutional neural network) has been shown to be effective for
text categorization (Johnson & Zhang, 2015). We view it as a special case of a
general framework which jointly trains a linear model with a non-linear feature
generator consisting of `text region embedding + pooling'. Under this
framework, we explore a more sophisticated region embedding method using Long
Short-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly
large) sizes, whereas the region size needs to be fixed in a CNN. We seek
effective and efficient use of LSTM for this purpose in the supervised and
semi-supervised settings. The best results were obtained by combining region
embeddings in the form of LSTM and convolution layers trained on unlabeled
data. The results indicate that on this task, embeddings of text regions, which
can convey complex concepts, are more useful than embeddings of single words in
isolation. We report performances exceeding the previous best results on four
benchmark datasets.
Theano-MPI: a Theano-based Distributed Training Framework
Authors: He Ma, Fei Mao, Graham W. Taylor
Date: 2016-05-26 00:00:00
Tasks: Deep Learning

We develop a scalable and extendable training framework that can utilize GPUs
across nodes in a cluster and accelerate the training of deep learning models
based on data parallelism. Both synchronous and asynchronous training are
implemented in our framework, where parameter exchange among GPUs is based on
CUDA-aware MPI. In this report, we analyze the convergence and capability of
the framework to reduce training time when scaling the synchronous training of
AlexNet and GoogLeNet from 2 GPUs to 8 GPUs. In addition, we explore novel ways
to reduce the communication overhead caused by exchanging parameters. Finally,
we release the framework as open-source for further research on distributed
deep learning
Aerial image geolocalization from recognition and matching of roads and intersections
Authors: Dragos Costea, Marius Leordeanu
Date: 2016-05-26 00:00:00

Aerial image analysis at a semantic level is important in many applications
with strong potential impact in industry and consumer use, such as automated
mapping, urban planning, real estate and environment monitoring, or disaster
relief. The problem is enjoying a great interest in computer vision and remote
sensing, due to increased computer power and improvement in automated image
understanding algorithms. In this paper we address the task of automatic
geolocalization of aerial images from recognition and matching of roads and
intersections. Our proposed method is a novel contribution in the literature
that could enable many applications of aerial image analysis when GPS data is
not available. We offer a complete pipeline for geolocalization, from the
detection of roads and intersections, to the identification of the enclosing
geographic region by matching detected intersections to previously learned
manually labeled ones, followed by accurate geometric alignment between the
detected roads and the manually labeled maps. We test on a novel dataset with
aerial images of two European cities and use the publicly available
OpenStreetMap project for collecting ground truth roads annotations. We show in
extensive experiments that our approach produces highly accurate localizations
in the challenging case when we train on images from one city and test on the
other and the quality of the aerial images is relatively poor. We also show
that the the alignment between detected roads and pre-stored manual annotations
can be effectively used for improving the quality of the road detection
results.
Subspace Learning with Partial Information
Authors: Alon Gonen, Dan Rosenbaum, Yonina Eldar, Shai Shalev-Shwartz
Date: 2014-02-19 00:00:00

The goal of subspace learning is to find a $k$-dimensional subspace of
$\mathbb{R}^d$, such that the expected squared distance between instance
vectors and the subspace is as small as possible. In this paper we study
subspace learning in a partial information setting, in which the learner can
only observe $r \le d$ attributes from each instance vector. We propose several
efficient algorithms for this task, and analyze their sample complexity
Discrete Deep Feature Extraction: A Theory and New Architectures
Authors: Thomas Wiatowski, Michael Tschannen, Aleksandar Stanić, Philipp Grohs, Helmut Bölcskei
Date: 2016-05-26 00:00:00
Tasks: Facial Landmark Detection, Feature Importance, General Classification, Translation

First steps towards a mathematical theory of deep convolutional neural
networks for feature extraction were made---for the continuous-time case---in
Mallat, 2012, and Wiatowski and B\"olcskei, 2015. This paper considers the
discrete case, introduces new convolutional neural network architectures, and
proposes a mathematical framework for their analysis. Specifically, we
establish deformation and translation sensitivity results of local and global
nature, and we investigate how certain structural properties of the input
signal are reflected in the corresponding feature vectors. Our theory applies
to general filters and general Lipschitz-continuous non-linearities and pooling
operators. Experiments on handwritten digit classification and facial landmark
detection---including feature importance evaluation---complement the
theoretical findings.
On the Min-cost Traveling Salesman Problem with Drone
Authors: Quang Minh Ha, Yves Deville, Quang Dung Pham, Minh Hoàng Hà
Date: 2015-12-04 00:00:00
Tasks: Traveling Salesman Problem

Once known to be used exclusively in military domain, unmanned aerial
vehicles (drones) have stepped up to become a part of new logistic method in
commercial sector called "last-mile delivery". In this novel approach, small
unmanned aerial vehicles (UAV), also known as drones, are deployed alongside
with trucks to deliver goods to customers in order to improve the service
quality or reduce the transportation cost. It gives rise to a new variant of
the traveling salesman problem (TSP), of which we call TSP with drone (TSP-D).
In this article, we consider a variant of TSP-D where the main objective is to
minimize the total transportation cost. We also propose two heuristics: "Drone
First, Truck Second" (DFTS) and "Truck First, Drone Second" (TFDS), to
effectively solve the problem. The former constructs route for drone first
while the latter constructs route for truck first. We solve a TSP to generate
route for truck and propose a mixed integer programming (MIP) formulation with
different profit functions to build route for drone. Numerical results obtained
on many instances with different sizes and characteristics are presented.
Recommendations on promising algorithm choices are also provided.
Low-rank tensor completion: a Riemannian manifold preconditioning approach
Authors: Hiroyuki Kasai, Bamdev Mishra
Date: 2016-05-26 00:00:00
Tasks: Riemannian optimization

We propose a novel Riemannian manifold preconditioning approach for the
tensor completion problem with rank constraint. A novel Riemannian metric or
inner product is proposed that exploits the least-squares structure of the cost
function and takes into account the structured symmetry that exists in Tucker
decomposition. The specific metric allows to use the versatile framework of
Riemannian optimization on quotient manifolds to develop preconditioned
nonlinear conjugate gradient and stochastic gradient descent algorithms for
batch and online setups, respectively. Concrete matrix representations of
various optimization-related ingredients are listed. Numerical comparisons
suggest that our proposed algorithms robustly outperform state-of-the-art
algorithms across different synthetic and real-world datasets.
cvpaper.challenge in 2015 - A review of CVPR2015 and DeepSurvey
Authors: Hirokatsu Kataoka, Yudai Miyashita, Tomoaki Yamabe, Soma Shirakabe, Shin'ichi Sato, Hironori Hoshino, Ryo Kato, Kaori Abe, Takaaki Imanari, Naomichi Kobayashi, Shinichiro Morita, Akio Nakamura
Date: 2016-05-26 00:00:00

The "cvpaper.challenge" is a group composed of members from AIST, Tokyo Denki
Univ. (TDU), and Univ. of Tsukuba that aims to systematically summarize papers
on computer vision, pattern recognition, and related fields. For this
particular review, we focused on reading the ALL 602 conference papers
presented at the CVPR2015, the premier annual computer vision event held in
June 2015, in order to grasp the trends in the field. Further, we are proposing
"DeepSurvey" as a mechanism embodying the entire process from the reading
through all the papers, the generation of ideas, and to the writing of paper.
Predicting online user behaviour using deep learning algorithms
Authors: Armando Vieira
Date: 2015-11-19 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Denoising

We propose a robust classifier to predict buying intentions based on user
behaviour within a large e-commerce website. In this work we compare
traditional machine learning techniques with the most advanced deep learning
approaches. We show that both Deep Belief Networks and Stacked Denoising
auto-Encoders achieved a substantial improvement by extracting features from
high dimensional data during the pre-train phase. They prove also to be more
convenient to deal with severe class imbalance.
Neighborhood Sensitive Mapping for Zero-Shot Classification using Independently Learned Semantic Embeddings
Authors: Gaurav Singh, Fabrizio Silvestri, John Shawe-Taylor
Date: 2016-05-26 00:00:00
Tasks: General Classification, zero-shot-classification, Zero-Shot Learning

In a traditional setting, classifiers are trained to approximate a target function $f:X \rightarrow Y$ where at least a sample for each $y \in Y$ is presented to the training algorithm. In a zero-shot setting we have a subset of the labels $\hat{Y} \subset Y$ for which we do not observe any corresponding training instance. Still, the function $f$ that we train must be able to correctly assign labels also on $\hat{Y}$. In practice, zero-shot problems are very important especially when the label set is large and the cost of editorially label samples for all possible values in the label set might be prohibitively high. Most recent approaches to zero-shot learning are based on finding and exploiting relationships between labels using semantic embeddings. We show in this paper that semantic embeddings, despite being very good at capturing relationships between labels, are not very good at capturing the relationships among labels in a data-dependent manner. For this reason, we propose a novel two-step process for learning a zero-shot classifier. In the first step, we learn what we call a \emph{property embedding space} capturing the "\emph{learnable}" features of the label set. Then, we exploit the learned properties in order to reduce the generalization error for a linear nearest neighbor-based classifier.
Exploiting Cyclic Symmetry in Convolutional Neural Networks
Authors: Sander Dieleman, Jeffrey De Fauw, Koray Kavukcuoglu
Date: 2016-02-08 00:00:00
Tasks: Data Augmentation, Translation

Many classes of images exhibit rotational symmetry. Convolutional neural
networks are sometimes trained using data augmentation to exploit this, but
they are still required to learn the rotation equivariance properties from the
data. Encoding these properties into the network architecture, as we are
already used to doing for translation equivariance by using convolutional
layers, could result in a more efficient use of the parameter budget by
relieving the model from learning them. We introduce four operations which can
be inserted into neural network models as layers, and which can be combined to
make these models partially equivariant to rotations. They also enable
parameter sharing across different orientations. We evaluate the effect of
these architectural modifications on three datasets which exhibit rotational
symmetry and demonstrate improved performance with smaller models.
Hierarchical Compound Poisson Factorization
Authors: Mehmet E. Basbug, Barbara E. Engelhardt
Date: 2016-04-13 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Non-negative matrix factorization models based on a hierarchical
Gamma-Poisson structure capture user and item behavior effectively in extremely
sparse data sets, making them the ideal choice for collaborative filtering
applications. Hierarchical Poisson factorization (HPF) in particular has proved
successful for scalable recommendation systems with extreme sparsity. HPF,
however, suffers from a tight coupling of sparsity model (absence of a rating)
and response model (the value of the rating), which limits the expressiveness
of the latter. Here, we introduce hierarchical compound Poisson factorization
(HCPF) that has the favorable Gamma-Poisson structure and scalability of HPF to
high-dimensional extremely sparse matrices. More importantly, HCPF decouples
the sparsity model from the response model, allowing us to choose the most
suitable distribution for the response. HCPF can capture binary, non-negative
discrete, non-negative continuous, and zero-inflated continuous responses. We
compare HCPF with HPF on nine discrete and three continuous data sets and
conclude that HCPF captures the relationship between sparsity and response
better than HPF.
Easy-First Dependency Parsing with Hierarchical Tree LSTMs
Authors: Eliyahu Kiperwasser, Yoav Goldberg
Date: 2016-03-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Dependency Parsing, Word Embeddings

We suggest a compositional vector representation of parse trees that relies
on a recursive combination of recurrent-neural network encoders. To demonstrate
its effectiveness, we use the representation as the backbone of a greedy,
bottom-up dependency parser, achieving state-of-the-art accuracies for English
and Chinese, without relying on external word embeddings. The parser's
implementation is available for download at the first author's webpage.
Solving Ridge Regression using Sketched Preconditioned SVRG
Authors: Alon Gonen, Francesco Orabona, Shai Shalev-Shwartz
Date: 2016-02-07 00:00:00
Tasks: regression

We develop a novel preconditioning method for ridge regression, based on
recent linear sketching methods. By equipping Stochastic Variance Reduced
Gradient (SVRG) with this preconditioning process, we obtain a significant
speed-up relative to fast stochastic methods such as SVRG, SDCA and SAG.
Highly-Smooth Zero-th Order Online Optimization Vianney Perchet
Authors: Francis Bach, Vianney Perchet
Date: 2016-05-26 00:00:00

The minimization of convex functions which are only available through partial
and noisy information is a key methodological problem in many disciplines. In
this paper we consider convex optimization with noisy zero-th order
information, that is noisy function evaluations at any desired point. We focus
on problems with high degrees of smoothness, such as logistic regression. We
show that as opposed to gradient-based algorithms, high-order smoothness may be
used to improve estimation rates, with a precise dependence of our upper-bounds
on the degree of smoothness. In particular, we show that for infinitely
differentiable functions, we recover the same dependence on sample size as
gradient-based algorithms, with an extra dimension-dependent factor. This is
done for both convex and strongly-convex functions, with finite horizon and
anytime algorithms. Finally, we also recover similar results in the online
optimization setting.
Dictionary Learning for Massive Matrix Factorization
Authors: Arthur Mensch, Julien Mairal, Bertrand Thirion, Gaël Varoquaux
Date: 2016-05-03 00:00:00
Tasks: Collaborative Filtering, Dictionary Learning, Matrix Completion, Recommendation Systems

Sparse matrix factorization is a popular tool to obtain interpretable data
decompositions, which are also effective to perform data completion or
denoising. Its applicability to large datasets has been addressed with online
and randomized methods, that reduce the complexity in one of the matrix
dimension, but not in both of them. In this paper, we tackle very large
matrices in both dimensions. We propose a new factoriza-tion method that scales
gracefully to terabyte-scale datasets, that could not be processed by previous
algorithms in a reasonable amount of time. We demonstrate the efficiency of our
approach on massive functional Magnetic Resonance Imaging (fMRI) data, and on
matrix completion problems for recommender systems, where we obtain significant
speed-ups compared to state-of-the art coordinate descent methods.
A single scale retinex based method for palm vein extraction
Authors: Chongyang Wang, Ming Peng, Lingfeng Xu, Tong Chen
Date: 2016-05-26 00:00:00

Palm vein recognition is a novel biometric identification technology. But how
to gain a better vein extraction result from the raw palm image is still a
challenging problem, especially when the raw data collection has the problem of
asymmetric illumination. This paper proposes a method based on single scale
Retinex algorithm to extract palm vein image when strong shadow presents due to
asymmetric illumination and uneven geometry of the palm. We test our method on
a multispectral palm image. The experimental result shows that the proposed
method is robust to the influence of illumination angle and shadow. Compared to
the traditional extraction methods, the proposed method can obtain palm vein
lines with better visualization performance (the contrast ratio increases by
18.4%, entropy increases by 1.07%, and definition increases by 18.8%).
DeepMovie: Using Optical Flow and Deep Neural Networks to Stylize Movies
Authors: Alexander G. Anderson, Cory P. Berg, Daniel P. Mossing, Bruno A. Olshausen
Date: 2016-05-26 00:00:00
Tasks: Optical Flow Estimation, Style Transfer

A recent paper by Gatys et al. describes a method for rendering an image in
the style of another image. First, they use convolutional neural network
features to build a statistical model for the style of an image. Then they
create a new image with the content of one image but the style statistics of
another image. Here, we extend this method to render a movie in a given
artistic style. The naive solution that independently renders each frame
produces poor results because the features of the style move substantially from
one frame to the next. The other naive method that initializes the optimization
for the next frame using the rendered version of the previous frame also
produces poor results because the features of the texture stay fixed relative
to the frame of the movie instead of moving with objects in the scene. The main
contribution of this paper is to use optical flow to initialize the style
transfer optimization so that the texture features move with the objects in the
video. Finally, we suggest a method to incorporate optical flow explicitly into
the cost function.
Cognitive Dynamic Systems: A Technical Review of Cognitive Radar
Authors: Krishanth Krishnan, Taralyn Schwering, Saman Sarraf
Date: 2016-05-26 00:00:00
Tasks: Management, Retrieval

We start with the history of cognitive radar, where origins of the PAC,
Fuster research on cognition and principals of cognition are provided. Fuster
describes five cognitive functions: perception, memory, attention, language,
and intelligence. We describe the Perception-Action Cyclec as it applies to
cognitive radar, and then discuss long-term memory, memory storage, memory
retrieval and working memory. A comparison between memory in human cognition
and cognitive radar is given as well. Attention is another function described
by Fuster, and we have given the comparison of attention in human cognition and
cognitive radar. We talk about the four functional blocks from the PAC:
Bayesian filter, feedback information, dynamic programming and state-space
model for the radar environment. Then, to show that the PAC improves the
tracking accuracy of Cognitive Radar over Traditional Active Radar, we have
provided simulation results. In the simulation, three nonlinear filters:
Cubature Kalman Filter, Unscented Kalman Filter and Extended Kalman Filter are
compared. Based on the results, radars implemented with CKF perform better than
the radars implemented with UKF or radars implemented with EKF. Further, radar
with EKF has the worst accuracy and has the biggest computation load because of
derivation and evaluation of Jacobian matrices. We suggest using the concept of
risk management to better control parameters and improve performance in
cognitive radar. We believe, spectrum sensing can be seen as a potential
interest to be used in cognitive radar and we propose a new approach
Probabilistic ICA which will presumably reduce noise based on estimation error
in cognitive radar. Parallel computing is a concept based on divide and
conquers mechanism, and we suggest using the parallel computing approach in
cognitive radar by doing complicated calculations or tasks to reduce processing
time.
Faster Eigenvector Computation via Shift-and-Invert Preconditioning
Authors: Dan Garber, Elad Hazan, Chi Jin, Sham M. Kakade, Cameron Musco, Praneeth Netrapalli, Aaron Sidford
Date: 2016-05-26 00:00:00
Tasks: Stochastic Optimization

We give faster algorithms and improved sample complexities for estimating the
top eigenvector of a matrix $\Sigma$ -- i.e. computing a unit vector $x$ such
that $x^T \Sigma x \ge (1-\epsilon)\lambda_1(\Sigma)$:
  Offline Eigenvector Estimation: Given an explicit $A \in \mathbb{R}^{n \times
d}$ with $\Sigma = A^TA$, we show how to compute an $\epsilon$ approximate top
eigenvector in time $\tilde O([nnz(A) + \frac{d*sr(A)}{gap^2} ]* \log
1/\epsilon )$ and $\tilde O([\frac{nnz(A)^{3/4} (d*sr(A))^{1/4}}{\sqrt{gap}} ]
* \log 1/\epsilon )$. Here $nnz(A)$ is the number of nonzeros in $A$, $sr(A)$
is the stable rank, $gap$ is the relative eigengap. By separating the $gap$
dependence from the $nnz(A)$ term, our first runtime improves upon the
classical power and Lanczos methods. It also improves prior work using fast
subspace embeddings [AC09, CW13] and stochastic optimization [Sha15c], giving
significantly better dependencies on $sr(A)$ and $\epsilon$. Our second running
time improves these further when $nnz(A) \le \frac{d*sr(A)}{gap^2}$.
  Online Eigenvector Estimation: Given a distribution $D$ with covariance
matrix $\Sigma$ and a vector $x_0$ which is an $O(gap)$ approximate top
eigenvector for $\Sigma$, we show how to refine to an $\epsilon$ approximation
using $ O(\frac{var(D)}{gap*\epsilon})$ samples from $D$. Here $var(D)$ is a
natural notion of variance. Combining our algorithm with previous work to
initialize $x_0$, we obtain improved sample complexity and runtime results
under a variety of assumptions on $D$.
  We achieve our results using a general framework that we believe is of
independent interest. We give a robust analysis of the classic method of
shift-and-invert preconditioning to reduce eigenvector computation to
approximately solving a sequence of linear systems. We then apply fast
stochastic variance reduced gradient (SVRG) based system solvers to achieve our
claims.