pwc-archive/papers-with-abstracts huggingface dataset:
Discovery of Latent Factors in High-dimensional Data Using Tensor Methods
Authors: Furong Huang
Date: 2016-06-10 00:00:00
Tasks: Dimensionality Reduction, Stochastic Block Model, Tensor Decomposition, Vocal Bursts Intensity Prediction

Unsupervised learning aims at the discovery of hidden structure that drives
the observations in the real world. It is essential for success in modern
machine learning. Latent variable models are versatile in unsupervised learning
and have applications in almost every domain. Training latent variable models
is challenging due to the non-convexity of the likelihood objective. An
alternative method is based on the spectral decomposition of low order moment
tensors. This versatile framework is guaranteed to estimate the correct model
consistently. My thesis spans both theoretical analysis of tensor decomposition
framework and practical implementation of various applications. This thesis
presents theoretical results on convergence to globally optimal solution of
tensor decomposition using the stochastic gradient descent, despite
non-convexity of the objective. This is the first work that gives global
convergence guarantees for the stochastic gradient descent on non-convex
functions with exponentially many local minima and saddle points. This thesis
also presents large-scale deployment of spectral methods carried out on various
platforms. Dimensionality reduction techniques such as random projection are
incorporated for a highly parallel and scalable tensor decomposition algorithm.
We obtain a gain in both accuracies and in running times by several orders of
magnitude compared to the state-of-art variational methods. To solve real world
problems, more advanced models and learning algorithms are proposed. This
thesis discusses generalization of LDA model to mixed membership stochastic
block model for learning user communities in social network, convolutional
dictionary model for learning word-sequence embeddings, hierarchical tensor
decomposition and latent tree structure model for learning disease hierarchy,
and spatial point process mixture model for detecting cell types in
neuroscience.
Causal Bandits: Learning Good Interventions via Causal Inference
Authors: Finnian Lattimore, Tor Lattimore, Mark D. Reid
Date: 2016-06-10 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Causal Inference

We study the problem of using causal models to improve the rate at which good
interventions can be learned online in a stochastic environment. Our formalism
combines multi-arm bandits and causal inference to model a novel type of bandit
feedback that is not exploited by existing approaches. We propose a new
algorithm that exploits the causal feedback and prove a bound on its simple
regret that is strictly better (in all quantities) than algorithms that do not
use the additional causal information.
Phase Retrieval via Incremental Truncated Wirtinger Flow
Authors: Ritesh Kolte, Ayfer Özgür
Date: 2016-06-10 00:00:00
Tasks: Retrieval

In the phase retrieval problem, an unknown vector is to be recovered given
quadratic measurements. This problem has received considerable attention in
recent times. In this paper, we present an algorithm to solve a nonconvex
formulation of the phase retrieval problem, that we call $\textit{Incremental
Truncated Wirtinger Flow}$. Given random Gaussian sensing vectors, we prove
that it converges linearly to the solution, with an optimal sample complexity.
We also provide stability guarantees of the algorithm under noisy measurements.
Performance and comparisons with existing algorithms are illustrated via
numerical experiments on simulated and real data, with both random and
structured sensing vectors.
PSDVec: a Toolbox for Incremental and Scalable Word Embedding
Authors: Shaohua Li, Jun Zhu, Chunyan Miao
Date: 2016-06-10 00:00:00
Tasks: Word Embeddings, Word Similarity

PSDVec is a Python/Perl toolbox that learns word embeddings, i.e. the mapping
of words in a natural language to continuous vectors which encode the
semantic/syntactic regularities between the words. PSDVec implements a word
embedding learning method based on a weighted low-rank positive semidefinite
approximation. To scale up the learning process, we implement a blockwise
online learning algorithm to learn the embeddings incrementally. This strategy
greatly reduces the learning time of word embeddings on a large vocabulary, and
can learn the embeddings of new words without re-learning the whole vocabulary.
On 9 word similarity/analogy benchmark sets and 2 Natural Language Processing
(NLP) tasks, PSDVec produces embeddings that has the best average performance
among popular word embedding tools. PSDVec provides a new option for NLP
practitioners.
Fuzzy-Klassen Model for Development Disparities Analysis based on Gross Regional Domestic Product Sector of a Region
Authors: Tb. Ai Munandar, Retantyo Wardoyo
Date: 2016-06-10 00:00:00

Analysis of regional development imbalances quadrant has a very important
meaning in order to see the extent of achievement of the development of certain
areas as well as the difference. Factors that could be used as a tool to
measure the inequality of development is to look at the average growth and
development contribution of each sector of Gross Regional Domestic Product
(GRDP) based on the analyzed region and the reference region. This study
discusses the development of a model to determine the regional development
imbalances using fuzzy approach system, and the rules of typology Klassen. The
model is then called fuzzy-Klassen. Implications Product Mamdani fuzzy system
is used in the model as an inference engine to generate output after
defuzzyfication process. Application of MATLAB is used as a tool of analysis in
this study. The test a result of Kota Cilegon is shows that there are
significant differences between traditional Klassen typology analyses with the
results of the model developed. Fuzzy model-Klassen shows GRDP sector
inequality Cilegon City is dominated by Quadrant I (K4), where status is the
sector forward and grows exponentially. While the traditional Klassen typology,
half of GRDP sector is dominated by Quadrant IV (K4) with a sector that is
lagging relative status.
Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation
Authors: Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji
Date: 2016-01-06 00:00:00
Proceeding: CONLL 2016 8
Tasks: Entity Disambiguation, Entity Linking

Named Entity Disambiguation (NED) refers to the task of resolving multiple
named entity mentions in a document to their correct references in a knowledge
base (KB) (e.g., Wikipedia). In this paper, we propose a novel embedding method
specifically designed for NED. The proposed method jointly maps words and
entities into the same continuous vector space. We extend the skip-gram model
by using two models. The KB graph model learns the relatedness of entities
using the link structure of the KB, whereas the anchor context model aims to
align vectors such that similar words and entities occur close to one another
in the vector space by leveraging KB anchors and their context words. By
combining contexts based on the proposed embedding with standard NED features,
we achieved state-of-the-art accuracy of 93.1% on the standard CoNLL dataset
and 85.2% on the TAC 2010 dataset.
Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays
Authors: Marek Rei, Ronan Cummins
Date: 2016-06-09 00:00:00
Proceeding: WS 2016 6
Tasks: Sentence, Sentence Similarity, Word Embeddings

We investigate the task of assessing sentence-level prompt relevance in
learner essays. Various systems using word overlap, neural embeddings and
neural compositional models are evaluated on two datasets of learner writing.
We propose a new method for sentence-level similarity calculation, which learns
to adjust the weights of pre-trained word embeddings for a specific task,
achieving substantially higher accuracy compared to other relevant baselines.
PerSum: Novel Systems for Document Summarization in Persian
Authors: Saeid Parvandeh, Shibamouli Lahiri, Fahimeh Boroumand
Date: 2016-06-09 00:00:00
Tasks: Articles, Document Summarization

In this paper we explore the problem of document summarization in Persian
language from two distinct angles. In our first approach, we modify a popular
and widely cited Persian document summarization framework to see how it works
on a realistic corpus of news articles. Human evaluation on generated summaries
shows that graph-based methods perform better than the modified systems. We
carry this intuition forward in our second approach, and probe deeper into the
nature of graph-based systems by designing several summarizers based on
centrality measures. Ad hoc evaluation using ROUGE score on these summarizers
suggests that there is a small class of centrality measures that perform better
than three strong unsupervised baselines.
Mutual Exclusivity Loss for Semi-Supervised Deep Learning
Authors: Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen
Date: 2016-06-09 00:00:00
Tasks: Deep Learning, Object Recognition

In this paper we consider the problem of semi-supervised learning with deep
Convolutional Neural Networks (ConvNets). Semi-supervised learning is motivated
on the observation that unlabeled data is cheap and can be used to improve the
accuracy of classifiers. In this paper we propose an unsupervised
regularization term that explicitly forces the classifier's prediction for
multiple classes to be mutually-exclusive and effectively guides the decision
boundary to lie on the low density space between the manifolds corresponding to
different classes of data. Our proposed approach is general and can be used
with any backpropagation-based learning method. We show through different
experiments that our method can improve the object recognition performance of
ConvNets using unlabeled data.
Fast, Robust and Non-convex Subspace Recovery
Authors: Gilad Lerman, Tyler Maunu
Date: 2014-06-24 00:00:00

This work presents a fast and non-convex algorithm for robust subspace
recovery. The data sets considered include inliers drawn around a
low-dimensional subspace of a higher dimensional ambient space, and a possibly
large portion of outliers that do not lie nearby this subspace. The proposed
algorithm, which we refer to as Fast Median Subspace (FMS), is designed to
robustly determine the underlying subspace of such data sets, while having
lower computational complexity than existing methods. We prove convergence of
the FMS iterates to a stationary point. Further, under a special model of data,
FMS converges to a point which is near to the global minimum with overwhelming
probability. Under this model, we show that the iteration complexity is
globally bounded and locally $r$-linear. The latter theorem holds for any fixed
fraction of outliers (less than 1) and any fixed positive distance between the
limit point and the global minimum. Numerical experiments on synthetic and real
data demonstrate its competitive speed and accuracy.
Symmetry-aware Depth Estimation using Deep Neural Networks
Authors: Guilin Liu, Chao Yang, Zimo Li, Duygu Ceylan, Qi-Xing Huang
Date: 2016-04-20 00:00:00
Tasks: Depth Estimation

Due to the abundance of 2D product images from the Internet, developing
efficient and scalable algorithms to recover the missing depth information is
central to many applications. Recent works have addressed the single-view depth
estimation problem by utilizing convolutional neural networks. In this paper,
we show that exploring symmetry information, which is ubiquitous in man made
objects, can significantly boost the quality of such depth predictions.
Specifically, we propose a new convolutional neural network architecture to
first estimate dense symmetric correspondences in a product image and then
propose an optimization which utilizes this information explicitly to
significantly improve the quality of single-view depth estimations. We have
evaluated our approach extensively, and experimental results show that this
approach outperforms state-of-the-art depth estimation techniques.
DopeLearning: A Computational Approach to Rap Lyrics Generation
Authors: Eric Malmi, Pyry Takala, Hannu Toivonen, Tapani Raiko, Aristides Gionis
Date: 2015-05-18 00:00:00
Tasks: Prediction

Writing rap lyrics requires both creativity to construct a meaningful,
interesting story and lyrical skills to produce complex rhyme patterns, which
form the cornerstone of good flow. We present a rap lyrics generation method
that captures both of these aspects. First, we develop a prediction model to
identify the next line of existing lyrics from a set of candidate next lines.
This model is based on two machine-learning techniques: the RankSVM algorithm
and a deep neural network model with a novel structure. Results show that the
prediction model can identify the true next line among 299 randomly selected
lines with an accuracy of 17%, i.e., over 50 times more likely than by random.
Second, we employ the prediction model to combine lines from existing songs,
producing lyrics with rhyme and a meaning. An evaluation of the produced lyrics
shows that in terms of quantitative rhyme density, the method outperforms the
best human rappers by 21%. The rap lyrics generator has been deployed as an
online tool called DeepBeat, and the performance of the tool has been assessed
by analyzing its usage logs. This analysis shows that machine-learned rankings
correlate with user preferences.
Efficient Robust Proper Learning of Log-concave Distributions
Authors: Ilias Diakonikolas, Daniel M. Kane, Alistair Stewart
Date: 2016-06-09 00:00:00

We study the {\em robust proper learning} of univariate log-concave
distributions (over continuous and discrete domains). Given a set of samples
drawn from an unknown target distribution, we want to compute a log-concave
hypothesis distribution that is as close as possible to the target, in total
variation distance. In this work, we give the first computationally efficient
algorithm for this learning problem. Our algorithm achieves the
information-theoretically optimal sample size (up to a constant factor), runs
in polynomial time, and is robust to model misspecification with nearly-optimal
error guarantees.
  Specifically, we give an algorithm that, on input $n=O(1/\eps^{5/2})$ samples
from an unknown distribution $f$, runs in time $\widetilde{O}(n^{8/5})$, and
outputs a log-concave hypothesis $h$ that (with high probability) satisfies
$\dtv(h, f) = O(\opt)+\eps$, where $\opt$ is the minimum total variation
distance between $f$ and the class of log-concave distributions. Our approach
to the robust proper learning problem is quite flexible and may be applicable
to many other univariate distribution families.
Convolutional Sketch Inversion
Authors: Yağmur Güçlütürk, Umut Güçlü, Rob Van Lier, Marcel A. J. van Gerven
Date: 2016-06-09 00:00:00
Tasks: Stochastic Optimization

In this paper, we use deep neural networks for inverting face sketches to
synthesize photorealistic face images. We first construct a semi-simulated
dataset containing a very large number of computer-generated face sketches with
different styles and corresponding face images by expanding existing
unconstrained face data sets. We then train models achieving state-of-the-art
results on both computer-generated sketches and hand-drawn sketches by
leveraging recent advances in deep learning such as batch normalization, deep
residual learning, perceptual losses and stochastic optimization in combination
with our new dataset. We finally demonstrate potential applications of our
models in fine arts and forensic arts. In contrast to existing patch-based
approaches, our deep-neural-network-based approach can be used for synthesizing
photorealistic face images by inverting face sketches in the wild.
The "Horse'' Inside: Seeking Causes Behind the Behaviours of Music Content Analysis Systems
Authors: Bob L. Sturm
Date: 2016-06-09 00:00:00

Building systems that possess the sensitivity and intelligence to identify
and describe high-level attributes in music audio signals continues to be an
elusive goal, but one that surely has broad and deep implications for a wide
variety of applications. Hundreds of papers have so far been published toward
this goal, and great progress appears to have been made. Some systems produce
remarkable accuracies at recognising high-level semantic concepts, such as
music style, genre and mood. However, it might be that these numbers do not
mean what they seem. In this paper, we take a state-of-the-art music content
analysis system and investigate what causes it to achieve exceptionally high
performance in a benchmark music audio dataset. We dissect the system to
understand its operation, determine its sensitivities and limitations, and
predict the kinds of knowledge it could and could not possess about music. We
perform a series of experiments to illuminate what the system has actually
learned to do, and to what extent it is performing the intended music listening
task. Our results demonstrate how the initial manifestation of music
intelligence in this state-of-the-art can be deceptive. Our work provides
constructive directions toward developing music content analysis systems that
can address the music information and creation needs of real-world users.
Supervised Incremental Hashing
Authors: Bahadir Ozdemir, Mahyar Najibi, Larry S. Davis
Date: 2016-04-25 00:00:00
Tasks: General Classification, Image Retrieval

We propose an incremental strategy for learning hash functions with kernels
for large-scale image search. Our method is based on a two-stage classification
framework that treats binary codes as intermediate variables between the
feature space and the semantic space. In the first stage of classification,
binary codes are considered as class labels by a set of binary SVMs; each
corresponds to one bit. In the second stage, binary codes become the input
space of a multi-class SVM. Hash functions are learned by an efficient
algorithm where the NP-hard problem of finding optimal binary codes is solved
via cyclic coordinate descent and SVMs are trained in a parallelized
incremental manner. For modifications like adding images from a previously
unseen class, we describe an incremental procedure for effective and efficient
updates to the previous hash functions. Experiments on three large-scale image
datasets demonstrate the effectiveness of the proposed hashing method,
Supervised Incremental Hashing (SIH), over the state-of-the-art supervised
hashing methods.
Feature-based Recursive Observer Design for Homography Estimation
Authors: Minh-Duc Hua, Jochen Trumpf, Tarek Hamel, Robert Mahony, Pascal Morin
Date: 2016-06-09 00:00:00
Tasks: Homography Estimation

This paper presents a new algorithm for online estimation of a sequence of
homographies applicable to image sequences obtained from robotic vehicles
equipped with vision sensors. The approach taken exploits the underlying
Special Linear group structure of the set of homographies along with gyroscope
measurements and direct point-feature correspondences between images to develop
temporal filter for the homography estimate. Theoretical analysis and
experimental results are provided to demonstrate the robustness of the proposed
algorithm. The experimental results show excellent performance even in the case
of very fast camera motion (relative to frame rate), severe occlusion, and in
the presence of specular reflections.
Implicit Tubular Surface Generation Guided by Centerline
Authors: Haoyin Zhou, James K. Min, Guanglei Xiong
Date: 2016-06-09 00:00:00
Tasks: Coronary Artery Segmentation, Segmentation

Most machine learning-based coronary artery segmentation methods represent
the vascular lumen surface in an implicit way by the centerline and the
associated lumen radii, which makes the subsequent modeling process to generate
a whole piece of watertight coronary artery tree model difficult. To solve this
problem, in this paper, we propose a modeling method with the learning-based
segmentation results by (1) considering mesh vertices as physical particles and
using interaction force model and particle expansion model to generate
uniformly distributed point cloud on the implicit lumen surface and; (2) doing
incremental Delaunay-based triangulation. Our method has the advantage of being
able to consider the complex shape of the coronary artery tree as a whole
piece; hence no extra stitching or intersection removal algorithm is needed to
generate a watertight model. Experiment results demonstrate that our method is
capable of generating high quality mesh model which is highly consistent with
the given implicit vascular lumen surface, with an average error of 0.08 mm.
MuFuRU: The Multi-Function Recurrent Unit
Authors: Dirk Weissenborn, Tim Rocktäschel
Date: 2016-06-09 00:00:00
Tasks: Language Modeling, Language Modelling, Sentiment Analysis

Recurrent neural networks such as the GRU and LSTM found wide adoption in
natural language processing and achieve state-of-the-art results for many
tasks. These models are characterized by a memory state that can be written to
and read from by applying gated composition operations to the current input and
the previous state. However, they only cover a small subset of potentially
useful compositions. We propose Multi-Function Recurrent Units (MuFuRUs) that
allow for arbitrary differentiable functions as composition operations.
Furthermore, MuFuRUs allow for an input- and state-dependent choice of these
composition operations that is learned. Our experiments demonstrate that the
additional functionality helps in different sequence modeling tasks, including
the evaluation of propositional logic formulae, language modeling and sentiment
analysis.
On Projected Stochastic Gradient Descent Algorithm with Weighted Averaging for Least Squares Regression
Authors: Kobi Cohen, Angelia Nedic, R. Srikant
Date: 2016-06-09 00:00:00
Tasks: regression

The problem of least squares regression of a $d$-dimensional unknown
parameter is considered. A stochastic gradient descent based algorithm with
weighted iterate-averaging that uses a single pass over the data is studied and
its convergence rate is analyzed. We first consider a bounded constraint set of
the unknown parameter. Under some standard regularity assumptions, we provide
an explicit $O(1/k)$ upper bound on the convergence rate, depending on the
variance (due to the additive noise in the measurements) and the size of the
constraint set. We show that the variance term dominates the error and
decreases with rate $1/k$, while the term which is related to the size of the
constraint set decreases with rate $\log k/k^2$. We then compare the asymptotic
ratio $\rho$ between the convergence rate of the proposed scheme and the
empirical risk minimizer (ERM) as the number of iterations approaches infinity.
We show that $\rho\leq 4$ under some mild conditions for all $d\geq 1$. We
further improve the upper bound by showing that $\rho\leq 4/3$ for the case of
$d=1$ and unbounded parameter set. Simulation results demonstrate strong
performance of the algorithm as compared to existing methods, and coincide with
$\rho\leq 4/3$ even for large $d$ in practice.
Question Answering on Freebase via Relation Extraction and Textual Evidence
Authors: Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang, Dongyan Zhao
Date: 2016-03-03 00:00:00
Proceeding: ACL 2016 8
Tasks: Question Answering, Relation, Relation Extraction, Semantic Parsing

Existing knowledge-based question answering systems often rely on small
annotated training data. While shallow methods like relation extraction are
robust to data scarcity, they are less expressive than the deep meaning
representation methods like semantic parsing, thereby failing at answering
questions involving multiple constraints. Here we alleviate this problem by
empowering a relation extraction method with additional evidence from
Wikipedia. We first present a neural network based relation extractor to
retrieve the candidate answers from Freebase, and then infer over Wikipedia to
validate these answers. Experiments on the WebQuestions question answering
dataset show that our method achieves an F_1 of 53.3%, a substantial
improvement over the state-of-the-art.