pwc-archive/papers-with-abstracts huggingface dataset:
Linear Models of Computation and Program Learning
Authors: Michael Bukatin, Steve Matthews
Date: 2015-12-15 00:00:00
Tasks: Probabilistic Programming

We consider two classes of computations which admit taking linear
combinations of execution runs: probabilistic sampling and generalized
animation. We argue that the task of program learning should be more tractable
for these architectures than for conventional deterministic programs. We look
at the recent advances in the "sampling the samplers" paradigm in higher-order
probabilistic programming. We also discuss connections between partial
inconsistency, non-monotonic inference, and vector semantics.
Max-margin Deep Generative Models
Authors: Chongxuan Li, Jun Zhu, Tianlin Shi, Bo Zhang
Date: 2015-04-26 00:00:00
Proceeding: NeurIPS 2015 12

Deep generative models (DGMs) are effective on learning multilayered
representations of complex data and performing inference of input data by
exploring the generative ability. However, little work has been done on
examining or empowering the discriminative ability of DGMs on making accurate
predictions. This paper presents max-margin deep generative models (mmDGMs),
which explore the strongly discriminative principle of max-margin learning to
improve the discriminative power of DGMs, while retaining the generative
capability. We develop an efficient doubly stochastic subgradient algorithm for
the piecewise linear objective. Empirical results on MNIST and SVHN datasets
demonstrate that (1) max-margin learning can significantly improve the
prediction performance of DGMs and meanwhile retain the generative ability; and
(2) mmDGMs are competitive to the state-of-the-art fully discriminative
networks by employing deep convolutional neural networks (CNNs) as both
recognition and generative models.
Semantic-enriched Visual Vocabulary Construction in a Weakly Supervised Context
Authors: Marian-Andrei Rizoiu, Julien Velcin, Stéphane Lallich
Date: 2015-12-14 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

One of the prevalent learning tasks involving images is content-based image
classification. This is a difficult task especially because the low-level
features used to digitally describe images usually capture little information
about the semantics of the images. In this paper, we tackle this difficulty by
enriching the semantic content of the image representation by using external
knowledge. The underlying hypothesis of our work is that creating a more
semantically rich representation for images would yield higher machine learning
performances, without the need to modify the learning algorithms themselves.
The external semantic information is presented under the form of non-positional
image labels, therefore positioning our work in a weakly supervised context.
Two approaches are proposed: the first one leverages the labels into the visual
vocabulary construction algorithm, the result being dedicated visual
vocabularies. The second approach adds a filtering phase as a pre-processing of
the vocabulary construction. Known positive and known negative sets are
constructed and features that are unlikely to be associated with the objects
denoted by the labels are filtered. We apply our proposition to the task of
content-based image classification and we show that semantically enriching the
image representation yields higher classification performances than the
baseline representation.
Relaxed Linearized Algorithms for Faster X-Ray CT Image Reconstruction
Authors: Hung Nien, Jeffrey A. Fessler
Date: 2015-12-14 00:00:00
Tasks: Computed Tomography (CT), Image Reconstruction

Statistical image reconstruction (SIR) methods are studied extensively for
X-ray computed tomography (CT) due to the potential of acquiring CT scans with
reduced X-ray dose while maintaining image quality. However, the longer
reconstruction time of SIR methods hinders their use in X-ray CT in practice.
To accelerate statistical methods, many optimization techniques have been
investigated. Over-relaxation is a common technique to speed up convergence of
iterative algorithms. For instance, using a relaxation parameter that is close
to two in alternating direction method of multipliers (ADMM) has been shown to
speed up convergence significantly. This paper proposes a relaxed linearized
augmented Lagrangian (AL) method that shows theoretical faster convergence rate
with over-relaxation and applies the proposed relaxed linearized AL method to
X-ray CT image reconstruction problems. Experimental results with both
simulated and real CT scan data show that the proposed relaxed algorithm (with
ordered-subsets [OS] acceleration) is about twice as fast as the existing
unrelaxed fast algorithms, with negligible computation and memory overhead.
Minimal Realization Problems for Hidden Markov Models
Authors: Qingqing Huang, Rong Ge, Sham Kakade, Munther Dahleh
Date: 2014-11-13 00:00:00
Tasks: Tensor Decomposition

Consider a stationary discrete random process with alphabet size d, which is
assumed to be the output process of an unknown stationary Hidden Markov Model
(HMM). Given the joint probabilities of finite length strings of the process,
we are interested in finding a finite state generative model to describe the
entire process. In particular, we focus on two classes of models: HMMs and
quasi-HMMs, which is a strictly larger class of models containing HMMs. In the
main theorem, we show that if the random process is generated by an HMM of
order less or equal than k, and whose transition and observation probability
matrix are in general position, namely almost everywhere on the parameter
space, both the minimal quasi-HMM realization and the minimal HMM realization
can be efficiently computed based on the joint probabilities of all the length
N strings, for N > 4 lceil log_d(k) rceil +1. In this paper, we also aim to
compare and connect the two lines of literature: realization theory of HMMs,
and the recent development in learning latent variable models with tensor
decomposition techniques.
Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs
Authors: Shuangfei Zhai, Zhongfei Zhang
Date: 2015-12-14 00:00:00
Tasks: Link Prediction, Multiview Learning

Matrix factorization (MF) and Autoencoder (AE) are among the most successful
approaches of unsupervised learning. While MF based models have been
extensively exploited in the graph modeling and link prediction literature, the
AE family has not gained much attention. In this paper we investigate both MF
and AE's application to the link prediction problem in sparse graphs. We show
the connection between AE and MF from the perspective of multiview learning,
and further propose MF+AE: a model training MF and AE jointly with shared
parameters. We apply dropout to training both the MF and AE parts, and show
that it can significantly prevent overfitting by acting as an adaptive
regularization. We conduct experiments on six real world sparse graph datasets,
and show that MF+AE consistently outperforms the competing methods, especially
on datasets that demonstrate strong non-cohesive structures.
Semisupervised Autoencoder for Sentiment Analysis
Authors: Shuangfei Zhai, Zhongfei Zhang
Date: 2015-12-14 00:00:00
Tasks: Sentiment Analysis

In this paper, we investigate the usage of autoencoders in modeling textual
data. Traditional autoencoders suffer from at least two aspects: scalability
with the high dimensionality of vocabulary size and dealing with
task-irrelevant words. We address this problem by introducing supervision via
the loss function of autoencoders. In particular, we first train a linear
classifier on the labeled data, then define a loss for the autoencoder with the
weights learned from the linear classifier. To reduce the bias brought by one
single classifier, we define a posterior probability distribution on the
weights of the classifier, and derive the marginalized loss of the autoencoder
with Laplace approximation. We show that our choice of loss function can be
rationalized from the perspective of Bregman Divergence, which justifies the
soundness of our model. We evaluate the effectiveness of our model on six
sentiment analysis datasets, and show that our model significantly outperforms
all the competing methods with respect to classification accuracy. We also show
that our model is able to take advantage of unlabeled dataset and get improved
performance. We further show that our model successfully learns highly
discriminative feature maps, which explains its superior performance.
Memory-based control with recurrent neural networks
Authors: Nicolas Heess, Jonathan J. Hunt, Timothy P. Lillicrap, David Silver
Date: 2015-12-14 00:00:00
Tasks: continuous-control, Continuous Control, Reinforcement Learning

Partially observed control problems are a challenging aspect of reinforcement
learning. We extend two related, model-free algorithms for continuous control
-- deterministic policy gradient and stochastic value gradient -- to solve
partially observed domains using recurrent neural networks trained with
backpropagation through time.
  We demonstrate that this approach, coupled with long-short term memory is
able to solve a variety of physical control problems exhibiting an assortment
of memory requirements. These include the short-term integration of information
from noisy sensors and the identification of system parameters, as well as
long-term memory problems that require preserving information over many time
steps. We also demonstrate success on a combined exploration and memory problem
in the form of a simplified version of the well-known Morris water maze task.
Finally, we show that our approach can deal with high-dimensional observations
by learning directly from pixels.
  We find that recurrent deterministic and stochastic policies are able to
learn similarly good solutions to these tasks, including the water maze where
the agent must learn effective search strategies.
Near-Optimal Bounds for Binary Embeddings of Arbitrary Sets
Authors: Samet Oymak, Ben Recht
Date: 2015-12-14 00:00:00

We study embedding a subset $K$ of the unit sphere to the Hamming cube
$\{-1,+1\}^m$. We characterize the tradeoff between distortion and sample
complexity $m$ in terms of the Gaussian width $\omega(K)$ of the set. For
subspaces and several structured sets we show that Gaussian maps provide the
optimal tradeoff $m\sim \delta^{-2}\omega^2(K)$, in particular for $\delta$
distortion one needs $m\approx\delta^{-2}{d}$ where $d$ is the subspace
dimension. For general sets, we provide sharp characterizations which reduces
to $m\approx{\delta^{-4}}{\omega^2(K)}$ after simplification. We provide
improved results for local embedding of points that are in close proximity of
each other which is related to locality sensitive hashing. We also discuss
faster binary embedding where one takes advantage of an initial sketching
procedure based on Fast Johnson-Lindenstauss Transform. Finally, we list
several numerical observations and discuss open problems.
Sentence Entailment in Compositional Distributional Semantics
Authors: Esma Balkir, Dimitri Kartsaklis, Mehrnoosh Sadrzadeh
Date: 2015-12-14 00:00:00
Tasks: Sentence

Distributional semantic models provide vector representations for words by
gathering co-occurrence frequencies from corpora of text. Compositional
distributional models extend these from words to phrases and sentences. In
categorical compositional distributional semantics, phrase and sentence
representations are functions of their grammatical structure and
representations of the words therein. In this setting, grammatical structures
are formalised by morphisms of a compact closed category and meanings of words
are formalised by objects of the same category. These can be instantiated in
the form of vectors or density matrices. This paper concerns the applications
of this model to phrase and sentence level entailment. We argue that
entropy-based distances of vectors and density matrices provide a good
candidate to measure word-level entailment, show the advantage of density
matrices over vectors for word level entailments, and prove that these
distances extend compositionally from words to phrases and sentences. We
exemplify our theoretical constructions on real data and a toy entailment
dataset and provide preliminary experimental evidence.
Sparse Representation of a Blur Kernel for Blind Image Restoration
Authors: Chia-Chen Lee, Wen-Liang Hwang
Date: 2015-12-14 00:00:00
Tasks: Image Restoration

Blind image restoration is a non-convex problem which involves restoration of
images from an unknown blur kernel. The factors affecting the performance of
this restoration are how much prior information about an image and a blur
kernel are provided and what algorithm is used to perform the restoration task.
Prior information on images is often employed to restore the sharpness of the
edges of an image. By contrast, no consensus is still present regarding what
prior information to use in restoring from a blur kernel due to complex image
blurring processes. In this paper, we propose modelling of a blur kernel as a
sparse linear combinations of basic 2-D patterns. Our approach has a
competitive edge over the existing blur kernel modelling methods because our
method has the flexibility to customize the dictionary design, which makes it
well-adaptive to a variety of applications. As a demonstration, we construct a
dictionary formed by basic patterns derived from the Kronecker product of
Gaussian sequences. We also compare our results with those derived by other
state-of-the-art methods, in terms of peak signal to noise ratio (PSNR).
Instance-aware Semantic Segmentation via Multi-task Network Cascades
Authors: Jifeng Dai, Kaiming He, Jian Sun
Date: 2015-12-14 00:00:00
Proceeding: CVPR 2016 6
Tasks: Multi-Human Parsing, object-detection, Object Detection, Segmentation, Semantic Segmentation

Semantic segmentation research has recently witnessed rapid progress, but
many leading methods are unable to identify object instances. In this paper, we
present Multi-task Network Cascades for instance-aware semantic segmentation.
Our model consists of three networks, respectively differentiating instances,
estimating masks, and categorizing objects. These networks form a cascaded
structure, and are designed to share their convolutional features. We develop
an algorithm for the nontrivial end-to-end training of this causal, cascaded
structure. Our solution is a clean, single-step training framework and can be
generalized to cascades that have more stages. We demonstrate state-of-the-art
instance-aware semantic segmentation accuracy on PASCAL VOC. Meanwhile, our
method takes only 360ms testing an image using VGG-16, which is two orders of
magnitude faster than previous systems for this challenging problem. As a by
product, our method also achieves compelling object detection results which
surpass the competitive Fast/Faster R-CNN systems.
  The method described in this paper is the foundation of our submissions to
the MS COCO 2015 segmentation competition, where we won the 1st place.
Image patch analysis of sunspots and active regions. I. Intrinsic dimension and correlation analysis
Authors: Kevin R. Moon, Jimmy J. Li, Veronique Delouille, Ruben De Visscher, Fraser Watson, Alfred O. Hero III
Date: 2015-03-13 00:00:00
Tasks: Clustering, Dictionary Learning

The flare-productivity of an active region is observed to be related to its
spatial complexity. Mount Wilson or McIntosh sunspot classifications measure
such complexity but in a categorical way, and may therefore not use all the
information present in the observations. Moreover, such categorical schemes
hinder a systematic study of an active region's evolution for example. We
propose fine-scale quantitative descriptors for an active region's complexity
and relate them to the Mount Wilson classification. We analyze the local
correlation structure within continuum and magnetogram data, as well as the
cross-correlation between continuum and magnetogram data. We compute the
intrinsic dimension, partial correlation, and canonical correlation analysis
(CCA) of image patches of continuum and magnetogram active region images taken
from the SOHO-MDI instrument. We use masks of sunspots derived from continuum
as well as larger masks of magnetic active regions derived from the magnetogram
to analyze separately the core part of an active region from its surrounding
part. We find the relationship between complexity of an active region as
measured by Mount Wilson and the intrinsic dimension of its image patches.
Partial correlation patterns exhibit approximately a third-order Markov
structure. CCA reveals different patterns of correlation between continuum and
magnetogram within the sunspots and in the region surrounding the sunspots.
These results also pave the way for patch-based dictionary learning with a view
towards automatic clustering of active regions.
Bayesian Policy Reuse
Authors: Benjamin Rosman, Majd Hawasly, Subramanian Ramamoorthy
Date: 2015-05-01 00:00:00
Tasks: Bayesian Optimisation

A long-lived autonomous agent should be able to respond online to novel
instances of tasks from a familiar domain. Acting online requires 'fast'
responses, in terms of rapid convergence, especially when the task instance has
a short duration, such as in applications involving interactions with humans.
These requirements can be problematic for many established methods for learning
to act. In domains where the agent knows that the task instance is drawn from a
family of related tasks, albeit without access to the label of any given
instance, it can choose to act through a process of policy reuse from a
library, rather than policy learning from scratch. In policy reuse, the agent
has prior knowledge of the class of tasks in the form of a library of policies
that were learnt from sample task instances during an offline training phase.
We formalise the problem of policy reuse, and present an algorithm for
efficiently responding to a novel task instance by reusing a policy from the
library of existing policies, where the choice is based on observed 'signals'
which correlate to policy performance. We achieve this by posing the problem as
a Bayesian choice problem with a corresponding notion of an optimal response,
but the computation of that response is in many cases intractable. Therefore,
to reduce the computation cost of the posterior, we follow a Bayesian
optimisation approach and define a set of policy selection functions, which
balance exploration in the policy library against exploitation of previously
tried policies, together with a model of expected performance of the policy
library on their corresponding task instances. We validate our method in
several simulated domains of interactive, short-duration episodic tasks,
showing rapid convergence in unknown task variations.
Decoding index finger position from EEG using random forests
Authors: Sebastian Weichwald, Timm Meyer, Bernhard Schölkopf, Tonio Ball, Moritz Grosse-Wentrup
Date: 2015-12-14 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Open-Ended Question Answering, Position

While invasively recorded brain activity is known to provide detailed
information on motor commands, it is an open question at what level of detail
information about positions of body parts can be decoded from non-invasively
acquired signals. In this work it is shown that index finger positions can be
differentiated from non-invasive electroencephalographic (EEG) recordings in
healthy human subjects. Using a leave-one-subject-out cross-validation
procedure, a random forest distinguished different index finger positions on a
numerical keyboard above chance-level accuracy. Among the different spectral
features investigated, high $\beta$-power (20-30 Hz) over contralateral
sensorimotor cortex carried most information about finger position. Thus, these
findings indicate that finger position is in principle decodable from
non-invasive features of brain activity that generalize across individuals.
A Proximal Approach for Sparse Multiclass SVM
Authors: G. Chierchia, Nelly Pustelnik, Jean-Christophe Pesquet, B. Pesquet-Popescu
Date: 2015-01-15 00:00:00

Sparsity-inducing penalties are useful tools to design multiclass support
vector machines (SVMs). In this paper, we propose a convex optimization
approach for efficiently and exactly solving the multiclass SVM learning
problem involving a sparse regularization and the multiclass hinge loss
formulated by Crammer and Singer. We provide two algorithms: the first one
dealing with the hinge loss as a penalty term, and the other one addressing the
case when the hinge loss is enforced through a constraint. The related convex
optimization problems can be efficiently solved thanks to the flexibility
offered by recent primal-dual proximal algorithms and epigraphical splitting
techniques. Experiments carried out on several datasets demonstrate the
interest of considering the exact expression of the hinge loss rather than a
smooth approximation. The efficiency of the proposed algorithms w.r.t. several
state-of-the-art methods is also assessed through comparisons of execution
times.
On the Relation between two Rotation Metrics
Authors: Thomas Ruland
Date: 2015-12-14 00:00:00
Tasks: global-optimization, LEMMA, Relation, Vocal Bursts Valence Prediction

In their work "Global Optimization through Rotation Space Search", Richard
Hartley and Fredrik Kahl introduce a global optimization strategy for problems
in geometric computer vision, based on rotation space search using a
branch-and-bound algorithm. In its core, Lemma 2 of their publication is the
important foundation for a class of global optimization algorithms, which is
adopted over a wide range of problems in subsequent publications. This lemma
relates a metric on rotations represented by rotation matrices with a metric on
rotations in axis-angle representation. This work focuses on a proof for this
relationship, which is based on Rodrigues' Rotation Theorem for the composition
of rotations in axis-angle representation.
Watch-Bot: Unsupervised Learning for Reminding Humans of Forgotten Actions
Authors: Chenxia Wu, Jiemi Zhang, Bart Selman, Silvio Savarese, Ashutosh Saxena
Date: 2015-12-14 00:00:00
Tasks: Action Segmentation, Object, Unsupervised Action Segmentation

We present a robotic system that watches a human using a Kinect v2 RGB-D
sensor, detects what he forgot to do while performing an activity, and if
necessary reminds the person using a laser pointer to point out the related
object. Our simple setup can be easily deployed on any assistive robot.
  Our approach is based on a learning algorithm trained in a purely
unsupervised setting, which does not require any human annotations. This makes
our approach scalable and applicable to variant scenarios. Our model learns the
action/object co-occurrence and action temporal relations in the activity, and
uses the learned rich relationships to infer the forgotten action and the
related object. We show that our approach not only improves the unsupervised
action segmentation and action cluster assignment performance, but also
effectively detects the forgotten actions on a challenging human activity RGB-D
video dataset. In robotic experiments, we show that our robot is able to remind
people of forgotten actions successfully.
Controlled Experiments for Word Embeddings
Authors: Benjamin J. Wilson, Adriaan M. J. Schakel
Date: 2015-10-09 00:00:00
Tasks: Word Embeddings

An experimental approach to studying the properties of word embeddings is
proposed. Controlled experiments, achieved through modifications of the
training corpus, permit the demonstration of direct relations between word
properties and word vector direction and length. The approach is demonstrated
using the word2vec CBOW model with experiments that independently vary word
frequency and word co-occurrence noise. The experiments reveal that word vector
length depends more or less linearly on both word frequency and the level of
noise in the co-occurrence distribution of the word. The coefficients of
linearity depend upon the word. The special point in feature space, defined by
the (artificial) word with pure noise in its co-occurrence distribution, is
found to be small but non-zero.