pwc-archive/papers-with-abstracts huggingface dataset:
DASA: Domain Adaptation in Stacked Autoencoders using Systematic Dropout
Authors: Abhijit Guha Roy, Debdoot Sheet
Date: 2016-03-19 00:00:00
Tasks: Domain Adaptation, Retinal Vessel Segmentation

Domain adaptation deals with adapting behaviour of machine learning based
systems trained using samples in source domain to their deployment in target
domain where the statistics of samples in both domains are dissimilar. The task
of directly training or adapting a learner in the target domain is challenged
by lack of abundant labeled samples. In this paper we propose a technique for
domain adaptation in stacked autoencoder (SAE) based deep neural networks (DNN)
performed in two stages: (i) unsupervised weight adaptation using systematic
dropouts in mini-batch training, (ii) supervised fine-tuning with limited
number of labeled samples in target domain. We experimentally evaluate
performance in the problem of retinal vessel segmentation where the SAE-DNN is
trained using large number of labeled samples in the source domain (DRIVE
dataset) and adapted using less number of labeled samples in target domain
(STARE dataset). The performance of SAE-DNN measured using $logloss$ in source
domain is $0.19$, without and with adaptation are $0.40$ and $0.18$, and $0.39$
when trained exclusively with limited samples in target domain. The area under
ROC curve is observed respectively as $0.90$, $0.86$, $0.92$ and $0.87$. The
high efficiency of vessel segmentation with DASA strongly substantiates our
claim.
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
Authors: Huijuan Xu, Kate Saenko
Date: 2015-11-17 00:00:00
Tasks: Image Captioning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We address the problem of Visual Question Answering (VQA), which requires
joint image and language understanding to answer a question about a given
photograph. Recent approaches have applied deep image captioning methods based
on convolutional-recurrent networks to this problem, but have failed to model
spatial inference. To remedy this, we propose a model we call the Spatial
Memory Network and apply it to the VQA task. Memory networks are recurrent
neural networks with an explicit attention mechanism that selects certain parts
of the information stored in memory. Our Spatial Memory Network stores neuron
activations from different spatial regions of the image in its memory, and uses
the question to choose relevant regions for computing the answer, a process of
which constitutes a single "hop" in the network. We propose a novel spatial
attention architecture that aligns words with image patches in the first hop,
and obtain improved results by adding a second attention hop which considers
the whole question to choose visual evidence based on the results of the first
hop. To better understand the inference process learned by the network, we
design synthetic questions that specifically require spatial inference and
visualize the attention weights. We evaluate our model on two published visual
question answering datasets, DAQUAR [1] and VQA [2], and obtain improved
results compared to a strong deep baseline model (iBOWIMG) which concatenates
image and question features to predict the answer [3].
L0-norm Sparse Graph-regularized SVD for Biclustering
Authors: Wenwen Min, Juan Liu, Shihua Zhang
Date: 2016-03-19 00:00:00
Tasks: Blocking

Learning the "blocking" structure is a central challenge for high dimensional
data (e.g., gene expression data). Recently, a sparse singular value
decomposition (SVD) has been used as a biclustering tool to achieve this goal.
However, this model ignores the structural information between variables (e.g.,
gene interaction graph). Although typical graph-regularized norm can
incorporate such prior graph information to get accurate discovery and better
interpretability, it fails to consider the opposite effect of variables with
different signs. Motivated by the development of sparse coding and
graph-regularized norm, we propose a novel sparse graph-regularized SVD as a
powerful biclustering tool for analyzing high-dimensional data. The key of this
method is to impose two penalties including a novel graph-regularized norm
($|\pmb{u}|\pmb{L}|\pmb{u}|$) and $L_0$-norm ($\|\pmb{u}\|_0$) on singular
vectors to induce structural sparsity and enhance interpretability. We design
an efficient Alternating Iterative Sparse Projection (AISP) algorithm to solve
it. Finally, we apply our method and related ones to simulated and real data to
show its efficiency in capturing natural blocking structures.
A Multi-scale Multiple Instance Video Description Network
Authors: Huijuan Xu, Subhashini Venugopalan, Vasili Ramanishka, Marcus Rohrbach, Kate Saenko
Date: 2015-05-21 00:00:00
Tasks: Image Segmentation, Multiple Instance Learning, Semantic Segmentation, Sentence, Video Description

Generating natural language descriptions for in-the-wild videos is a
challenging task. Most state-of-the-art methods for solving this problem borrow
existing deep convolutional neural network (CNN) architectures (AlexNet,
GoogLeNet) to extract a visual representation of the input video. However,
these deep CNN architectures are designed for single-label centered-positioned
object classification. While they generate strong semantic features, they have
no inherent structure allowing them to detect multiple objects of different
sizes and locations in the frame. Our paper tries to solve this problem by
integrating the base CNN into several fully convolutional neural networks
(FCNs) to form a multi-scale network that handles multiple receptive field
sizes in the original image. FCNs, previously applied to image segmentation,
can generate class heat-maps efficiently compared to sliding window mechanisms,
and can easily handle multiple scales. To further handle the ambiguity over
multiple objects and locations, we incorporate the Multiple Instance Learning
mechanism (MIL) to consider objects in different positions and at different
scales simultaneously. We integrate our multi-scale multi-instance architecture
with a sequence-to-sequence recurrent neural network to generate sentence
descriptions based on the visual representation. Ours is the first end-to-end
trainable architecture that is capable of multi-scale region processing.
Evaluation on a Youtube video dataset shows the advantage of our approach
compared to the original single-scale whole frame CNN model. Our flexible and
efficient architecture can potentially be extended to support other video
processing tasks.
New Optimisation Methods for Machine Learning
Authors: Aaron Defazio
Date: 2015-10-09 00:00:00
Tasks: BIG-bench Machine Learning, Philosophy

A thesis submitted for the degree of Doctor of Philosophy of The Australian
National University.
  In this work we introduce several new optimisation methods for problems in
machine learning. Our algorithms broadly fall into two categories: optimisation
of finite sums and of graph structured objectives. The finite sum problem is
simply the minimisation of objective functions that are naturally expressed as
a summation over a large number of terms, where each term has a similar or
identical weight. Such objectives most often appear in machine learning in the
empirical risk minimisation framework in the non-online learning setting. The
second category, that of graph structured objectives, consists of objectives
that result from applying maximum likelihood to Markov random field models.
Unlike the finite sum case, all the non-linearity is contained within a
partition function term, which does not readily decompose into a summation.
  For the finite sum problem, we introduce the Finito and SAGA algorithms, as
well as variants of each.
  For graph-structured problems, we take three complementary approaches. We
look at learning the parameters for a fixed structure, learning the structure
independently, and learning both simultaneously. Specifically, for the combined
approach, we introduce a new method for encouraging graph structures with the
"scale-free" property. For the structure learning problem, we establish
SHORTCUT, a O(n^{2.5}) expected time approximate structure learning method for
Gaussian graphical models. For problems where the structure is known but the
parameters unknown, we introduce an approximate maximum likelihood learning
algorithm that is capable of learning a useful subclass of Gaussian graphical
models.
Readability-based Sentence Ranking for Evaluating Text Simplification
Authors: Sowmya Vajjala, Detmar Meurers
Date: 2016-03-18 00:00:00
Tasks: Articles, Cross-corpus, Sentence, Text Simplification

We propose a new method for evaluating the readability of simplified
sentences through pair-wise ranking. The validity of the method is established
through in-corpus and cross-corpus evaluation experiments. The approach
correctly identifies the ranking of simplified and unsimplified sentences in
terms of their reading level with an accuracy of over 80%, significantly
outperforming previous results. To gain qualitative insights into the nature of
simplification at the sentence level, we studied the impact of specific
linguistic features. We empirically confirm that both word-level and syntactic
features play a role in comparing the degree of simplification of authentic
data. To carry out this research, we created a new sentence-aligned corpus from
professionally simplified news articles. The new corpus resource enriches the
empirical basis of sentence-level simplification research, which so far relied
on a single resource. Most importantly, it facilitates cross-corpus evaluation
for simplification, a key step towards generalizable results.
A Message Passing Algorithm for the Problem of Path Packing in Graphs
Authors: Patrick Eschenfeldt, David Gamarnik
Date: 2016-03-18 00:00:00

We consider the problem of packing node-disjoint directed paths in a directed
graph. We consider a variant of this problem where each path starts within a
fixed subset of root nodes, subject to a given bound on the length of paths.
This problem is motivated by the so-called kidney exchange problem, but has
potential other applications and is interesting in its own right.
  We propose a new algorithm for this problem based on the message
passing/belief propagation technique. A priori this problem does not have an
associated graphical model, so in order to apply a belief propagation algorithm
we provide a novel representation of the problem as a graphical model. Standard
belief propagation on this model has poor scaling behavior, so we provide an
efficient implementation that significantly decreases the complexity. We
provide numerical results comparing the performance of our algorithm on both
artificially created graphs and real world networks to several alternative
algorithms, including algorithms based on integer programming (IP) techniques.
These comparisons show that our algorithm scales better to large instances than
IP-based algorithms and often finds better solutions than a simple algorithm
that greedily selects the longest path from each root node. In some cases it
also finds better solutions than the ones found by IP-based algorithms even
when the latter are allowed to run significantly longer than our algorithm.
Linear Dimensionality Reduction: Survey, Insights, and Generalizations
Authors: John P. Cunningham, Zoubin Ghahramani
Date: 2014-06-03 00:00:00
Tasks: Dimensionality Reduction, Metric Learning, Survey

Linear dimensionality reduction methods are a cornerstone of analyzing high
dimensional data, due to their simple geometric interpretations and typically
attractive computational properties. These methods capture many data features
of interest, such as covariance, dynamical structure, correlation between data
sets, input-output relationships, and margin between data classes. Methods have
been developed with a variety of names and motivations in many fields, and
perhaps as a result the connections between all these methods have not been
highlighted. Here we survey methods from this disparate literature as
optimization programs over matrix manifolds. We discuss principal component
analysis, factor analysis, linear multidimensional scaling, Fisher's linear
discriminant analysis, canonical correlations analysis, maximum autocorrelation
factors, slow feature analysis, sufficient dimensionality reduction,
undercomplete independent component analysis, linear regression, distance
metric learning, and more. This optimization framework gives insight to some
rarely discussed shortcomings of well-known methods, such as the suboptimality
of certain eigenvector solutions. Modern techniques for optimization over
matrix manifolds enable a generic linear dimensionality reduction solver, which
accepts as input data and an objective to be optimized, and returns, as output,
an optimal low-dimensional projection of the data. This simple optimization
framework further allows straightforward generalizations and novel variants of
classical methods, which we demonstrate here by creating an
orthogonal-projection canonical correlations analysis. More broadly, this
survey and generic solver suggest that linear dimensionality reduction can move
toward becoming a blackbox, objective-agnostic numerical technology.
Best-of-K Bandits
Authors: Max Simchowitz, Kevin Jamieson, Benjamin Recht
Date: 2016-03-09 00:00:00

This paper studies the Best-of-K Bandit game: At each time the player chooses
a subset S among all N-choose-K possible options and observes reward max(X(i) :
i in S) where X is a random vector drawn from a joint distribution. The
objective is to identify the subset that achieves the highest expected reward
with high probability using as few queries as possible. We present
distribution-dependent lower bounds based on a particular construction which
force a learner to consider all N-choose-K subsets, and match naive extensions
of known upper bounds in the bandit setting obtained by treating each subset as
a separate arm. Nevertheless, we present evidence that exhaustive search may be
avoided for certain, favorable distributions because the influence of
high-order order correlations may be dominated by lower order statistics.
Finally, we present an algorithm and analysis for independent arms, which
mitigates the surprising non-trivial information occlusion that occurs due to
only observing the max in the subset. This may inform strategies for more
general dependent measures, and we complement these result with independent-arm
lower bounds.
Document Neural Autoregressive Distribution Estimation
Authors: Stanislas Lauly, Yin Zheng, Alexandre Allauzen, Hugo Larochelle
Date: 2016-03-18 00:00:00

We present an approach based on feed-forward neural networks for learning the
distribution of textual documents. This approach is inspired by the Neural
Autoregressive Distribution Estimator(NADE) model, which has been shown to be a
good estimator of the distribution of discrete-valued igh-dimensional vectors.
In this paper, we present how NADE can successfully be adapted to the case of
textual data, retaining from NADE the property that sampling or computing the
probability of observations can be done exactly and efficiently. The approach
can also be used to learn deep representations of documents that are
competitive to those learned by the alternative topic modeling approaches.
Finally, we describe how the approach can be combined with a regular neural
network N-gram model and substantially improve its performance, by making its
learned representation sensitive to the larger, document-specific context.
Transferring Learned Microcalcification Group Detection from 2D Mammography to 3D Digital Breast Tomosynthesis Using a Hierarchical Model and Scope-based Normalization Features
Authors: Yin Yin, Sergei V. Fotin, Hrishikesh Haldankar, Jeffrey W. Hoffmeister, Senthil Periaswamy
Date: 2016-03-18 00:00:00

A novel hierarchical model is introduced to solve a general problem of
detecting groups of similar objects. Under this model, detection of groups is
performed in hierarchically organized layers while each layer represents a
scope for target objects. The processing of these layers involves sequential
extraction of appearance features for an individual object, consistency
measurement features for nearby objects, and finally the distribution features
for all objects within the group. Using the concept of scope-based
normalization, the extracted features not only enhance local contrast of an
individual object, but also provide consistent characterization for all related
objects. As an example, a microcalcification group detection system for 2D
mammography was developed, and then the learned model was transferred to 3D
digital breast tomosynthesis without any retraining or fine-tuning. The
detection system demonstrated state-of-the-art performance and detected 96% of
cancerous lesions at the rate of 1.2 false positives per volume as measured on
an independent tomosynthesis test set.
Geometric Hypergraph Learning for Visual Tracking
Authors: Dawei Du, Honggang Qi, Longyin Wen, Qi Tian, Qingming Huang, Siwei Lyu
Date: 2016-03-18 00:00:00
Tasks: Visual Tracking

Graph based representation is widely used in visual tracking field by finding
correct correspondences between target parts in consecutive frames. However,
most graph based trackers consider pairwise geometric relations between local
parts. They do not make full use of the target's intrinsic structure, thereby
making the representation easily disturbed by errors in pairwise affinities
when large deformation and occlusion occur. In this paper, we propose a
geometric hypergraph learning based tracking method, which fully exploits
high-order geometric relations among multiple correspondences of parts in
consecutive frames. Then visual tracking is formulated as the mode-seeking
problem on the hypergraph in which vertices represent correspondence hypotheses
and hyperedges describe high-order geometric relations. Besides, a
confidence-aware sampling method is developed to select representative vertices
and hyperedges to construct the geometric hypergraph for more robustness and
scalability. The experiments are carried out on two challenging datasets
(VOT2014 and Deform-SOT) to demonstrate that the proposed method performs
favorable against other existing trackers.
Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
Authors: Kyle Cranmer, Juan Pavez, Gilles Louppe
Date: 2015-06-06 00:00:00
Tasks: Dimensionality Reduction

In many fields of science, generalized likelihood ratio tests are established
tools for statistical inference. At the same time, it has become increasingly
common that a simulator (or generative model) is used to describe complex
processes that tie parameters $\theta$ of an underlying theory and measurement
apparatus to high-dimensional observations $\mathbf{x}\in \mathbb{R}^p$.
However, simulator often do not provide a way to evaluate the likelihood
function for a given observation $\mathbf{x}$, which motivates a new class of
likelihood-free inference algorithms. In this paper, we show that likelihood
ratios are invariant under a specific class of dimensionality reduction maps
$\mathbb{R}^p \mapsto \mathbb{R}$. As a direct consequence, we show that
discriminative classifiers can be used to approximate the generalized
likelihood ratio statistic when only a generative model for the data is
available. This leads to a new machine learning-based approach to
likelihood-free inference that is complementary to Approximate Bayesian
Computation, and which does not require a prior on the model parameters.
Experimental results on artificial problems with known exact likelihoods
illustrate the potential of the proposed method.
Predicting Grades
Authors: Yannick Meier, Jie Xu, Onur Atan, Mihaela van der Schaar
Date: 2015-08-16 00:00:00
Tasks: Prediction

To increase efficacy in traditional classroom courses as well as in Massive
Open Online Courses (MOOCs), automated systems supporting the instructor are
needed. One important problem is to automatically detect students that are
going to do poorly in a course early enough to be able to take remedial
actions. Existing grade prediction systems focus on maximizing the accuracy of
the prediction while overseeing the importance of issuing timely and
personalized predictions. This paper proposes an algorithm that predicts the
final grade of each student in a class. It issues a prediction for each student
individually, when the expected accuracy of the prediction is sufficient. The
algorithm learns online what is the optimal prediction and time to issue a
prediction based on past history of students' performance in a course. We
derive a confidence estimate for the prediction accuracy and demonstrate the
performance of our algorithm on a dataset obtained based on the performance of
approximately 700 UCLA undergraduate students who have taken an introductory
digital signal processing over the past 7 years. We demonstrate that for 85% of
the students we can predict with 76% accuracy whether they are going do well or
poorly in the class after the 4th course week. Using data obtained from a pilot
course, our methodology suggests that it is effective to perform early in-class
assessments such as quizzes, which result in timely performance prediction for
each student, thereby enabling timely interventions by the instructor (at the
student or class level) when necessary.
Multivariate Median Filters and Partial Differential Equations
Authors: Martin Welk
Date: 2015-09-27 00:00:00

Multivariate median filters have been proposed as generalisations of the
well-established median filter for grey-value images to multi-channel images.
As multivariate median, most of the recent approaches use the $L^1$ median,
i.e.\ the minimiser of an objective function that is the sum of distances to
all input points. Many properties of univariate median filters generalise to
such a filter. However, the famous result by Guichard and Morel about
approximation of the mean curvature motion PDE by median filtering does not
have a comparably simple counterpart for $L^1$ multivariate median filtering.
We discuss the affine equivariant Oja median and the affine equivariant
transformation--retransformation $L^1$ median as alternatives to $L^1$ median
filtering. We analyse multivariate median filters in a space-continuous
setting, including the formulation of a space-continuous version of the
transformation--retransformation $L^1$ median, and derive PDEs approximated by
these filters in the cases of bivariate planar images, three-channel volume
images and three-channel planar images. The PDEs for the affine equivariant
filters can be interpreted geometrically as combinations of a diffusion and a
principal-component-wise curvature motion contribution with a cross-effect term
based on torsions of principal components. Numerical experiments are presented
that demonstrate the validity of the approximation results.
An Infinite Restricted Boltzmann Machine
Authors: Marc-Alexandre Côté, Hugo Larochelle
Date: 2015-02-09 00:00:00

We present a mathematical construction for the restricted Boltzmann machine
(RBM) that doesn't require specifying the number of hidden units. In fact, the
hidden layer size is adaptive and can grow during training. This is obtained by
first extending the RBM to be sensitive to the ordering of its hidden units.
Then, thanks to a carefully chosen definition of the energy function, we show
that the limit of infinitely many hidden units is well defined. As with RBM,
approximate maximum likelihood training can be performed, resulting in an
algorithm that naturally and adaptively adds trained hidden units during
learning. We empirically study the behaviour of this infinite RBM, showing that
its performance is competitive to that of the RBM, while not requiring the
tuning of a hidden layer size.