pwc-archive/papers-with-abstracts huggingface dataset:
Alzheimer's Disease Diagnostics by Adaptation of 3D Convolutional Network
Authors: Ehsan Hosseini-Asl, Robert Keynto, Ayman El-Baz
Date: 2016-07-02 00:00:00
Tasks: General Classification, Hippocampus, Skull Stripping

Early diagnosis, playing an important role in preventing progress and
treating the Alzheimer\{'}s disease (AD), is based on classification of
features extracted from brain images. The features have to accurately capture
main AD-related variations of anatomical brain structures, such as, e.g.,
ventricles size, hippocampus shape, cortical thickness, and brain volume. This
paper proposed to predict the AD with a deep 3D convolutional neural network
(3D-CNN), which can learn generic features capturing AD biomarkers and adapt to
different domain datasets. The 3D-CNN is built upon a 3D convolutional
autoencoder, which is pre-trained to capture anatomical shape variations in
structural brain MRI scans. Fully connected upper layers of the 3D-CNN are then
fine-tuned for each task-specific AD classification. Experiments on the
CADDementia MRI dataset with no skull-stripping preprocessing have shown our
3D-CNN outperforms several conventional classifiers by accuracy. Abilities of
the 3D-CNN to generalize the features learnt and adapt to other domains have
been validated on the ADNI dataset.
Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms
Authors: Jianwen Xie, Pamela K. Douglas, Ying Nian Wu, Arthur L. Brody, Ariana E. Anderson
Date: 2016-07-01 00:00:00
Tasks: Time Series Analysis

Brain networks in fMRI are typically identified using spatial independent
component analysis (ICA), yet mathematical constraints such as sparse coding
and positivity both provide alternate biologically-plausible frameworks for
generating brain networks. Non-negative Matrix Factorization (NMF) would
suppress negative BOLD signal by enforcing positivity. Spatial sparse coding
algorithms ($L1$ Regularized Learning and K-SVD) would impose local
specialization and a discouragement of multitasking, where the total observed
activity in a single voxel originates from a restricted number of possible
brain networks.
  The assumptions of independence, positivity, and sparsity to encode
task-related brain networks are compared; the resulting brain networks for
different constraints are used as basis functions to encode the observed
functional activity at a given time point. These encodings are decoded using
machine learning to compare both the algorithms and their assumptions, using
the time series weights to predict whether a subject is viewing a video,
listening to an audio cue, or at rest, in 304 fMRI scans from 51 subjects.
  For classifying cognitive activity, the sparse coding algorithm of $L1$
Regularized Learning consistently outperformed 4 variations of ICA across
different numbers of networks and noise levels (p$<$0.001). The NMF algorithms,
which suppressed negative BOLD signal, had the poorest accuracy. Within each
algorithm, encodings using sparser spatial networks (containing more
zero-valued voxels) had higher classification accuracy (p$<$0.001). The success
of sparse coding algorithms may suggest that algorithms which enforce sparse
coding, discourage multitasking, and promote local specialization may capture
better the underlying source processes than those which allow inexhaustible
local processes such as ICA.
Situated Structure Learning of a Bayesian Logic Network for Commonsense Reasoning
Authors: Haley Garrison, Sonia Chernova
Date: 2016-07-01 00:00:00

This paper details the implementation of an algorithm for automatically
generating a high-level knowledge network to perform commonsense reasoning,
specifically with the application of robotic task repair. The network is
represented using a Bayesian Logic Network (BLN) (Jain, Waldherr, and Beetz
2009), which combines a set of directed relations between abstract concepts,
including IsA, AtLocation, HasProperty, and UsedFor, with a corresponding
probability distribution that models the uncertainty inherent in these
relations. Inference over this network enables reasoning over the abstract
concepts in order to perform appropriate object substitution or to locate
missing objects in the robot's environment. The structure of the network is
generated by combining information from two existing knowledge sources:
ConceptNet (Speer and Havasi 2012), and WordNet (Miller 1995). This is done in
a "situated" manner by only including information relevant a given context.
Results show that the generated network is able to accurately predict object
categories, locations, properties, and affordances in three different household
scenarios.
Learning Relational Dependency Networks for Relation Extraction
Authors: Dileep Viswanathan, Ameet Soni, Jude Shavlik, Sriraam Natarajan
Date: 2016-07-01 00:00:00
Tasks: Knowledge Base Construction, Relation, Relation Extraction, slot-filling, Slot Filling

We consider the task of KBP slot filling -- extracting relation information
from newswire documents for knowledge base construction. We present our
pipeline, which employs Relational Dependency Networks (RDNs) to learn
linguistic patterns for relation extraction. Additionally, we demonstrate how
several components such as weak supervision, word2vec features, joint learning
and the use of human advice, can be incorporated in this relational framework.
We evaluate the different components in the benchmark KBP 2015 task and show
that RDNs effectively model a diverse set of features and perform competitively
with current state-of-the-art relation extraction.
Learning Dynamic Classes of Events using Stacked Multilayer Perceptron Networks
Authors: Nattiya Kanhabua, Huamin Ren, Thomas B. Moeslund
Date: 2016-06-23 00:00:00
Tasks: Retrieval

People often use a web search engine to find information about events of
interest, for example, sport competitions, political elections, festivals and
entertainment news. In this paper, we study a problem of detecting
event-related queries, which is the first step before selecting a suitable
time-aware retrieval model. In general, event-related information needs can be
observed in query streams through various temporal patterns of user search
behavior, e.g., spiky peaks for popular events, and periodicities for
repetitive events. However, it is also common that users search for non-popular
events, which may not exhibit temporal variations in query streams, e.g., past
events recently occurred, historical events triggered by anniversaries or
similar events, and future events anticipated to happen. To address the
challenge of detecting dynamic classes of events, we propose a novel deep
learning model to classify a given query into a predetermined set of multiple
event types. Our proposed model, a Stacked Multilayer Perceptron (S-MLP)
network, consists of multilayer perceptron used as a basic learning unit. We
assemble stacked units to further learn complex relationships between neutrons
in successive layers. To evaluate our proposed model, we conduct experiments
using real-world queries and a set of manually created ground truth.
Preliminary results have shown that our proposed deep learning model
outperforms the state-of-the-art classification models significantly.
Continuous Adaptation of Multi-Camera Person Identification Models through Sparse Non-redundant Representative Selection
Authors: Abir Das, Rameswar Panda, Amit K. Roy-Chowdhury
Date: 2016-07-01 00:00:00
Tasks: Person Identification, Person Re-Identification

The problem of image-base person identification/recognition is to provide an
identity to the image of an individual based on learned models that describe
his/her appearance. Most traditional person identification systems rely on
learning a static model on tediously labeled training data. Though labeling
manually is an indispensable part of a supervised framework, for a large scale
identification system labeling huge amount of data is a significant overhead.
For large multi-sensor data as typically encountered in camera networks,
labeling a lot of samples does not always mean more information, as redundant
images are labeled several times. In this work, we propose a convex
optimization based iterative framework that progressively and judiciously
chooses a sparse but informative set of samples for labeling, with minimal
overlap with previously labeled images. We also use a structure preserving
sparse reconstruction based classifier to reduce the training burden typically
seen in discriminative classifiers. The two stage approach leads to a novel
framework for online update of the classifiers involving only the incorporation
of new labeled data rather than any expensive training phase. We demonstrate
the effectiveness of our approach on multi-camera person re-identification
datasets, to demonstrate the feasibility of learning online classification
models in multi-camera big data applications. Using three benchmark datasets,
we validate our approach and demonstrate that our framework achieves superior
performance with significantly less amount of manual labeling.
Domain Adaptation for Neural Networks by Parameter Augmentation
Authors: Yusuke Watanabe, Kazuma Hashimoto, Yoshimasa Tsuruoka
Date: 2016-07-01 00:00:00
Proceeding: WS 2016 8
Tasks: Caption Generation, Domain Adaptation

We propose a simple domain adaptation method for neural networks in a
supervised setting. Supervised domain adaptation is a way of improving the
generalization performance on the target domain by using the source domain
dataset, assuming that both of the datasets are labeled. Recently, recurrent
neural networks have been shown to be successful on a variety of NLP tasks such
as caption generation; however, the existing domain adaptation techniques are
limited to (1) tune the model parameters by the target dataset after the
training by the source dataset, or (2) design the network to have dual output,
one for the source domain and the other for the target domain. Reformulating
the idea of the domain adaptation technique proposed by Daume (2007), we
propose a simple domain adaptation method, which can be applied to neural
networks trained with a cross-entropy loss. On captioning datasets, we show
performance improvements over other domain adaptation methods.
Tracking Dynamic Point Processes on Networks
Authors: Eric C. Hall, Rebecca M. Willett
Date: 2014-08-29 00:00:00
Tasks: Point Processes

Cascading chains of events are a salient feature of many real-world social,
biological, and financial networks. In social networks, social reciprocity
accounts for retaliations in gang interactions, proxy wars in nation-state
conflicts, or Internet memes shared via social media. Neuron spikes stimulate
or inhibit spike activity in other neurons. Stock market shocks can trigger a
contagion of volatility throughout a financial network. In these and other
examples, only individual events associated with network nodes are observed,
usually without knowledge of the underlying dynamic relationships between
nodes. This paper addresses the challenge of tracking how events within such
networks stimulate or influence future events. The proposed approach is an
online learning framework well-suited to streaming data, using a multivariate
Hawkes point process model to encapsulate autoregressive features of observed
events within the social network. Recent work on online learning in dynamic
environments is leveraged not only to exploit the dynamics within the
underlying network, but also to track the network structure as it evolves.
Regret bounds and experimental results demonstrate that the proposed method
performs nearly as well as an oracle or batch algorithm.
A scaled Bregman theorem with applications
Authors: Richard Nock, Aditya Krishna Menon, Cheng Soon Ong
Date: 2016-07-01 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: BIG-bench Machine Learning, Clustering

Bregman divergences play a central role in the design and analysis of a range
of machine learning algorithms. This paper explores the use of Bregman
divergences to establish reductions between such algorithms and their analyses.
We present a new scaled isodistortion theorem involving Bregman divergences
(scaled Bregman theorem for short) which shows that certain "Bregman
distortions'" (employing a potentially non-convex generator) may be exactly
re-written as a scaled Bregman divergence computed over transformed data.
Admissible distortions include geodesic distances on curved manifolds and
projections or gauge-normalisation, while admissible data include scalars,
vectors and matrices.
  Our theorem allows one to leverage to the wealth and convenience of Bregman
divergences when analysing algorithms relying on the aforementioned Bregman
distortions. We illustrate this with three novel applications of our theorem: a
reduction from multi-class density ratio to class-probability estimation, a new
adaptive projection free yet norm-enforcing dual norm mirror descent algorithm,
and a reduction from clustering on flat manifolds to clustering on curved
manifolds. Experiments on each of these domains validate the analyses and
suggest that the scaled Bregman theorem might be a worthy addition to the
popular handful of Bregman divergence properties that have been pervasive in
machine learning.
Moving Toward High Precision Dynamical Modelling in Hidden Markov Models
Authors: Sébastien Gagnon, Jean Rouat
Date: 2016-07-01 00:00:00
Tasks: speech-recognition, Speech Recognition, Vocal Bursts Intensity Prediction

Hidden Markov Model (HMM) is often regarded as the dynamical model of choice
in many fields and applications. It is also at the heart of most
state-of-the-art speech recognition systems since the 70's. However, from
Gaussian mixture models HMMs (GMM-HMM) to deep neural network HMMs (DNN-HMM),
the underlying Markovian chain of state-of-the-art models did not changed much.
The "left-to-right" topology is mostly always employed because very few other
alternatives exist. In this paper, we propose that finely-tuned HMM topologies
are essential for precise temporal modelling and that this approach should be
investigated in state-of-the-art HMM system. As such, we propose a
proof-of-concept framework for learning efficient topologies by pruning down
complex generic models. Speech recognition experiments that were conducted
indicate that complex time dependencies can be better learned by this approach
than with classical "left-to-right" models.
A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation
Authors: Qiang Liu, Jason D. Lee, Michael. I. Jordan
Date: 2016-02-10 00:00:00

We derive a new discrepancy statistic for measuring differences between two
probability distributions based on combining Stein's identity with the
reproducing kernel Hilbert space theory. We apply our result to test how well a
probabilistic model fits a set of observations, and derive a new class of
powerful goodness-of-fit tests that are widely applicable for complex and high
dimensional distributions, even for those with computationally intractable
normalization constants. Both theoretical and empirical properties of our
methods are studied thoroughly.
Convergence Rate of Frank-Wolfe for Non-Convex Objectives
Authors: Simon Lacoste-Julien
Date: 2016-07-01 00:00:00

We give a simple proof that the Frank-Wolfe algorithm obtains a stationary
point at a rate of $O(1/\sqrt{t})$ on non-convex objectives with a Lipschitz
continuous gradient. Our analysis is affine invariant and is the first, to the
best of our knowledge, giving a similar rate to what was already proven for
projected gradient methods (though on slightly different measures of
stationarity).
A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression
Authors: Eran Treister, Javier S. Turek, Irad Yavneh
Date: 2016-07-01 00:00:00
Tasks: regression

Solving l1 regularized optimization problems is common in the fields of
computational biology, signal processing and machine learning. Such l1
regularization is utilized to find sparse minimizers of convex functions. A
well-known example is the LASSO problem, where the l1 norm regularizes a
quadratic function. A multilevel framework is presented for solving such l1
regularized sparse optimization problems efficiently. We take advantage of the
expected sparseness of the solution, and create a hierarchy of problems of
similar type, which is traversed in order to accelerate the optimization
process. This framework is applied for solving two problems: (1) the sparse
inverse covariance estimation problem, and (2) l1-regularized logistic
regression. In the first problem, the inverse of an unknown covariance matrix
of a multivariate normal distribution is estimated, under the assumption that
it is sparse. To this end, an l1 regularized log-determinant optimization
problem needs to be solved. This task is challenging especially for large-scale
datasets, due to time and memory limitations. In the second problem, the
l1-regularization is added to the logistic regression classification objective
to reduce overfitting to the data and obtain a sparse model. Numerical
experiments demonstrate the efficiency of the multilevel framework in
accelerating existing iterative solvers for both of these problems.
Binary embeddings with structured hashed projections
Authors: Anna Choromanska, Krzysztof Choromanski, Mariusz Bojarski, Tony Jebara, Sanjiv Kumar, Yann Lecun
Date: 2015-11-16 00:00:00
Tasks: LEMMA

We consider the hashing mechanism for constructing binary embeddings, that
involves pseudo-random projections followed by nonlinear (sign function)
mappings. The pseudo-random projection is described by a matrix, where not all
entries are independent random variables but instead a fixed "budget of
randomness" is distributed across the matrix. Such matrices can be efficiently
stored in sub-quadratic or even linear space, provide reduction in randomness
usage (i.e. number of required random values), and very often lead to
computational speed ups. We prove several theoretical results showing that
projections via various structured matrices followed by nonlinear mappings
accurately preserve the angular distance between input high-dimensional
vectors. To the best of our knowledge, these results are the first that give
theoretical ground for the use of general structured matrices in the nonlinear
setting. In particular, they generalize previous extensions of the
Johnson-Lindenstrauss lemma and prove the plausibility of the approach that was
so far only heuristically confirmed for some special structured matrices.
Consequently, we show that many structured matrices can be used as an efficient
information compression mechanism. Our findings build a better understanding of
certain deep architectures, which contain randomly weighted and untrained
layers, and yet achieve high performance on different learning tasks. We
empirically verify our theoretical findings and show the dependence of learning
via structured hashed projections on the performance of neural network as well
as nearest neighbor classifier.
Crossings as a side effect of dependency lengths
Authors: Ramon Ferrer-i-Cancho, Carlos Gómez-Rodríguez
Date: 2015-08-26 00:00:00
Tasks: Sentence

The syntactic structure of sentences exhibits a striking regularity:
dependencies tend to not cross when drawn above the sentence. We investigate
two competing explanations. The traditional hypothesis is that this trend
arises from an independent principle of syntax that reduces crossings
practically to zero. An alternative to this view is the hypothesis that
crossings are a side effect of dependency lengths, i.e. sentences with shorter
dependency lengths should tend to have fewer crossings. We are able to reject
the traditional view in the majority of languages considered. The alternative
hypothesis can lead to a more parsimonious theory of language.
Meaningful Models: Utilizing Conceptual Structure to Improve Machine Learning Interpretability
Authors: Nick Condry
Date: 2016-07-01 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

The last decade has seen huge progress in the development of advanced machine
learning models; however, those models are powerless unless human users can
interpret them. Here we show how the mind's construction of concepts and
meaning can be used to create more interpretable machine learning models. By
proposing a novel method of classifying concepts, in terms of 'form' and
'function', we elucidate the nature of meaning and offer proposals to improve
model understandability. As machine learning begins to permeate daily life,
interpretable models may serve as a bridge between domain-expert authors and
non-expert users.
A new analytical approach to consistency and overfitting in regularized empirical risk minimization
Authors: Nicolas Garcia Trillos, Ryan Murray
Date: 2016-07-01 00:00:00
Tasks: Binary Classification, General Classification

This work considers the problem of binary classification: given training data
$x_1, \dots, x_n$ from a certain population, together with associated labels
$y_1,\dots, y_n \in \left\{0,1 \right\}$, determine the best label for an
element $x$ not among the training data. More specifically, this work considers
a variant of the regularized empirical risk functional which is defined
intrinsically to the observed data and does not depend on the underlying
population. Tools from modern analysis are used to obtain a concise proof of
asymptotic consistency as regularization parameters are taken to zero at rates
related to the size of the sample. These analytical tools give a new framework
for understanding overfitting and underfitting, and rigorously connect the
notion of overfitting with a loss of compactness.
Noise Models in Feature-based Stereo Visual Odometry
Authors: Pablo F. Alcantarilla, Oliver J. Woodford
Date: 2016-07-01 00:00:00
Tasks: Visual Odometry

Feature-based visual structure and motion reconstruction pipelines, common in
visual odometry and large-scale reconstruction from photos, use the location of
corresponding features in different images to determine the 3D structure of the
scene, as well as the camera parameters associated with each image. The noise
model, which defines the likelihood of the location of each feature in each
image, is a key factor in the accuracy of such pipelines, alongside
optimization strategy. Many different noise models have been proposed in the
literature; in this paper we investigate the performance of several. We
evaluate these models specifically w.r.t. stereo visual odometry, as this task
is both simple (camera intrinsics are constant and known; geometry can be
initialized reliably) and has datasets with ground truth readily available
(KITTI Odometry and New Tsukuba Stereo Dataset). Our evaluation shows that
noise models which are more adaptable to the varying nature of noise generally
perform better.