pwc-archive/papers-with-abstracts huggingface dataset:
Regularization for Unsupervised Deep Neural Nets
Authors: Baiyang Wang, Diego Klabjan
Date: 2016-08-15 00:00:00
Tasks: feature selection, General Classification

Unsupervised neural networks, such as restricted Boltzmann machines (RBMs)
and deep belief networks (DBNs), are powerful tools for feature selection and
pattern recognition tasks. We demonstrate that overfitting occurs in such
models just as in deep feedforward neural networks, and discuss possible
regularization methods to reduce overfitting. We also propose a "partial"
approach to improve the efficiency of Dropout/DropConnect in this scenario, and
discuss the theoretical justification of these methods from model convergence
and likelihood bounds. Finally, we compare the performance of these methods
based on their likelihood and classification error rates for various pattern
recognition data sets.
Precise Recovery of Latent Vectors from Generative Adversarial Networks
Authors: Zachary C. Lipton, Subarna Tripathi
Date: 2017-02-15 00:00:00

Generative adversarial networks (GANs) transform latent vectors into visually
plausible images. It is generally thought that the original GAN formulation
gives no out-of-the-box method to reverse the mapping, projecting images back
into latent space. We introduce a simple, gradient-based technique called
stochastic clipping. In experiments, for images generated by the GAN, we
precisely recover their latent vector pre-images 100% of the time. Additional
experiments demonstrate that this method is robust to noise. Finally, we show
that even for unseen images, our method appears to recover unique encodings.
Trace Norm Regularised Deep Multi-Task Learning
Authors: Yongxin Yang, Timothy M. Hospedales
Date: 2016-06-13 00:00:00
Tasks: Multi-Task Learning

We propose a framework for training multiple neural networks simultaneously.
The parameters from all models are regularised by the tensor trace norm, so
that each neural network is encouraged to reuse others' parameters if possible
-- this is the main motivation behind multi-task learning. In contrast to many
deep multi-task learning models, we do not predefine a parameter sharing
strategy by specifying which layers have tied parameters. Instead, our
framework considers sharing for all shareable layers, and the sharing strategy
is learned in a data-driven way.
Cloud-based Deep Learning of Big EEG Data for Epileptic Seizure Prediction
Authors: Mohammad-Parsa Hosseini, Hamid Soltanian-Zadeh, Kost Elisevich, Dario Pompili
Date: 2017-02-17 00:00:00
Tasks: Brain Computer Interface, Cloud Computing, Dimensionality Reduction, EEG, Electroencephalogram (EEG), General Classification, Seizure prediction

Developing a Brain-Computer Interface~(BCI) for seizure prediction can help
epileptic patients have a better quality of life. However, there are many
difficulties and challenges in developing such a system as a real-life support
for patients. Because of the nonstationary nature of EEG signals, normal and
seizure patterns vary across different patients. Thus, finding a group of
manually extracted features for the prediction task is not practical. Moreover,
when using implanted electrodes for brain recording massive amounts of data are
produced. This big data calls for the need for safe storage and high
computational resources for real-time processing. To address these challenges,
a cloud-based BCI system for the analysis of this big EEG data is presented.
First, a dimensionality-reduction technique is developed to increase
classification accuracy as well as to decrease the communication bandwidth and
computation time. Second, following a deep-learning approach, a stacked
autoencoder is trained in two steps for unsupervised feature extraction and
classification. Third, a cloud-computing solution is proposed for real-time
analysis of big EEG data. The results on a benchmark clinical dataset
illustrate the superiority of the proposed patient-specific BCI as an
alternative method and its expected usefulness in real-life support of epilepsy
patients.
The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime
Authors: Max Simchowitz, Kevin Jamieson, Benjamin Recht
Date: 2017-02-16 00:00:00

We propose a novel technique for analyzing adaptive sampling called the {\em Simulator}. Our approach differs from the existing methods by considering not how much information could be gathered by any fixed sampling strategy, but how difficult it is to distinguish a good sampling strategy from a bad one given the limited amount of data collected up to any given time. This change of perspective allows us to match the strength of both Fano and change-of-measure techniques, without succumbing to the limitations of either method. For concreteness, we apply our techniques to a structured multi-arm bandit problem in the fixed-confidence pure exploration setting, where we show that the constraints on the means imply a substantial gap between the moderate-confidence sample complexity, and the asymptotic sample complexity as $\delta \to 0$ found in the literature. We also prove the first instance-based lower bounds for the top-k problem which incorporate the appropriate log-factors. Moreover, our lower bounds zero-in on the number of times each \emph{individual} arm needs to be pulled, uncovering new phenomena which are drowned out in the aggregate sample complexity. Our new analysis inspires a simple and near-optimal algorithm for the best-arm and top-k identification, the first {\em practical} algorithm of its kind for the latter problem which removes extraneous log factors, and outperforms the state-of-the-art in experiments.
Completing a joint PMF from projections: a low-rank coupled tensor factorization approach
Authors: Nikos Kargas, Nicholas D. Sidiropoulos
Date: 2017-02-16 00:00:00
Tasks: Recommendation Systems

There has recently been considerable interest in completing a low-rank matrix
or tensor given only a small fraction (or few linear combinations) of its
entries. Related approaches have found considerable success in the area of
recommender systems, under machine learning. From a statistical estimation
point of view, the gold standard is to have access to the joint probability
distribution of all pertinent random variables, from which any desired optimal
estimator can be readily derived. In practice high-dimensional joint
distributions are very hard to estimate, and only estimates of low-dimensional
projections may be available. We show that it is possible to identify
higher-order joint PMFs from lower-order marginalized PMFs using coupled
low-rank tensor factorization. Our approach features guaranteed identifiability
when the full joint PMF is of low-enough rank, and effective approximation
otherwise. We provide an algorithmic approach to compute the sought factors,
and illustrate the merits of our approach using rating prediction as an
example.
RIPML: A Restricted Isometry Property based Approach to Multilabel Learning
Authors: Akshay Soni, Yashar Mehdad
Date: 2017-02-16 00:00:00
Tasks: Dimensionality Reduction

The multilabel learning problem with large number of labels, features, and
data-points has generated a tremendous interest recently. A recurring theme of
these problems is that only a few labels are active in any given datapoint as
compared to the total number of labels. However, only a small number of
existing work take direct advantage of this inherent extreme sparsity in the
label space. By the virtue of Restricted Isometry Property (RIP), satisfied by
many random ensembles, we propose a novel procedure for multilabel learning
known as RIPML. During the training phase, in RIPML, labels are projected onto
a random low-dimensional subspace followed by solving a least-square problem in
this subspace. Inference is done by a k-nearest neighbor (kNN) based approach.
We demonstrate the effectiveness of RIPML by conducting extensive simulations
and comparing results with the state-of-the-art linear dimensionality reduction
based approaches.
Distance-Penalized Active Learning Using Quantile Search
Authors: John Lipor, Brandon Wong, Donald Scavia, Branko Kerkez, Laura Balzano
Date: 2015-09-28 00:00:00
Tasks: Active Learning

Adaptive sampling theory has shown that, with proper assumptions on the
signal class, algorithms exist to reconstruct a signal in $\mathbb{R}^{d}$ with
an optimal number of samples. We generalize this problem to the case of spatial
signals, where the sampling cost is a function of both the number of samples
taken and the distance traveled during estimation. This is motivated by our
work studying regions of low oxygen concentration in the Great Lakes. We show
that for one-dimensional threshold classifiers, a tradeoff between the number
of samples taken and distance traveled can be achieved using a generalization
of binary search, which we refer to as quantile search. We characterize both
the estimation error after a fixed number of samples and the distance traveled
in the noiseless case, as well as the estimation error in the case of noisy
measurements. We illustrate our results in both simulations and experiments and
show that our method outperforms existing algorithms in the majority of
practical scenarios.
Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
Authors: Michal Drozdzal, Gabriel Chartrand, Eugene Vorontsov, Lisa Di Jorio, An Tang, Adriana Romero, Yoshua Bengio, Chris Pal, Samuel Kadoury
Date: 2017-02-16 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

In this paper, we introduce a simple, yet powerful pipeline for medical image
segmentation that combines Fully Convolutional Networks (FCNs) with Fully
Convolutional Residual Networks (FC-ResNets). We propose and examine a design
that takes particular advantage of recent advances in the understanding of both
Convolutional Neural Networks as well as ResNets. Our approach focuses upon the
importance of a trainable pre-processing when using FC-ResNets and we show that
a low-capacity FCN model can serve as a pre-processor to normalize medical
input data. In our image segmentation pipeline, we use FCNs to obtain
normalized images, which are then iteratively refined by means of a FC-ResNet
to generate a segmentation prediction. As in other fully convolutional
approaches, our pipeline can be used off-the-shelf on different image
modalities. We show that using this pipeline, we exhibit state-of-the-art
performance on the challenging Electron Microscopy benchmark, when compared to
other 2D methods. We improve segmentation results on CT images of liver
lesions, when contrasting with standard FCN methods. Moreover, when applying
our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we
reach results that are competitive even when compared to 3D methods. The
obtained results illustrate the strong potential and versatility of the
pipeline by achieving highly accurate results on multi-modality images from
different anatomical regions and organs.
Learning Invariant Representations Of Planar Curves
Authors: Gautam Pai, Aaron Wetzler, Ron Kimmel
Date: 2016-11-23 00:00:00
Tasks: Metric Learning

We propose a metric learning framework for the construction of invariant
geometric functions of planar curves for the Eucledian and Similarity group of
transformations. We leverage on the representational power of convolutional
neural networks to compute these geometric quantities. In comparison with
axiomatic constructions, we show that the invariants approximated by the
learning architectures have better numerical qualities such as robustness to
noise, resiliency to sampling, as well as the ability to adapt to occlusion and
partiality. Finally, we develop a novel multi-scale representation in a
similarity metric learning paradigm.
Broad Context Language Modeling as Reading Comprehension
Authors: Zewei Chu, Hai Wang, Kevin Gimpel, David Mcallester
Date: 2016-10-26 00:00:00
Proceeding: EACL 2017 4
Tasks: coreference-resolution, Coreference Resolution, LAMBADA, Language Modeling, Language Modelling, Reading Comprehension, Sentence

Progress in text understanding has been driven by large datasets that test
particular capabilities, like recent datasets for reading comprehension
(Hermann et al., 2015). We focus here on the LAMBADA dataset (Paperno et al.,
2016), a word prediction task requiring broader context than the immediate
sentence. We view LAMBADA as a reading comprehension problem and apply
comprehension models based on neural networks. Though these models are
constrained to choose a word from the context, they improve the state of the
art on LAMBADA from 7.3% to 49%. We analyze 100 instances, finding that neural
network readers perform well in cases that involve selecting a name from the
context based on dialogue or discourse cues but struggle when coreference
resolution or external knowledge is needed.
An Analysis of Parallelized Motion Masking Using Dual-Mode Single Gaussian Models
Authors: Peter Henderson, Matthew Vertescher
Date: 2017-02-16 00:00:00
Tasks: Activity Recognition, Motion Detection, Optical Flow Estimation

Motion detection in video is important for a number of applications and
fields. In video surveillance, motion detection is an essential accompaniment
to activity recognition for early warning systems. Robotics also has much to
gain from motion detection and segmentation, particularly in high speed motion
tracking for tactile systems. There are a myriad of techniques for detecting
and masking motion in an image. Successful systems have used Gaussian Models to
discern background from foreground in an image (motion from static imagery).
However, particularly in the case of a moving camera or frame of reference, it
is necessary to compensate for the motion of the camera when attempting to
discern objects moving in the foreground. For example, it is possible to
estimate motion of the camera through optical flow methods or temporal
differencing and then compensate for this motion in a background subtraction
model. We selection a method by Yi et al. using Dual-Mode Single Gaussian
Models which does just this. We implement the technique in Intel's Thread
Building Blocks (TBB) and NVIDIA's CUDA libraries. We then compare
parallelization improvements with a theoretical analysis of speedups based on
the characteristics of our selected model and attributes of both TBB and CUDA.
We make our implementation available to the public.
Deep Multi-task Representation Learning: A Tensor Factorisation Approach
Authors: Yongxin Yang, Timothy Hospedales
Date: 2016-05-20 00:00:00
Tasks: Deep Learning, Multi-Task Learning, Representation Learning

Most contemporary multi-task learning methods assume linear models. This
setting is considered shallow in the era of deep learning. In this paper, we
present a new deep multi-task representation learning framework that learns
cross-task sharing structure at every layer in a deep network. Our approach is
based on generalising the matrix factorisation techniques explicitly or
implicitly used by many conventional MTL algorithms to tensor factorisation, to
realise automatic learning of end-to-end knowledge sharing in deep networks.
This is in contrast to existing deep learning approaches that need a
user-defined multi-task sharing strategy. Our approach applies to both
homogeneous and heterogeneous MTL. Experiments demonstrate the efficacy of our
deep multi-task representation learning in terms of both higher accuracy and
fewer design choices.
Automatic Handgun Detection Alarm in Videos Using Deep Learning
Authors: Roberto Olmos, Siham Tabik, Francisco Herrera
Date: 2017-02-16 00:00:00
Tasks: Deep Learning, Region Proposal

Current surveillance and control systems still require human supervision and
intervention. This work presents a novel automatic handgun detection system in
videos appropriate for both, surveillance and control purposes. We reformulate
this detection problem into the problem of minimizing false positives and solve
it by building the key training data-set guided by the results of a deep
Convolutional Neural Networks (CNN) classifier, then assessing the best
classification model under two approaches, the sliding window approach and
region proposal approach. The most promising results are obtained by Faster
R-CNN based model trained on our new database. The best detector show a high
potential even in low quality youtube videos and provides satisfactory results
as automatic alarm system. Among 30 scenes, it successfully activates the alarm
after five successive true positives in less than 0.2 seconds, in 27 scenes. We
also define a new metric, Alarm Activation per Interval (AApI), to assess the
performance of a detection model as an automatic detection system in videos.
Semi-supervised Learning for Discrete Choice Models
Authors: Jie Yang, Sergey Shebalov, Diego Klabjan
Date: 2017-02-16 00:00:00
Tasks: Clustering, Discrete Choice Models

We introduce a semi-supervised discrete choice model to calibrate discrete
choice models when relatively few requests have both choice sets and stated
preferences but the majority only have the choice sets. Two classic
semi-supervised learning algorithms, the expectation maximization algorithm and
the cluster-and-label algorithm, have been adapted to our choice modeling
problem setting. We also develop two new algorithms based on the
cluster-and-label algorithm. The new algorithms use the Bayesian Information
Criterion to evaluate a clustering setting to automatically adjust the number
of clusters. Two computational studies including a hotel booking case and a
large-scale airline itinerary shopping case are presented to evaluate the
prediction accuracy and computational effort of the proposed algorithms.
Algorithmic recommendations are rendered under various scenarios.
OntoMath Digital Ecosystem: Ontologies, Mathematical Knowledge Analytics and Management
Authors: Alexander Elizarov, Alexander Kirillovich, Evgeny Lipachev, Olga Nevzorova
Date: 2017-02-16 00:00:00
Tasks: Management, Recommendation Systems

In this article we consider the basic ideas, approaches and results of
developing of mathematical knowledge management technologies based on
ontologies. These solutions form the basis of a specialized digital ecosystem
OntoMath which consists of the ontology of the logical structure of
mathematical documents Mocassin and ontology of mathematical knowledge
OntoMathPRO, tools of text analysis, recommender system and other applications
to manage mathematical knowledge. The studies are in according to the ideas of
creating a distributed system of interconnected repositories of digitized
versions of mathematical documents and project to create a World Digital
Mathematical Library.
Improving Text Proposals for Scene Images with Fully Convolutional Networks
Authors: Dena Bazazian, Raul Gomez, Anguelos Nicolaou, Lluis Gomez, Dimosthenis Karatzas, Andrew D. Bagdanov
Date: 2017-02-16 00:00:00
Tasks: Object, Scene Text Recognition

Text Proposals have emerged as a class-dependent version of object proposals
- efficient approaches to reduce the search space of possible text object
locations in an image. Combined with strong word classifiers, text proposals
currently yield top state of the art results in end-to-end scene text
recognition. In this paper we propose an improvement over the original Text
Proposals algorithm of Gomez and Karatzas (2016), combining it with Fully
Convolutional Networks to improve the ranking of proposals. Results on the
ICDAR RRC and the COCO-text datasets show superior performance over current
state-of-the-art.
KEPLER: Keypoint and Pose Estimation of Unconstrained Faces by Learning Efficient H-CNN Regressors
Authors: Amit Kumar, Azadeh Alavi, Rama Chellappa
Date: 2017-02-16 00:00:00
Tasks: Face Alignment, Head Pose Estimation, Keypoint Detection, Keypoint Estimation, Pose Estimation, Pose Prediction

Keypoint detection is one of the most important pre-processing steps in tasks
such as face modeling, recognition and verification. In this paper, we present
an iterative method for Keypoint Estimation and Pose prediction of
unconstrained faces by Learning Efficient H-CNN Regressors (KEPLER) for
addressing the face alignment problem. Recent state of the art methods have
shown improvements in face keypoint detection by employing Convolution Neural
Networks (CNNs). Although a simple feed forward neural network can learn the
mapping between input and output spaces, it cannot learn the inherent
structural dependencies. We present a novel architecture called H-CNN
(Heatmap-CNN) which captures structured global and local features and thus
favors accurate keypoint detecion. HCNN is jointly trained on the visibility,
fiducials and 3D-pose of the face. As the iterations proceed, the error
decreases making the gradients small and thus requiring efficient training of
DCNNs to mitigate this. KEPLER performs global corrections in pose and
fiducials for the first four iterations followed by local corrections in the
subsequent stage. As a by-product, KEPLER also provides 3D pose (pitch, yaw and
roll) of the face accurately. In this paper, we show that without using any 3D
information, KEPLER outperforms state of the art methods for alignment on
challenging datasets such as AFW and AFLW.
Discovering objects and their relations from entangled scene representations
Authors: David Raposo, Adam Santoro, David Barrett, Razvan Pascanu, Timothy Lillicrap, Peter Battaglia
Date: 2017-02-16 00:00:00
Tasks: Object, One-Shot Learning, Relation

Our world can be succinctly and compactly described as structured scenes of
objects and relations. A typical room, for example, contains salient objects
such as tables, chairs and books, and these objects typically relate to each
other by their underlying causes and semantics. This gives rise to correlated
features, such as position, function and shape. Humans exploit knowledge of
objects and their relations for learning a wide spectrum of tasks, and more
generally when learning the structure underlying observed data. In this work,
we introduce relation networks (RNs) - a general purpose neural network
architecture for object-relation reasoning. We show that RNs are capable of
learning object relations from scene description data. Furthermore, we show
that RNs can act as a bottleneck that induces the factorization of objects from
entangled scene description inputs, and from distributed deep representations
of scene images provided by a variational autoencoder. The model can also be
used in conjunction with differentiable memory mechanisms for implicit relation
discovery in one-shot learning tasks. Our results suggest that relation
networks are a potentially powerful architecture for solving a variety of
problems that require object relation reasoning.
Structured Attention Networks
Authors: Yoon Kim, Carl Denton, Luong Hoang, Alexander M. Rush
Date: 2017-02-03 00:00:00
Tasks: Machine Translation, Natural Language Inference, Question Answering, Translation

Attention networks have proven to be an effective approach for embedding
categorical inference within a deep neural network. However, for many tasks we
may want to model richer structural dependencies without abandoning end-to-end
training. In this work, we experiment with incorporating richer structural
distributions, encoded using graphical models, within deep networks. We show
that these structured attention networks are simple extensions of the basic
attention procedure, and that they allow for extending attention beyond the
standard soft-selection approach, such as attending to partial segmentations or
to subtrees. We experiment with two different classes of structured attention
networks: a linear-chain conditional random field and a graph-based parsing
model, and describe how these models can be practically implemented as neural
network layers. Experiments show that this approach is effective for
incorporating structural biases, and structured attention networks outperform
baseline attention models on a variety of synthetic and real tasks: tree
transduction, neural machine translation, question answering, and natural
language inference. We further find that models trained in this way learn
interesting unsupervised hidden representations that generalize simple
attention.