pwc-archive/papers-with-abstracts huggingface dataset:
An Artificial Agent for Robust Image Registration
Authors: Rui Liao, Shun Miao, Pierre de Tournemire, Sasa Grbic, Ali Kamen, Tommaso Mansi, Dorin Comaniciu
Date: 2016-11-30 00:00:00
Tasks: Image Registration, Medical Image Registration

3-D image registration, which involves aligning two or more images, is a
critical step in a variety of medical applications from diagnosis to therapy.
Image registration is commonly performed by optimizing an image matching metric
as a cost function. However, this task is challenging due to the non-convex
nature of the matching metric over the plausible registration parameter space
and insufficient approaches for a robust optimization. As a result, current
approaches are often customized to a specific problem and sensitive to image
quality and artifacts. In this paper, we propose a completely different
approach to image registration, inspired by how experts perform the task. We
first cast the image registration problem as a "strategy learning" process,
where the goal is to find the best sequence of motion actions (e.g. up, down,
etc.) that yields image alignment. Within this approach, an artificial agent is
learned, modeled using deep convolutional neural networks, with 3D raw image
data as the input, and the next optimal action as the output. To cope with the
dimensionality of the problem, we propose a greedy supervised approach for an
end-to-end training, coupled with attention-driven hierarchical strategy. The
resulting registration approach inherently encodes both a data-driven matching
metric and an optimal registration strategy (policy). We demonstrate, on two
3-D/3-D medical image registration examples with drastically different nature
of challenges, that the artificial agent outperforms several state-of-art
registration methods by a large margin in terms of both accuracy and
robustness.
Reliable Evaluation of Neural Network for Multiclass Classification of Real-world Data
Authors: Siddharth Dinesh, Tirtharaj Dash
Date: 2016-11-30 00:00:00
Tasks: General Classification

This paper presents a systematic evaluation of Neural Network (NN) for
classification of real-world data. In the field of machine learning, it is
often seen that a single parameter that is 'predictive accuracy' is being used
for evaluating the performance of a classifier model. However, this parameter
might not be considered reliable given a dataset with very high level of
skewness. To demonstrate such behavior, seven different types of datasets have
been used to evaluate a Multilayer Perceptron (MLP) using twelve(12) different
parameters which include micro- and macro-level estimation. In the present
study, the most common problem of prediction called 'multiclass' classification
has been considered. The results that are obtained for different parameters for
each of the dataset could demonstrate interesting findings to support the
usability of these set of performance evaluation parameters.
Comparing Apples and Oranges: Two Examples of the Limits of Statistical Inference, With an Application to Google Advertising Markets
Authors: John Mount, Nina Zumel
Date: 2016-11-30 00:00:00

We show how the classic Cramer-Rao bound limits how accurately one can
simultaneously estimate values of a large number of Google Ad campaigns (or
similarly limit the measurement rate of many confounding A/B tests).
The observer-assisted method for adjusting hyper-parameters in deep learning algorithms
Authors: Maciej Wielgosz
Date: 2016-11-30 00:00:00
Tasks: Deep Learning

This paper presents a concept of a novel method for adjusting
hyper-parameters in Deep Learning (DL) algorithms. An external agent-observer
monitors a performance of a selected Deep Learning algorithm. The observer
learns to model the DL algorithm using a series of random experiments.
Consequently, it may be used for predicting a response of the DL algorithm in
terms of a selected quality measurement to a set of hyper-parameters. This
allows to construct an ensemble composed of a series of evaluators which
constitute an observer-assisted architecture. The architecture may be used to
gradually iterate towards to the best achievable quality score in tiny steps
governed by a unit of progress. The algorithm is stopped when the maximum
number of steps is reached or no further progress is made.
DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model
Authors: Eldar Insafutdinov, Leonid Pishchulin, Bjoern Andres, Mykhaylo Andriluka, Bernt Schiele
Date: 2016-05-10 00:00:00
Tasks: Keypoint Detection, Multi-Person Pose Estimation, Pose Estimation

The goal of this paper is to advance the state-of-the-art of articulated pose
estimation in scenes with multiple people. To that end we contribute on three
fronts. We propose (1) improved body part detectors that generate effective
bottom-up proposals for body parts; (2) novel image-conditioned pairwise terms
that allow to assemble the proposals into a variable number of consistent body
part configurations; and (3) an incremental optimization strategy that explores
the search space more efficiently thus leading both to better performance and
significant speed-up factors. Evaluation is done on two single-person and two
multi-person pose estimation benchmarks. The proposed approach significantly
outperforms best known multi-person pose estimation results while demonstrating
competitive performance on the task of single person pose estimation. Models
and code available at http://pose.mpi-inf.mpg.de
Influential Node Detection in Implicit Social Networks using Multi-task Gaussian Copula Models
Authors: Qunwei Li, Bhavya Kailkhura, Jayaraman J. Thiagarajan, Zhenliang Zhang, Pramod K. Varshney
Date: 2016-11-30 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Prediction

Influential node detection is a central research topic in social network
analysis. Many existing methods rely on the assumption that the network
structure is completely known \textit{a priori}. However, in many applications,
network structure is unavailable to explain the underlying information
diffusion phenomenon. To address the challenge of information diffusion
analysis with incomplete knowledge of network structure, we develop a
multi-task low rank linear influence model. By exploiting the relationships
between contagions, our approach can simultaneously predict the volume (i.e.
time series prediction) for each contagion (or topic) and automatically
identify the most influential nodes for each contagion. The proposed model is
validated using synthetic data and an ISIS twitter dataset. In addition to
improving the volume prediction performance significantly, we show that the
proposed approach can reliably infer the most influential users for specific
contagions.
Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions
Authors: Marcin Junczys-Dowmunt, Tomasz Dwojak, Hieu Hoang
Date: 2016-10-04 00:00:00
Proceeding: IWSLT 2016 12
Tasks: Decoder, Machine Translation, Sentence, Translation

In this paper we provide the largest published comparison of translation
quality for phrase-based SMT and neural machine translation across 30
translation directions. For ten directions we also include hierarchical
phrase-based MT. Experiments are performed for the recently published United
Nations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus.
In the second part of the paper we investigate aspects of translation speed,
introducing AmuNMT, our efficient neural machine translation decoder. We
demonstrate that current neural machine translation could already be used for
in-production systems when comparing words-per-second ratios.
Bandit algorithms to emulate human decision making using probabilistic distortions
Authors: Ravi Kumar Kolla, Prashanth L. A., Aditya Gopalan, Krishna Jagannathan, Michael Fu, Steve Marcus
Date: 2016-11-30 00:00:00
Tasks: Decision Making, Multi-Armed Bandits

Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the reward distributions: the classic $K$-armed bandit and the linearly parameterized bandit settings. We consider the aforementioned problems in the regret minimization as well as best arm identification framework for multi-armed bandits. For the regret minimization setting in $K$-armed as well as linear bandit problems, we propose algorithms that are inspired by Upper Confidence Bound (UCB) algorithms, incorporate reward distortions, and exhibit sublinear regret. For the $K$-armed bandit setting, we derive an upper bound on the expected regret for our proposed algorithm, and then we prove a matching lower bound to establish the order-optimality of our algorithm. For the linearly parameterized setting, our algorithm achieves a regret upper bound that is of the same order as that of regular linear bandit algorithm called Optimism in the Face of Uncertainty Linear (OFUL) bandit algorithm, and unlike OFUL, our algorithm handles distortions and an arm-dependent noise model. For the best arm identification problem in the $K$-armed bandit setting, we propose algorithms, derive guarantees on their performance, and also show that these algorithms are order optimal by proving matching fundamental limits on performance. For best arm identification in linear bandits, we propose an algorithm and establish sample complexity guarantees. Finally, we present simulation experiments which demonstrate the advantages resulting from using distortion-aware learning algorithms in a vehicular traffic routing application.
A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Authors: Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J. Mankowitz, Shie Mannor
Date: 2016-04-25 00:00:00
Tasks: Deep Reinforcement Learning, Lifelong learning, Minecraft, Reinforcement Learning

We propose a lifelong learning system that has the ability to reuse and
transfer knowledge from one task to another while efficiently retaining the
previously learned knowledge-base. Knowledge is transferred by learning
reusable skills to solve tasks in Minecraft, a popular video game which is an
unsolved and high-dimensional lifelong learning problem. These reusable skills,
which we refer to as Deep Skill Networks, are then incorporated into our novel
Hierarchical Deep Reinforcement Learning Network (H-DRLN) architecture using
two techniques: (1) a deep skill array and (2) skill distillation, our novel
variation of policy distillation (Rusu et. al. 2015) for learning skills. Skill
distillation enables the HDRLN to efficiently retain knowledge and therefore
scale in lifelong learning, by accumulating knowledge and encapsulating
multiple reusable skills into a single distilled network. The H-DRLN exhibits
superior performance and lower learning sample complexity compared to the
regular Deep Q Network (Mnih et. al. 2015) in sub-domains of Minecraft.
Enabling Dark Energy Science with Deep Generative Models of Galaxy Images
Authors: Siamak Ravanbakhsh, Francois Lanusse, Rachel Mandelbaum, Jeff Schneider, Barnabas Poczos
Date: 2016-09-19 00:00:00

Understanding the nature of dark energy, the mysterious force driving the
accelerated expansion of the Universe, is a major challenge of modern
cosmology. The next generation of cosmological surveys, specifically designed
to address this issue, rely on accurate measurements of the apparent shapes of
distant galaxies. However, shape measurement methods suffer from various
unavoidable biases and therefore will rely on a precise calibration to meet the
accuracy requirements of the science analysis. This calibration process remains
an open challenge as it requires large sets of high quality galaxy images. To
this end, we study the application of deep conditional generative models in
generating realistic galaxy images. In particular we consider variations on
conditional variational autoencoder and introduce a new adversarial objective
for training of conditional generative networks. Our results suggest a reliable
alternative to the acquisition of expensive high quality observations for
generating the calibration data needed by the next generation of cosmological
surveys.
Learning to learn by gradient descent by gradient descent
Authors: Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando de Freitas
Date: 2016-06-14 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Meta-Learning

The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Gaussian Attention Model and Its Application to Knowledge Base Embedding and Question Answering
Authors: Liwen Zhang, John Winn, Ryota Tomioka
Date: 2016-11-07 00:00:00
Tasks: Question Answering

We propose the Gaussian attention model for content-based neural memory
access. With the proposed attention model, a neural network has the additional
degree of freedom to control the focus of its attention from a laser sharp
attention to a broad attention. It is applicable whenever we can assume that
the distance in the latent space reflects some notion of semantics. We use the
proposed attention model as a scoring function for the embedding of a knowledge
base into a continuous vector space and then train a model that performs
question answering about the entities in the knowledge base. The proposed
attention model can handle both the propagation of uncertainty when following a
series of relations and also the conjunction of conditions in a natural way. On
a dataset of soccer players who participated in the FIFA World Cup 2014, we
demonstrate that our model can handle both path queries and conjunctive queries
well.
Reliably Learning the ReLU in Polynomial Time
Authors: Surbhi Goel, Varun Kanade, Adam Klivans, Justin Thaler
Date: 2016-11-30 00:00:00

We give the first dimension-efficient algorithms for learning Rectified
Linear Units (ReLUs), which are functions of the form $\mathbf{x} \mapsto
\max(0, \mathbf{w} \cdot \mathbf{x})$ with $\mathbf{w} \in \mathbb{S}^{n-1}$.
Our algorithm works in the challenging Reliable Agnostic learning model of
Kalai, Kanade, and Mansour (2009) where the learner is given access to a
distribution $\cal{D}$ on labeled examples but the labeling may be arbitrary.
We construct a hypothesis that simultaneously minimizes the false-positive rate
and the loss on inputs given positive labels by $\cal{D}$, for any convex,
bounded, and Lipschitz loss function.
  The algorithm runs in polynomial-time (in $n$) with respect to any
distribution on $\mathbb{S}^{n-1}$ (the unit sphere in $n$ dimensions) and for
any error parameter $\epsilon = \Omega(1/\log n)$ (this yields a PTAS for a
question raised by F. Bach on the complexity of maximizing ReLUs). These
results are in contrast to known efficient algorithms for reliably learning
linear threshold functions, where $\epsilon$ must be $\Omega(1)$ and strong
assumptions are required on the marginal distribution. We can compose our
results to obtain the first set of efficient algorithms for learning
constant-depth networks of ReLUs.
  Our techniques combine kernel methods and polynomial approximations with a
"dual-loss" approach to convex programming. As a byproduct we obtain a number
of applications including the first set of efficient algorithms for "convex
piecewise-linear fitting" and the first efficient algorithms for noisy
polynomial reconstruction of low-weight polynomials on the unit sphere.
Behavior-Based Machine-Learning: A Hybrid Approach for Predicting Human Decision Making
Authors: Gali Noti, Effi Levi, Yoav Kolumbus, Amit Daniely
Date: 2016-11-30 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

A large body of work in behavioral fields attempts to develop models that
describe the way people, as opposed to rational agents, make decisions. A
recent Choice Prediction Competition (2015) challenged researchers to suggest a
model that captures 14 classic choice biases and can predict human decisions
under risk and ambiguity. The competition focused on simple decision problems,
in which human subjects were asked to repeatedly choose between two gamble
options.
  In this paper we present our approach for predicting human decision behavior:
we suggest to use machine learning algorithms with features that are based on
well-established behavioral theories. The basic idea is that these
psychological features are essential for the representation of the data and are
important for the success of the learning process. We implement a vanilla model
in which we train SVM models using behavioral features that rely on the
psychological properties underlying the competition baseline model. We show
that this basic model captures the 14 choice biases and outperforms all the
other learning-based models in the competition. The preliminary results suggest
that such hybrid models can significantly improve the prediction of human
decision making, and are a promising direction for future research.
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Authors: Yani Ioannou, Duncan Robertson, Roberto Cipolla, Antonio Criminisi
Date: 2016-05-20 00:00:00
Proceeding: CVPR 2017 7
Tasks: CPU, GPU

We propose a new method for creating computationally efficient and compact
convolutional neural networks (CNNs) using a novel sparse connection structure
that resembles a tree root. This allows a significant reduction in
computational cost and number of parameters compared to state-of-the-art deep
CNNs, without compromising accuracy, by exploiting the sparsity of inter-layer
filter dependencies. We validate our approach by using it to train more
efficient variants of state-of-the-art CNN architectures, evaluated on the
CIFAR10 and ILSVRC datasets. Our results show similar or higher accuracy than
the baseline architectures with much less computation, as measured by CPU and
GPU timings. For example, for ResNet 50, our model has 40% fewer parameters,
45% fewer floating point operations, and is 31% (12%) faster on a CPU (GPU).
For the deeper ResNet 200 our model has 25% fewer floating point operations and
44% fewer parameters, while maintaining state-of-the-art accuracy. For
GoogLeNet, our model has 7% fewer parameters and is 21% (16%) faster on a CPU
(GPU).
Effective Quantization Methods for Recurrent Neural Networks
Authors: Qinyao He, He Wen, Shuchang Zhou, Yuxin Wu, Cong Yao, Xinyu Zhou, Yuheng Zou
Date: 2016-11-30 00:00:00
Tasks: Quantization

Reducing bit-widths of weights, activations, and gradients of a Neural
Network can shrink its storage size and memory usage, and also allow for faster
training and inference by exploiting bitwise operations. However, previous
attempts for quantization of RNNs show considerable performance degradation
when using low bit-width weights and activations. In this paper, we propose
methods to quantize the structure of gates and interlinks in LSTM and GRU
cells. In addition, we propose balanced quantization methods for weights to
further reduce performance degradation. Experiments on PTB and IMDB datasets
confirm effectiveness of our methods as performances of our models match or
surpass the previous state-of-the-art of quantized RNN.
Stability selection for component-wise gradient boosting in multiple dimensions
Authors: Janek Thomas, Andreas Mayr, Bernd Bischl, Matthias Schmid, Adam Smith, Benjamin Hofner
Date: 2016-11-30 00:00:00
Tasks: Additive models

We present a new algorithm for boosting generalized additive models for
location, scale and shape (GAMLSS) that allows to incorporate stability
selection, an increasingly popular way to obtain stable sets of covariates
while controlling the per-family error rate (PFER). The model is fitted
repeatedly to subsampled data and variables with high selection frequencies are
extracted. To apply stability selection to boosted GAMLSS, we develop a new
"noncyclical" fitting algorithm that incorporates an additional selection step
of the best-fitting distribution parameter in each iteration. This new
algorithms has the additional advantage that optimizing the tuning parameters
of boosting is reduced from a multi-dimensional to a one-dimensional problem
with vastly decreased complexity. The performance of the novel algorithm is
evaluated in an extensive simulation study. We apply this new algorithm to a
study to estimate abundance of common eider in Massachusetts, USA, featuring
excess zeros, overdispersion, non-linearity and spatio-temporal structures.
Eider abundance is estimated via boosted GAMLSS, allowing both mean and
overdispersion to be regressed on covariates. Stability selection is used to
obtain a sparse set of stable predictors.
Classifiers for centrality determination in proton-nucleus and nucleus-nucleus collisions
Authors: Igor Altsybeev, Vladimir Kovalenko
Date: 2016-11-30 00:00:00

Centrality, as a geometrical property of the collision, is crucial for the
physical interpretation of nucleus-nucleus and proton-nucleus experimental
data. However, it cannot be directly accessed in event-by-event data analysis.
Common methods for centrality estimation in A-A and p-A collisions usually rely
on a single detector (either on the signal in zero-degree calorimeters or on
the multiplicity in some semi-central rapidity range). In the present work, we
made an attempt to develop an approach for centrality determination that is
based on machine-learning techniques and utilizes information from several
detector subsystems simultaneously. Different event classifiers are suggested
and evaluated for their selectivity power in terms of the number of
nucleons-participants and the impact parameter of the collision. Finer
centrality resolution may allow to reduce impact from so-called volume
fluctuations on physical observables being studied in heavy-ion experiments
like ALICE at the LHC and fixed target experiment NA61/SHINE on SPS.
Deep encoding of etymological information in TEI
Authors: Jack Bowers, Laurent Romary
Date: 2016-11-30 00:00:00

This paper aims to provide a comprehensive modeling and representation of
etymological data in digital dictionaries. The purpose is to integrate in one
coherent framework both digital representations of legacy dictionaries, and
also born-digital lexical databases that are constructed manually or
semi-automatically. We want to propose a systematic and coherent set of
modeling principles for a variety of etymological phenomena that may contribute
to the creation of a continuum between existing and future lexical constructs,
where anyone interested in tracing the history of words and their meanings will
be able to seamlessly query lexical resources.Instead of designing an ad hoc
model and representation language for digital etymological data, we will focus
on identifying all the possibilities offered by the TEI guidelines for the
representation of lexical information.