pwc-archive/papers-with-abstracts huggingface dataset:
Review Networks for Caption Generation
Authors: Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W. Cohen
Date: 2016-05-25 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Caption Generation, Decoder, Image Captioning

We propose a novel extension of the encoder-decoder framework, called a
review network. The review network is generic and can enhance any existing
encoder- decoder model: in this paper, we consider RNN decoders with both CNN
and RNN encoders. The review network performs a number of review steps with
attention mechanism on the encoder hidden states, and outputs a thought vector
after each review step; the thought vectors are used as the input of the
attention mechanism in the decoder. We show that conventional encoder-decoders
are a special case of our framework. Empirically, we show that our framework
improves over state-of- the-art encoder-decoder systems on the tasks of image
captioning and source code captioning.
Double Thompson Sampling for Dueling Bandits
Authors: Huasen Wu, Xin Liu
Date: 2016-04-25 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Thompson Sampling

In this paper, we propose a Double Thompson Sampling (D-TS) algorithm for
dueling bandit problems. As indicated by its name, D-TS selects both the first
and the second candidates according to Thompson Sampling. Specifically, D-TS
maintains a posterior distribution for the preference matrix, and chooses the
pair of arms for comparison by sampling twice from the posterior distribution.
This simple algorithm applies to general Copeland dueling bandits, including
Condorcet dueling bandits as its special case. For general Copeland dueling
bandits, we show that D-TS achieves $O(K^2 \log T)$ regret. For Condorcet
dueling bandits, we further simplify the D-TS algorithm and show that the
simplified D-TS algorithm achieves $O(K \log T + K^2 \log \log T)$ regret.
Simulation results based on both synthetic and real-world data demonstrate the
efficiency of the proposed D-TS algorithm.
Detecting People in Artwork with CNNs
Authors: Nicholas Westlake, Hongping Cai, Peter Hall
Date: 2016-10-27 00:00:00
Tasks: Object, object-detection, Object Detection, Object Detection in Artwork

CNNs have massively improved performance in object detection in photographs.
However research into object detection in artwork remains limited. We show
state-of-the-art performance on a challenging dataset, People-Art, which
contains people from photos, cartoons and 41 different artwork movements. We
achieve this high performance by fine-tuning a CNN for this task, thus also
demonstrating that training CNNs on photos results in overfitting for photos:
only the first three or four layers transfer from photos to artwork. Although
the CNN's performance is the highest yet, it remains less than 60\% AP,
suggesting further work is needed for the cross-depiction problem. The final
publication is available at Springer via
http://dx.doi.org/10.1007/978-3-319-46604-0_57
Road Curb Extraction from Mobile LiDAR Point Clouds
Authors: Sheng Xu, Ruisheng Wang, Han Zheng
Date: 2016-10-15 00:00:00

Automatic extraction of road curbs from uneven, unorganized, noisy and
massive 3D point clouds is a challenging task. Existing methods often project
3D point clouds onto 2D planes to extract curbs. However, the projection causes
loss of 3D information which degrades the performance of the detection. This
paper presents a robust, accurate and efficient method to extract road curbs
from 3D mobile LiDAR point clouds. Our method consists of two steps: 1)
extracting the candidate points of curbs based on the proposed novel energy
function and 2) refining the candidate points using the proposed least cost
path model. We evaluated our method on a large-scale of residential area
(16.7GB, 300 million points) and an urban area (1.07GB, 20 million points)
mobile LiDAR point clouds. Results indicate that the proposed method is
superior to the state-of-the-art methods in terms of robustness, accuracy and
efficiency. The proposed curb extraction method achieved a completeness of
78.62% and a correctness of 83.29%. These experiments demonstrate that the
proposed method is a promising solution to extract road curbs from mobile LiDAR
point clouds.
On Bochner's and Polya's Characterizations of Positive-Definite Kernels and the Respective Random Feature Maps
Authors: Jie Chen, Dehua Cheng, Yan Liu
Date: 2016-10-27 00:00:00
Tasks: Gaussian Processes

Positive-definite kernel functions are fundamental elements of kernel methods
and Gaussian processes. A well-known construction of such functions comes from
Bochner's characterization, which connects a positive-definite function with a
probability distribution. Another construction, which appears to have attracted
less attention, is Polya's criterion that characterizes a subset of these
functions. In this paper, we study the latter characterization and derive a
number of novel kernels little known previously.
  In the context of large-scale kernel machines, Rahimi and Recht (2007)
proposed a random feature map (random Fourier) that approximates a kernel
function, through independent sampling of the probability distribution in
Bochner's characterization. The authors also suggested another feature map
(random binning), which, although not explicitly stated, comes from Polya's
characterization. We show that with the same number of random samples, the
random binning map results in an Euclidean inner product closer to the kernel
than does the random Fourier map. The superiority of the random binning map is
confirmed empirically through regressions and classifications in the
reproducing kernel Hilbert space.
Contextual Bandits with Latent Confounders: An NMF Approach
Authors: Rajat Sen, Karthikeyan Shanmugam, Murat Kocaoglu, Alexandros G. Dimakis, Sanjay Shakkottai
Date: 2016-06-01 00:00:00
Tasks: Matrix Completion, Multi-Armed Bandits

Motivated by online recommendation and advertising systems, we consider a
causal model for stochastic contextual bandits with a latent low-dimensional
confounder. In our model, there are $L$ observed contexts and $K$ arms of the
bandit. The observed context influences the reward obtained through a latent
confounder variable with cardinality $m$ ($m \ll L,K$). The arm choice and the
latent confounder causally determines the reward while the observed context is
correlated with the confounder. Under this model, the $L \times K$ mean reward
matrix $\mathbf{U}$ (for each context in $[L]$ and each arm in $[K]$)
factorizes into non-negative factors $\mathbf{A}$ ($L \times m$) and
$\mathbf{W}$ ($m \times K$). This insight enables us to propose an
$\epsilon$-greedy NMF-Bandit algorithm that designs a sequence of interventions
(selecting specific arms), that achieves a balance between learning this
low-dimensional structure and selecting the best arm to minimize regret. Our
algorithm achieves a regret of $\mathcal{O}\left(L\mathrm{poly}(m, \log K) \log
T \right)$ at time $T$, as compared to $\mathcal{O}(LK\log T)$ for conventional
contextual bandits, assuming a constant gap between the best arm and the rest
for each context. These guarantees are obtained under mild sufficiency
conditions on the factors that are weaker versions of the well-known
Statistical RIP condition. We further propose a class of generative models that
satisfy our sufficient conditions, and derive a lower bound of
$\mathcal{O}\left(Km\log T\right)$. These are the first regret guarantees for
online matrix completion with bandit feedback, when the rank is greater than
one. We further compare the performance of our algorithm with the state of the
art, on synthetic and real world data-sets.
Tool and Phase recognition using contextual CNN features
Authors: Manish Sahu, Anirban Mukhopadhyay, Angelika Szengel, Stefan Zachow
Date: 2016-10-27 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Transfer Learning

A transfer learning method for generating features suitable for surgical
tools and phase recognition from the ImageNet classification features [1] is
proposed here. In addition, methods are developed for generating contextual
features and combining them with time series analysis for final classification
using multi-class random forest. The proposed pipeline is tested over the
training and testing datasets of M2CAI16 challenges: tool and phase detection.
Encouraging results are obtained by leave-one-out cross validation evaluation
on the training dataset.
Personalized Risk Scoring for Critical Care Prognosis using Mixtures of Gaussian Processes
Authors: Ahmed M. Alaa, Jinsung Yoon, Scott Hu, Mihaela van der Schaar
Date: 2016-10-27 00:00:00
Tasks: Gaussian Processes, Prognosis, Transfer Learning

Objective: In this paper, we develop a personalized real-time risk scoring
algorithm that provides timely and granular assessments for the clinical acuity
of ward patients based on their (temporal) lab tests and vital signs; the
proposed risk scoring system ensures timely intensive care unit (ICU)
admissions for clinically deteriorating patients. Methods: The risk scoring
system learns a set of latent patient subtypes from the offline electronic
health record data, and trains a mixture of Gaussian Process (GP) experts,
where each expert models the physiological data streams associated with a
specific patient subtype. Transfer learning techniques are used to learn the
relationship between a patient's latent subtype and her static admission
information (e.g. age, gender, transfer status, ICD-9 codes, etc). Results:
Experiments conducted on data from a heterogeneous cohort of 6,321 patients
admitted to Ronald Reagan UCLA medical center show that our risk score
significantly and consistently outperforms the currently deployed risk scores,
such as the Rothman index, MEWS, APACHE and SOFA scores, in terms of
timeliness, true positive rate (TPR), and positive predictive value (PPV).
Conclusion: Our results reflect the importance of adopting the concepts of
personalized medicine in critical care settings; significant accuracy and
timeliness gains can be achieved by accounting for the patients' heterogeneity.
Significance: The proposed risk scoring methodology can confer huge clinical
and social benefits on more than 200,000 critically ill inpatient who exhibit
cardiac arrests in the US every year.
Single- and Multi-Task Architectures for Tool Presence Detection Challenge at M2CAI 2016
Authors: Andru P. Twinanda, Didier Mutter, Jacques Marescaux, Michel de Mathelin, Nicolas Padoy
Date: 2016-10-27 00:00:00

The tool presence detection challenge at M2CAI 2016 consists of identifying
the presence/absence of seven surgical tools in the images of cholecystectomy
videos. Here, we propose to use deep architectures that are based on our
previous work where we presented several architectures to perform multiple
recognition tasks on laparoscopic videos. In this technical report, we present
the tool presence detection results using two architectures: (1) a single-task
architecture designed to perform solely the tool presence detection task and
(2) a multi-task architecture designed to perform jointly phase recognition and
tool presence detection. The results show that the multi-task network only
slightly improves the tool presence detection results. In constrast, a
significant improvement is obtained when there are more data available to train
the networks. This significant improvement can be regarded as a call for action
for other institutions to start working toward publishing more datasets into
the community, so that better models could be generated to perform the task.
A Category Space Approach to Supervised Dimensionality Reduction
Authors: Anthony O. Smith, Anand Rangarajan
Date: 2016-10-27 00:00:00
Tasks: Dimensionality Reduction, Supervised dimensionality reduction

Supervised dimensionality reduction has emerged as an important theme in the
last decade. Despite the plethora of models and formulations, there is a lack
of a simple model which aims to project the set of patterns into a space
defined by the classes (or categories). To this end, we set up a model in which
each class is represented as a 1D subspace of the vector space formed by the
features. Assuming the set of classes does not exceed the cardinality of the
features, the model results in multi-class supervised learning in which the
features of each class are projected into the class subspace. Class
discrimination is automatically guaranteed via the imposition of orthogonality
of the 1D class sub-spaces. The resulting optimization problem - formulated as
the minimization of a sum of quadratic functions on a Stiefel manifold - while
being non-convex (due to the constraints), nevertheless has a structure for
which we can identify when we have reached a global minimum. After formulating
a version with standard inner products, we extend the formulation to
reproducing kernel Hilbert spaces in a straightforward manner. The optimization
approach also extends in a similar fashion to the kernel version. Results and
comparisons with the multi-class Fisher linear (and kernel) discriminants and
principal component analysis (linear and kernel) showcase the relative merits
of this approach to dimensionality reduction.
Dependence and Relevance: A probabilistic view
Authors: Dan Geiger, David Heckerman
Date: 2016-10-27 00:00:00
Tasks: Sentence

We examine three probabilistic concepts related to the sentence "two
variables have no bearing on each other". We explore the relationships between
these three concepts and establish their relevance to the process of
constructing similarity networks---a tool for acquiring probabilistic knowledge
from human experts. We also establish a precise relationship between
connectedness in Bayesian networks and relevance in probability.
Level-based Analysis of Genetic Algorithms and other Search Processes
Authors: Dogan Corus, Duc-Cuong Dang, Anton V. Eremeev, Per Kristian Lehre
Date: 2014-07-29 00:00:00
Tasks: Evolutionary Algorithms

Understanding how the time-complexity of evolutionary algorithms (EAs) depend
on their parameter settings and characteristics of fitness landscapes is a
fundamental problem in evolutionary computation. Most rigorous results were
derived using a handful of key analytic techniques, including drift analysis.
However, since few of these techniques apply effortlessly to population-based
EAs, most time-complexity results concern simplified EAs, such as the (1+1) EA.
  This paper describes the level-based theorem, a new technique tailored to
population-based processes. It applies to any non-elitist process where
offspring are sampled independently from a distribution depending only on the
current population. Given conditions on this distribution, our technique
provides upper bounds on the expected time until the process reaches a target
state.
  We demonstrate the technique on several pseudo-Boolean functions, the sorting
problem, and approximation of optimal solutions in combinatorial optimisation.
The conditions of the theorem are often straightforward to verify, even for
Genetic Algorithms and Estimation of Distribution Algorithms which were
considered highly non-trivial to analyse. Finally, we prove that the theorem is
nearly optimal for the processes considered. Given the information the theorem
requires about the process, a much tighter bound cannot be proved.
Sparse Signal Subspace Decomposition Based on Adaptive Over-complete Dictionary
Authors: Hong Sun, Cheng-Wei Sang, Didier Le Ruyet
Date: 2016-10-27 00:00:00
Tasks: Denoising, Image Denoising, Retrieval

This paper proposes a subspace decomposition method based on an over-complete
dictionary in sparse representation, called "Sparse Signal Subspace
Decomposition" (or 3SD) method. This method makes use of a novel criterion
based on the occurrence frequency of atoms of the dictionary over the data set.
This criterion, well adapted to subspace-decomposition over a dependent basis
set, adequately re ects the intrinsic characteristic of regularity of the
signal. The 3SD method combines variance, sparsity and component frequency
criteria into an unified framework. It takes benefits from using an
over-complete dictionary which preserves details and from subspace
decomposition which rejects strong noise. The 3SD method is very simple with a
linear retrieval operation. It does not require any prior knowledge on
distributions or parameters. When applied to image denoising, it demonstrates
high performances both at preserving fine details and suppressing strong noise.
RankMap: A Platform-Aware Framework for Distributed Learning from Dense Datasets
Authors: Azalia Mirhoseini, Eva L. Dyer, Ebrahim. M. Songhori, Richard G. Baraniuk, Farinaz Koushanfar
Date: 2015-03-27 00:00:00
Tasks: Distributed Computing, Scheduling

This paper introduces RankMap, a platform-aware end-to-end framework for
efficient execution of a broad class of iterative learning algorithms for
massive and dense datasets. Our framework exploits data structure to factorize
it into an ensemble of lower rank subspaces. The factorization creates sparse
low-dimensional representations of the data, a property which is leveraged to
devise effective mapping and scheduling of iterative learning algorithms on the
distributed computing machines. We provide two APIs, one matrix-based and one
graph-based, which facilitate automated adoption of the framework for
performing several contemporary learning applications. To demonstrate the
utility of RankMap, we solve sparse recovery and power iteration problems on
various real-world datasets with up to 1.8 billion non-zeros. Our evaluations
are performed on Amazon EC2 and IBM iDataPlex servers using up to 244 cores.
The results demonstrate up to two orders of magnitude improvements in memory
usage, execution speed, and bandwidth compared with the best reported prior
work, while achieving the same level of learning accuracy.
Domain Adaptation with Soft-margin multiple feature-kernel learning beats Deep Learning for surveillance face recognition
Authors: Samik Banerjee, Sukhendu Das
Date: 2016-10-05 00:00:00
Tasks: Domain Adaptation, Face Recognition

Face recognition (FR) is the most preferred mode for biometric-based
surveillance, due to its passive nature of detecting subjects, amongst all
different types of biometric traits. FR under surveillance scenario does not
give satisfactory performance due to low contrast, noise and poor illumination
conditions on probes, as compared to the training samples. A state-of-the-art
technology, Deep Learning, even fails to perform well in these scenarios. We
propose a novel soft-margin based learning method for multiple feature-kernel
combinations, followed by feature transformed using Domain Adaptation, which
outperforms many recent state-of-the-art techniques, when tested using three
real-world surveillance face datasets.
Evaluation of Denoising Techniques for EOG signals based on SNR Estimation
Authors: Anirban Dasgupta, Suvodip Chakrborty, Aritra Chaudhuri, Aurobinda Routray
Date: 2015-06-16 00:00:00
Tasks: Denoising

This paper evaluates four algorithms for denoising raw Electrooculography
(EOG) data based on the Signal to Noise Ratio (SNR). The SNR is computed using
the eigenvalue method. The filtering algorithms are a) Finite Impulse Response
(FIR) bandpass filters, b) Stationary Wavelet Transform, c) Empirical Mode
Decomposition (EMD) d) FIR Median Hybrid Filters. An EOG dataset has been
prepared where the subject is asked to perform letter cancelation test on 20
subjects.
Things Bayes can't do
Authors: Daniil Ryabko
Date: 2016-10-26 00:00:00

The problem of forecasting conditional probabilities of the next event given
the past is considered in a general probabilistic setting. Given an arbitrary
(large, uncountable) set C of predictors, we would like to construct a single
predictor that performs asymptotically as well as the best predictor in C, on
any data. Here we show that there are sets C for which such predictors exist,
but none of them is a Bayesian predictor with a prior concentrated on C. In
other words, there is a predictor with sublinear regret, but every Bayesian
predictor must have a linear regret. This negative finding is in sharp contrast
with previous results that establish the opposite for the case when one of the
predictors in $C$ achieves asymptotically vanishing error. In such a case, if
there is a predictor that achieves asymptotically vanishing error for any
measure in C, then there is a Bayesian predictor that also has this property,
and whose prior is concentrated on (a countable subset of) C.
GPflow: A Gaussian process library using TensorFlow
Authors: Alexander G. de G. Matthews, Mark van der Wilk, Tom Nickson, Keisuke Fujii, Alexis Boukouvalas, Pablo León-Villagrá, Zoubin Ghahramani, James Hensman
Date: 2016-10-27 00:00:00
Tasks: Gaussian Processes, GPU, software testing, Variational Inference

GPflow is a Gaussian process library that uses TensorFlow for its core
computations and Python for its front end. The distinguishing features of
GPflow are that it uses variational inference as the primary approximation
method, provides concise code through the use of automatic differentiation, has
been engineered with a particular emphasis on software testing and is able to
exploit GPU hardware.
e-Distance Weighted Support Vector Regression
Authors: Yan Wang, Ge Ou, Wei Pang, Lan Huang, George Macleod Coghill
Date: 2016-07-21 00:00:00
Tasks: regression

We propose a novel support vector regression approach called e-Distance
Weighted Support Vector Regression (e-DWSVR).e-DWSVR specifically addresses two
challenging issues in support vector regression: first, the process of noisy
data; second, how to deal with the situation when the distribution of boundary
data is different from that of the overall data. The proposed e-DWSVR optimizes
the minimum margin and the mean of functional margin simultaneously to tackle
these two issues. In addition, we use both dual coordinate descent (CD) and
averaged stochastic gradient descent (ASGD) strategies to make e-DWSVR scalable
to large scale problems. We report promising results obtained by e-DWSVR in
comparison with existing methods on several benchmark datasets.
Embodiment of Learning in Electro-Optical Signal Processors
Authors: Michiel Hermans, Piotr Antonik, Marc Haelterman, Serge Massar
Date: 2016-10-20 00:00:00
Tasks: speech-recognition, Speech Recognition

Delay-coupled electro-optical systems have received much attention for their
dynamical properties and their potential use in signal processing. In
particular it has recently been demonstrated, using the artificial intelligence
algorithm known as reservoir computing, that photonic implementations of such
systems solve complex tasks such as speech recognition. Here we show how the
backpropagation algorithm can be physically implemented on the same
electro-optical delay-coupled architecture used for computation with only minor
changes to the original design. We find that, compared when the backpropagation
algorithm is not used, the error rate of the resulting computing device,
evaluated on three benchmark tasks, decreases considerably. This demonstrates
that electro-optical analog computers can embody a large part of their own
training process, allowing them to be applied to new, more difficult tasks.
A random version of principal component analysis in data clustering
Authors: Luigi Leonardo Palese
Date: 2016-10-27 00:00:00
Tasks: Clustering

Principal component analysis (PCA) is a widespread technique for data
analysis that relies on the covariance-correlation matrix of the analyzed data.
However to properly work with high-dimensional data, PCA poses severe
mathematical constraints on the minimum number of different replicates or
samples that must be included in the analysis. Here we show that a modified
algorithm works not only on well dimensioned datasets, but also on degenerated
ones.