pwc-archive/papers-with-abstracts huggingface dataset:
Near-Optimal Active Learning of Multi-Output Gaussian Processes
Authors: Yehong Zhang, Trong Nghia Hoang, Kian Hsiang Low, Mohan Kankanhalli
Date: 2015-11-21 00:00:00
Tasks: Active Learning, Gaussian Processes

This paper addresses the problem of active learning of a multi-output
Gaussian process (MOGP) model representing multiple types of coexisting
correlated environmental phenomena. In contrast to existing works, our active
learning problem involves selecting not just the most informative sampling
locations to be observed but also the types of measurements at each selected
location for minimizing the predictive uncertainty (i.e., posterior joint
entropy) of a target phenomenon of interest given a sampling budget.
Unfortunately, such an entropy criterion scales poorly in the numbers of
candidate sampling locations and selected observations when optimized. To
resolve this issue, we first exploit a structure common to sparse MOGP models
for deriving a novel active learning criterion. Then, we exploit a relaxed form
of submodularity property of our new criterion for devising a polynomial-time
approximation algorithm that guarantees a constant-factor approximation of that
achieved by the optimal set of selected observations. Empirical evaluation on
real-world datasets shows that our proposed approach outperforms existing
algorithms for active learning of MOGP and single-output GP models.
Mouse Pose Estimation From Depth Images
Authors: Ashwin Nanjappa, Li Cheng, Wei Gao, Chi Xu, Adam Claridge-Chang, Zoe Bichler
Date: 2015-11-24 00:00:00
Tasks: 3D Pose Estimation, Pose Estimation

We focus on the challenging problem of efficient mouse 3D pose estimation
based on static images, and especially single depth images. We introduce an
approach to discriminatively train the split nodes of trees in random forest to
improve their performance on estimation of 3D joint positions of mouse. Our
algorithm is capable of working with different types of rodents and with
different types of depth cameras and imaging setups. In particular, it is
demonstrated in this paper that when a top-mounted depth camera is combined
with a bottom-mounted color camera, the final system is capable of delivering
full-body pose estimation including four limbs and the paws. Empirical
examinations on synthesized and real-world depth images confirm the
applicability of our approach on mouse pose estimation, as well as the closely
related task of part-based labeling of mouse.
Picking a Conveyor Clean by an Autonomously Learning Robot
Authors: Janne V. Kujala, Tuomas J. Lukka, Harri Holopainen
Date: 2015-11-24 00:00:00

We present a research picking prototype related to our company's industrial
waste sorting application. The goal of the prototype is to be as autonomous as
possible and it both calibrates itself and improves its picking with minimal
human intervention. The system learns to pick objects better based on a
feedback sensor in its gripper and uses machine learning to choosing the best
proposal from a random sample produced by simple hard-coded geometric models.
We show experimentally the system improving its picking autonomously by
measuring the pick success rate as function of time. We also show how this
system can pick a conveyor belt clean, depositing 70 out of 80 objects in a
difficult to manipulate pile of novel objects into the correct chute. We
discuss potential improvements and next steps in this direction.
Parsing Occluded People by Flexible Compositions
Authors: Xianjie Chen, Alan Yuille
Date: 2014-12-04 00:00:00
Proceeding: CVPR 2015 6

This paper presents an approach to parsing humans when there is significant
occlusion. We model humans using a graphical model which has a tree structure
building on recent work [32, 6] and exploit the connectivity prior that, even
in presence of occlusion, the visible nodes form a connected subtree of the
graphical model. We call each connected subtree a flexible composition of
object parts. This involves a novel method for learning occlusion cues. During
inference we need to search over a mixture of different flexible models. By
exploiting part sharing, we show that this inference can be done extremely
efficiently requiring only twice as many computations as searching for the
entire object (i.e., not modeling occlusion). We evaluate our model on the
standard benchmarked "We Are Family" Stickmen dataset and obtain significant
performance improvements over the best alternative algorithms.
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Authors: Justin Johnson, Andrej Karpathy, Li Fei-Fei
Date: 2015-11-24 00:00:00
Proceeding: CVPR 2016 6
Tasks: Dense Captioning, Image Captioning, Language Modeling, Language Modelling, object-detection, Object Detection, Retrieval

We introduce the dense captioning task, which requires a computer vision
system to both localize and describe salient regions in images in natural
language. The dense captioning task generalizes object detection when the
descriptions consist of a single word, and Image Captioning when one predicted
region covers the full image. To address the localization and description task
jointly we propose a Fully Convolutional Localization Network (FCLN)
architecture that processes an image with a single, efficient forward pass,
requires no external regions proposals, and can be trained end-to-end with a
single round of optimization. The architecture is composed of a Convolutional
Network, a novel dense localization layer, and Recurrent Neural Network
language model that generates the label sequences. We evaluate our network on
the Visual Genome dataset, which comprises 94,000 images and 4,100,000
region-grounded captions. We observe both speed and accuracy improvements over
baselines based on current state of the art approaches in both generation and
retrieval settings.
Learning Simple Algorithms from Examples
Authors: Wojciech Zaremba, Tomas Mikolov, Armand Joulin, Rob Fergus
Date: 2015-11-23 00:00:00
Tasks: Q-Learning

We present an approach for learning simple algorithms such as copying,
multi-digit addition and single digit multiplication directly from examples.
Our framework consists of a set of interfaces, accessed by a controller.
Typical interfaces are 1-D tapes or 2-D grids that hold the input and output
data. For the controller, we explore a range of neural network-based models
which vary in their ability to abstract the underlying algorithm from training
instances and generalize to test examples with many thousands of digits. The
controller is trained using $Q$-learning with several enhancements and we show
that the bottleneck is in the capabilities of the controller rather than in the
search incurred by $Q$-learning.
Functional Gaussian Process Model for Bayesian Nonparametric Analysis
Authors: Leo L. Duan, Xia Wang, Rhonda D. Szczesniak
Date: 2015-02-10 00:00:00
Tasks: Clustering, model, valid

Gaussian process is a theoretically appealing model for nonparametric
analysis, but its computational cumbersomeness hinders its use in large scale
and the existing reduced-rank solutions are usually heuristic. In this work, we
propose a novel construction of Gaussian process as a projection from fixed
discrete frequencies to any continuous location. This leads to a valid
stochastic process that has a theoretic support with the reduced rank in the
spectral density, as well as a high-speed computing algorithm. Our method
provides accurate estimates for the covariance parameters and concise form of
predictive distribution for spatial prediction. For non-stationary data, we
adopt the mixture framework with a customized spectral dependency structure.
This enables clustering based on local stationarity, while maintains the joint
Gaussianness. Our work is directly applicable in solving some of the challenges
in the spatial data, such as large scale computation, anisotropic covariance,
spatio-temporal modeling, etc. We illustrate the uses of the model via
simulations and an application on a massive dataset.
Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
Authors: Yanshuai Cao, David J. Fleet
Date: 2014-10-28 00:00:00
Tasks: Gaussian Processes, valid

In this work, we propose a generalized product of experts (gPoE) framework
for combining the predictions of multiple probabilistic models. We identify
four desirable properties that are important for scalability, expressiveness
and robustness, when learning and inferring with a combination of multiple
models. Through analysis and experiments, we show that gPoE of Gaussian
processes (GP) have these qualities, while no other existing combination
schemes satisfy all of them at the same time. The resulting GP-gPoE is highly
scalable as individual GP experts can be independently learned in parallel;
very expressive as the way experts are combined depends on the input rather
than fixed; the combined prediction is still a valid probabilistic model with
natural interpretation; and finally robust to unreliable predictions from
individual experts.
Transductive Log Opinion Pool of Gaussian Process Experts
Authors: Yanshuai Cao, David J. Fleet
Date: 2015-11-24 00:00:00

We introduce a framework for analyzing transductive combination of Gaussian
process (GP) experts, where independently trained GP experts are combined in a
way that depends on test point location, in order to scale GPs to big data. The
framework provides some theoretical justification for the generalized product
of GP experts (gPoE-GP) which was previously shown to work well in practice but
lacks theoretical basis. Based on the proposed framework, an improvement over
gPoE-GP is introduced and empirically validated.
Constrained Deep Metric Learning for Person Re-identification
Authors: Hailin Shi, Xiangyu Zhu, Shengcai Liao, Zhen Lei, Yang Yang, Stan Z. Li
Date: 2015-11-24 00:00:00
Tasks: Metric Learning, Person Re-Identification

Person re-identification aims to re-identify the probe image from a given set
of images under different camera views. It is challenging due to large
variations of pose, illumination, occlusion and camera view. Since the
convolutional neural networks (CNN) have excellent capability of feature
extraction, certain deep learning methods have been recently applied in person
re-identification. However, in person re-identification, the deep networks
often suffer from the over-fitting problem. In this paper, we propose a novel
CNN-based method to learn a discriminative metric with good robustness to the
over-fitting problem in person re-identification. Firstly, a novel deep
architecture is built where the Mahalanobis metric is learned with a weight
constraint. This weight constraint is used to regularize the learning, so that
the learned metric has a better generalization ability. Secondly, we find that
the selection of intra-class sample pairs is crucial for learning but has
received little attention. To cope with the large intra-class variations in
pedestrian images, we propose a novel training strategy named moderate positive
mining to prevent the training process from over-fitting to the extreme samples
in intra-class pairs. Experiments show that our approach significantly
outperforms state-of-the-art methods on several benchmarks of person
re-identification.
Skip-Thought Memory Networks
Authors: Ethan Caballero
Date: 2015-11-19 00:00:00
Tasks: Question Answering, Sentence

Question Answering (QA) is fundamental to natural language processing in that
most nlp problems can be phrased as QA (Kumar et al., 2015). Current weakly
supervised memory network models that have been proposed so far struggle at
answering questions that involve relations among multiple entities (such as
facebook's bAbi qa5-three-arg-relations in (Weston et al., 2015)). To address
this problem of learning multi-argument multi-hop semantic relations for the
purpose of QA, we propose a method that combines the jointly learned long-term
read-write memory and attentive inference components of end-to-end memory
networks (MemN2N) (Sukhbaatar et al., 2015) with distributed sentence vector
representations encoded by a Skip-Thought model (Kiros et al., 2015). This
choice to append Skip-Thought Vectors to the existing MemN2N framework is
motivated by the fact that Skip-Thought Vectors have been shown to accurately
model multi-argument semantic relations (Kiros et al., 2015).
The Limitations of Deep Learning in Adversarial Settings
Authors: Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, Ananthram Swami
Date: 2015-11-24 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, Deep Learning

Deep learning takes advantage of large datasets and computationally efficient
training algorithms to outperform other approaches at various machine learning
tasks. However, imperfections in the training phase of deep neural networks
make them vulnerable to adversarial samples: inputs crafted by adversaries with
the intent of causing deep neural networks to misclassify. In this work, we
formalize the space of adversaries against deep neural networks (DNNs) and
introduce a novel class of algorithms to craft adversarial samples based on a
precise understanding of the mapping between inputs and outputs of DNNs. In an
application to computer vision, we show that our algorithms can reliably
produce samples correctly classified by human subjects but misclassified in
specific targets by a DNN with a 97% adversarial success rate while only
modifying on average 4.02% of the input features per sample. We then evaluate
the vulnerability of different sample classes to adversarial perturbations by
defining a hardness measure. Finally, we describe preliminary work outlining
defenses against adversarial samples by defining a predictive measure of
distance between a benign input and a target classification.
Noise-adaptive Margin-based Active Learning and Lower Bounds under Tsybakov Noise Condition
Authors: Yining Wang, Aarti Singh
Date: 2014-06-20 00:00:00
Tasks: Active Learning

We present a simple noise-robust margin-based active learning algorithm to
find homogeneous (passing the origin) linear separators and analyze its error
convergence when labels are corrupted by noise. We show that when the imposed
noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others
1999; Tsybakov 2004) the algorithm is able to adapt to unknown level of noise
and achieves optimal statistical rate up to poly-logarithmic factors. We also
derive lower bounds for margin based active learning algorithms under Tsybakov
noise conditions (TNC) for the membership query synthesis scenario (Angluin
1988). Our result implies lower bounds for the stream based selective sampling
scenario (Cohn 1990) under TNC for some fairly simple data distributions. Quite
surprisingly, we show that the sample complexity cannot be improved even if the
underlying data distribution is as simple as the uniform distribution on the
unit ball. Our proof involves the construction of a well separated hypothesis
set on the d-dimensional unit ball along with carefully designed label
distributions for the Tsybakov noise condition. Our analysis might provide
insights for other forms of lower bounds as well.
Constrained Structured Regression with Convolutional Neural Networks
Authors: Deepak Pathak, Philipp Krähenbühl, Stella X. Yu, Trevor Darrell
Date: 2015-11-23 00:00:00
Tasks: Intrinsic Image Decomposition, regression

Convolutional Neural Networks (CNNs) have recently emerged as the dominant
model in computer vision. If provided with enough training data, they predict
almost any visual quantity. In a discrete setting, such as classification, CNNs
are not only able to predict a label but often predict a confidence in the form
of a probability distribution over the output space. In continuous regression
tasks, such a probability estimate is often lacking. We present a regression
framework which models the output distribution of neural networks. This output
distribution allows us to infer the most likely labeling following a set of
physical or modeling constraints. These constraints capture the intricate
interplay between different input and output variables, and complement the
output of a CNN. However, they may not hold everywhere. Our setup further
allows to learn a confidence with which a constraint holds, in the form of a
distribution of the constrain satisfaction. We evaluate our approach on the
problem of intrinsic image decomposition, and show that constrained structured
regression significantly increases the state-of-the-art.
Predicting People's 3D Poses from Short Sequences
Authors: Bugra Tekin, Xiaolu Sun, Xinchao Wang, Vincent Lepetit, Pascal Fua
Date: 2015-04-30 00:00:00

We propose an efficient approach to exploiting motion information from
consecutive frames of a video sequence to recover the 3D pose of people.
Instead of computing candidate poses in individual frames and then linking
them, as is often done, we regress directly from a spatio-temporal block of
frames to a 3D pose in the central one. We will demonstrate that this approach
allows us to effectively overcome ambiguities and to improve upon the
state-of-the-art on challenging sequences.
Symmetric Tensor Completion from Multilinear Entries and Learning Product Mixtures over the Hypercube
Authors: Tselil Schramm, Benjamin Weitz
Date: 2015-06-09 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

We give an algorithm for completing an order-$m$ symmetric low-rank tensor
from its multilinear entries in time roughly proportional to the number of
tensor entries. We apply our tensor completion algorithm to the problem of
learning mixtures of product distributions over the hypercube, obtaining new
algorithmic results. If the centers of the product distribution are linearly
independent, then we recover distributions with as many as $\Omega(n)$ centers
in polynomial time and sample complexity. In the general case, we recover
distributions with as many as $\tilde\Omega(n)$ centers in quasi-polynomial
time, answering an open problem of Feldman et al. (SIAM J. Comp.) for the
special case of distributions with incoherent bias vectors.
  Our main algorithmic tool is the iterated application of a low-rank matrix
completion algorithm for matrices with adversarially missing entries.
On the Structure, Covering, and Learning of Poisson Multinomial Distributions
Authors: Constantinos Daskalakis, Gautam Kamath, Christos Tzamos
Date: 2015-04-30 00:00:00

An $(n,k)$-Poisson Multinomial Distribution (PMD) is the distribution of the
sum of $n$ independent random vectors supported on the set ${\cal
B}_k=\{e_1,\ldots,e_k\}$ of standard basis vectors in $\mathbb{R}^k$. We prove
a structural characterization of these distributions, showing that, for all
$\varepsilon >0$, any $(n, k)$-Poisson multinomial random vector is
$\varepsilon$-close, in total variation distance, to the sum of a discretized
multidimensional Gaussian and an independent $(\text{poly}(k/\varepsilon),
k)$-Poisson multinomial random vector. Our structural characterization extends
the multi-dimensional CLT of Valiant and Valiant, by simultaneously applying to
all approximation requirements $\varepsilon$. In particular, it overcomes
factors depending on $\log n$ and, importantly, the minimum eigenvalue of the
PMD's covariance matrix from the distance to a multidimensional Gaussian random
variable.
  We use our structural characterization to obtain an $\varepsilon$-cover, in
total variation distance, of the set of all $(n, k)$-PMDs, significantly
improving the cover size of Daskalakis and Papadimitriou, and obtaining the
same qualitative dependence of the cover size on $n$ and $\varepsilon$ as the
$k=2$ cover of Daskalakis and Papadimitriou. We further exploit this structure
to show that $(n,k)$-PMDs can be learned to within $\varepsilon$ in total
variation distance from $\tilde{O}_k(1/\varepsilon^2)$ samples, which is
near-optimal in terms of dependence on $\varepsilon$ and independent of $n$. In
particular, our result generalizes the single-dimensional result of Daskalakis,
Diakonikolas, and Servedio for Poisson Binomials to arbitrary dimension.
Stick-Breaking Policy Learning in Dec-POMDPs
Authors: Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, Jonathan P. How
Date: 2015-05-01 00:00:00

Expectation maximization (EM) has recently been shown to be an efficient
algorithm for learning finite-state controllers (FSCs) in large decentralized
POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often
converge to maxima that are far from optimal. This paper considers a
variable-size FSC to represent the local policy of each agent. These
variable-size FSCs are constructed using a stick-breaking prior, leading to a
new framework called \emph{decentralized stick-breaking policy representation}
(Dec-SBPR). This approach learns the controller parameters with a variational
Bayesian algorithm without having to assume that the Dec-POMDP model is
available. The performance of Dec-SBPR is demonstrated on several benchmark
problems, showing that the algorithm scales to large problems while
outperforming other state-of-the-art methods.
Compact Convolutional Neural Network Cascade for Face Detection
Authors: Ilya Kalinovskii, Vladimir Spitsyn
Date: 2015-08-06 00:00:00
Tasks: 4k, Computational Efficiency, CPU, Face Detection, GPU

The problem of faces detection in images or video streams is a classical
problem of computer vision. The multiple solutions of this problem have been
proposed, but the question of their optimality is still open. Many algorithms
achieve a high quality face detection, but at the cost of high computational
complexity. This restricts their application in the real-time systems. This
paper presents a new solution of the frontal face detection problem based on
compact convolutional neural networks cascade. The test results on FDDB dataset
show that it is competitive with state-of-the-art algorithms. This proposed
detector is implemented using three technologies: SSE/AVX/AVX2 instruction sets
for Intel CPUs, Nvidia CUDA, OpenCL. The detection speed of our approach
considerably exceeds all the existing CPU-based and GPU-based algorithms.
Because of high computational efficiency, our detector can processing 4K Ultra
HD video stream in real time (up to 27 fps) on mobile platforms (Intel Ivy
Bridge CPUs and Nvidia Kepler GPUs) in searching objects with the dimension
60x60 pixels or higher. At the same time its performance weakly dependent on
the background and number of objects in scene. This is achieved by the
asynchronous computation of stages in the cascade.
What is the plausibility of probability?(revised 2003, 2015)
Authors: Stefan Arnborg, Gunnar Sjödin
Date: 2015-11-23 00:00:00

We present and examine a result related to uncertainty reasoning, namely that
a certain plausibility space of Cox's type can be uniquely embedded in a
minimal ordered field. This, although a purely mathematical result, can be
claimed to imply that every rational method to reason with uncertainty must be
based on sets of extended probability distributions, where extended probability
is standard probability extended with infinitesimals.
  This claim must be supported by some argumentation of non-mathematical type,
however, since pure mathematics does not tell us anything about the world. We
propose one such argumentation, and relate it to results from the literature of
uncertainty and statistics.
  In an added retrospective section we discuss some developments in the area
regarding countable additivity, partially ordered domains and robustness, and
philosophical stances on the Cox/Jaynes approach since 2003. We also show that
the most general partially ordered plausibility calculus embeddable in a ring
can be represented as a set of extended probability distributions or, in
algebraic terms, is a subdirect sum of ordered fields. In other words, the
robust Bayesian approach is universal. This result is exemplified by relating
Dempster-Shafer's evidence theory to robust Bayesian analysis.