pwc-archive/papers-with-abstracts huggingface dataset:
Learning What Information to Give in Partially Observed Domains
Authors: Rohan Chitnis, Leslie Pack Kaelbling, Tomás Lozano-Pérez
Date: 2018-05-21 00:00:00

In many robotic applications, an autonomous agent must act within and explore
a partially observed environment that is unobserved by its human teammate. We
consider such a setting in which the agent can, while acting, transmit
declarative information to the human that helps them understand aspects of this
unseen environment. In this work, we address the algorithmic question of how
the agent should plan out what actions to take and what information to
transmit. Naturally, one would expect the human to have preferences, which we
model information-theoretically by scoring transmitted information based on the
change it induces in weighted entropy of the human's belief state. We formulate
this setting as a belief MDP and give a tractable algorithm for solving it
approximately. Then, we give an algorithm that allows the agent to learn the
human's preferences online, through exploration. We validate our approach
experimentally in simulated discrete and continuous partially observed
search-and-recover domains. Visit http://tinyurl.com/chitnis-corl-18 for a
supplementary video.
Kernel machines that adapt to GPUs for effective large batch training
Authors: Siyuan Ma, Mikhail Belkin
Date: 2018-06-15 00:00:00
Tasks: GPU

Modern machine learning models are typically trained using Stochastic
Gradient Descent (SGD) on massively parallel computing resources such as GPUs.
Increasing mini-batch size is a simple and direct way to utilize the parallel
computing capacity. For small batch an increase in batch size results in the
proportional reduction in the training time, a phenomenon known as linear
scaling. However, increasing batch size beyond a certain value leads to no
further improvement in training time. In this paper we develop the first
analytical framework that extends linear scaling to match the parallel
computing capacity of a resource. The framework is designed for a class of
classical kernel machines. It automatically modifies a standard kernel machine
to output a mathematically equivalent prediction function, yet allowing for
extended linear scaling, i.e., higher effective parallelization and faster
training time on given hardware.
  The resulting algorithms are accurate, principled and very fast. For example,
using a single Titan Xp GPU, training on ImageNet with $1.3\times 10^6$ data
points and $1000$ labels takes under an hour, while smaller datasets, such as
MNIST, take seconds. As the parameters are chosen analytically, based on the
theoretical bounds, little tuning beyond selecting the kernel and the kernel
parameter is needed, further facilitating the practical use of these methods.
Sparse Pseudo-input Local Kriging for Large Spatial Datasets with Exogenous Variables
Authors: Babak Farmanesh, Arash Pourhabib
Date: 2015-08-05 00:00:00

We study large-scale spatial systems that contain exogenous variables, e.g. environmental factors that are significant predictors in spatial processes. Building predictive models for such processes is challenging because the large numbers of observations present makes it inefficient to apply full Kriging. In order to reduce computational complexity, this paper proposes Sparse Pseudo-input Local Kriging (SPLK), which utilizes hyperplanes to partition a domain into smaller subdomains and then applies a sparse approximation of the full Kriging to each subdomain. We also develop an optimization procedure to find the desired hyperplanes. To alleviate the problem of discontinuity in the global predictor, we impose continuity constraints on the boundaries of the neighboring subdomains. Furthermore, partitioning the domain into smaller subdomains makes it possible to use different parameter values for the covariance function in each region and, therefore, the heterogeneity in the data structure can be effectively captured. Numerical experiments demonstrate that SPLK outperforms, or is comparable to, the algorithms commonly applied to spatial datasets.
Possibility results for graph clustering: A novel consistency axiom
Authors: Fabio Strazzeri, Rubén J. Sánchez-García
Date: 2018-06-15 00:00:00
Tasks: Clustering, Graph Clustering

Kleinberg introduced three natural clustering properties, or axioms, and showed they cannot be simultaneously satisfied by any clustering algorithm. We present a new clustering property, Monotonic Consistency, which avoids the well-known problematic behaviour of Kleinberg's Consistency axiom, and the impossibility result. Namely, we describe a clustering algorithm, Morse Clustering, inspired by Morse Theory in Differential Topology, which satisfies Kleinberg's original axioms with Consistency replaced by Monotonic Consistency. Morse clustering uncovers the underlying flow structure on a set or graph and returns a partition into trees representing basins of attraction of critical vertices. We also generalise Kleinberg's axiomatic approach to sparse graphs, showing an impossibility result for Consistency, and a possibility result for Monotonic Consistency and Morse clustering.
Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
Authors: Stefan Depeweg, José Miguel Hernández-Lobato, Finale Doshi-Velez, Steffen Udluft
Date: 2017-10-19 00:00:00
Proceeding: ICML 2018 7
Tasks: Active Learning, Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Bayesian neural networks with latent variables are scalable and flexible
probabilistic models: They account for uncertainty in the estimation of the
network weights and, by making use of latent variables, can capture complex
noise patterns in the data. We show how to extract and decompose uncertainty
into epistemic and aleatoric components for decision-making purposes. This
allows us to successfully identify informative points for active learning of
functions with heteroscedastic and bimodal noise. Using the decomposition we
further define a novel risk-sensitive criterion for reinforcement learning to
identify policies that balance expected cost, model-bias and noise aversion.
High Dimensional Data Enrichment: Interpretable, Fast, and Data-Efficient
Authors: Amir Asiaee, Samet Oymak, Kevin R. Coombes, Arindam Banerjee
Date: 2018-06-11 00:00:00
Tasks: Multi-Task Learning, Vocal Bursts Intensity Prediction

We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational properties for the problem of multiple connected linear regressions known as Data Enrichment/Sharing. The between-tasks connections are captured by a cross-tasks \emph{common parameter}, which gets refined by per-task \emph{individual parameters}. Any convex function, e.g., norm, can characterize the structure of both common and individual parameters. We delineate the sample complexity of our estimator and provide a high probability non-asymptotic bound for estimation error of all parameters under a geometric condition. We show that the recovery of the common parameter benefits from \emph{all} of the pooled samples. We propose an iterative estimation algorithm with a geometric convergence rate and supplement our theoretical analysis with experiments on synthetic data. Overall, we present a first thorough statistical and computational analysis of inference in the data-sharing model.
Near-Lossless Deep Feature Compression for Collaborative Intelligence
Authors: Hyomin Choi, Ivan V. Bajic
Date: 2018-04-26 00:00:00
Tasks: Feature Compression

Collaborative intelligence is a new paradigm for efficient deployment of deep
neural networks across the mobile-cloud infrastructure. By dividing the network
between the mobile and the cloud, it is possible to distribute the
computational workload such that the overall energy and/or latency of the
system is minimized. However, this necessitates sending deep feature data from
the mobile to the cloud in order to perform inference. In this work, we examine
the differences between the deep feature data and natural image data, and
propose a simple and effective near-lossless deep feature compressor. The
proposed method achieves up to 5% bit rate reduction compared to HEVC-Intra and
even more against other popular image codecs. Finally, we suggest an approach
for reconstructing the input image from compressed deep features in the cloud,
that could serve to supplement the inference performed by the deep model.
Teaching Machines to Code: Neural Markup Generation with Visual Attention
Authors: Sumeet S. Singh
Date: 2018-02-15 00:00:00
Tasks: Math, Optical Character Recognition (OCR)

We present a neural transducer model with visual attention that learns to
generate LaTeX markup of a real-world math formula given its image. Applying
sequence modeling and transduction techniques that have been very successful
across modalities such as natural language, image, handwriting, speech and
audio; we construct an image-to-markup model that learns to produce
syntactically and semantically correct LaTeX markup code over 150 words long
and achieves a BLEU score of 89%; improving upon the previous state-of-art for
the Im2Latex problem. We also demonstrate with heat-map visualization how
attention helps in interpreting the model and can pinpoint (detect and
localize) symbols on the image accurately despite having been trained without
any bounding box data.
Supervised Fuzzy Partitioning
Authors: Pooya Ashtari, Fateme Nateghi Haredasht, Hamid Beigy
Date: 2018-06-15 00:00:00
Tasks: Clustering, Computational Efficiency, General Classification

Centroid-based methods including k-means and fuzzy c-means are known as effective and easy-to-implement approaches to clustering purposes in many applications. However, these algorithms cannot be directly applied to supervised tasks. This paper thus presents a generative model extending the centroid-based clustering approach to be applicable to classification and regression tasks. Given an arbitrary loss function, the proposed approach, termed Supervised Fuzzy Partitioning (SFP), incorporates labels information into its objective function through a surrogate term penalizing the empirical risk. Entropy-based regularization is also employed to fuzzify the partition and to weight features, enabling the method to capture more complex patterns, identify significant features, and yield better performance facing high-dimensional data. An iterative algorithm based on block coordinate descent scheme is formulated to efficiently find a local optimum. Extensive classification experiments on synthetic, real-world, and high-dimensional datasets demonstrate that the predictive performance of SFP is competitive with state-of-the-art algorithms such as SVM and random forest. SFP has a major advantage over such methods, in that it not only leads to a flexible, nonlinear model but also can exploit any convex loss function in the training phase without compromising computational efficiency.
On the Relationship between Data Efficiency and Error for Uncertainty Sampling
Authors: Stephen Mussmann, Percy Liang
Date: 2018-06-15 00:00:00
Proceeding: ICML 2018 7
Tasks: Active Learning, regression

While active learning offers potential cost savings, the actual data
efficiency---the reduction in amount of labeled data needed to obtain the same
error rate---observed in practice is mixed. This paper poses a basic question:
when is active learning actually helpful? We provide an answer for logistic
regression with the popular active learning algorithm, uncertainty sampling.
Empirically, on 21 datasets from OpenML, we find a strong inverse correlation
between data efficiency and the error rate of the final classifier.
Theoretically, we show that for a variant of uncertainty sampling, the
asymptotic data efficiency is within a constant factor of the inverse error
rate of the limiting classifier.
Fairness Under Composition
Authors: Cynthia Dwork, Christina Ilvento
Date: 2018-06-15 00:00:00
Tasks: Fairness

Algorithmic fairness, and in particular the fairness of scoring and
classification algorithms, has become a topic of increasing social concern and
has recently witnessed an explosion of research in theoretical computer
science, machine learning, statistics, the social sciences, and law. Much of
the literature considers the case of a single classifier (or scoring function)
used once, in isolation. In this work, we initiate the study of the fairness
properties of systems composed of algorithms that are fair in isolation; that
is, we study fairness under composition. We identify pitfalls of naive
composition and give general constructions for fair composition, demonstrating
both that classifiers that are fair in isolation do not necessarily compose
into fair systems and also that seemingly unfair components may be carefully
combined to construct fair systems. We focus primarily on the individual
fairness setting proposed in [Dwork, Hardt, Pitassi, Reingold, Zemel, 2011],
but also extend our results to a large class of group fairness definitions
popular in the recent literature, exhibiting several cases in which group
fairness definitions give misleading signals under composition.
Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
Authors: Guokun Lai, Bohan Li, Guoqing Zheng, Yiming Yang
Date: 2018-06-15 00:00:00

How to model distribution of sequential data, including but not limited to
speech and human motions, is an important ongoing research problem. It has been
demonstrated that model capacity can be significantly enhanced by introducing
stochastic latent variables in the hidden states of recurrent neural networks.
Simultaneously, WaveNet, equipped with dilated convolutions, achieves
astonishing empirical performance in natural speech generation task. In this
paper, we combine the ideas from both stochastic latent variables and dilated
convolutions, and propose a new architecture to model sequential data, termed
as Stochastic WaveNet, where stochastic latent variables are injected into the
WaveNet structure. We argue that Stochastic WaveNet enjoys powerful
distribution modeling capacity and the advantage of parallel training from
dilated convolutions. In order to efficiently infer the posterior distribution
of the latent variables, a novel inference network structure is designed based
on the characteristics of WaveNet architecture. State-of-the-art performances
on benchmark datasets are obtained by Stochastic WaveNet on natural speech
modeling and high quality human handwriting samples can be generated as well.
MV-YOLO: Motion Vector-aided Tracking by Semantic Object Detection
Authors: Saeed Ranjbar Alvar, Ivan V. Bajić
Date: 2018-04-30 00:00:00
Tasks: Object, object-detection, Object Detection, Object Tracking

Object tracking is the cornerstone of many visual analytics systems. While
considerable progress has been made in this area in recent years, robust,
efficient, and accurate tracking in real-world video remains a challenge. In
this paper, we present a hybrid tracker that leverages motion information from
the compressed video stream and a general-purpose semantic object detector
acting on decoded frames to construct a fast and efficient tracking engine. The
proposed approach is compared with several well-known recent trackers on the
OTB tracking dataset. The results indicate advantages of the proposed method in
terms of speed and/or accuracy.Other desirable features of the proposed method
are its simplicity and deployment efficiency, which stems from the fact that it
reuses the resources and information that may already exist in the system for
other reasons.
Non-Negative Networks Against Adversarial Attacks
Authors: William Fleshman, Edward Raff, Jared Sylvester, Steven Forsyth, Mark McLean
Date: 2018-06-15 00:00:00
Tasks: Binary Classification, Classification, General Classification, image-classification, Image Classification, Spam detection

Adversarial attacks against neural networks are a problem of considerable
importance, for which effective defenses are not yet readily available. We make
progress toward this problem by showing that non-negative weight constraints
can be used to improve resistance in specific scenarios. In particular, we show
that they can provide an effective defense for binary classification problems
with asymmetric cost, such as malware or spam detection. We also show the
potential for non-negativity to be helpful to non-binary problems by applying
it to image classification.
The Limits of Post-Selection Generalization
Authors: Kobbi Nissim, Adam Smith, Thomas Steinke, Uri Stemmer, Jonathan Ullman
Date: 2018-06-15 00:00:00
Proceeding: NeurIPS 2018 12

While statistics and machine learning offers numerous methods for ensuring
generalization, these methods often fail in the presence of adaptivity---the
common practice in which the choice of analysis depends on previous
interactions with the same dataset. A recent line of work has introduced
powerful, general purpose algorithms that ensure post hoc generalization (also
called robust or post-selection generalization), which says that, given the
output of the algorithm, it is hard to find any statistic for which the data
differs significantly from the population it came from.
  In this work we show several limitations on the power of algorithms
satisfying post hoc generalization. First, we show a tight lower bound on the
error of any algorithm that satisfies post hoc generalization and answers
adaptively chosen statistical queries, showing a strong barrier to progress in
post selection data analysis. Second, we show that post hoc generalization is
not closed under composition, despite many examples of such algorithms
exhibiting strong composition properties.
Unsupervised Training for 3D Morphable Model Regression
Authors: Kyle Genova, Forrester Cole, Aaron Maschinot, Aaron Sarna, Daniel Vlasic, William T. Freeman
Date: 2018-06-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Face Reconstruction, model, regression

We present a method for training a regression network from image pixels to 3D
morphable model coordinates using only unlabeled photographs. The training loss
is based on features from a facial recognition network, computed on-the-fly by
rendering the predicted faces with a differentiable renderer. To make training
from features feasible and avoid network fooling effects, we introduce three
objectives: a batch distribution loss that encourages the output distribution
to match the distribution of the morphable model, a loopback loss that ensures
the network can correctly reinterpret its own output, and a multi-view identity
loss that compares the features of the predicted 3D face and the input
photograph from multiple viewing angles. We train a regression network using
these objectives, a set of unlabeled photographs, and the morphable model
itself, and demonstrate state-of-the-art results.
Integrating Human-Provided Information Into Belief State Representation Using Dynamic Factorization
Authors: Rohan Chitnis, Leslie Pack Kaelbling, Tomás Lozano-Pérez
Date: 2018-02-28 00:00:00

In partially observed environments, it can be useful for a human to provide
the robot with declarative information that represents probabilistic relational
constraints on properties of objects in the world, augmenting the robot's
sensory observations. For instance, a robot tasked with a search-and-rescue
mission may be informed by the human that two victims are probably in the same
room. An important question arises: how should we represent the robot's
internal knowledge so that this information is correctly processed and combined
with raw sensory information? In this paper, we provide an efficient belief
state representation that dynamically selects an appropriate factoring,
combining aspects of the belief when they are correlated through information
and separating them when they are not. This strategy works in open domains, in
which the set of possible objects is not known in advance, and provides
significant improvements in inference time over a static factoring, leading to
more efficient planning for complex partially observed tasks. We validate our
approach experimentally in two open-domain planning problems: a 2D discrete
gridworld task and a 3D continuous cooking task. A supplementary video can be
found at http://tinyurl.com/chitnis-iros-18.
Crime Event Embedding with Unsupervised Feature Selection
Authors: Shixiang Zhu, Yao Xie
Date: 2018-06-15 00:00:00
Tasks: feature selection

We present a novel event embedding algorithm for crime data that can jointly
capture time, location, and the complex free-text component of each event. The
embedding is achieved by regularized Restricted Boltzmann Machines (RBMs), and
we introduce a new way to regularize by imposing a $\ell_1$ penalty on the
conditional distributions of the observed variables of RBMs. This choice of
regularization performs feature selection and it also leads to efficient
computation since the gradient can be computed in a closed form. The feature
selection forces embedding to be based on the most important keywords, which
captures the common modus operandi (M. O.) in crime series. Using numerical
experiments on a large-scale crime dataset, we show that our regularized RBMs
can achieve better event embedding and the selected features are highly
interpretable from human understanding.
Training VAEs Under Structured Residuals
Authors: Garoe Dorta, Sara Vicente, Lourdes Agapito, Neill D. F. Campbell, Ivor Simpson
Date: 2018-04-03 00:00:00

Variational auto-encoders (VAEs) are a popular and powerful deep generative
model. Previous works on VAEs have assumed a factorized likelihood model,
whereby the output uncertainty of each pixel is assumed to be independent. This
approximation is clearly limited as demonstrated by observing a residual image
from a VAE reconstruction, which often possess a high level of structure. This
paper demonstrates a novel scheme to incorporate a structured Gaussian
likelihood prediction network within the VAE that allows the residual
correlations to be modeled. Our novel architecture, with minimal increase in
complexity, incorporates the covariance matrix prediction within the VAE. We
also propose a new mechanism for allowing structured uncertainty on color
images. Furthermore, we provide a scheme for effectively training this model,
and include some suggestions for improving performance in terms of efficiency
or modeling longer range correlations.
Minibatch Gibbs Sampling on Large Graphical Models
Authors: Christopher De Sa, Vincent Chen, Wing Wong
Date: 2018-06-15 00:00:00
Proceeding: ICML 2018 7

Gibbs sampling is the de facto Markov chain Monte Carlo method used for
inference and learning on large scale graphical models. For complicated factor
graphs with lots of factors, the performance of Gibbs sampling can be limited
by the computational cost of executing a single update step of the Markov
chain. This cost is proportional to the degree of the graph, the number of
factors adjacent to each variable. In this paper, we show how this cost can be
reduced by using minibatching: subsampling the factors to form an estimate of
their sum. We introduce several minibatched variants of Gibbs, show that they
can be made unbiased, prove bounds on their convergence rates, and show that
under some conditions they can result in asymptotic single-update-run-time
speedups over plain Gibbs sampling.