pwc-archive/papers-with-abstracts huggingface dataset:
A Strongly Asymptotically Optimal Agent in General Environments
Authors: Michael K. Cohen, Elliot Catt, Marcus Hutter
Date: 2019-03-04 00:00:00
Tasks: Reinforcement Learning

Reinforcement Learning agents are expected to eventually perform well. Typically, this takes the form of a guarantee about the asymptotic behavior of an algorithm given some assumptions about the environment. We present an algorithm for a policy whose value approaches the optimal value with probability 1 in all computable probabilistic environments, provided the agent has a bounded horizon. This is known as strong asymptotic optimality, and it was previously unknown whether it was possible for a policy to be strongly asymptotically optimal in the class of all computable probabilistic environments. Our agent, Inquisitive Reinforcement Learner (Inq), is more likely to explore the more it expects an exploratory action to reduce its uncertainty about which environment it is in, hence the term inquisitive. Exploring inquisitively is a strategy that can be applied generally; for more manageable environment classes, inquisitiveness is tractable. We conducted experiments in "grid-worlds" to compare the Inquisitive Reinforcement Learner to other weakly asymptotically optimal agents.
Hand Pose Estimation: A Survey
Authors: Bardia Doosti
Date: 2019-03-03 00:00:00
Tasks: All, Hand Pose Estimation, Pose Estimation, Survey

The success of Deep Convolutional Neural Networks (CNNs) in recent years in almost all the Computer Vision tasks on one hand, and the popularity of low-cost consumer depth cameras on the other, has made Hand Pose Estimation a hot topic in computer vision field. In this report, we will first explain the hand pose estimation problem and will review major approaches solving this problem, especially the two different problems of using depth maps or RGB images. We will survey the most important papers in each field and will discuss the strengths and weaknesses of each. Finally, we will explain the biggest datasets in this field in detail and list 22 datasets with all their properties. To the best of our knowledge this is the most complete list of all the datasets in the hand pose estimation field.
Budgeted Reinforcement Learning in Continuous State Space
Authors: Nicolas Carrara, Edouard Leurent, Romain Laroche, Tanguy Urvoy, Odalric-Ambrym Maillard, Olivier Pietquin
Date: 2019-03-03 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Autonomous Driving, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemented in the shape of a cost signal constrained to lie below an - adjustable - threshold. So far, BMDPs could only be solved in the case of finite state spaces with known dynamics. This work extends the state-of-the-art to continuous spaces environments and unknown dynamics. We show that the solution to a BMDP is a fixed point of a novel Budgeted Bellman Optimality operator. This observation allows us to introduce natural extensions of Deep Reinforcement Learning algorithms to address large-scale BMDPs. We validate our approach on two simulated applications: spoken dialogue and autonomous driving.
Hacking Google reCAPTCHA v3 using Reinforcement Learning
Authors: Ismail Akrout, Amal Feriani, Mohamed Akrout
Date: 2019-03-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present a Reinforcement Learning (RL) methodology to bypass Google
reCAPTCHA v3. We formulate the problem as a grid world where the agent learns
how to move the mouse and click on the reCAPTCHA button to receive a high
score. We study the performance of the agent when we vary the cell size of the
grid world and show that the performance drops when the agent takes big steps
toward the goal. Finally, we used a divide and conquer strategy to defeat the
reCAPTCHA system for any grid resolution. Our proposed method achieves a
success rate of 97.4% on a 100x100 grid and 96.7% on a 1000x1000 screen
resolution.
Self-Supervised Learning of Face Representations for Video Face Clustering
Authors: Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
Date: 2019-03-03 00:00:00
Tasks: Clustering, Diversity, Face Clustering, Self-Supervised Learning

Analyzing the story behind TV series and movies often requires understanding
who the characters are and what they are doing. With improving deep face
models, this may seem like a solved problem. However, as face detectors get
better, clustering/identification needs to be revisited to address increasing
diversity in facial appearance. In this paper, we address video face clustering
using unsupervised methods. Our emphasis is on distilling the essential
information, identity, from the representations obtained using deep pre-trained
face networks. We propose a self-supervised Siamese network that can be trained
without the need for video/track based supervision, and thus can also be
applied to image collections. We evaluate our proposed method on three video
face clustering datasets. The experiments show that our methods outperform
current state-of-the-art methods on all datasets. Video face clustering is
lacking a common benchmark as current works are often evaluated with different
metrics and/or different sets of face tracks.
X-Section: Cross-Section Prediction for Enhanced RGBD Fusion
Authors: Andrea Nicastro, Ronald Clark, Stefan Leutenegger
Date: 2019-03-03 00:00:00
Tasks: 3D Reconstruction, Object

Detailed 3D reconstruction is an important challenge with application to robotics, augmented and virtual reality, which has seen impressive progress throughout the past years. Advancements were driven by the availability of depth cameras (RGB-D), as well as increased compute power, e.g.\ in the form of GPUs -- but also thanks to inclusion of machine learning in the process. Here, we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep learning to make object-level predictions about thicknesses that can be readily integrated into a volumetric multi-view fusion process, where we propose an extension to the popular KinectFusion approach. In essence, our method allows to complete shape in general indoor scenes behind what is sensed by the RGB-D camera, which may be crucial e.g.\ for robotic manipulation tasks or efficient scene exploration. Predicting object thicknesses rather than volumes allows us to work with comparably high spatial resolution without exploding memory and training data requirements on the employed Convolutional Neural Networks. In a series of qualitative and quantitative evaluations, we demonstrate how we accurately predict object thickness and reconstruct general 3D scenes containing multiple objects.
Classification via local manifold approximation
Authors: Didong Li, David B. Dunson
Date: 2019-03-03 00:00:00
Tasks: Classification, General Classification, Single Particle Analysis

Classifiers label data as belonging to one of a set of groups based on input
features. It is challenging to obtain accurate classification performance when
the feature distributions in the different classes are complex, with nonlinear,
overlapping and intersecting supports. This is particularly true when training
data are limited. To address this problem, this article proposes a new type of
classifier based on obtaining a local approximation to the support of the data
within each class in a neighborhood of the feature to be classified, and
assigning the feature to the class having the closest support. This general
algorithm is referred to as LOcal Manifold Approximation (LOMA) classification.
As a simple and theoretically supported special case having excellent
performance in a broad variety of examples, we use spheres for local
approximation, obtaining a SPherical Approximation (SPA) classifier. We
illustrate substantial gains for SPA over competitors on a variety of
challenging simulated and real data examples.
Analysis of a Generalized Expectation-Maximization Algorithm for Gaussian Mixture Models: A Control Systems Perspective
Authors: Sarthak Chatterjee, Orlando Romero, Sérgio Pequito
Date: 2019-03-03 00:00:00
Tasks: Clustering

The Expectation-Maximization (EM) algorithm is one of the most popular methods used to solve the problem of parametric distribution-based clustering in unsupervised learning. In this paper, we propose to analyze a generalized EM (GEM) algorithm in the context of Gaussian mixture models, where the maximization step in the EM is replaced by an increasing step. We show that this GEM algorithm can be understood as a linear time-invariant (LTI) system with a feedback nonlinearity. Therefore, we explore some of its convergence properties by leveraging tools from robust control theory. Lastly, we explain how the proposed GEM can be designed, and present a pedagogical example to understand the advantages of the proposed approach.
Anytime Online-to-Batch Conversions, Optimism, and Acceleration
Authors: Ashok Cutkosky
Date: 2019-03-03 00:00:00

A standard way to obtain convergence guarantees in stochastic convex
optimization is to run an online learning algorithm and then output the average
of its iterates: the actual iterates of the online learning algorithm do not
come with individual guarantees. We close this gap by introducing a black-box
modification to any online learning algorithm whose iterates converge to the
optimum in stochastic scenarios. We then consider the case of smooth losses,
and show that combining our approach with optimistic online learning algorithms
immediately yields a fast convergence rate of $O(L/T^{3/2}+\sigma/\sqrt{T})$ on
$L$-smooth problems with $\sigma^2$ variance in the gradients. Finally, we
provide a reduction that converts any adaptive online algorithm into one that
obtains the optimal accelerated rate of $\tilde O(L/T^2 + \sigma/\sqrt{T})$,
while still maintaining $\tilde O(1/\sqrt{T})$ convergence in the non-smooth
setting. Importantly, our algorithms adapt to $L$ and $\sigma$ automatically:
they do not need to know either to obtain these rates.
Matching Thermal to Visible Face Images Using a Semantic-Guided Generative Adversarial Network
Authors: Cunjian Chen, Arun Ross
Date: 2019-03-03 00:00:00
Tasks: Face Generation, Face Parsing, Face Recognition, Generative Adversarial Network

Designing face recognition systems that are capable of matching face images
obtained in the thermal spectrum with those obtained in the visible spectrum is
a challenging problem. In this work, we propose the use of semantic-guided
generative adversarial network (SG-GAN) to automatically synthesize visible
face images from their thermal counterparts. Specifically, semantic labels,
extracted by a face parsing network, are used to compute a semantic loss
function to regularize the adversarial network during training. These semantic
cues denote high-level facial component information associated with each pixel.
Further, an identity extraction network is leveraged to generate multi-scale
features to compute an identity loss function. To achieve photo-realistic
results, a perceptual loss function is introduced during network training to
ensure that the synthesized visible face is perceptually similar to the target
visible face image. We extensively evaluate the benefits of individual loss
functions, and combine them effectively to learn the mapping from thermal to
visible face images. Experiments involving two multispectral face datasets show
that the proposed method achieves promising results in both face synthesis and
cross-spectral face matching.
End-to-End Game-Focused Learning of Adversary Behavior in Security Games
Authors: Andrew Perrault, Bryan Wilder, Eric Ewing, Aditya Mate, Bistra Dilkina, Milind Tambe
Date: 2019-03-03 00:00:00

Stackelberg security games are a critical tool for maximizing the utility of limited defense resources to protect important targets from an intelligent adversary. Motivated by green security, where the defender may only observe an adversary's response to defense on a limited set of targets, we study the problem of learning a defense that generalizes well to a new set of targets with novel feature values and combinations. Traditionally, this problem has been addressed via a two-stage approach where an adversary model is trained to maximize predictive accuracy without considering the defender's optimization problem. We develop an end-to-end game-focused approach, where the adversary model is trained to maximize a surrogate for the defender's expected utility. We show both in theory and experimental results that our game-focused approach achieves higher defender expected utility than the two-stage alternative when there is limited data.
Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks
Authors: Jonas Rothfuss, Fabio Ferreira, Simon Walther, Maxim Ulrich
Date: 2019-03-03 00:00:00
Tasks: Density Estimation

Given a set of empirical observations, conditional density estimation aims to
capture the statistical relationship between a conditional variable
$\mathbf{x}$ and a dependent variable $\mathbf{y}$ by modeling their
conditional probability $p(\mathbf{y}|\mathbf{x})$. The paper develops best
practices for conditional density estimation for finance applications with
neural networks, grounded on mathematical insights and empirical evaluations.
In particular, we introduce a noise regularization and data normalization
scheme, alleviating problems with over-fitting, initialization and
hyper-parameter sensitivity of such estimators. We compare our proposed
methodology with popular semi- and non-parametric density estimators, underpin
its effectiveness in various benchmarks on simulated and Euro Stoxx 50 data and
show its superior performance. Our methodology allows to obtain high-quality
estimators for statistical expectations of higher moments, quantiles and
non-linear return transformations, with very little assumptions about the
return dynamic.
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies
Authors: Ethan Wilcox, Peng Qian, Richard Futrell, Miguel Ballesteros, Roger Levy
Date: 2019-03-03 00:00:00
Proceeding: NAACL 2019 6
Tasks: Language Modeling, Language Modelling

State-of-the-art LSTM language models trained on large corpora learn
sequential contingencies in impressive detail and have been shown to acquire a
number of non-local grammatical dependencies with some success. Here we
investigate whether supervision with hierarchical structure enhances learning
of a range of grammatical dependencies, a question that has previously been
addressed only for subject-verb agreement. Using controlled experimental
methods from psycholinguistics, we compare the performance of word-based LSTM
models versus two models that represent hierarchical structure and deploy it in
left-to-right processing: Recurrent Neural Network Grammars (RNNGs) (Dyer et
al., 2016) and a incrementalized version of the Parsing-as-Language-Modeling
configuration from Chariak et al., (2016). Models are tested on a diverse range
of configurations for two classes of non-local grammatical dependencies in
English---Negative Polarity licensing and Filler--Gap Dependencies. Using the
same training data across models, we find that structurally-supervised models
outperform the LSTM, with the RNNG demonstrating best results on both types of
grammatical dependencies and even learning many of the Island Constraints on
the filler--gap dependency. Structural supervision thus provides data
efficiency advantages over purely string-based training of neural language
models in acquiring human-like generalizations about non-local grammatical
dependencies.
Bernoulli Race Particle Filters
Authors: Sebastian M. Schmon, Arnaud Doucet, George Deligiannidis
Date: 2019-03-03 00:00:00
Tasks: valid

When the weights in a particle filter are not available analytically,
standard resampling methods cannot be employed. To circumvent this problem
state-of-the-art algorithms replace the true weights with non-negative unbiased
estimates. This algorithm is still valid but at the cost of higher variance of
the resulting filtering estimates in comparison to a particle filter using the
true weights. We propose here a novel algorithm that allows for resampling
according to the true intractable weights when only an unbiased estimator of
the weights is available. We demonstrate our algorithm on several examples.
The Role of Artificial Intelligence (AI) in Adaptive eLearning System (AES) Content Formation: Risks and Opportunities involved
Authors: Suleiman Adamu, Jamilu Awwalu
Date: 2019-03-03 00:00:00
Tasks: Retrieval

Artificial Intelligence (AI) plays varying roles in supporting both existing
and emerging technologies. In the area of Learning and Tutoring, it plays key
role in Intelligent Tutoring Systems (ITS). The fusion of ITS with Adaptive
Hypermedia and Multimedia (AHAM) form the backbone of Adaptive eLearning
Systems (AES) which provides personalized experiences to learners. This
experience is important because it facilitates the accurate delivery of the
learning modules in specific to the learner capacity and readiness. AES types
vary, with Adaptive Web Based eLearning Systems (AWBES) being the popular type
because of wider access offered by the web technology.The retrieval and
aggregation of contents for any eLearning system is critical whichis determined
by the relevance of learning material to the needs of the learner.In this
paper, we discuss components of AES, role of AI in AES content aggregation,
possible risks and available opportunities.
Detecting dementia in Mandarin Chinese using transfer learning from a parallel corpus
Authors: Bai Li, Yi-Te Hsu, Frank Rudzicz
Date: 2019-03-03 00:00:00
Proceeding: NAACL 2019 6
Tasks: BIG-bench Machine Learning, Machine Translation, Transfer Learning, Translation

Machine learning has shown promise for automatic detection of Alzheimer's disease (AD) through speech; however, efforts are hampered by a scarcity of data, especially in languages other than English. We propose a method to learn a correspondence between independently engineered lexicosyntactic features in two languages, using a large parallel corpus of out-of-domain movie dialogue data. We apply it to dementia detection in Mandarin Chinese, and demonstrate that our method outperforms both unilingual and machine translation-based baselines. This appears to be the first study that transfers feature domains in detecting cognitive decline.
Accelerating Training of Deep Neural Networks with a Standardization Loss
Authors: Jasmine Collins, Johannes Balle, Jonathon Shlens
Date: 2019-03-03 00:00:00
Tasks: image-classification, Image Classification

A significant advance in accelerating neural network training has been the
development of normalization methods, permitting the training of deep models
both faster and with better accuracy. These advances come with practical
challenges: for instance, batch normalization ties the prediction of individual
examples with other examples within a batch, resulting in a network that is
heavily dependent on batch size. Layer normalization and group normalization
are data-dependent and thus must be continually used, even at test-time. To
address the issues that arise from using explicit normalization techniques, we
propose to replace existing normalization methods with a simple, secondary
objective loss that we term a standardization loss. This formulation is
flexible and robust across different batch sizes and surprisingly, this
secondary objective accelerates learning on the primary training objective.
Because it is a training loss, it is simply removed at test-time, and no
further effort is needed to maintain normalized activations. We find that a
standardization loss accelerates training on both small- and large-scale image
classification experiments, works with a variety of architectures, and is
largely robust to training across different batch sizes.
Pancreas segmentation with probabilistic map guided bi-directional recurrent UNet
Authors: Jun Li, Xiaozhu Lin, Hui Che, Hao Li, Xiaohua Qian
Date: 2019-03-03 00:00:00
Tasks: Pancreas Segmentation, Segmentation

Pancreas segmentation in medical imaging data is of great significance for clinical pancreas diagnostics and treatment. However, the large population variations in the pancreas shape and volume cause enormous segmentation difficulties, even for state-of-the-art algorithms utilizing fully-convolutional neural networks (FCNs). Specifically, pancreas segmentation suffers from the loss of spatial information in 2D methods, and the high computational cost of 3D methods. To alleviate these problems, we propose a probabilistic-map-guided bi-directional recurrent UNet (PBR-UNet) architecture, which fuses intra-slice information and inter-slice probabilistic maps into a local 3D hybrid regularization scheme, which is followed by bi-directional recurrent network optimization. The PBR-UNet method consists of an initial estimation module for efficiently extracting pixel-level probabilistic maps and a primary segmentation module for propagating hybrid information through a 2.5D U-Net architecture. Specifically, local 3D information is inferred by combining an input image with the probabilistic maps of the adjacent slices into multichannel hybrid data, and then hierarchically aggregating the hybrid information of the entire segmentation network. Besides, a bi-directional recurrent optimization mechanism is developed to update the hybrid information in both the forward and the backward directions. This allows the proposed network to make full and optimal use of the local context information. Quantitative and qualitative evaluation was performed on the NIH Pancreas-CT dataset, and our proposed PBR-UNet method achieved better segmentation results with less computational cost compared to other state-of-the-art methods.
3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting
Authors: Bing Yu, Mengzhang Li, Jiyong Zhang, Zhanxing Zhu
Date: 2019-03-03 00:00:00
Tasks: 4D Spatio Temporal Semantic Segmentation, Time Series, Time Series Analysis, Traffic Prediction

Spatio-temporal prediction plays an important role in many application areas
especially in traffic domain. However, due to complicated spatio-temporal
dependency and high non-linear dynamics in road networks, traffic prediction
task is still challenging. Existing works either exhibit heavy training cost or
fail to accurately capture the spatio-temporal patterns, also ignore the
correlation between distant roads that share the similar patterns. In this
paper, we propose a novel deep learning framework to overcome these issues: 3D
Temporal Graph Convolutional Networks (3D-TGCN). Two novel components of our
model are introduced. (1) Instead of constructing the road graph based on
spatial information, we learn it by comparing the similarity between time
series for each road, thus providing a spatial information free framework. (2)
We propose an original 3D graph convolution model to model the spatio-temporal
data more accurately. Empirical results show that 3D-TGCN could outperform
state-of-the-art baselines.