pwc-archive/papers-with-abstracts huggingface dataset:
Guiding Neuroevolution with Structural Objectives
Authors: Kai Olav Ellefsen, Joost Huizinga, Jim Torresen
Date: 2019-02-12 00:00:00
Tasks: Diversity, Evolutionary Algorithms

The structure and performance of neural networks are intimately connected,
and by use of evolutionary algorithms, neural network structures optimally
adapted to a given task can be explored. Guiding such neuroevolution with
additional objectives related to network structure has been shown to improve
performance in some cases, especially when modular neural networks are
beneficial. However, apart from objectives aiming to make networks more
modular, such structural objectives have not been widely explored. We propose
two new structural objectives and test their ability to guide evolving neural
networks on two problems which can benefit from decomposition into subtasks.
The first structural objective guides evolution to align neural networks with a
user-recommended decomposition pattern. Intuitively, this should be a powerful
guiding target for problems where human users can easily identify a structure.
The second structural objective guides evolution towards a population with a
high diversity in decomposition patterns. This results in exploration of many
different ways to decompose a problem, allowing evolution to find good
decompositions faster. Tests on our target problems reveal that both methods
perform well on a problem with a very clear and decomposable structure.
However, on a problem where the optimal decomposition is less obvious, the
structural diversity objective is found to outcompete other structural
objectives -- and this technique can even increase performance on problems
without any decomposable structure at all.
Gaussian Mean Field Regularizes by Limiting Learned Information
Authors: Julius Kunze, Louis Kirsch, Hippolyt Ritter, David Barber
Date: 2019-02-12 00:00:00
Tasks: Variational Inference

Variational inference with a factorized Gaussian posterior estimate is a
widely used approach for learning parameters and hidden variables. Empirically,
a regularizing effect can be observed that is poorly understood. In this work,
we show how mean field inference improves generalization by limiting mutual
information between learned parameters and the data through noise. We quantify
a maximum capacity when the posterior variance is either fixed or learned and
connect it to generalization error, even when the KL-divergence in the
objective is rescaled. Our experiments demonstrate that bounding information
between parameters and data effectively regularizes neural networks on both
supervised and unsupervised tasks.
Winning the Big Data Technologies Horizon Prize: Fast and reliable forecasting of electricity grid traffic by identification of recurrent fluctuations
Authors: Jose M. G. Vilar
Date: 2019-02-12 00:00:00
Tasks: regression

This paper provides a description of the approach and methodology I used in
winning the European Union Big Data Technologies Horizon Prize on data-driven
prediction of electricity grid traffic. The methodology relies on identifying
typical short-term recurrent fluctuations, which is subsequently refined
through a regression-of-fluctuations approach. The key points and strategic
considerations that led to selecting or discarding different methodological
aspects are also discussed. The criteria include adaptability to changing
conditions, reliability with outliers and missing data, robustness to noise,
and efficiency in implementation.
Hyperbolic Disk Embeddings for Directed Acyclic Graphs
Authors: Ryota Suzuki, Ryusuke Takahama, Shun Onoda
Date: 2019-02-12 00:00:00

Obtaining continuous representations of structural data such as directed acyclic graphs (DAGs) has gained attention in machine learning and artificial intelligence. However, embedding complex DAGs in which both ancestors and descendants of nodes are exponentially increasing is difficult. Tackling in this problem, we develop Disk Embeddings, which is a framework for embedding DAGs into quasi-metric spaces. Existing state-of-the-art methods, Order Embeddings and Hyperbolic Entailment Cones, are instances of Disk Embedding in Euclidean space and spheres respectively. Furthermore, we propose a novel method Hyperbolic Disk Embeddings to handle exponential growth of relations. The results of our experiments show that our Disk Embedding models outperform existing methods especially in complex DAGs other than trees.
An In-Vehicle KWS System with Multi-Source Fusion for Vehicle Applications
Authors: Yue Tan, Kan Zheng, Lei Lei
Date: 2019-02-12 00:00:00
Tasks: General Classification, Keyword Spotting

In order to maximize detection precision rate as well as the recall rate,
this paper proposes an in-vehicle multi-source fusion scheme in Keyword
Spotting (KWS) System for vehicle applications. Vehicle information, as a new
source for the original system, is collected by an in-vehicle data acquisition
platform while the user is driving. A Deep Neural Network (DNN) is trained to
extract acoustic features and make a speech classification. Based on the
posterior probabilities obtained from DNN, the vehicle information including
the speed and direction of vehicle is applied to choose the suitable parameter
from a pair of sensitivity values for the KWS system. The experimental results
show that the KWS system with the proposed multi-source fusion scheme can
achieve better performances in term of precision rate, recall rate, and mean
square error compared to the system without it.
GAN- vs. JPEG2000 Image Compression for Distributed Automotive Perception: Higher Peak SNR Does Not Mean Better Semantic Segmentation
Authors: Jonas Löhdefink, Andreas Bär, Nico M. Schmidt, Fabian Hüger, Peter Schlicht, Tim Fingscheidt
Date: 2019-02-12 00:00:00
Tasks: Autonomous Driving, Decoder, Image Compression, Segmentation, Semantic Segmentation, SSIM

The high amount of sensors required for autonomous driving poses enormous
challenges on the capacity of automotive bus systems. There is a need to
understand tradeoffs between bitrate and perception performance. In this paper,
we compare the image compression standards JPEG, JPEG2000, and WebP to a modern
encoder/decoder image compression approach based on generative adversarial
networks (GANs). We evaluate both the pure compression performance using
typical metrics such as peak signal-to-noise ratio (PSNR), structural
similarity (SSIM) and others, but also the performance of a subsequent
perception function, namely a semantic segmentation (characterized by the mean
intersection over union (mIoU) measure). Not surprisingly, for all investigated
compression methods, a higher bitrate means better results in all investigated
quality metrics. Interestingly, however, we show that the semantic segmentation
mIoU of the GAN autoencoder in the highly relevant low-bitrate regime (at
0.0625 bit/pixel) is better by 3.9% absolute than JPEG2000, although the latter
still is considerably better in terms of PSNR (5.91 dB difference). This effect
can greatly be enlarged by training the semantic segmentation model with images
originating from the decoder, so that the mIoU using the segmentation model
trained by GAN reconstructions exceeds the use of the model trained with
original images by almost 20% absolute. We conclude that distributed perception
in future autonomous driving will most probably not provide a solution to the
automotive bus capacity bottleneck by using standard compression schemes such
as JPEG2000, but requires modern coding approaches, with the GAN
encoder/decoder method being a promising candidate.
Unpriortized Autoencoder For Image Generation
Authors: Jaeyoung Yoo, Hojun Lee, Nojun Kwak
Date: 2019-02-12 00:00:00
Tasks: Density Estimation, Image Generation, Incremental Learning

In this paper, we treat the image generation task using an autoencoder, a representative latent model. Unlike many studies regularizing the latent variable's distribution by assuming a manually specified prior, we approach the image generation task using an autoencoder by directly estimating the latent distribution. To this end, we introduce 'latent density estimator' which captures latent distribution explicitly and propose its structure. Through experiments, we show that our generative model generates images with the improved visual quality compared to previous autoencoder-based generative models.
Towards Self-Supervised High Level Sensor Fusion
Authors: Qadeer Khan, Torsten Schön, Patrick Wenzel
Date: 2019-02-12 00:00:00
Tasks: Self-Driving Cars, Sensor Fusion, Vocal Bursts Intensity Prediction

In this paper, we present a framework to control a self-driving car by fusing
raw information from RGB images and depth maps. A deep neural network
architecture is used for mapping the vision and depth information,
respectively, to steering commands. This fusion of information from two sensor
sources allows to provide redundancy and fault tolerance in the presence of
sensor failures. Even if one of the input sensors fails to produce the correct
output, the other functioning sensor would still be able to maneuver the car.
Such redundancy is crucial in the critical application of self-driving cars.
The experimental results have showed that our method is capable of learning to
use the relevant sensor information even when one of the sensors fail without
any explicit signal.
Reading Protocol: Understanding what has been Read in Interactive Information Retrieval Tasks
Authors: Daniel Hienert, Dagmar Kern, Matthew Mitsui, Chirag Shah, Nicholas J. Belkin
Date: 2019-02-12 00:00:00
Tasks: Information Retrieval, Retrieval

In Interactive Information Retrieval (IIR) experiments the user's gaze motion
on web pages is often recorded with eye tracking. The data is used to analyze
gaze behavior or to identify Areas of Interest (AOI) the user has looked at. So
far, tools for analyzing eye tracking data have certain limitations in
supporting the analysis of gaze behavior in IIR experiments. Experiments often
consist of a huge number of different visited web pages. In existing analysis
tools the data can only be analyzed in videos or images and AOIs for every
single web page have to be specified by hand, in a very time consuming process.
In this work, we propose the reading protocol software which breaks eye
tracking data down to the textual level by considering the HTML structure of
the web pages. This has a lot of advantages for the analyst. First and
foremost, it can easily be identified on a large scale what has actually been
viewed and read on the stimuli pages by the subjects. Second, the web page
structure can be used to filter to AOIs. Third, gaze data of multiple users can
be presented on the same page, and fourth, fixation times on text can be
exported and further processed in other tools. We present the software, its
validation, and example use cases with data from three existing IIR
experiments.
Table2answer: Read the database and answer without SQL
Authors: Tong Guo, Huilin Gao
Date: 2019-02-12 00:00:00
Tasks: Form, Question Answering, Semantic Parsing

Semantic parsing is the task of mapping natural language to logic form. In question answering, semantic parsing can be used to map the question to logic form and execute the logic form to get the answer. One key problem for semantic parsing is the hard label work. We study this problem in another way: we do not use the logic form any more. Instead we only use the schema and answer info. We think that the logic form step can be injected into the deep model. The reason why we think removing the logic form step is possible is that human can do the task without explicit logic form. We use BERT-based model and do the experiment in the WikiSQL dataset, which is a large natural language to SQL dataset. Our experimental evaluations that show that our model can achieves the baseline results in WikiSQL dataset.
NAIL: A General Interactive Fiction Agent
Authors: Matthew Hausknecht, Ricky Loynd, Greg Yang, Adith Swaminathan, Jason D. Williams
Date: 2019-02-12 00:00:00
Tasks: Decision Making

Interactive Fiction (IF) games are complex textual decision making problems.
This paper introduces NAIL, an autonomous agent for general parser-based IF
games. NAIL won the 2018 Text Adventure AI Competition, where it was evaluated
on twenty unseen games. This paper describes the architecture, development, and
insights underpinning NAIL's performance.
A system for generating complex physically accurate sensor images for automotive applications
Authors: Zhenyi Liu, Minghao Shen, Jia-Qi Zhang, Shuangting Liu, Henryk Blasinski, Trisha Lian, Brian Wandell
Date: 2019-02-12 00:00:00
Tasks: General Classification, object-detection, Object Detection

We describe an open-source simulator that creates sensor irradiance and
sensor images of typical automotive scenes in urban settings. The purpose of
the system is to support camera design and testing for automotive applications.
The user can specify scene parameters (e.g., scene type, road type, traffic
density, time of day) to assemble a large number of random scenes from graphics
assets stored in a database. The sensor irradiance is generated using
quantitative computer graphics methods, and the sensor images are created using
image systems sensor simulation. The synthetic sensor images have pixel level
annotations; hence, they can be used to train and evaluate neural networks for
imaging tasks, such as object detection and classification. The end-to-end
simulation system supports quantitative assessment, from scene to camera to
network accuracy, for automotive applications.
Deep Reinforcement Learning from Policy-Dependent Human Feedback
Authors: Dilip Arumugam, Jun Ki Lee, Sophie Saskin, Michael L. Littman
Date: 2019-02-12 00:00:00
Tasks: Deep Reinforcement Learning, Minecraft, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

To widen their accessibility and increase their utility, intelligent agents
must be able to learn complex behaviors as specified by (non-expert) human
users. Moreover, they will need to learn these behaviors within a reasonable
amount of time while efficiently leveraging the sparse feedback a human trainer
is capable of providing. Recent work has shown that human feedback can be
characterized as a critique of an agent's current behavior rather than as an
alternative reward signal to be maximized, culminating in the COnvergent
Actor-Critic by Humans (COACH) algorithm for making direct policy updates based
on human feedback. Our work builds on COACH, moving to a setting where the
agent's policy is represented by a deep neural network. We employ a series of
modifications on top of the original COACH algorithm that are critical for
successfully learning behaviors from high-dimensional observations, while also
satisfying the constraint of obtaining reduced sample complexity. We
demonstrate the effectiveness of our Deep COACH algorithm in the rich 3D world
of Minecraft with an agent that learns to complete tasks by mapping from raw
pixels to actions using only real-time human feedback in 10-15 minutes of
interaction.
A Theory of Selective Prediction
Authors: Mingda Qiao, Gregory Valiant
Date: 2019-02-12 00:00:00
Tasks: Open-Ended Question Answering, Prediction

We consider a model of selective prediction, where the prediction algorithm is given a data sequence in an online fashion and asked to predict a pre-specified statistic of the upcoming data points. The algorithm is allowed to choose when to make the prediction as well as the length of the prediction window, possibly depending on the observations so far. We prove that, even without any distributional assumption on the input data stream, a large family of statistics can be estimated to non-trivial accuracy. To give one concrete example, suppose that we are given access to an arbitrary binary sequence $x_1, \ldots, x_n$ of length $n$. Our goal is to accurately predict the average observation, and we are allowed to choose the window over which the prediction is made: for some $t < n$ and $m \le n - t$, after seeing $t$ observations we predict the average of $x_{t+1}, \ldots, x_{t+m}$. This particular problem was first studied in Drucker (2013) and referred to as the "density prediction game". We show that the expected squared error of our prediction can be bounded by $O(\frac{1}{\log n})$ and prove a matching lower bound, which resolves an open question raised in Drucker (2013). This result holds for any sequence (that is not adaptive to when the prediction is made, or the predicted value), and the expectation of the error is with respect to the randomness of the prediction algorithm. Our results apply to more general statistics of a sequence of observations, and we highlight several open directions for future work.
Thompson Sampling with Information Relaxation Penalties
Authors: Seungki Min, Costis Maglaras, Ciamac C. Moallemi
Date: 2019-02-12 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Thompson Sampling

We consider a finite-horizon multi-armed bandit (MAB) problem in a Bayesian setting, for which we propose an information relaxation sampling framework. With this framework, we define an intuitive family of control policies that include Thompson sampling (TS) and the Bayesian optimal policy as endpoints. Analogous to TS, which, at each decision epoch pulls an arm that is best with respect to the randomly sampled parameters, our algorithms sample entire future reward realizations and take the corresponding best action. However, this is done in the presence of "penalties" that seek to compensate for the availability of future information. We develop several novel policies and performance bounds for MAB problems that vary in terms of improving performance and increasing computational complexity between the two endpoints. Our policies can be viewed as natural generalizations of TS that simultaneously incorporate knowledge of the time horizon and explicitly consider the exploration-exploitation trade-off. We prove associated structural results on performance bounds and suboptimality gaps. Numerical experiments suggest that this new class of policies perform well, in particular in settings where the finite time horizon introduces significant exploration-exploitation tension into the problem. Finally, inspired by the finite-horizon Gittins index, we propose an index policy that builds on our framework that particularly outperforms the state-of-the-art algorithms in our numerical experiments.
Post-Data Augmentation to Improve Deep Pose Estimation of Extreme and Wild Motions
Authors: Kohei Toyoda, Michinari Kono, Jun Rekimoto
Date: 2019-02-12 00:00:00
Tasks: Data Augmentation, GPU, Pose Estimation

Contributions of recent deep-neural-network (DNN) based techniques have been
playing a significant role in human-computer interaction (HCI) and user
interface (UI) domains. One of the commonly used DNNs is human pose estimation.
This kind of technique is widely used for motion capturing of humans, and to
generate or modify virtual avatars. However, in order to gain accuracy and to
use such systems, large and precise datasets are required for the machine
learning (ML) procedure. This can be especially difficult for extreme/wild
motions such as acrobatic movements or motions in specific sports, which are
difficult to estimate in typically provided training models. In addition,
training may take a long duration, and will require a high-grade GPU for
sufficient speed. To address these issues, we propose a method to improve the
pose estimation accuracy for extreme/wild motions by using pre-trained models,
i.e., without performing the training procedure by yourselves. We assume our
method to encourage usage of these DNN techniques for users in application
areas that are out of the ML field, and to help users without high-end
computers to apply them for personal and end use cases.
Sparse Feature Selection in Kernel Discriminant Analysis via Optimal Scoring
Authors: Alexander F. Lapanowski, Irina Gaynanova
Date: 2019-02-12 00:00:00
Tasks: Classification, feature selection, General Classification

We consider the two-group classification problem and propose a kernel
classifier based on the optimal scoring framework. Unlike previous approaches,
we provide theoretical guarantees on the expected risk consistency of the
method. We also allow for feature selection by imposing structured sparsity
using weighted kernels. We propose fully-automated methods for selection of all
tuning parameters, and in particular adapt kernel shrinkage ideas for ridge
parameter selection. Numerical studies demonstrate the superior classification
performance of the proposed approach compared to existing nonparametric
classifiers.
PAC-Bayes Analysis of Sentence Representation
Authors: Kento Nozawa, Issei Sato
Date: 2019-02-12 00:00:00
Tasks: BIG-bench Machine Learning, Sentence, Transfer Learning

Learning sentence vectors from an unlabeled corpus has attracted attention
because such vectors can represent sentences in a lower dimensional and
continuous space. Simple heuristics using pre-trained word vectors are widely
applied to machine learning tasks. However, they are not well understood from a
theoretical perspective. We analyze learning sentence vectors from a transfer
learning perspective by using a PAC-Bayes bound that enables us to understand
existing heuristics. We show that simple heuristics such as averaging and
inverse document frequency weighted averaging are derived by our formulation.
Moreover, we propose novel sentence vector learning algorithms on the basis of
our PAC-Bayes analysis.
VERIFAI: A Toolkit for the Design and Analysis of Artificial Intelligence-Based Systems
Authors: Tommaso Dreossi, Daniel J. Fremont, Shromona Ghosh, Edward Kim, Hadi Ravanbakhsh, Marcell Vazquez-Chanlatte, Sanjit A. Seshia
Date: 2019-02-12 00:00:00
Tasks: BIG-bench Machine Learning

We present VERIFAI, a software toolkit for the formal design and analysis of
systems that include artificial intelligence (AI) and machine learning (ML)
components. VERIFAI particularly seeks to address challenges with applying
formal methods to perception and ML components, including those based on neural
networks, and to model and analyze system behavior in the presence of
environment uncertainty. We describe the initial version of VERIFAI which
centers on simulation guided by formal models and specifications. Several use
cases are illustrated with examples, including temporal-logic falsification,
model-based systematic fuzz testing, parameter synthesis, counterexample
analysis, and data set augmentation.
Enhancement Mask for Hippocampus Detection and Segmentation
Authors: Dengsheng Chen, Wenxi Liu, You Huang, Tong Tong, Yuanlong Yu
Date: 2019-02-12 00:00:00
Tasks: Hippocampus, Segmentation

Detection and segmentation of the hippocampal structures in volumetric brain
images is a challenging problem in the area of medical imaging. In this paper,
we propose a two-stage 3D fully convolutional neural network that efficiently
detects and segments the hippocampal structures. In particular, our approach
first localizes the hippocampus from the whole volumetric image while obtaining
a proposal for a rough segmentation. After localization, we apply the proposal
as an enhancement mask to extract the fine structure of the hippocampus. The
proposed method has been evaluated on a public dataset and compares with
state-of-the-art approaches. Results indicate the effectiveness of the proposed
method, which yields mean Dice Similarity Coefficients (i.e. DSC) of $0.897$
and $0.900$ for the left and right hippocampus, respectively. Furthermore,
extensive experiments manifest that the proposed enhancement mask layer has
remarkable benefits for accelerating training process and obtaining more
accurate segmentation results.