pwc-archive/papers-with-abstracts huggingface dataset:
Using NLU in Context for Question Answering: Improving on Facebook's bAbI Tasks
Authors: John S. Ball
Date: 2017-09-13 00:00:00
Tasks: Natural Language Understanding, Question Answering, Text Generation

For the next step in human to machine interaction, Artificial Intelligence
(AI) should interact predominantly using natural language because, if it
worked, it would be the fastest way to communicate. Facebook's toy tasks (bAbI)
provide a useful benchmark to compare implementations for conversational AI.
While the published experiments so far have been based on exploiting the
distributional hypothesis with machine learning, our model exploits natural
language understanding (NLU) with the decomposition of language based on Role
and Reference Grammar (RRG) and the brain-based Patom theory. Our combinatorial
system for conversational AI based on linguistics has many advantages: passing
bAbI task tests without parsing or statistics while increasing scalability. Our
model validates both the training and test data to find 'garbage' input and
output (GIGO). It is not rules-based, nor does it use parts of speech, but
instead relies on meaning. While Deep Learning is difficult to debug and fix,
every step in our model can be understood and changed like any non-statistical
computer program. Deep Learning's lack of explicable reasoning has raised
opposition to AI, partly due to fear of the unknown. To support the goals of
AI, we propose extended tasks to use human-level statements with tense, aspect
and voice, and embedded clauses with junctures: and answers to be natural
language generation (NLG) instead of keywords. While machine learning permits
invalid training data to produce incorrect test responses, our system cannot
because the context tracking would need to be intentionally broken. We believe
no existing learning systems can currently solve these extended natural
language tests. There appears to be a knowledge gap between NLP researchers and
linguists, but ongoing competitive results such as these promise to narrow that
gap.
Variational Memory Addressing in Generative Models
Authors: Jörg Bornschein, andriy mnih, Daniel Zoran, Danilo J. Rezende
Date: 2017-09-21 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Few-Shot Learning, Representation Learning, Variational Inference

Aiming to augment generative models with external memory, we interpret the
output of a memory module with stochastic addressing as a conditional mixture
distribution, where a read operation corresponds to sampling a discrete memory
address and retrieving the corresponding content from memory. This perspective
allows us to apply variational inference to memory addressing, which enables
effective training of the memory module by using the target information to
guide memory lookups. Stochastic addressing is particularly well-suited for
generative models as it naturally encourages multimodality which is a prominent
aspect of most high-dimensional datasets. Treating the chosen address as a
latent variable also allows us to quantify the amount of information gained
with a memory lookup and measure the contribution of the memory module to the
generative process. To illustrate the advantages of this approach we
incorporate it into a variational autoencoder and apply the resulting model to
the task of generative few-shot learning. The intuition behind this
architecture is that the memory module can pick a relevant template from memory
and the continuous part of the model can concentrate on modeling remaining
variations. We demonstrate empirically that our model is able to identify and
access the relevant memory contents even with hundreds of unseen Omniglot
characters in memory
Cost Adaptation for Robust Decentralized Swarm Behaviour
Authors: Peter Henderson, Matthew Vertescher, David Meger, Mark Coates
Date: 2017-09-21 00:00:00
Tasks: Meta-Learning

Decentralized receding horizon control (D-RHC) provides a mechanism for
coordination in multi-agent settings without a centralized command center.
However, combining a set of different goals, costs, and constraints to form an
efficient optimization objective for D-RHC can be difficult. To allay this
problem, we use a meta-learning process -- cost adaptation -- which generates
the optimization objective for D-RHC to solve based on a set of human-generated
priors (cost and constraint functions) and an auxiliary heuristic. We use this
adaptive D-RHC method for control of mesh-networked swarm agents. This
formulation allows a wide range of tasks to be encoded and can account for
network delays, heterogeneous capabilities, and increasingly large swarms
through the adaptation mechanism. We leverage the Unity3D game engine to build
a simulator capable of introducing artificial networking failures and delays in
the swarm. Using the simulator we validate our method on an example coordinated
exploration task. We demonstrate that cost adaptation allows for more efficient
and safer task completion under varying environment conditions and increasingly
large swarm sizes. We release our simulator and code to the community for
future work.
Opposition based Ensemble Micro Differential Evolution
Authors: Hojjat Salehinejad, Shahryar Rahnamayan, Hamid. R. Tizhoosh
Date: 2017-09-08 00:00:00
Tasks: Benchmarking, Diversity

Differential evolution (DE) algorithm with a small population size is called
Micro-DE (MDE). A small population size decreases the computational complexity
but also reduces the exploration ability of DE by limiting the population
diversity. In this paper, we propose the idea of combining ensemble mutation
scheme selection and opposition-based learning concepts to enhance the
diversity of population in MDE at mutation and selection stages. The proposed
algorithm enhances the diversity of population by generating a random mutation
scale factor per individual and per dimension, randomly assigning a mutation
scheme to each individual in each generation, and diversifying individuals
selection using opposition-based learning. This approach is easy to implement
and does not require the setting of mutation scheme selection and mutation
scale factor. Experimental results are conducted for a variety of objective
functions with low and high dimensionality on the CEC Black- Box Optimization
Benchmarking 2015 (CEC-BBOB 2015). The results show superior performance of the
proposed algorithm compared to the other micro-DE algorithms.
Supervised Learning with Indefinite Topological Kernels
Authors: Tullia Padellini, Pierpaolo Brutti
Date: 2017-09-20 00:00:00
Tasks: General Classification, regression, Topological Data Analysis

Topological Data Analysis (TDA) is a recent and growing branch of statistics
devoted to the study of the shape of the data. In this work we investigate the
predictive power of TDA in the context of supervised learning. Since
topological summaries, most noticeably the Persistence Diagram, are typically
defined in complex spaces, we adopt a kernel approach to translate them into
more familiar vector spaces. We define a topological exponential kernel, we
characterize it, and we show that, despite not being positive semi-definite, it
can be successfully used in regression and classification tasks.
Persistence Flamelets: multiscale Persistent Homology for kernel density exploration
Authors: Tullia Padellini, Pierpaolo Brutti
Date: 2017-09-20 00:00:00
Tasks: Object, Topological Data Analysis

In recent years there has been noticeable interest in the study of the "shape
of data". Among the many ways a "shape" could be defined, topology is the most
general one, as it describes an object in terms of its connectivity structure:
connected components (topological features of dimension 0), cycles (features of
dimension 1) and so on. There is a growing number of techniques, generally
denoted as Topological Data Analysis, aimed at estimating topological
invariants of a fixed object; when we allow this object to change, however,
little has been done to investigate the evolution in its topology. In this work
we define the Persistence Flamelets, a multiscale version of one of the most
popular tool in TDA, the Persistence Landscape. We examine its theoretical
properties and we show how it could be used to gain insights on KDEs bandwidth
parameter.
Practical Machine Learning for Cloud Intrusion Detection: Challenges and the Way Forward
Authors: Ram Shankar Siva Kumar, Andrew Wicker, Matt Swann
Date: 2017-09-20 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Intrusion Detection

Operationalizing machine learning based security detections is extremely
challenging, especially in a continuously evolving cloud environment.
Conventional anomaly detection does not produce satisfactory results for
analysts that are investigating security incidents in the cloud. Model
evaluation alone presents its own set of problems due to a lack of benchmark
datasets. When deploying these detections, we must deal with model compliance,
localization, and data silo issues, among many others. We pose the problem of
"attack disruption" as a way forward in the security data science space. In
this paper, we describe the framework, challenges, and open questions
surrounding the successful operationalization of machine learning based
security detections in a cloud environment and provide some insights on how we
have addressed them.
Near Optimal Sketching of Low-Rank Tensor Regression
Authors: Jarvis Haupt, Xingguo Li, David P. Woodruff
Date: 2017-09-20 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Dimensionality Reduction, regression

We study the least squares regression problem \begin{align*} \min_{\Theta \in
\mathcal{S}_{\odot D,R}} \|A\Theta-b\|_2, \end{align*} where
$\mathcal{S}_{\odot D,R}$ is the set of $\Theta$ for which $\Theta =
\sum_{r=1}^{R} \theta_1^{(r)} \circ \cdots \circ \theta_D^{(r)}$ for vectors
$\theta_d^{(r)} \in \mathbb{R}^{p_d}$ for all $r \in [R]$ and $d \in [D]$, and
$\circ$ denotes the outer product of vectors. That is, $\Theta$ is a
low-dimensional, low-rank tensor. This is motivated by the fact that the number
of parameters in $\Theta$ is only $R \cdot \sum_{d=1}^D p_d$, which is
significantly smaller than the $\prod_{d=1}^{D} p_d$ number of parameters in
ordinary least squares regression. We consider the above CP decomposition model
of tensors $\Theta$, as well as the Tucker decomposition. For both models we
show how to apply data dimensionality reduction techniques based on {\it
sparse} random projections $\Phi \in \mathbb{R}^{m \times n}$, with $m \ll n$,
to reduce the problem to a much smaller problem $\min_{\Theta} \|\Phi A \Theta
- \Phi b\|_2$, for which if $\Theta'$ is a near-optimum to the smaller problem,
then it is also a near optimum to the original problem. We obtain significantly
smaller dimension and sparsity in $\Phi$ than is possible for ordinary least
squares regression, and we also provide a number of numerical simulations
supporting our theory.
On Compiling DNNFs without Determinism
Authors: Umut Oztok, Adnan Darwiche
Date: 2017-09-20 00:00:00

State-of-the-art knowledge compilers generate deterministic subsets of DNNF,
which have been recently shown to be exponentially less succinct than DNNF. In
this paper, we propose a new method to compile DNNFs without enforcing
determinism necessarily. Our approach is based on compiling deterministic DNNFs
with the addition of auxiliary variables to the input formula. These variables
are then existentially quantified from the deterministic structure in linear
time, which would lead to a DNNF that is equivalent to the input formula and
not necessarily deterministic. On the theoretical side, we show that the new
method could generate exponentially smaller DNNFs than deterministic ones, even
by adding a single auxiliary variable. Further, we show that various existing
techniques that introduce auxiliary variables to the input formulas can be
employed in our framework. On the practical side, we empirically demonstrate
that our new method can significantly advance DNNF compilation on certain
benchmarks.
Deep Voice 2: Multi-Speaker Neural Text-to-Speech
Authors: Sercan Arik, Gregory Diamos, Andrew Gibiansky, John Miller, Kainan Peng, Wei Ping, Jonathan Raiman, Yanqi Zhou
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Speech Synthesis, text-to-speech, Text to Speech

We introduce a technique for augmenting neural text-to-speech (TTS) with
lowdimensional trainable speaker embeddings to generate different voices from a
single model. As a starting point, we show improvements over the two
state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and
Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with
Deep Voice 1, but constructed with higher performance building blocks and
demonstrates a significant audio quality improvement over Deep Voice 1. We
improve Tacotron by introducing a post-processing neural vocoder, and
demonstrate a significant audio quality improvement. We then demonstrate our
technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron
on two multi-speaker TTS datasets. We show that a single neural TTS system can
learn hundreds of unique voices from less than half an hour of data per
speaker, while achieving high audio quality synthesis and preserving the
speaker identities almost perfectly.
On the Design of LQR Kernels for Efficient Controller Learning
Authors: Alonso Marco, Philipp Hennig, Stefan Schaal, Sebastian Trimpe
Date: 2017-09-20 00:00:00
Tasks: Bayesian Optimization

Finding optimal feedback controllers for nonlinear dynamic systems from data
is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful
framework for direct controller tuning from experimental trials. For selecting
the next query point and finding the global optimum, BO relies on a
probabilistic description of the latent objective function, typically a
Gaussian process (GP). As is shown herein, GPs with a common kernel choice can,
however, lead to poor learning outcomes on standard quadratic control problems.
For a first-order system, we construct two kernels that specifically leverage
the structure of the well-known Linear Quadratic Regulator (LQR), yet retain
the flexibility of Bayesian nonparametric learning. Simulations of uncertain
linear and nonlinear systems demonstrate that the LQR kernels yield superior
learning performance.
Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging
Authors: Youngjun Cho, Simon J. Julier, Nicolai Marquardt, Nadia Bianchi-Berthouze
Date: 2017-05-08 00:00:00
Tasks: Physiological Computing, Quantization, Thermal Infrared Object Tracking

The ability to monitor respiratory rate is extremely important for medical
treatment, healthcare and fitness sectors. In many situations, mobile methods,
which allow users to undertake every day activities, are required. However,
current monitoring systems can be obtrusive, requiring users to wear
respiration belts or nasal probes. Recent advances in thermographic systems
have shrunk their size, weight and cost, to the point where it is possible to
create smart-phone based respiration rate monitoring devices that are not
affected by lighting conditions. However, mobile thermal imaging is challenged
in scenes with high thermal dynamic ranges. This challenge is further amplified
by general problems such as motion artifacts and low spatial resolution,
leading to unreliable breathing signals. In this paper, we propose a novel and
robust approach for respiration tracking which compensates for the negative
effects of variations in the ambient temperature and motion artifacts and can
accurately extract breathing rates in highly dynamic thermal scenes. It has
three main contributions. The first is a novel Optimal Quantization technique
which adaptively constructs a color mapping of absolute temperature to improve
segmentation, classification and tracking. The second is the Thermal Gradient
Flow method that computes thermal gradient magnitude maps to enhance accuracy
of the nostril region tracking. Finally, we introduce the Thermal Voxel method
to increase the reliability of the captured respiration signals compared to the
traditional averaging method. We demonstrate the extreme robustness of our
system to track the nostril-region and measure the respiratory rate in high
dynamic range scenes.
A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
Authors: Giorgio Stampa, Marta Arias, David Sanchez-Charles, Victor Muntes-Mulero, Albert Cabellos
Date: 2017-09-20 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper we design and evaluate a Deep-Reinforcement Learning agent that
optimizes routing. Our agent adapts automatically to current traffic conditions
and proposes tailored configurations that attempt to minimize the network
delay. Experiments show very promising performance. Moreover, this approach
provides important operational advantages with respect to traditional
optimization algorithms.
Estimated Depth Map Helps Image Classification
Authors: Yihui He
Date: 2017-09-20 00:00:00
Tasks: Classification, Depth Estimation, General Classification, image-classification, Image Classification, Transfer Learning

We consider image classification with estimated depth. This problem falls
into the domain of transfer learning, since we are using a model trained on a
set of depth images to generate depth maps (additional features) for use in
another classification problem using another disjoint set of images. It's
challenging as no direct depth information is provided. Though depth estimation
has been well studied, none have attempted to aid image classification with
estimated depth. Therefore, we present a way of transferring domain knowledge
on depth estimation to a separate image classification task over a disjoint set
of train, and test data. We build a RGBD dataset based on RGB dataset and do
image classification on it. Then evaluation the performance of neural networks
on the RGBD dataset compared to the RGB dataset. From our experiments, the
benefit is significant with shallow and deep networks. It improves ResNet-20 by
0.55% and ResNet-56 by 0.53%. Our code and dataset are available publicly.
Multi-camera Multi-Object Tracking
Authors: Wenqian Liu, Octavia Camps, Mario Sznaier
Date: 2017-09-20 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking, Visual Tracking

In this paper, we propose a pipeline for multi-target visual tracking under
multi-camera system. For multi-camera system tracking problem, efficient data
association across cameras, and at the same time, across frames becomes more
important than single-camera system tracking. However, most of the multi-camera
tracking algorithms emphasis on single camera across frame data association.
Thus in our work, we model our tracking problem as a global graph, and adopt
Generalized Maximum Multi Clique optimization problem as our core algorithm to
take both across frame and across camera data correlation into account all
together. Furthermore, in order to compute good similarity scores as the input
of our graph model, we extract both appearance and dynamic motion similarities.
For appearance feature, Local Maximal Occurrence Representation(LOMO) feature
extraction algorithm for ReID is conducted. When it comes to capturing the
dynamic information, we build Hankel matrix for each tracklet of target and
apply rank estimation with Iterative Hankel Total Least Squares(IHTLS)
algorithm to it. We evaluate our tracker on the challenging Terrace Sequences
from EPFL CVLAB as well as recently published Duke MTMC dataset.
Inter-Subject Analysis: Inferring Sparse Interactions with Dense Intra-Graphs
Authors: Cong Ma, Junwei Lu, Han Liu
Date: 2017-09-20 00:00:00
Tasks: Functional Connectivity, valid

We develop a new modeling framework for Inter-Subject Analysis (ISA). The
goal of ISA is to explore the dependency structure between different subjects
with the intra-subject dependency as nuisance. It has important applications in
neuroscience to explore the functional connectivity between brain regions under
natural stimuli. Our framework is based on the Gaussian graphical models, under
which ISA can be converted to the problem of estimation and inference of the
inter-subject precision matrix. The main statistical challenge is that we do
not impose sparsity constraint on the whole precision matrix and we only assume
the inter-subject part is sparse. For estimation, we propose to estimate an
alternative parameter to get around the non-sparse issue and it can achieve
asymptotic consistency even if the intra-subject dependency is dense. For
inference, we propose an "untangle and chord" procedure to de-bias our
estimator. It is valid without the sparsity assumption on the inverse Hessian
of the log-likelihood function. This inferential method is general and can be
applied to many other statistical problems, thus it is of independent
theoretical interest. Numerical experiments on both simulated and brain imaging
data validate our methods and theory.
Do semantic parts emerge in Convolutional Neural Networks?
Authors: Abel Gonzalez-Garcia, Davide Modolo, Vittorio Ferrari
Date: 2016-07-13 00:00:00
Tasks: Object Recognition

Semantic object parts can be useful for several visual recognition tasks.
Lately, these tasks have been addressed using Convolutional Neural Networks
(CNN), achieving outstanding results. In this work we study whether CNNs learn
semantic parts in their internal representation. We investigate the responses
of convolutional filters and try to associate their stimuli with semantic
parts. We perform two extensive quantitative analyses. First, we use
ground-truth part bounding-boxes from the PASCAL-Part dataset to determine how
many of those semantic parts emerge in the CNN. We explore this emergence for
different layers, network depths, and supervision levels. Second, we collect
human judgements in order to study what fraction of all filters systematically
fire on any semantic part, even if not annotated in PASCAL-Part. Moreover, we
explore several connections between discriminative power and semantics. We find
out which are the most discriminative filters for object recognition, and
analyze whether they respond to semantic parts or to other image patches. We
also investigate the other direction: we determine which semantic parts are the
most discriminative and whether they correspond to those parts emerging in the
network. This enables to gain an even deeper understanding of the role of
semantic parts in the network.
Text Compression for Sentiment Analysis via Evolutionary Algorithms
Authors: Emmanuel Dufourq, Bruce A. Bassett
Date: 2017-09-20 00:00:00
Tasks: Data Compression, Evolutionary Algorithms, General Classification, Sentiment Analysis, Sentiment Classification, Text Compression

Can textual data be compressed intelligently without losing accuracy in
evaluating sentiment? In this study, we propose a novel evolutionary
compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression),
which makes use of Parts-of-Speech tags to compress text in a way that
sacrifices minimal classification accuracy when used in conjunction with
sentiment analysis algorithms. An analysis of PARSEC with eight commercial and
non-commercial sentiment analysis algorithms on twelve English sentiment data
sets reveals that accurate compression is possible with (0%, 1.3%, 3.3%) loss
in sentiment classification accuracy for (20%, 50%, 75%) data compression with
PARSEC using LingPipe, the most accurate of the sentiment algorithms. Other
sentiment analysis algorithms are more severely affected by compression. We
conclude that significant compression of text data is possible for sentiment
analysis depending on the accuracy demands of the specific application and the
specific sentiment analysis algorithm used.