pwc-archive/papers-with-abstracts huggingface dataset:
Learning Belief Network Structure From Data under Causal Insufficiency
Authors: Mieczysław Kłopotek
Date: 2017-05-29 00:00:00
Tasks: Open-Ended Question Answering

Though a belief network (a representation of the joint probability
distribution, see [3]) and a causal network (a representation of causal
relationships [14]) are intended to mean different things, they are closely
related. Both assume an underlying dag (directed acyclic graph) structure of
relations among variables and if Markov condition and faithfulness condition
[15] are met, then a causal network is in fact a belief network. The difference
comes to appearance when we recover belief network and causal network structure
from data.
  A causal network structure may be impossible to recover completely from data
as not all directions of causal links may be uniquely determined [15].
Fortunately, if we deal with causally sufficient sets of variables (that is
whenever significant influence variables are not omitted from observation),
then there exists the possibility to identify the family of belief networks a
causal network belongs to [16]. Regrettably, to our knowledge, a similar result
is not directly known for causally insufficient sets of variables. Spirtes,
Glymour and Scheines developed a CI algorithm to handle this situation, but it
leaves some important questions open.
  The big open question is whether or not the bidirectional edges (that is
indications of a common cause) are the only ones necessary to develop a belief
network out of the product of CI, or must there be some other hidden variables
added (e.g. by guessing). This paper is devoted to settling this question.
Consistent Kernel Density Estimation with Non-Vanishing Bandwidth
Authors: Efrén Cruz Cortés, Clayton Scott
Date: 2017-05-24 00:00:00
Tasks: Density Estimation

Consistency of the kernel density estimator requires that the kernel
bandwidth tends to zero as the sample size grows. In this paper we investigate
the question of whether consistency is possible when the bandwidth is fixed, if
we consider a more general class of weighted KDEs. To answer this question in
the affirmative, we introduce the fixed-bandwidth KDE (fbKDE), obtained by
solving a quadratic program, and prove that it consistently estimates any
continuous square-integrable density. We also establish rates of convergence
for the fbKDE with radial kernels and the box kernel under appropriate
smoothness assumptions. Furthermore, in an experimental study we demonstrate
that the fbKDE compares favorably to the standard KDE and the previously
proposed variable bandwidth KDE.
Feature Incay for Representation Regularization
Authors: Yuhui Yuan, Kuiyuan Yang, Chao Zhang
Date: 2017-05-29 00:00:00
Proceeding: ICLR 2018 1
Tasks: Multi-class Classification, Representation Learning

Softmax loss is widely used in deep neural networks for multi-class
classification, where each class is represented by a weight vector, a sample is
represented as a feature vector, and the feature vector has the largest
projection on the weight vector of the correct category when the model
correctly classifies a sample. To ensure generalization, weight decay that
shrinks the weight norm is often used as regularizer. Different from
traditional learning algorithms where features are fixed and only weights are
tunable, features are also tunable as representation learning in deep learning.
Thus, we propose feature incay to also regularize representation learning,
which favors feature vectors with large norm when the samples can be correctly
classified. With the feature incay, feature vectors are further pushed away
from the origin along the direction of their corresponding weight vectors,
which achieves better inter-class separability. In addition, the proposed
feature incay encourages intra-class compactness along the directions of weight
vectors by increasing the small feature norm faster than the large ones.
Empirical results on MNIST, CIFAR10 and CIFAR100 demonstrate feature incay can
improve the generalization ability.
Towards Visual Ego-motion Learning in Robots
Authors: Sudeep Pillai, John J. Leonard
Date: 2017-05-29 00:00:00
Tasks: Motion Estimation, Optical Flow Estimation, Self-Supervised Learning, Sensor Fusion, Visual Odometry

Many model-based Visual Odometry (VO) algorithms have been proposed in the
past decade, often restricted to the type of camera optics, or the underlying
motion manifold observed. We envision robots to be able to learn and perform
these tasks, in a minimally supervised setting, as they gain more experience.
To this end, we propose a fully trainable solution to visual ego-motion
estimation for varied camera optics. We propose a visual ego-motion learning
architecture that maps observed optical flow vectors to an ego-motion density
estimate via a Mixture Density Network (MDN). By modeling the architecture as a
Conditional Variational Autoencoder (C-VAE), our model is able to provide
introspective reasoning and prediction for ego-motion induced scene-flow.
Additionally, our proposed model is especially amenable to bootstrapped
ego-motion learning in robots where the supervision in ego-motion estimation
for a particular camera sensor can be obtained from standard navigation-based
sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through
experiments, we show the utility of our proposed approach in enabling the
concept of self-supervised learning for visual ego-motion estimation in
autonomous robots.
Who's to say what's funny? A computer using Language Models and Deep Learning, That's Who!
Authors: Xinru Yan, Ted Pedersen
Date: 2017-05-29 00:00:00
Tasks: Language Modeling, Language Modelling

Humor is a defining characteristic of human beings. Our goal is to develop
methods that automatically detect humorous statements and rank them on a
continuous scale. In this paper we report on results using a Language Model
approach, and outline our plans for using methods from Deep Learning.
Algorithms for stochastic optimization with functional or expectation constraints
Authors: Guanghui Lan, Zhiqiang Zhou
Date: 2016-04-13 00:00:00
Tasks: Stochastic Optimization

This paper considers the problem of minimizing an expectation function over a closed convex set, coupled with a {\color{black} functional or expectation} constraint on either decision variables or problem parameters. We first present a new stochastic approximation (SA) type algorithm, namely the cooperative SA (CSA), to handle problems with the constraint on devision variables. We show that this algorithm exhibits the optimal ${\cal O}(1/\epsilon^2)$ rate of convergence, in terms of both optimality gap and constraint violation, when the objective and constraint functions are generally convex, where $\epsilon$ denotes the optimality gap and infeasibility. Moreover, we show that this rate of convergence can be improved to ${\cal O}(1/\epsilon)$ if the objective and constraint functions are strongly convex. We then present a variant of CSA, namely the cooperative stochastic parameter approximation (CSPA) algorithm, to deal with the situation when the constraint is defined over problem parameters and show that it exhibits similar optimal rate of convergence to CSA. It is worth noting that CSA and CSPA are primal methods which do not require the iterations on the dual space and/or the estimation on the size of the dual variables. To the best of our knowledge, this is the first time that such optimal SA methods for solving functional or expectation constrained stochastic optimization are presented in the literature.
Character-Based Text Classification using Top Down Semantic Model for Sentence Representation
Authors: Zhenzhou Wu, Xin Zheng, Daniel Dahlmeier
Date: 2017-05-29 00:00:00
Tasks: Deep Learning, General Classification, Sentence, text-classification, Text Classification

Despite the success of deep learning on many fronts especially image and
speech, its application in text classification often is still not as good as a
simple linear SVM on n-gram TF-IDF representation especially for smaller
datasets. Deep learning tends to emphasize on sentence level semantics when
learning a representation with models like recurrent neural network or
recursive neural network, however from the success of TF-IDF representation, it
seems a bag-of-words type of representation has its strength. Taking advantage
of both representions, we present a model known as TDSM (Top Down Semantic
Model) for extracting a sentence representation that considers both the
word-level semantics by linearly combining the words with attention weights and
the sentence-level semantics with BiLSTM and use it on text classification. We
apply the model on characters and our results show that our model is better
than all the other character-based and word-based convolutional neural network
models by \cite{zhang15} across seven different datasets with only 1\% of their
parameters. We also demonstrate that this model beats traditional linear models
on TF-IDF vectors on small and polished datasets like news article in which
typically deep learning models surrender.
Robust Stochastic Configuration Networks with Kernel Density Estimation
Authors: Dianhui Wang, Ming Li
Date: 2017-02-15 00:00:00
Tasks: Density Estimation

Neural networks have been widely used as predictive models to fit data
distribution, and they could be implemented through learning a collection of
samples. In many applications, however, the given dataset may contain noisy
samples or outliers which may result in a poor learner model in terms of
generalization. This paper contributes to a development of robust stochastic
configuration networks (RSCNs) for resolving uncertain data regression
problems. RSCNs are built on original stochastic configuration networks with
weighted least squares method for evaluating the output weights, and the input
weights and biases are incrementally and randomly generated by satisfying with
a set of inequality constrains. The kernel density estimation (KDE) method is
employed to set the penalty weights for each training samples, so that some
negative impacts, caused by noisy data or outliers, on the resulting learner
model can be reduced. The alternating optimization technique is applied for
updating a RSCN model with improved penalty weights computed from the kernel
density estimation function. Performance evaluation is carried out by a
function approximation, four benchmark datasets and a case study on engineering
application. Comparisons to other robust randomised neural modelling
techniques, including the probabilistic robust learning algorithm for neural
networks with random weights and improved RVFL networks, indicate that the
proposed RSCNs with KDE perform favourably and demonstrate good potential for
real-world applications.
Deep Learning for Patient-Specific Kidney Graft Survival Analysis
Authors: Margaux Luck, Tristan Sylvain, Héloïse Cardinal, Andrea Lodi, Yoshua Bengio
Date: 2017-05-29 00:00:00
Tasks: Decision Making, Deep Learning, Multi-Task Learning, Survival Analysis

An accurate model of patient-specific kidney graft survival distributions can
help to improve shared-decision making in the treatment and care of patients.
In this paper, we propose a deep learning method that directly models the
survival function instead of estimating the hazard function to predict survival
times for graft patients based on the principle of multi-task learning. By
learning to jointly predict the time of the event, and its rank in the cox
partial log likelihood framework, our deep learning approach outperforms, in
terms of survival time prediction quality and concordance index, other common
methods for survival analysis, including the Cox Proportional Hazards model and
a network trained on the cox partial log-likelihood.
Differentially Private Bayesian Learning on Distributed Data
Authors: Mikko Heikkilä, Eemil Lagerspetz, Samuel Kaski, Kana Shimizu, Sasu Tarkoma, Antti Honkela
Date: 2017-03-03 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Bayesian Inference

Many applications of machine learning, for example in health care, would
benefit from methods that can guarantee privacy of data subjects. Differential
privacy (DP) has become established as a standard for protecting learning
results. The standard DP algorithms require a single trusted party to have
access to the entire data, which is a clear weakness. We consider DP Bayesian
learning in a distributed setting, where each party only holds a single sample
or a few samples of the data. We propose a learning strategy based on a secure
multi-party sum function for aggregating summaries from data holders and the
Gaussian mechanism for DP. Our method builds on an asymptotically optimal and
practically efficient DP Bayesian inference with rapidly diminishing extra
cost.
Latent Intention Dialogue Models
Authors: Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, Steve Young
Date: 2017-05-29 00:00:00
Proceeding: ICML 2017 8
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Variational Inference

Developing a dialogue agent that is capable of making autonomous decisions
and communicating by natural language is one of the long-term goals of machine
learning research. Traditional approaches either rely on hand-crafting a small
state-action set for applying reinforcement learning that is not scalable or
constructing deterministic models for learning dialogue sentences that fail to
capture natural conversational variability. In this paper, we propose a Latent
Intention Dialogue Model (LIDM) that employs a discrete latent variable to
learn underlying dialogue intentions in the framework of neural variational
inference. In a goal-oriented dialogue scenario, these latent intentions can be
interpreted as actions guiding the generation of machine responses, which can
be further refined autonomously by reinforcement learning. The experimental
evaluation of LIDM shows that the model out-performs published benchmarks for
both corpus-based and human evaluation, demonstrating the effectiveness of
discrete latent variable models for learning goal-oriented dialogues.
Optimizing Memory Efficiency for Convolution Kernels on Kepler GPUs
Authors: Xiaoming Chen, Jianxu Chen, Danny Z. Chen, Xiaobo Sharon Hu
Date: 2017-05-29 00:00:00

Convolution is a fundamental operation in many applications, such as computer
vision, natural language processing, image processing, etc. Recent successes of
convolutional neural networks in various deep learning applications put even
higher demand on fast convolution. The high computation throughput and memory
bandwidth of graphics processing units (GPUs) make GPUs a natural choice for
accelerating convolution operations. However, maximally exploiting the
available memory bandwidth of GPUs for convolution is a challenging task. This
paper introduces a general model to address the mismatch between the memory
bank width of GPUs and computation data width of threads. Based on this model,
we develop two convolution kernels, one for the general case and the other for
a special case with one input channel. By carefully optimizing memory access
patterns and computation patterns, we design a communication-optimized kernel
for the special case and a communication-reduced kernel for the general case.
Experimental data based on implementations on Kepler GPUs show that our kernels
achieve 5.16X and 35.5% average performance improvement over the latest cuDNN
library, for the special case and the general case, respectively.
Automatic White-Box Testing of First-Order Logic Ontologies
Authors: Javier Álvez, Montserrat Hermo, Paqui Lucio, German Rigau
Date: 2017-05-29 00:00:00

Formal ontologies are axiomatizations in a logic-based formalism. The
development of formal ontologies, and their important role in the Semantic Web
area, is generating considerable research on the use of automated reasoning
techniques and tools that help in ontology engineering. One of the main aims is
to refine and to improve axiomatizations for enabling automated reasoning tools
to efficiently infer reliable information. Defects in the axiomatization can
not only cause wrong inferences, but can also hinder the inference of expected
information, either by increasing the computational cost of, or even
preventing, the inference. In this paper, we introduce a novel, fully automatic
white-box testing framework for first-order logic ontologies. Our methodology
is based on the detection of inference-based redundancies in the given
axiomatization. The application of the proposed testing method is fully
automatic since a) the automated generation of tests is guided only by the
syntax of axioms and b) the evaluation of tests is performed by automated
theorem provers. Our proposal enables the detection of defects and serves to
certify the grade of suitability --for reasoning purposes-- of every axiom. We
formally define the set of tests that are generated from any axiom and prove
that every test is logically related to redundancies in the axiom from which
the test has been generated. We have implemented our method and used this
implementation to automatically detect several non-trivial defects that were
hidden in various first-order logic ontologies. Throughout the paper we provide
illustrative examples of these defects, explain how they were found, and how
each proof --given by an automated theorem-prover-- provides useful hints on
the nature of each defect. Additionally, by correcting all the detected
defects, we have obtained an improved version of one of the tested ontologies:
Adimen-SUMO.
Enhanced Experience Replay Generation for Efficient Reinforcement Learning
Authors: Vincent Huang, Tobias Ley, Martha Vlachou-Konchylaki, Wenfeng Hu
Date: 2017-05-23 00:00:00
Tasks: Deep Reinforcement Learning, Generative Adversarial Network, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Applying deep reinforcement learning (RL) on real systems suffers from slow
data sampling. We propose an enhanced generative adversarial network (EGAN) to
initialize an RL agent in order to achieve faster learning. The EGAN utilizes
the relation between states and actions to enhance the quality of data samples
generated by a GAN. Pre-training the agent with the EGAN shows a steeper
learning curve with a 20% improvement of training time in the beginning of
learning, compared to no pre-training, and an improvement compared to training
with GAN by about 5% with smaller variations. For real time systems with sparse
and slow data sampling the EGAN could be used to speed up the early phases of
the training process.
On Multilingual Training of Neural Dependency Parsers
Authors: Michał Zapotoczny, Paweł Rychlikowski, Jan Chorowski
Date: 2017-05-29 00:00:00

We show that a recently proposed neural dependency parser can be improved by
joint training on multiple languages from the same family. The parser is
implemented as a deep neural network whose only input is orthographic
representations of words. In order to successfully parse, the network has to
discover how linguistically relevant concepts can be inferred from word
spellings. We analyze the representations of characters and words that are
learned by the network to establish which properties of languages were
accounted for. In particular we show that the parser has approximately learned
to associate Latin characters with their Cyrillic counterparts and that it can
group Polish and Russian words that have a similar grammatical function.
Finally, we evaluate the parser on selected languages from the Universal
Dependencies dataset and show that it is competitive with other recently
proposed state-of-the art methods, while having a simple structure.
Fast learning rate of deep learning via a kernel perspective
Authors: Taiji Suzuki
Date: 2017-05-29 00:00:00
Tasks: Deep Learning

We develop a new theoretical framework to analyze the generalization error of
deep learning, and derive a new fast learning rate for two representative
algorithms: empirical risk minimization and Bayesian deep learning. The series
of theoretical analyses of deep learning has revealed its high expressive power
and universal approximation capability. Although these analyses are highly
nonparametric, existing generalization error analyses have been developed
mainly in a fixed dimensional parametric model. To compensate this gap, we
develop an infinite dimensional model that is based on an integral form as
performed in the analysis of the universal approximation capability. This
allows us to define a reproducing kernel Hilbert space corresponding to each
layer. Our point of view is to deal with the ordinary finite dimensional deep
neural network as a finite approximation of the infinite dimensional one. The
approximation error is evaluated by the degree of freedom of the reproducing
kernel Hilbert space in each layer. To estimate a good finite dimensional
model, we consider both of empirical risk minimization and Bayesian deep
learning. We derive its generalization error bound and it is shown that there
appears bias-variance trade-off in terms of the number of parameters of the
finite dimensional approximation. We show that the optimal width of the
internal layers can be determined through the degree of freedom and the
convergence rate can be faster than $O(1/\sqrt{n})$ rate which has been shown
in the existing studies.
Automatic Generation of Grounded Visual Questions
Authors: Shijie Zhang, Lizhen Qu, ShaoDi You, Zhenglu Yang, Jiawan Zhang
Date: 2016-12-20 00:00:00
Tasks: Diversity, Question Generation, Question-Generation

In this paper, we propose the first model to be able to generate visually
grounded questions with diverse types for a single image. Visual question
generation is an emerging topic which aims to ask questions in natural language
based on visual input. To the best of our knowledge, it lacks automatic methods
to generate meaningful questions with various types for the same visual input.
To circumvent the problem, we propose a model that automatically generates
visually grounded questions with varying types. Our model takes as input both
images and the captions generated by a dense caption model, samples the most
probable question types, and generates the questions in sequel. The
experimental results on two real world datasets show that our model outperforms
the strongest baseline in terms of both correctness and diversity with a wide
margin.
A Multi-Armed Bandit to Smartly Select a Training Set from Big Medical Data
Authors: Benjamín Gutiérrez, Loïc Peter, Tassilo Klein, Christian Wachinger
Date: 2017-05-23 00:00:00
Tasks: Thompson Sampling

With the availability of big medical image data, the selection of an adequate
training set is becoming more important to address the heterogeneity of
different datasets. Simply including all the data does not only incur high
processing costs but can even harm the prediction. We formulate the smart and
efficient selection of a training dataset from big medical image data as a
multi-armed bandit problem, solved by Thompson sampling. Our method assumes
that image features are not available at the time of the selection of the
samples, and therefore relies only on meta information associated with the
images. Our strategy simultaneously exploits data sources with high chances of
yielding useful samples and explores new data regions. For our evaluation, we
focus on the application of estimating the age from a brain MRI. Our results on
7,250 subjects from 10 datasets show that our approach leads to higher accuracy
while only requiring a fraction of the training data.
Tangent Cones to TT Varieties
Authors: Benjamin Kutschan
Date: 2017-05-29 00:00:00

As already done for the matrix case for example in [Joe Harris, Algebraic
Geometry - A first course, p.256] we give a parametrization of the Bouligand
tangent cone of the variety of tensors of bounded TT rank. We discuss how the
proof generalizes to any binary hierarchical format. The parametrization can be
rewritten as an orthogonal sum of TT tensors. Its retraction onto the variety
is particularly easy to compose. We also give an implicit description of the
tangent cone as the solution of a system of polynomial equations.