pwc-archive/papers-with-abstracts huggingface dataset:
Finite-Sample Analysis for SARSA with Linear Function Approximation
Authors: Shaofeng Zou, Tengyu Xu, Yingbin Liang
Date: 2019-02-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Q-Learning, Reinforcement Learning

SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.\ data, where a single sample trajectory is available. With a Lipschitz continuous policy improvement operator that is smooth enough, SARSA has been shown to converge asymptotically \cite{perkins2003convergent,melo2008analysis}. However, its non-asymptotic analysis is challenging and remains unsolved due to the non-i.i.d. samples and the fact that the behavior policy changes dynamically with time. In this paper, we develop a novel technique to explicitly characterize the stochastic bias of a type of stochastic approximation procedures with time-varying Markov transition kernels. Our approach enables non-asymptotic convergence analyses of this type of stochastic approximation algorithms, which may be of independent interest. Using our bias characterization technique and a gradient descent type of analysis, we provide the finite-sample analysis on the mean square error of the SARSA algorithm. We then further study a fitted SARSA algorithm, which includes the original SARSA algorithm and its variant in \cite{perkins2003convergent} as special cases. This fitted SARSA algorithm provides a more general framework for \textit{iterative} on-policy fitted policy iteration, which is more memory and computationally efficient. For this fitted SARSA algorithm, we also provide its finite-sample analysis.
Robust One-Class Kernel Spectral Regression
Authors: Shervin Rahimzadeh Arashloo, Josef Kittler
Date: 2019-02-06 00:00:00
Tasks: General Classification, One-Class Classification, regression

The kernel null-space technique and its regression-based formulation (called
one-class kernel spectral regression, a.k.a. OC-KSR) is known to be an
effective and computationally attractive one-class classification framework.
Despite its outstanding performance, the applicability of kernel null-space
method is limited due to its susceptibility to possible training data
corruptions and inability to rank training observations according to their
conformity with the model. This work addresses these shortcomings by studying
the effect of regularising the solution of the null-space kernel Fisher
methodology in the context of its regression-based formulation (OC-KSR). In
this respect, first, the effect of a Tikhonov regularisation in the Hilbert
space is analysed where the one-class learning problem in presence of
contaminations in the training set is posed as a sensitivity analysis problem.
Next, driven by the success of the sparse representation methodology, the
effect of a sparsity regularisation on the solution is studied. For both
alternative regularisation schemes, iterative algorithms are proposed which
recursively update label confidences and rank training observations based on
their fit with the model. Through extensive experiments conducted on different
data sets, the proposed methodology is found to enhance robustness against
contamination in the training set as compared with the baseline kernel
null-space technique as well as other existing approaches in a one-class
classification paradigm while providing the functionality to rank training
samples effectively.
Labelling Vertebrae with 2D Reformations of Multidetector CT Images: An Adversarial Approach for Incorporating Prior Knowledge of Spine Anatomy
Authors: Anjany Sekuboyina, Markus Rempfler, Alexander Valentinitsch, Bjoern H. Menze, Jan S. Kirschke
Date: 2019-02-06 00:00:00
Tasks: Anatomy, Benchmarking, Computed Tomography (CT)

Purpose: To use and test a labelling algorithm that operates on two-dimensional (2D) reformations, rather than three-dimensional (3D) data to locate and identify vertebrae. Methods: We improved the Btrfly Net (described by Sekuboyina et al) that works on sagittal and coronal maximum intensity projections (MIP) and augmented it with two additional components: spine-localization and adversarial a priori-learning. Furthermore, we explored two variants of adversarial training schemes that incorporated the anatomical a priori knowledge into the Btrfly Net. We investigated the superiority of the proposed approach for labelling vertebrae on three datasets: a public benchmarking dataset of 302 CT scans and two in-house datasets with a total of 238 CT scans. We employed Wilcoxon signed-rank test to compute the statistical significance of the improvement in performance observed due to various architectural components in our approach. Results: On the public dataset, our approach using the described Btrfly(pe-eb) network performed on par with current state-of-the-art methods achieving a statistically significant (p < .001) vertebrae identification rate of 88.5+/-0.2 % and localization distances of less than 7-mm. On the in-house datasets that had a higher inter-scan data variability, we obtained an identification rate of 85.1+/-1.2%. Conclusion: An identification performance comparable to existing 3D approaches was achieved when labelling vertebrae on 2D MIPs. The performance was further improved using the proposed adversarial training regime that effectively enforced local spine a priori knowledge during training. Lastly, spine-localization increased the generalizability of our approach by homogenizing the content in the MIPs.
Neural-Network Guided Expression Transformation
Authors: Romain Edelmann, Viktor Kunčak
Date: 2019-02-06 00:00:00

Optimizing compilers, as well as other translator systems, often work by
rewriting expressions according to equivalence preserving rules. Given an input
expression and its optimized form, finding the sequence of rules that were
applied is a non-trivial task. Most of the time, the tools provide no proof, of
any kind, of the equivalence between the original expression and its optimized
form. In this work, we propose to reconstruct proofs of equivalence of simple
mathematical expressions, after the fact, by finding paths of equivalence
preserving transformations between expressions. We propose to find those
sequences of transformations using a search algorithm, guided by a neural
network heuristic. Using a Tree-LSTM recursive neural network, we learn a
distributed representation of expressions where the Manhattan distance between
vectors approximately corresponds to the rewrite distance between expressions.
We then show how the neural network can be efficiently used to search for
transformation paths, leading to substantial gain in speed compared to an
uninformed exhaustive search. In one of our experiments, our neural-network
guided search algorithm is able to solve more instances with a 2 seconds
timeout per instance than breadth-first search does with a 5 minutes timeout
per instance.
Distilling Policy Distillation
Authors: Wojciech Marian Czarnecki, Razvan Pascanu, Simon Osindero, Siddhant M. Jayakumar, Grzegorz Swirszcz, Max Jaderberg
Date: 2019-02-06 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

The transfer of knowledge from one policy to another is an important tool in
Deep Reinforcement Learning. This process, referred to as distillation, has
been used to great success, for example, by enhancing the optimisation of
agents, leading to stronger performance faster, on harder domains [26, 32, 5,
8]. Despite the widespread use and conceptual simplicity of distillation, many
different formulations are used in practice, and the subtle variations between
them can often drastically change the performance and the resulting objective
that is being optimised. In this work, we rigorously explore the entire
landscape of policy distillation, comparing the motivations and strengths of
each variant through theoretical and empirical analysis. Our results point to
three distillation techniques, that are preferred depending on specifics of the
task. Specifically a newly proposed expected entropy regularised distillation
allows for quicker learning in a wide range of situations, while still
guaranteeing convergence.
Unstructured Multi-View Depth Estimation Using Mask-Based Multiplane Representation
Authors: Yuxin Hou, Arno Solin, Juho Kannala
Date: 2019-02-06 00:00:00
Tasks: Depth Estimation

This paper presents a novel method, MaskMVS, to solve depth estimation for
unstructured multi-view image-pose pairs. In the plane-sweep procedure, the
depth planes are sampled by histogram matching that ensures covering the depth
range of interest. Unlike other plane-sweep methods, we do not rely on a cost
metric to explicitly build the cost volume, but instead infer a multiplane mask
representation which regularizes the learning. Compared to many previous
approaches, we show that our method is lightweight and generalizes well without
requiring excessive training. We outperform the current state-of-the-art and
show results on the sun3d, scenes11, MVS, and RGBD test data sets.
Progressive Generative Adversarial Networks for Medical Image Super resolution
Authors: Dwarikanath Mahapatra, Behzad Bozorgtabar
Date: 2019-02-06 00:00:00
Tasks: Anatomy, Image Super-Resolution, Super-Resolution, Triplet

Anatomical landmark segmentation and pathology localization are important
steps in automated analysis of medical images. They are particularly
challenging when the anatomy or pathology is small, as in retinal images and
cardiac MRI, or when the image is of low quality due to device acquisition
parameters as in magnetic resonance (MR) scanners. We propose an image
super-resolution method using progressive generative adversarial networks
(P-GAN) that can take as input a low-resolution image and generate a high
resolution image of desired scaling factor. The super resolved images can be
used for more accurate detection of landmarks and pathology. Our primary
contribution is in proposing a multistage model where the output image quality
of one stage is progressively improved in the next stage by using a triplet
loss function. The triplet loss enables stepwise image quality improvement by
using the output of the previous stage as the baseline. This facilitates
generation of super resolved images of high scaling factor while maintaining
good image quality. Experimental results for image super-resolution show that
our proposed multistage P-GAN outperforms competing methods and baseline GAN.
The FA Quantifier Fuzzification Mechanism: analysis of convergence and efficient implementations
Authors: Félix Díaz-Hermida, Marcos Matabuena, Juan. C. Vidal
Date: 2019-02-06 00:00:00

The fuzzy quantification model FA has been identified as one of the best
behaved quantification models in several revisions of the field of fuzzy
quantification. This model is, to our knowledge, the unique one fulfilling the
strict Determiner Fuzzification Scheme axiomatic framework that does not induce
the standard min and max operators. The main contribution of this paper is the
proof of a convergence result that links this quantification model with the
Zadeh's model when the size of the input sets tends to infinite. The
convergence proof is, in any case, more general than the convergence to the
Zadeh's model, being applicable to any quantitative quantifier. In addition,
recent revisions papers have presented some doubts about the existence of
suitable computational implementations to evaluate the FA model in practical
applications. In order to prove that this model is not only a theoretical
approach, we show exact algorithmic solutions for the most common linguistic
quantifiers as well as an approximate implementation by means of Monte Carlo.
Additionally, we will also give a general overview of the main properties
fulfilled by the FA model, as a single compendium integrating the whole set of
properties fulfilled by it has not been previously published.
Transfer Learning From Sound Representations For Anger Detection in Speech
Authors: Mohamed Ezzeldin A. ElShaer, Scott Wisdom, Taniya Mishra
Date: 2019-02-06 00:00:00
Tasks: Transfer Learning

In this work, we train fully convolutional networks to detect anger in
speech. Since training these deep architectures requires large amounts of data
and the size of emotion datasets is relatively small, we use transfer learning.
However, unlike previous approaches that use speech or emotion-based tasks for
the source model, we instead use SoundNet, a fully convolutional neural network
trained multimodally on a massive video dataset to classify audio, with
ground-truth labels provided by vision-based classifiers. As a result of
transfer learning from SoundNet, our trained anger detection model improves
performance and generalizes well on a variety of acted, elicited, and natural
emotional speech datasets. We also test the cross-lingual effectiveness of our
model by evaluating our English-trained model on Mandarin Chinese speech
emotion data. Furthermore, our proposed system has low latency suitable for
real-time applications, only requiring 1.2 seconds of audio to make a reliable
classification.
Mol-CycleGAN - a generative model for molecular optimization
Authors: Łukasz Maziarka, Agnieszka Pocha, Jan Kaczmarczyk, Krzysztof Rataj, Michał Warchoł
Date: 2019-02-06 00:00:00
Proceeding: ICLR 2019 5

Designing a molecule with desired properties is one of the biggest challenges
in drug development, as it requires optimization of chemical compound
structures with respect to many complex properties. To augment the compound
design process we introduce Mol-CycleGAN - a CycleGAN-based model that
generates optimized compounds with high structural similarity to the original
ones. Namely, given a molecule our model generates a structurally similar one
with an optimized value of the considered property. We evaluate the performance
of the model on selected optimization objectives related to structural
properties (presence of halogen groups, number of aromatic rings) and to a
physicochemical property (penalized logP). In the task of optimization of
penalized logP of drug-like molecules our model significantly outperforms
previous results.
Latent Space Cartography: Generalised Metric-Inspired Measures and Measure-Based Transformations for Generative Models
Authors: Max F. Frenzel, Bogdan Teleaga, Asahi Ushio
Date: 2019-02-06 00:00:00

Deep generative models are universal tools for learning data distributions on
high dimensional data spaces via a mapping to lower dimensional latent spaces.
We provide a study of latent space geometries and extend and build upon
previous results on Riemannian metrics. We show how a class of heuristic
measures gives more flexibility in finding meaningful, problem-specific
distances, and how it can be applied to diverse generator types such as
autoregressive generators commonly used in e.g. language and other sequence
modeling. We further demonstrate how a diffusion-inspired transformation
previously studied in cartography can be used to smooth out latent spaces,
stretching them according to a chosen measure. In addition to providing more
meaningful distances directly in latent space, this also provides a unique tool
for novel kinds of data visualizations. We believe that the proposed methods
can be a valuable tool for studying the structure of latent spaces and learned
data distributions of generative models.
BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
Authors: Lars Maaløe, Marco Fraccaro, Valentin Liévin, Ole Winther
Date: 2019-02-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Anomaly Detection, Attribute, Image Generation

With the introduction of the variational autoencoder (VAE), probabilistic latent variable models have received renewed attention as powerful generative models. However, their performance in terms of test likelihood and quality of generated samples has been surpassed by autoregressive models without stochastic units. Furthermore, flow-based models have recently been shown to be an attractive alternative that scales well to high-dimensional data. In this paper we close the performance gap by constructing VAE models that can effectively utilize a deep hierarchy of stochastic variables and model complex covariance structures. We introduce the Bidirectional-Inference Variational Autoencoder (BIVA), characterized by a skip-connected generative model and an inference network formed by a bidirectional stochastic inference path. We show that BIVA reaches state-of-the-art test likelihoods, generates sharp and coherent natural images, and uses the hierarchy of latent variables to capture different aspects of the data distribution. We observe that BIVA, in contrast to recent results, can be used for anomaly detection. We attribute this to the hierarchy of latent variables which is able to extract high-level semantic features. Finally, we extend BIVA to semi-supervised classification tasks and show that it performs comparably to state-of-the-art results by generative adversarial networks.
Space Navigator: a Tool for the Optimization of Collision Avoidance Maneuvers
Authors: Leonid Gremyachikh, Dmitrii Dubov, Nikita Kazeev, Andrey Kulibaba, Andrey Skuratov, Anton Tereshkin, Andrey Ustyuzhanin, Lubov Shiryaeva, Sergej Shishkin
Date: 2019-02-06 00:00:00
Tasks: Collision Avoidance, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The number of space objects will grow several times in a few years due to the
planned launches of constellations of thousands microsatellites. It leads to a
significant increase in the threat of satellite collisions. Spacecraft must
undertake collision avoidance maneuvers to mitigate the risk. According to
publicly available information, conjunction events are now manually handled by
operators on the Earth. The manual maneuver planning requires qualified
personnel and will be impractical for constellations of thousands satellites.
In this paper we propose a new modular autonomous collision avoidance system
called "Space Navigator". It is based on a novel maneuver optimization approach
that combines domain knowledge with Reinforcement Learning methods.
GEN-SLAM: Generative Modeling for Monocular Simultaneous Localization and Mapping
Authors: Punarjay Chakravarty, Praveen Narayanan, Tom Roussel
Date: 2019-02-06 00:00:00
Tasks: Depth Estimation, Simultaneous Localization and Mapping

We present a Deep Learning based system for the twin tasks of localization
and obstacle avoidance essential to any mobile robot. Our system learns from
conventional geometric SLAM, and outputs, using a single camera, the
topological pose of the camera in an environment, and the depth map of
obstacles around it. We use a CNN to localize in a topological map, and a
conditional VAE to output depth for a camera image, conditional on this
topological location estimation. We demonstrate the effectiveness of our
monocular localization and depth estimation system on simulated and real
datasets.
Widely Linear Kernels for Complex-Valued Kernel Activation Functions
Authors: Simone Scardapane, Steven Van Vaerenbergh, Danilo Comminiello, Aurelio Uncini
Date: 2019-02-06 00:00:00
Tasks: image-classification, Image Classification

Complex-valued neural networks (CVNNs) have been shown to be powerful
nonlinear approximators when the input data can be properly modeled in the
complex domain. One of the major challenges in scaling up CVNNs in practice is
the design of complex activation functions. Recently, we proposed a novel
framework for learning these activation functions neuron-wise in a
data-dependent fashion, based on a cheap one-dimensional kernel expansion and
the idea of kernel activation functions (KAFs). In this paper we argue that,
despite its flexibility, this framework is still limited in the class of
functions that can be modeled in the complex domain. We leverage the idea of
widely linear complex kernels to extend the formulation, allowing for a richer
expressiveness without an increase in the number of adaptable parameters. We
test the resulting model on a set of complex-valued image classification
benchmarks. Experimental results show that the resulting CVNNs can achieve
higher accuracy while at the same time converging faster.
Word Embeddings for Entity-annotated Texts
Authors: Satya Almasian, Andreas Spitz, Michael Gertz
Date: 2019-02-06 00:00:00
Tasks: Clustering, Entity Embeddings, Information Retrieval, Retrieval, Word Embeddings, Word Similarity

Learned vector representations of words are useful tools for many information retrieval and natural language processing tasks due to their ability to capture lexical semantics. However, while many such tasks involve or even rely on named entities as central components, popular word embedding models have so far failed to include entities as first-class citizens. While it seems intuitive that annotating named entities in the training corpus should result in more intelligent word features for downstream tasks, performance issues arise when popular embedding approaches are naively applied to entity annotated corpora. Not only are the resulting entity embeddings less useful than expected, but one also finds that the performance of the non-entity word embeddings degrades in comparison to those trained on the raw, unannotated corpus. In this paper, we investigate approaches to jointly train word and entity embeddings on a large corpus with automatically annotated and linked entities. We discuss two distinct approaches to the generation of such embeddings, namely the training of state-of-the-art embeddings on raw-text and annotated versions of the corpus, as well as node embeddings of a co-occurrence graph representation of the annotated corpus. We compare the performance of annotated embeddings and classical word embeddings on a variety of word similarity, analogy, and clustering evaluation tasks, and investigate their performance in entity-specific tasks. Our findings show that it takes more than training popular word embedding models on an annotated corpus to create entity embeddings with acceptable performance on common test cases. Based on these results, we discuss how and when node embeddings of the co-occurrence graph representation of the text can restore the performance.
Common Mode Patterns for Supervised Tensor Subspace Learning
Authors: Konstantinos Makantasis, Anastasios Doulamis, Nikolaos Doulamis, Athanasios Voulodimos
Date: 2019-02-06 00:00:00
Tasks: Binary Classification, General Classification

In this work we propose a method for reducing the dimensionality of tensor
objects in a binary classification framework. The proposed Common Mode Patterns
method takes into consideration the labels' information, and ensures that
tensor objects that belong to different classes do not share common features
after the reduction of their dimensionality. We experimentally validate the
proposed supervised subspace learning technique and compared it against
Multilinear Principal Component Analysis using a publicly available
hyperspectral imaging dataset. Experimental results indicate that the proposed
CMP method can efficiently reduce the dimensionality of tensor objects, while,
at the same time, increasing the inter-class separability.
Knowledge-Based Regularization in Generative Modeling
Authors: Naoya Takeishi, Yoshinobu Kawahara
Date: 2019-02-06 00:00:00

Prior domain knowledge can greatly help to learn generative models. However, it is often too costly to hard-code prior knowledge as a specific model architecture, so we often have to use general-purpose models. In this paper, we propose a method to incorporate prior knowledge of feature relations into the learning of general-purpose generative models. To this end, we formulate a regularizer that makes the marginals of a generative model to follow prescribed relative dependence of features. It can be incorporated into off-the-shelf learning methods of many generative models, including variational autoencoders and generative adversarial networks, as its gradients can be computed using standard backpropagation techniques. We show the effectiveness of the proposed method with experiments on multiple types of datasets and generative models.