pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Acquire Information
Authors: Yewen Pu, Leslie P. Kaelbling, Armando Solar-Lezama
Date: 2017-04-20 00:00:00

We consider the problem of diagnosis where a set of simple observations are
used to infer a potentially complex hidden hypothesis. Finding the optimal
subset of observations is intractable in general, thus we focus on the problem
of active diagnosis, where the agent selects the next most-informative
observation based on the results of previous observations. We show that under
the assumption of uniform observation entropy, one can build an implication
model which directly predicts the outcome of the potential next observation
conditioned on the results of past observations, and selects the observation
with the maximum entropy. This approach enjoys reduced computation complexity
by bypassing the complicated hypothesis space, and can be trained on
observation data alone, learning how to query without knowledge of the hidden
hypothesis.
On the function approximation error for risk-sensitive reinforcement learning
Authors: Prasenjit Karmakar, Shalabh Bhatnagar
Date: 2016-12-22 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper we obtain several informative error bounds on function
approximation for the policy evaluation algorithm proposed by Basu et al. when
the aim is to find the risk-sensitive cost represented using exponential
utility. The main idea is to use classical Bapat's inequality and to use
Perron-Frobenius eigenvectors (exists if we assume irreducible Markov chain) to
get the new bounds. The novelty of our approach is that we use the
irreduciblity of Markov chain to get the new bounds whereas the earlier work by
Basu et al. used spectral variation bound which is true for any matrix. We also
give examples where all our bounds achieve the "actual error" whereas the
earlier bound given by Basu et al. is much weaker in comparison. We show that
this happens due to the absence of difference term in the earlier bound which
is always present in all our bounds when the state space is large.
Additionally, we discuss how all our bounds compare with each other.
Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks
Authors: Morten Kolbæk, Dong Yu, Zheng-Hua Tan, Jesper Jensen
Date: 2017-03-18 00:00:00
Tasks: Clustering, Deep Clustering, Speech Separation

In this paper we propose the utterance-level Permutation Invariant Training
(uPIT) technique. uPIT is a practically applicable, end-to-end, deep learning
based solution for speaker independent multi-talker speech separation.
Specifically, uPIT extends the recently proposed Permutation Invariant Training
(PIT) technique with an utterance-level cost function, hence eliminating the
need for solving an additional permutation problem during inference, which is
otherwise required by frame-level PIT. We achieve this using Recurrent Neural
Networks (RNNs) that, during training, minimize the utterance-level separation
error, hence forcing separated frames belonging to the same speaker to be
aligned to the same output stream. In practice, this allows RNNs, trained with
uPIT, to separate multi-talker mixed speech without any prior knowledge of
signal duration, number of speakers, speaker identity or gender. We evaluated
uPIT on the WSJ0 and Danish two- and three-talker mixed-speech separation tasks
and found that uPIT outperforms techniques based on Non-negative Matrix
Factorization (NMF) and Computational Auditory Scene Analysis (CASA), and
compares favorably with Deep Clustering (DPCL) and the Deep Attractor Network
(DANet). Furthermore, we found that models trained with uPIT generalize well to
unseen speakers and languages. Finally, we found that a single model, trained
with uPIT, can handle both two-speaker, and three-speaker speech mixtures.
Named Entity Recognition with stack residual LSTM and trainable bias decoding
Authors: Quan Tran, Andrew MacKinlay, Antonio Jimeno Yepes
Date: 2017-06-23 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Recurrent Neural Network models are the state-of-the-art for Named Entity
Recognition (NER). We present two innovations to improve the performance of
these models. The first innovation is the introduction of residual connections
between the Stacked Recurrent Neural Network model to address the degradation
problem of deep neural networks. The second innovation is a bias decoding
mechanism that allows the trained system to adapt to non-differentiable and
externally computed objectives, such as the entity-based F-measure. Our work
improves the state-of-the-art results for both Spanish and English languages on
the standard train/development/test split of the CoNLL 2003 Shared Task NER
dataset.
Deductive and Analogical Reasoning on a Semantically Embedded Knowledge Graph
Authors: Douglas Summers-Stay
Date: 2017-07-11 00:00:00

Representing knowledge as high-dimensional vectors in a continuous semantic
vector space can help overcome the brittleness and incompleteness of
traditional knowledge bases. We present a method for performing deductive
reasoning directly in such a vector space, combining analogy, association, and
deduction in a straightforward way at each step in a chain of reasoning,
drawing on knowledge from diverse sources and ontologies.
A non-projective greedy dependency parser with bidirectional LSTMs
Authors: David Vilares, Carlos Gómez-Rodríguez
Date: 2017-07-11 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, POS, POS Tagging

The LyS-FASTPARSE team presents BIST-COVINGTON, a neural implementation of
the Covington (2001) algorithm for non-projective dependency parsing. The
bidirectional LSTM approach by Kipperwasser and Goldberg (2016) is used to
train a greedy parser with a dynamic oracle to mitigate error propagation. The
model participated in the CoNLL 2017 UD Shared Task. In spite of not using any
ensemble methods and using the baseline segmentation and PoS tagging, the
parser obtained good results on both macro-average LAS and UAS in the big
treebanks category (55 languages), ranking 7th out of 33 teams. In the all
treebanks category (LAS and UAS) we ranked 16th and 12th. The gap between the
all and big categories is mainly due to the poor performance on four parallel
PUD treebanks, suggesting that some `suffixed' treebanks (e.g. Spanish-AnCora)
perform poorly on cross-treebank settings, which does not occur with the
corresponding `unsuffixed' treebank (e.g. Spanish). By changing that, we obtain
the 11th best LAS among all runs (official and unofficial). The code is made
available at https://github.com/CoNLL-UD-2017/LyS-FASTPARSE
Changepoint Detection in the Presence of Outliers
Authors: Paul Fearnhead, Guillem Rigaill
Date: 2016-09-23 00:00:00

Many traditional methods for identifying changepoints can struggle in the
presence of outliers, or when the noise is heavy-tailed. Often they will infer
additional changepoints in order to fit the outliers. To overcome this problem,
data often needs to be pre-processed to remove outliers, though this is
difficult for applications where the data needs to be analysed online. We
present an approach to changepoint detection that is robust to the presence of
outliers. The idea is to adapt existing penalised cost approaches for detecting
changes so that they use loss functions that are less sensitive to outliers. We
argue that loss functions that are bounded, such as the classical biweight
loss, are particularly suitable -- as we show that only bounded loss functions
are robust to arbitrarily extreme outliers. We present an efficient dynamic
programming algorithm that can find the optimal segmentation under our
penalised cost criteria. Importantly, this algorithm can be used in settings
where the data needs to be analysed online. We show that we can consistently
estimate the number of changepoints, and accurately estimate their locations,
using the biweight loss function. We demonstrate the usefulness of our approach
for applications such as analysing well-log data, detecting copy number
variation, and detecting tampering of wireless devices.
DeepTrend: A Deep Hierarchical Neural Network for Traffic Flow Prediction
Authors: Xingyuan Dai, Rui Fu, Yilun Lin, Li Li, Fei-Yue Wang
Date: 2017-07-11 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis

In this paper, we consider the temporal pattern in traffic flow time series,
and implement a deep learning model for traffic flow prediction. Detrending
based methods decompose original flow series into trend and residual series, in
which trend describes the fixed temporal pattern in traffic flow and residual
series is used for prediction. Inspired by the detrending method, we propose
DeepTrend, a deep hierarchical neural network used for traffic flow prediction
which considers and extracts the time-variant trend. DeepTrend has two stacked
layers: extraction layer and prediction layer. Extraction layer, a fully
connected layer, is used to extract the time-variant trend in traffic flow by
feeding the original flow series concatenated with corresponding simple average
trend series. Prediction layer, an LSTM layer, is used to make flow prediction
by feeding the obtained trend from the output of extraction layer and
calculated residual series. To make the model more effective, DeepTrend needs
first pre-trained layer-by-layer and then fine-tuned in the entire network.
Experiments show that DeepTrend can noticeably boost the prediction performance
compared with some traditional prediction models and LSTM with detrending based
methods.
Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices
Authors: Mehmet Süzen, Cornelius Weber, Joan J. Cerdà
Date: 2017-04-25 00:00:00
Tasks: Deep Learning

In this work a novel method to quantify spectral ergodicity for random
matrices is presented. The new methodology combines approaches rooted in the
metrics of Thirumalai-Mountain (TM) and Kullbach-Leibler (KL) divergence. The
method is applied to a general study of deep and recurrent neural networks via
the analysis of random matrix ensembles mimicking typical weight matrices of
those systems. In particular, we examine circular random matrix ensembles:
circular unitary ensemble (CUE), circular orthogonal ensemble (COE), and
circular symplectic ensemble (CSE). Eigenvalue spectra and spectral ergodicity
are computed for those ensembles as a function of network size. It is observed
that as the matrix size increases the level of spectral ergodicity of the
ensemble rises, i.e., the eigenvalue spectra obtained for a single realisation
at random from the ensemble is closer to the spectra obtained averaging over
the whole ensemble. Based on previous results we conjecture that success of
deep learning architectures is strongly bound to the concept of spectral
ergodicity. The method to compute spectral ergodicity proposed in this work
could be used to optimise the size and architecture of deep as well as
recurrent neural networks.
Adversarial training and dilated convolutions for brain MRI segmentation
Authors: Pim Moeskops, Mitko Veta, Maxime W. Lafarge, Koen A. J. Eppenhof, Josien P. W. Pluim
Date: 2017-07-11 00:00:00
Tasks: Image Segmentation, Medical Image Analysis, MRI segmentation, Segmentation, Semantic Segmentation

Convolutional neural networks (CNNs) have been applied to various automatic
image segmentation tasks in medical image analysis, including brain MRI
segmentation. Generative adversarial networks have recently gained popularity
because of their power in generating images that are difficult to distinguish
from real images.
  In this study we use an adversarial training approach to improve CNN-based
brain MRI segmentation. To this end, we include an additional loss function
that motivates the network to generate segmentations that are difficult to
distinguish from manual segmentations. During training, this loss function is
optimised together with the conventional average per-voxel cross entropy loss.
  The results show improved segmentation performance using this adversarial
training procedure for segmentation of two different sets of images and using
two different network architectures, both visually and in terms of Dice
coefficients.
Towards an automated method based on Iterated Local Search optimization for tuning the parameters of Support Vector Machines
Authors: Sergio Consoli, Jacek Kustra, Pieter Vos, Monique Hendriks, Dimitrios Mavroeidis
Date: 2017-07-11 00:00:00

We provide preliminary details and formulation of an optimization strategy
under current development that is able to automatically tune the parameters of
a Support Vector Machine over new datasets. The optimization strategy is a
heuristic based on Iterated Local Search, a modification of classic hill
climbing which iterates calls to a local search routine.
Accelerated Variance Reduced Stochastic ADMM
Authors: Yuanyuan Liu, Fanhua Shang, James Cheng
Date: 2017-07-11 00:00:00

Recently, many variance reduced stochastic alternating direction method of
multipliers (ADMM) methods (e.g.\ SAG-ADMM, SDCA-ADMM and SVRG-ADMM) have made
exciting progress such as linear convergence rates for strongly convex
problems. However, the best known convergence rate for general convex problems
is O(1/T) as opposed to O(1/T^2) of accelerated batch algorithms, where $T$ is
the number of iterations. Thus, there still remains a gap in convergence rates
between existing stochastic ADMM and batch algorithms. To bridge this gap, we
introduce the momentum acceleration trick for batch optimization into the
stochastic variance reduced gradient based ADMM (SVRG-ADMM), which leads to an
accelerated (ASVRG-ADMM) method. Then we design two different momentum term
update rules for strongly convex and general convex cases. We prove that
ASVRG-ADMM converges linearly for strongly convex problems. Besides having a
low per-iteration complexity as existing stochastic ADMM methods, ASVRG-ADMM
improves the convergence rate on general convex problems from O(1/T) to
O(1/T^2). Our experimental results show the effectiveness of ASVRG-ADMM.
A Survey on Resilient Machine Learning
Authors: Atul Kumar, Sameep Mehta
Date: 2017-07-11 00:00:00
Tasks: BIG-bench Machine Learning, Blocking, Computer Security, Survey

Machine learning based system are increasingly being used for sensitive tasks
such as security surveillance, guiding autonomous vehicle, taking investment
decisions, detecting and blocking network intrusion and malware etc. However,
recent research has shown that machine learning models are venerable to attacks
by adversaries at all phases of machine learning (eg, training data collection,
training, operation). All model classes of machine learning systems can be
misled by providing carefully crafted inputs making them wrongly classify
inputs. Maliciously created input samples can affect the learning process of a
ML system by either slowing down the learning process, or affecting the
performance of the learned mode, or causing the system make error(s) only in
attacker's planned scenario. Because of these developments, understanding
security of machine learning algorithms and systems is emerging as an important
research area among computer security and machine learning researchers and
practitioners. We present a survey of this emerging area in machine learning.
Dataset for a Neural Natural Language Interface for Databases (NNLIDB)
Authors: Florin Brad, Radu Iacob, Ionel Hosu, Traian Rebedea
Date: 2017-07-11 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Translation

Progress in natural language interfaces to databases (NLIDB) has been slow
mainly due to linguistic issues (such as language ambiguity) and domain
portability. Moreover, the lack of a large corpus to be used as a standard
benchmark has made data-driven approaches difficult to develop and compare. In
this paper, we revisit the problem of NLIDBs and recast it as a sequence
translation problem. To this end, we introduce a large dataset extracted from
the Stack Exchange Data Explorer website, which can be used for training neural
natural language interfaces for databases. We also report encouraging baseline
results on a smaller manually annotated test corpus, obtained using an
attention-based sequence-to-sequence neural network.
RegNet: Multimodal Sensor Registration Using Deep Neural Networks
Authors: Nick Schneider, Florian Piewak, Christoph Stiller, Uwe Franke
Date: 2017-07-11 00:00:00
Tasks: Translation

In this paper, we present RegNet, the first deep convolutional neural network
(CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between
multimodal sensors, exemplified using a scanning LiDAR and a monocular camera.
Compared to existing approaches, RegNet casts all three conventional
calibration steps (feature extraction, feature matching and global regression)
into a single real-time capable CNN. Our method does not require any human
interaction and bridges the gap between classical offline and target-less
online calibration approaches as it provides both a stable initial estimation
as well as a continuous online correction of the extrinsic parameters. During
training we randomly decalibrate our system in order to train RegNet to infer
the correspondence between projected depth measurements and RGB image and
finally regress the extrinsic calibration. Additionally, with an iterative
execution of multiple CNNs, that are trained on different magnitudes of
decalibration, our approach compares favorably to state-of-the-art methods in
terms of a mean calibration error of 0.28 degrees for the rotational and 6 cm
for the translation components even for large decalibrations up to 1.5 m and 20
degrees.
Foreground Detection in Camouflaged Scenes
Authors: Shuai Li, Dinei Florencio, Yaqin Zhao, Chris Cook, Wanqing Li
Date: 2017-07-11 00:00:00

Foreground detection has been widely studied for decades due to its
importance in many practical applications. Most of the existing methods assume
foreground and background show visually distinct characteristics and thus the
foreground can be detected once a good background model is obtained. However,
there are many situations where this is not the case. Of particular interest in
video surveillance is the camouflage case. For example, an active attacker
camouflages by intentionally wearing clothes that are visually similar to the
background. In such cases, even given a decent background model, it is not
trivial to detect foreground objects. This paper proposes a texture guided
weighted voting (TGWV) method which can efficiently detect foreground objects
in camouflaged scenes. The proposed method employs the stationary wavelet
transform to decompose the image into frequency bands. We show that the small
and hardly noticeable differences between foreground and background in the
image domain can be effectively captured in certain wavelet frequency bands. To
make the final foreground decision, a weighted voting scheme is developed based
on intensity and texture of all the wavelet bands with weights carefully
designed. Experimental results demonstrate that the proposed method achieves
superior performance compared to the current state-of-the-art results.
Efficient mixture model for clustering of sparse high dimensional binary data
Authors: Marek Śmieja, Krzysztof Hajto, Jacek Tabor
Date: 2017-07-11 00:00:00
Tasks: Clustering, Vocal Bursts Intensity Prediction

In this paper we propose a mixture model, SparseMix, for clustering of sparse
high dimensional binary data, which connects model-based with centroid-based
clustering. Every group is described by a representative and a probability
distribution modeling dispersion from this representative. In contrast to
classical mixture models based on EM algorithm, SparseMix:
  -is especially designed for the processing of sparse data,
  -can be efficiently realized by an on-line Hartigan optimization algorithm,
  -is able to automatically reduce unnecessary clusters.
  We perform extensive experimental studies on various types of data, which
confirm that SparseMix builds partitions with higher compatibility with
reference grouping than related methods. Moreover, constructed representatives
often better reveal the internal structure of data.
Least Square Variational Bayesian Autoencoder with Regularization
Authors: Gautam Ramachandra
Date: 2017-07-11 00:00:00
Proceeding: arXiv 2017 7

In recent years Variation Autoencoders have become one of the most popular
unsupervised learning of complicated distributions.Variational Autoencoder
(VAE) provides more efficient reconstructive performance over a traditional
autoencoder. Variational auto enocders make better approximaiton than MCMC. The
VAE defines a generative process in terms of ancestral sampling through a
cascade of hidden stochastic layers. They are a directed graphic models.
Variational autoencoder is trained to maximise the variational lower bound.
Here we are trying maximise the likelihood and also at the same time we are
trying to make a good approximation of the data. Its basically trading of the
data log-likelihood and the KL divergence from the true posterior. This paper
describes the scenario in which we wish to find a point-estimate to the
parameters $\theta$ of some parametric model in which we generate each
observations by first sampling a local latent variable and then sampling the
associated observation. Here we use least square loss function with
regularization in the the reconstruction of the image, the least square loss
function was found to give better reconstructed images and had a faster
training time.
Impulsive noise removal from color images with morphological filtering
Authors: Alexey Ruchay, Vitaly Kober
Date: 2017-07-11 00:00:00
Tasks: Color Image Denoising, Denoising, Image Denoising, Image Restoration

This paper deals with impulse noise removal from color images. The proposed
noise removal algorithm employs a novel approach with morphological filtering
for color image denoising; that is, detection of corrupted pixels and removal
of the detected noise by means of morphological filtering. With the help of
computer simulation we show that the proposed algorithm can effectively remove
impulse noise. The performance of the proposed algorithm is compared in terms
of image restoration metrics and processing speed with that of common
successful algorithms.
Distance-to-Mean Continuous Conditional Random Fields to Enhance Prediction Problem in Traffic Flow Data
Authors: Sumarsih Condroayu Purbarani, Hadaiq Rolis Sanabila, Wisnu Jatmiko
Date: 2017-07-11 00:00:00
Tasks: Time Series, Time Series Analysis

The increase of vehicle in highways may cause traffic congestion as well as
in the normal roadways. Predicting the traffic flow in highways especially, is
demanded to solve this congestion problem. Predictions on time-series
multivariate data, such as in the traffic flow dataset, have been largely
accomplished through various approaches. The approach with conventional
prediction algorithms, such as with Support Vector Machine (SVM), is only
capable of accommodating predictions that are independent in each time unit.
Hence, the sequential relationships in this time series data is hardly
explored. Continuous Conditional Random Field (CCRF) is one of Probabilistic
Graphical Model (PGM) algorithms which can accommodate this problem. The
neighboring aspects of sequential data such as in the time series data can be
expressed by CCRF so that its predictions are more reliable. In this article, a
novel approach called DM-CCRF is adopted by modifying the CCRF prediction
algorithm to strengthen the probability of the predictions made by the baseline
regressor. The result shows that DM-CCRF is superior in performance compared to
CCRF. This is validated by the error decrease of the baseline up to 9%
significance. This is twice the standard CCRF performance which can only
decrease baseline error by 4.582% at most.