pwc-archive/papers-with-abstracts huggingface dataset:
Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization
Authors: Thanh Huy Nguyen, Umut Şimşekli, Gaël Richard
Date: 2019-01-22 00:00:00

Recent studies on diffusion-based sampling methods have shown that Langevin
Monte Carlo (LMC) algorithms can be beneficial for non-convex optimization, and
rigorous theoretical guarantees have been proven for both asymptotic and
finite-time regimes. Algorithmically, LMC-based algorithms resemble the
well-known gradient descent (GD) algorithm, where the GD recursion is perturbed
by an additive Gaussian noise whose variance has a particular form. Fractional
Langevin Monte Carlo (FLMC) is a recently proposed extension of LMC, where the
Gaussian noise is replaced by a heavy-tailed {\alpha}-stable noise. As opposed
to its Gaussian counterpart, these heavy-tailed perturbations can incur large
jumps and it has been empirically demonstrated that the choice of
{\alpha}-stable noise can provide several advantages in modern machine learning
problems, both in optimization and sampling contexts. However, as opposed to
LMC, only asymptotic convergence properties of FLMC have been yet established.
In this study, we analyze the non-asymptotic behavior of FLMC for non-convex
optimization and prove finite-time bounds for its expected suboptimality. Our
results show that the weak-error of FLMC increases faster than LMC, which
suggests using smaller step-sizes in FLMC. We finally extend our results to the
case where the exact gradients are replaced by stochastic gradients and show
that similar results hold in this setting as well.
An Exact Reformulation of Feature-Vector-based Radial-Basis-Function Networks for Graph-based Observations
Authors: Isaac J. Sledge, Jose C. Principe
Date: 2019-01-22 00:00:00

Radial-basis-function networks are traditionally defined for sets of vector-based observations. In this short paper, we reformulate such networks so that they can be applied to adjacency-matrix representations of weighted, directed graphs that represent the relationships between object pairs. We re-state the sum-of-squares objective function so that it is purely dependent on entries from the adjacency matrix. From this objective function, we derive a gradient descent update for the network weights. We also derive a gradient update that simulates the repositioning of the radial basis prototypes and changes in the radial basis prototype parameters. An important property of our radial basis function networks is that they are guaranteed to yield the same responses as conventional radial-basis networks trained on a corresponding vector realization of the relationships encoded by the adjacency-matrix. Such a vector realization only needs to provably exist for this property to hold, which occurs whenever the relationships correspond to distances from some arbitrary metric applied to a latent set of vectors. We therefore completely avoid needing to actually construct vectorial realizations via multi-dimensional scaling, which ensures that the underlying relationships are totally preserved.
Debugging Frame Semantic Role Labeling
Authors: Alexandre Kabbach
Date: 2019-01-22 00:00:00
Tasks: Semantic Parsing, Semantic Role Labeling

We propose a quantitative and qualitative analysis of the performances of
statistical models for frame semantic structure extraction. We report on a
replication study on FrameNet 1.7 data and show that preprocessing toolkits
play a major role in argument identification performances, observing gains
similar in their order of magnitude to those reported by recent models for
frame semantic parsing. We report on the robustness of a recent statistical
classifier for frame semantic parsing to lexical configurations of
predicate-argument structures, relying on an artificially augmented dataset
generated using a rule-based algorithm combining valence pattern matching and
lexical substitution. We prove that syntactic pre-processing plays a major role
in the performances of statistical classifiers to argument identification, and
discuss the core reasons of syntactic mismatch between dependency parsers
output and FrameNet syntactic formalism. Finally, we suggest new leads for
improving statistical models for frame semantic parsing, including joint
syntax-semantic parsing relying on FrameNet syntactic formalism, latent classes
inference via split-and-merge algorithms and neural network architectures
relying on rich input representations of words.
Pedestrian Attribute Recognition: A Survey
Authors: Xiao Wang, Shaofei Zheng, Rui Yang, Aihua Zheng, Zhe Chen, Jin Tang, Bin Luo
Date: 2019-01-22 00:00:00
Tasks: Attribute, Multi-Label Learning, Multi-Task Learning, Pedestrian Attribute Recognition, Survey

Recognizing pedestrian attributes is an important task in the computer vision community due to it plays an important role in video surveillance. Many algorithms have been proposed to handle this task. The goal of this paper is to review existing works using traditional methods or based on deep learning networks. Firstly, we introduce the background of pedestrian attribute recognition (PAR, for short), including the fundamental concepts of pedestrian attributes and corresponding challenges. Secondly, we introduce existing benchmarks, including popular datasets and evaluation criteria. Thirdly, we analyze the concept of multi-task learning and multi-label learning and also explain the relations between these two learning algorithms and pedestrian attribute recognition. We also review some popular network architectures which have been widely applied in the deep learning community. Fourthly, we analyze popular solutions for this task, such as attributes group, part-based, etc. Fifthly, we show some applications that take pedestrian attributes into consideration and achieve better performance. Finally, we summarize this paper and give several possible research directions for pedestrian attribute recognition. We continuously update the following GitHub to keep tracking the most cutting-edge related works on pedestrian attribute recognition~\url{https://github.com/wangxiao5791509/Pedestrian-Attribute-Recognition-Paper-List}
Use of First and Third Person Views for Deep Intersection Classification
Authors: Koji Takeda, Kanji Tanaka
Date: 2019-01-22 00:00:00
Tasks: General Classification

We explore the problem of intersection classification using monocular
on-board passive vision, with the goal of classifying traffic scenes with
respect to road topology. We divide the existing approaches into two broad
categories according to the type of input data: (a) first person vision (FPV)
approaches, which use an egocentric view sequence as the intersection is
passed; and (b) third person vision (TPV) approaches, which use a single view
immediately before entering the intersection. The FPV and TPV approaches each
have advantages and disadvantages. Therefore, we aim to combine them into a
unified deep learning framework. Experimental results show that the proposed
FPV-TPV scheme outperforms previous methods and only requires minimal FPV/TPV
measurements.
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
Authors: Bugra Can, Mert Gurbuzbalaban, Lingjiong Zhu
Date: 2019-01-22 00:00:00

Momentum methods such as Polyak's heavy ball (HB) method, Nesterov's accelerated gradient (AG) as well as accelerated projected gradient (APG) method have been commonly used in machine learning practice, but their performance is quite sensitive to noise in the gradients. We study these methods under a first-order stochastic oracle model where noisy estimates of the gradients are available. For strongly convex problems, we show that the distribution of the iterates of AG converges with the accelerated $O(\sqrt{\kappa}\log(1/\varepsilon))$ linear rate to a ball of radius $\varepsilon$ centered at a unique invariant distribution in the 1-Wasserstein metric where $\kappa$ is the condition number as long as the noise variance is smaller than an explicit upper bound we can provide. Our analysis also certifies linear convergence rates as a function of the stepsize, momentum parameter and the noise variance; recovering the accelerated rates in the noiseless case and quantifying the level of noise that can be tolerated to achieve a given performance. In the special case of strongly convex quadratic objectives, we can show accelerated linear rates in the $p$-Wasserstein metric for any $p\geq 1$ with improved sensitivity to noise for both AG and HB through a non-asymptotic analysis under some additional assumptions on the noise structure. Our analysis for HB and AG also leads to improved non-asymptotic convergence bounds in suboptimality for both deterministic and stochastic settings which is of independent interest. To the best of our knowledge, these are the first linear convergence results for stochastic momentum methods under the stochastic oracle model. We also extend our results to the APG method and weakly convex functions showing accelerated rates when the noise magnitude is sufficiently small.
PadChest: A large chest x-ray image dataset with multi-label annotated reports
Authors: Aurelia Bustos, Antonio Pertusa, Jose-Maria Salinas, Maria de la Iglesia-Vayá
Date: 2019-01-22 00:00:00

We present a labeled large-scale, high resolution chest x-ray dataset for the
automated exploration of medical images along with their associated reports.
This dataset includes more than 160,000 images obtained from 67,000 patients
that were interpreted and reported by radiologists at Hospital San Juan
Hospital (Spain) from 2009 to 2017, covering six different position views and
additional information on image acquisition and patient demography. The reports
were labeled with 174 different radiographic findings, 19 differential
diagnoses and 104 anatomic locations organized as a hierarchical taxonomy and
mapped onto standard Unified Medical Language System (UMLS) terminology. Of
these reports, 27% were manually annotated by trained physicians and the
remaining set was labeled using a supervised method based on a recurrent neural
network with attention mechanisms. The labels generated were then validated in
an independent test set achieving a 0.93 Micro-F1 score. To the best of our
knowledge, this is one of the largest public chest x-ray database suitable for
training supervised models concerning radiographs, and the first to contain
radiographic reports in Spanish. The PadChest dataset can be downloaded from
http://bimcv.cipf.es/bimcv-projects/padchest/.
Multiple Graph Adversarial Learning
Authors: Bo Jiang, Ziyan Zhang, Jin Tang, Bin Luo
Date: 2019-01-22 00:00:00

Recently, Graph Convolutional Networks (GCNs) have been widely studied for
graph-structured data representation and learning. However, in many real
applications, data are coming with multiple graphs, and it is non-trivial to
adapt GCNs to deal with data representation with multiple graph structures. One
main challenge for multi-graph representation is how to exploit both structure
information of each individual graph and correlation information across
multiple graphs simultaneously. In this paper, we propose a novel Multiple
Graph Adversarial Learning (MGAL) framework for multi-graph representation and
learning. MGAL aims to learn an optimal structure-invariant and consistent
representation for multiple graphs in a common subspace via a novel adversarial
learning framework, which thus incorporates both structure information of
intra-graph and correlation information of inter-graphs simultaneously. Based
on MGAL, we then provide a unified network for semi-supervised learning task.
Promising experimental results demonstrate the effectiveness of MGAL model.
Deep learning and sub-word-unit approach in written art generation
Authors: Krzysztof Wołk, Emilia Zawadzka-Gosk, Wojciech Czarnowski
Date: 2019-01-22 00:00:00
Tasks: Deep Learning

Automatic poetry generation is novel and interesting application of natural
language processing research. It became more popular during the last few years
due to the rapid development of technology and neural computing power. This
line of research can be applied to the study of linguistics and literature, for
social science experiments, or simply for entertainment. The most effective
known method of artificial poem generation uses recurrent neural networks
(RNN). We also used RNNs to generate poems in the style of Adam Mickiewicz. Our
network was trained on the Sir Thaddeus poem. For data pre-processing, we used
a specialized stemming tool, which is one of the major innovations and
contributions of this work. Our experiment was conducted on the source text,
divided into sub-word units (at a level of resolution close to syllables). This
approach is novel and is not often employed in the published literature. The
subwords units seem to be a natural choice for analysis of the Polish language,
as the language is morphologically rich due to cases, gender forms and a large
vocabulary. Moreover, Sir Thaddeus contains rhymes, so the analysis of
syllables can be meaningful. We verified our model with different settings for
the temperature parameter, which controls the randomness of the generated text.
We also compared our results with similar models trained on the same text but
divided into characters (which is the most common approach alongside the use of
full word units). The differences were tremendous. Our solution generated much
better poems that were able to follow the metre and vocabulary of the source
data text.
Simultaneous lesion and neuroanatomy segmentation in Multiple Sclerosis using deep neural networks
Authors: Richard McKinley, Rik Wepfer, Fabian Aschwanden, Lorenz Grunder, Raphaela Muri, Christian Rummel, Rajeev Verma, Christian Weisstanner, Mauricio Reyes, Anke Salmen, Andrew Chan, Franca Wagner, Roland Wiest
Date: 2019-01-22 00:00:00

Segmentation of white matter lesions and deep grey matter structures is an important task in the quantification of magnetic resonance imaging in multiple sclerosis. In this paper we explore segmentation solutions based on convolutional neural networks (CNNs) for providing fast, reliable segmentations of lesions and grey-matter structures in multi-modal MR imaging, and the performance of these methods when applied to out-of-centre data. We trained two state-of-the-art fully convolutional CNN architectures on the 2016 MSSEG training dataset, which was annotated by seven independent human raters: a reference implementation of a 3D Unet, and a more recently proposed 3D-to-2D architecture (DeepSCAN). We then retrained those methods on a larger dataset from a single centre, with and without labels for other brain structures. We quantified changes in performance owing to dataset shift, and changes in performance by adding the additional brain-structure labels. We also compared performance with freely available reference methods. Both fully-convolutional CNN methods substantially outperform other approaches in the literature when trained and evaluated in cross-validation on the MSSEG dataset, showing agreement with human raters in the range of human inter-rater variability. Both architectures showed drops in performance when trained on single-centre data and tested on the MSSEG dataset. When trained with the addition of weak anatomical labels derived from Freesurfer, the performance of the 3D Unet degraded, while the performance of the DeepSCAN net improved. Overall, the DeepSCAN network predicting both lesion and anatomical labels was the best-performing network examined.
CAMR: Coded Aggregated MapReduce
Authors: Konstantinos Konstantinidis, Aditya Ramamoorthy
Date: 2019-01-22 00:00:00

Many big data algorithms executed on MapReduce-like systems have a shuffle phase that often dominates the overall job execution time. Recent work has demonstrated schemes where the communication load in the shuffle phase can be traded off for the computation load in the map phase. In this work, we focus on a class of distributed algorithms, broadly used in deep learning, where intermediate computations of the same task can be combined. Even though prior techniques reduce the communication load significantly, they require a number of jobs that grows exponentially in the system parameters. This limitation is crucial and may diminish the load gains as the algorithm scales. We propose a new scheme which achieves the same load as the state-of-the-art while ensuring that the number of jobs as well as the number of subfiles that the data set needs to be split into remain small.
On Connected Sublevel Sets in Deep Learning
Authors: Quynh Nguyen
Date: 2019-01-22 00:00:00
Tasks: Deep Learning

This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large global valley.
Lower bounds for testing graphical models: colorings and antiferromagnetic Ising models
Authors: Ivona Bezakova, Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda
Date: 2019-01-22 00:00:00

We study the identity testing problem in the context of spin systems or undirected graphical models, where it takes the following form: given the parameter specification of the model $M$ and a sampling oracle for the distribution $\mu_{\hat{M}}$ of an unknown model $\hat{M}$, can we efficiently determine if the two models $M$ and $\hat{M}$ are the same? We consider identity testing for both soft-constraint and hard-constraint systems. In particular, we prove hardness results in two prototypical cases, the Ising model and proper colorings, and explore whether identity testing is any easier than structure learning. For the ferromagnetic (attractive) Ising model, Daskalakis et al. (2018) presented a polynomial time algorithm for identity testing. We prove hardness results in the antiferromagnetic (repulsive) setting in the same regime of parameters where structure learning is known to require a super-polynomial number of samples. In particular, for $n$-vertex graphs of maximum degree $d$, we prove that if $|\beta| d = \omega(\log{n})$ (where $\beta$ is the inverse temperature parameter), then there is no polynomial running time identity testing algorithm unless $RP=NP$. We also establish computational lower bounds for a broader set of parameters under the (randomized) exponential time hypothesis. Our proofs utilize insights into the design of gadgets using random graphs in recent works concerning the hardness of approximate counting by Sly (2010). In the hard-constraint setting, we present hardness results for identity testing for proper colorings. Our results are based on the presumed hardness of #BIS, the problem of (approximately) counting independent sets in bipartite graphs. In particular, we prove that identity testing is hard in the same range of parameters where structure learning is known to be hard.
Reducing state updates via Gaussian-gated LSTMs
Authors: Matthew Thornton, Jithendar Anumula, Shih-Chii Liu
Date: 2019-01-22 00:00:00

Recurrent neural networks can be difficult to train on long sequence data due
to the well-known vanishing gradient problem. Some architectures incorporate
methods to reduce RNN state updates, therefore allowing the network to preserve
memory over long temporal intervals. To address these problems of convergence,
this paper proposes a timing-gated LSTM RNN model, called the Gaussian-gated
LSTM (g-LSTM). The time gate controls when a neuron can be updated during
training, enabling longer memory persistence and better error-gradient flow.
This model captures long-temporal dependencies better than an LSTM and the time
gate parameters can be learned even from non-optimal initialization values.
Because the time gate limits the updates of the neuron state, the number of
computes needed for the network update is also reduced. By adding a
computational budget term to the training loss, we can obtain a network which
further reduces the number of computes by at least 10x. Finally, by employing a
temporal curriculum learning schedule for the g-LSTM, we can reduce the
convergence time of the equivalent LSTM network on long sequences.
The autofeat Python Library for Automated Feature Engineering and Selection
Authors: Franziska Horn, Robert Pack, Michael Rieger
Date: 2019-01-22 00:00:00
Tasks: Automated Feature Engineering, Feature Engineering, regression

This paper describes the autofeat Python library, which provides scikit-learn style linear regression and classification models with automated feature engineering and selection capabilities. Complex non-linear machine learning models, such as neural networks, are in practice often difficult to train and even harder to explain to non-statisticians, who require transparent analysis results as a basis for important business decisions. While linear models are efficient and intuitive, they generally provide lower prediction accuracies. Our library provides a multi-step feature engineering and selection process, where first a large pool of non-linear features is generated, from which then a small and robust set of meaningful features is selected, which improve the prediction accuracy of a linear model while retaining its interpretability.
Modal clustering asymptotics with applications to bandwidth selection
Authors: Alessandro Casa, José E. Chacón, Giovanna Menardi
Date: 2019-01-22 00:00:00
Tasks: Clustering, Density Estimation

Density-based clustering relies on the idea of linking groups to some
specific features of the probability distribution underlying the data. The
reference to a true, yet unknown, population structure allows to frame the
clustering problem in a standard inferential setting, where the concept of
ideal population clustering is defined as the partition induced by the true
density function. The nonparametric formulation of this approach, known as
modal clustering, draws a correspondence between the groups and the domains of
attraction of the density modes. Operationally, a nonparametric density
estimate is required and a proper selection of the amount of smoothing,
governing the shape of the density and hence possibly the modal structure, is
crucial to identify the final partition. In this work, we address the issue of
density estimation for modal clustering from an asymptotic perspective. A
natural and easy to interpret metric to measure the distance between
density-based partitions is discussed, its asymptotic approximation explored,
and employed to study the problem of bandwidth selection for nonparametric
modal clustering.
Online Estimation of Multiple Dynamic Graphs in Pattern Sequences
Authors: Jimmy Gaudreault, Arunabh Saxena, Hideaki Shimazaki
Date: 2019-01-22 00:00:00
Tasks: Time Series, Time Series Analysis

Sequences of correlated binary patterns can represent many time-series data
including text, movies, and biological signals. These patterns may be described
by weighted combinations of a few dominant structures that underpin specific
interactions among the binary elements. To extract the dominant correlation
structures and their contributions to generating data in a time-dependent
manner, we model the dynamics of binary patterns using the state-space model of
an Ising-type network that is composed of multiple undirected graphs. We
provide a sequential Bayes algorithm to estimate the dynamics of weights on the
graphs while gaining the graph structures online. This model can uncover
overlapping graphs underlying the data better than a traditional orthogonal
decomposition method, and outperforms an original time-dependent Ising model.
We assess the performance of the method by simulated data, and demonstrate that
spontaneous activity of cultured hippocampal neurons is represented by dynamics
of multiple graphs.