pwc-archive/papers-with-abstracts huggingface dataset:
Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form
Authors: Maxim Naumov
Date: 2017-09-16 00:00:00
Tasks: Form

In this paper we focus on the linear algebra theory behind feedforward (FNN)
and recurrent (RNN) neural networks. We review backward propagation, including
backward propagation through time (BPTT). Also, we obtain a new exact
expression for Hessian, which represents second order effects. We show that for
$t$ time steps the weight gradient can be expressed as a rank-$t$ matrix, while
the weight Hessian is as a sum of $t^{2}$ Kronecker products of rank-$1$ and
$W^{T}AW$ matrices, for some matrix $A$ and weight matrix $W$. Also, we show
that for a mini-batch of size $r$, the weight update can be expressed as a
rank-$rt$ matrix. Finally, we briefly comment on the eigenvalues of the Hessian
matrix.
Multivariate Gaussian Network Structure Learning
Authors: Xingqi Du, Subhashis Ghosal
Date: 2017-09-16 00:00:00

We consider a graphical model where a multivariate normal vector is
associated with each node of the underlying graph and estimate the graphical
structure. We minimize a loss function obtained by regressing the vector at
each node on those at the remaining ones under a group penalty. We show that
the proposed estimator can be computed by a fast convex optimization algorithm.
We show that as the sample size increases, the estimated regression
coefficients and the correct graphical structure are correctly estimated with
probability tending to one. By extensive simulations, we show the superiority
of the proposed method over comparable procedures. We apply the technique on
two real datasets. The first one is to identify gene and protein networks
showing up in cancer cell lines, and the second one is to reveal the
connections among different industries in the US.
Forecasting of commercial sales with large scale Gaussian Processes
Authors: Rodrigo Rivera, Evgeny Burnaev
Date: 2017-09-16 00:00:00
Tasks: Decision Making, Gaussian Processes, Management

This paper argues that there has not been enough discussion in the field of
applications of Gaussian Process for the fast moving consumer goods industry.
Yet, this technique can be important as it e.g., can provide automatic feature
relevance determination and the posterior mean can unlock insights on the data.
Significant challenges are the large size and high dimensionality of commercial
data at a point of sale. The study reviews approaches in the Gaussian Processes
modeling for large data sets, evaluates their performance on commercial sales
and shows value of this type of models as a decision-making tool for
management.
Latent Gaussian Process Regression
Authors: Erik Bodin, Neill D. F. Campbell, Carl Henrik Ek
Date: 2017-07-18 00:00:00
Tasks: regression

We introduce Latent Gaussian Process Regression which is a latent variable
extension allowing modelling of non-stationary multi-modal processes using GPs.
The approach is built on extending the input space of a regression problem with
a latent variable that is used to modulate the covariance function over the
training data. We show how our approach can be used to model multi-modal and
non-stationary processes. We exemplify the approach on a set of synthetic data
and provide results on real data from motion capture and geostatistics.
DeepLung: 3D Deep Convolutional Nets for Automated Pulmonary Nodule Detection and Classification
Authors: Wentao Zhu, Chaochun Liu, Wei Fan, Xiaohui Xie
Date: 2017-09-16 00:00:00
Tasks: Automated Pulmonary Nodule Detection And Classification, Classification, Decoder, General Classification

In this work, we present a fully automated lung CT cancer diagnosis system,
DeepLung. DeepLung contains two parts, nodule detection and classification.
Considering the 3D nature of lung CT data, two 3D networks are designed for the
nodule detection and classification respectively. Specifically, a 3D Faster
R-CNN is designed for nodule detection with a U-net-like encoder-decoder
structure to effectively learn nodule features. For nodule classification,
gradient boosting machine (GBM) with 3D dual path network (DPN) features is
proposed. The nodule classification subnetwork is validated on a public dataset
from LIDC-IDRI, on which it achieves better performance than state-of-the-art
approaches, and surpasses the average performance of four experienced doctors.
For the DeepLung system, candidate nodules are detected first by the nodule
detection subnetwork, and nodule diagnosis is conducted by the classification
subnetwork. Extensive experimental results demonstrate the DeepLung is
comparable to the experienced doctors both for the nodule-level and
patient-level diagnosis on the LIDC-IDRI dataset.
A Comparison of Public Causal Search Packages on Linear, Gaussian Data with No Latent Variables
Authors: Joseph D. Ramsey, Bryan Andrews
Date: 2017-09-13 00:00:00

We compare Tetrad (Java) algorithms to the other public software packages BNT
(Bayes Net Toolbox, Matlab), pcalg (R), bnlearn (R) on the \vanilla" task of
recovering DAG structure to the extent possible from data generated recursively
from linear, Gaussian structure equation models (SEMs) with no latent
variables, for random graphs, with no additional knowledge of variable order or
adjacency structure, and without additional specification of intervention
information. Each one of the above packages offers at least one implementation
suitable to this purpose. We compare them on adjacency and orientation accuracy
as well as time performance, for fixed datasets. We vary the number of
variables, the number of samples, and the density of graph, for a total of 27
combinations, averaging all statistics over 10 runs, for a total of 270
datasets. All runs are carried out on the same machine and on their native
platforms. An interactive visualization tool is provided for the reader who
wishes to know more than can be documented explicitly in this report.
AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline
Authors: Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, Hao Zheng
Date: 2017-09-16 00:00:00
Tasks: speech-recognition, Speech Recognition

An open-source Mandarin speech corpus called AISHELL-1 is released. It is by
far the largest corpus which is suitable for conducting the speech recognition
research and building speech recognition systems for Mandarin. The recording
procedure, including audio capturing devices and environments are presented in
details. The preparation of the related resources, including transcriptions and
lexicon are described. The corpus is released with a Kaldi recipe. Experimental
results implies that the quality of audio recordings and transcriptions are
promising.
Approximating persistent homology for a cloud of $n$ points in a subquadratic time
Authors: Vitaliy Kurlin
Date: 2013-12-05 00:00:00
Tasks: Topological Data Analysis

The Vietoris-Rips filtration for an $n$-point metric space is a sequence of
large simplicial complexes adding a topological structure to the otherwise
disconnected space. The persistent homology is a key tool in topological data
analysis and studies topological features of data that persist over many
scales. The fastest algorithm for computing persistent homology of a filtration
has time $O(M(u)+u^2\log^2 u)$, where $u$ is the number of updates (additions
or deletions of simplices), $M(u)=O(u^{2.376})$ is the time for multiplication
of $u\times u$ matrices. For a space of $n$ points given by their pairwise
distances, we approximate the Vietoris-Rips filtration by a zigzag filtration
consisting of $u=o(n)$ updates, which is sublinear in $n$. The constant depends
on a given error of approximation and on the doubling dimension of the metric
space. Then the persistent homology of this sublinear-size filtration can be
computed in time $o(n^2)$, which is subquadratic in $n$.
Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
Authors: Hiroyuki Kasai, Hiroyuki Sato, Bamdev Mishra
Date: 2017-03-15 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

Stochastic variance reduction algorithms have recently become popular for
minimizing the average of a large, but finite number of loss functions. The
present paper proposes a Riemannian stochastic quasi-Newton algorithm with
variance reduction (R-SQN-VR). The key challenges of averaging, adding, and
subtracting multiple gradients are addressed with notions of retraction and
vector transport. We present convergence analyses of R-SQN-VR on both
non-convex and retraction-convex functions under retraction and vector
transport operators. The proposed algorithm is evaluated on the Karcher mean
computation on the symmetric positive-definite manifold and the low-rank matrix
completion on the Grassmann manifold. In all cases, the proposed algorithm
outperforms the state-of-the-art Riemannian batch and stochastic gradient
algorithms.
Role of Morphology Injection in Statistical Machine Translation
Authors: Sreelekha. S, Pushpak Bhattacharyya
Date: 2017-09-16 00:00:00
Tasks: Machine Translation, Translation

Phrase-based Statistical models are more commonly used as they perform
optimally in terms of both, translation quality and complexity of the system.
Hindi and in general all Indian languages are morphologically richer than
English. Hence, even though Phrase-based systems perform very well for the less
divergent language pairs, for English to Indian language translation, we need
more linguistic information (such as morphology, parse tree, parts of speech
tags, etc.) on the source side. Factored models seem to be useful in this case,
as Factored models consider word as a vector of factors. These factors can
contain any information about the surface word and use it while translating.
Hence, the objective of this work is to handle morphological inflections in
Hindi and Marathi using Factored translation models while translating from
English. SMT approaches face the problem of data sparsity while translating
into a morphologically rich language. It is very unlikely for a parallel corpus
to contain all morphological forms of words. We propose a solution to generate
these unseen morphological forms and inject them into original training
corpora. In this paper, we study factored models and the problem of sparseness
in context of translation to morphologically rich languages. We propose a
simple and effective solution which is based on enriching the input with
various morphological forms of words. We observe that morphology injection
improves the quality of translation in terms of both adequacy and fluency. We
verify this with the experiments on two morphologically rich languages: Hindi
and Marathi, while translating from English.
A segmental framework for fully-unsupervised large-vocabulary speech recognition
Authors: Herman Kamper, Aren Jansen, Sharon Goldwater
Date: 2016-06-22 00:00:00
Tasks: Language Modelling, Speech Recognition, Unsupervised Speech Recognition

Zero-resource speech technology is a growing research area that aims to
develop methods for speech processing in the absence of transcriptions,
lexicons, or language modelling text. Early term discovery systems focused on
identifying isolated recurring patterns in a corpus, while more recent
full-coverage systems attempt to completely segment and cluster the audio into
word-like units---effectively performing unsupervised speech recognition. This
article presents the first attempt we are aware of to apply such a system to
large-vocabulary multi-speaker data. Our system uses a Bayesian modelling
framework with segmental word representations: each word segment is represented
as a fixed-dimensional acoustic embedding obtained by mapping the sequence of
feature frames to a single embedding vector. We compare our system on English
and Xitsonga datasets to state-of-the-art baselines, using a variety of
measures including word error rate (obtained by mapping the unsupervised output
to ground truth transcriptions). Very high word error rates are reported---in
the order of 70--80% for speaker-dependent and 80--95% for speaker-independent
systems---highlighting the difficulty of this task. Nevertheless, in terms of
cluster quality and word segmentation metrics, we show that by imposing a
consistent top-down segmentation while also using bottom-up knowledge from
detected syllable boundaries, both single-speaker and multi-speaker versions of
our system outperform a purely bottom-up single-speaker syllable-based
approach. We also show that the discovered clusters can be made less speaker-
and gender-specific by using an unsupervised autoencoder-like feature extractor
to learn better frame-level features (prior to embedding). Our system's
discovered clusters are still less pure than those of unsupervised term
discovery systems, but provide far greater coverage.
Subset Labeled LDA for Large-Scale Multi-Label Classification
Authors: Yannis Papanikolaou, Grigorios Tsoumakas
Date: 2017-09-16 00:00:00
Tasks: Classification, Extreme Multi-Label Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

Labeled Latent Dirichlet Allocation (LLDA) is an extension of the standard
unsupervised Latent Dirichlet Allocation (LDA) algorithm, to address
multi-label learning tasks. Previous work has shown it to perform in par with
other state-of-the-art multi-label methods. Nonetheless, with increasing label
sets sizes LLDA encounters scalability issues. In this work, we introduce
Subset LLDA, a simple variant of the standard LLDA algorithm, that not only can
effectively scale up to problems with hundreds of thousands of labels but also
improves over the LLDA state-of-the-art. We conduct extensive experiments on
eight data sets, with label sets sizes ranging from hundreds to hundreds of
thousands, comparing our proposed algorithm with the previously proposed LLDA
algorithms (Prior--LDA, Dep--LDA), as well as the state of the art in extreme
multi-label classification. The results show a steady advantage of our method
over the other LLDA algorithms and competitive results compared to the extreme
multi-label classification algorithms.
Difficulty Adjustable and Scalable Constrained Multi-objective Test Problem Toolkit
Authors: Zhun Fan, Wenji Li, Xinye Cai, Hui Li, Caimin Wei, Qingfu Zhang, Kalyanmoy Deb, Erik D. Goodman
Date: 2016-12-21 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Multi-objective evolutionary algorithms (MOEAs) have progressed significantly in recent decades, but most of them are designed to solve unconstrained multi-objective optimization problems. In fact, many real-world multi-objective problems contain a number of constraints. To promote research on constrained multi-objective optimization, we first propose a problem classification scheme with three primary types of difficulty, which reflect various types of challenges presented by real-world optimization problems, in order to characterize the constraint functions in constrained multi-objective optimization problems (CMOPs). These are feasibility-hardness, convergence-hardness and diversity-hardness. We then develop a general toolkit to construct difficulty-adjustable and scalable CMOPs (DAS-CMOPs, or DAS-CMaOPs when the number of objectives is greater than three) with three types of parameterized constraint functions developed to capture the three proposed types of difficulty. Based on this toolkit, we suggest nine difficulty-adjustable and scalable CMOPs and nine CMaOPs. The experimental results reveal that mechanisms in MOEA/D-CDP may be more effective in solving convergence-hard DAS-CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving DAS-CMOPs with simultaneous diversity-, feasibility- and convergence-hardness. Mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness. In addition, none of them can solve these problems efficiently, which stimulates us to continue to develop new CMOEAs and CMaOEAs to solve the suggested DAS-CMOPs and DAS-CMaOPs.
Long-Term Ensemble Learning of Visual Place Classifiers
Authors: Xiaoxiao Fei, Kanji Tanaka, Yichu Fang, Akitaka Takayama
Date: 2017-09-16 00:00:00
Tasks: Ensemble Learning, Scheduling, Transfer Learning

This paper addresses the problem of cross-season visual place classification
(VPC) from a novel perspective of long-term map learning. Our goal is to enable
transfer learning efficiently from one season to the next, at a small constant
cost, and without wasting the robot's available long-term-memory by memorizing
very large amounts of training data. To realize a good tradeoff between
generalization and specialization abilities, we employ an ensemble of
convolutional neural network (DCN) classifiers and consider the task of
scheduling (when and which classifiers to retrain), given a previous season's
DCN classifiers as the sole prior knowledge. We present a unified framework for
retraining scheduling and discuss practical implementation strategies.
Furthermore, we address the task of partitioning a robot's workspace into
places to define place classes in an unsupervised manner, rather than using
uniform partitioning, so as to maximize VPC performance. Experiments using the
publicly available NCLT dataset revealed that retraining scheduling of a DCN
classifier ensemble is crucial and performance is significantly increased by
using planned scheduling.
Face Retrieval using Frequency Decoded Local Descriptor
Authors: Shiv Ram Dubey
Date: 2017-09-16 00:00:00
Tasks: Decoder, Retrieval

The local descriptors have been the backbone of most of the computer vision
problems. Most of the existing local descriptors are generated over the raw
input images. In order to increase the discriminative power of the local
descriptors, some researchers converted the raw image into multiple images with
the help of some high and low pass frequency filters, then the local
descriptors are computed over each filtered image and finally concatenated into
a single descriptor. By doing so, these approaches do not utilize the inter
frequency relationship which causes the less improvement in the discriminative
power of the descriptor that could be achieved. In this paper, this problem is
solved by utilizing the decoder concept of multi-channel decoded local binary
pattern over the multi-frequency patterns. A frequency decoded local binary
pattern (FDLBP) is proposed with two decoders. Each decoder works with one low
frequency pattern and two high frequency patterns. Finally, the descriptors
from both decoders are concatenated to form the single descriptor. The face
retrieval experiments are conducted over four benchmarks and challenging
databases such as PaSC, LFW, PubFig, and ESSEX. The experimental results
confirm the superiority of the FDLBP descriptor as compared to the
state-of-the-art descriptors such as LBP, SOBEL_LBP, BoF_LBP, SVD_S_LBP, mdLBP,
etc.
Acquiring Background Knowledge to Improve Moral Value Prediction
Authors: Ying Lin, Joe Hoover, Morteza Dehghani, Marlon Mooijman, Heng Ji
Date: 2017-09-16 00:00:00
Tasks: Prediction, Value prediction

In this paper, we address the problem of detecting expressions of moral
values in tweets using content analysis. This is a particularly challenging
problem because moral values are often only implicitly signaled in language,
and tweets contain little contextual information due to length constraints. To
address these obstacles, we present a novel approach to automatically acquire
background knowledge from an external knowledge base to enrich input texts and
thus improve moral value prediction. By combining basic text features with
background knowledge, our overall context-aware framework achieves performance
comparable to a single human annotator. To the best of our knowledge, this is
the first attempt to incorporate background knowledge for the prediction of
implicit psychological variables in the area of computational social science.
Statistical inference on random dot product graphs: a survey
Authors: Avanti Athreya, Donniell E. Fishkind, Keith Levin, Vince Lyzinski, Youngser Park, Yichen Qin, Daniel L. Sussman, Minh Tang, Joshua T. Vogelstein, Carey E. Priebe
Date: 2017-09-16 00:00:00
Tasks: Community Detection, Survey

The random dot product graph (RDPG) is an independent-edge random graph that
is analytically tractable and, simultaneously, either encompasses or can
successfully approximate a wide range of random graphs, from relatively simple
stochastic block models to complex latent position graphs. In this survey
paper, we describe a comprehensive paradigm for statistical inference on random
dot product graphs, a paradigm centered on spectral embeddings of adjacency and
Laplacian matrices. We examine the analogues, in graph inference, of several
canonical tenets of classical Euclidean inference: in particular, we summarize
a body of existing results on the consistency and asymptotic normality of the
adjacency and Laplacian spectral embeddings, and the role these spectral
embeddings can play in the construction of single- and multi-sample hypothesis
tests for graph data. We investigate several real-world applications, including
community detection and classification in large social networks and the
determination of functional and biologically relevant network properties from
an exploratory data analysis of the Drosophila connectome. We outline requisite
background and current open problems in spectral graph inference.
Network Model Selection for Task-Focused Attributed Network Inference
Authors: Ivan Brugere, Chris Kanich, Tanya Y. Berger-Wolf
Date: 2017-08-21 00:00:00
Tasks: Model Selection

Networks are models representing relationships between entities. Often these
relationships are explicitly given, or we must learn a representation which
generalizes and predicts observed behavior in underlying individual data (e.g.
attributes or labels). Whether given or inferred, choosing the best
representation affects subsequent tasks and questions on the network. This work
focuses on model selection to evaluate network representations from data,
focusing on fundamental predictive tasks on networks. We present a modular
methodology using general, interpretable network models, task neighborhood
functions found across domains, and several criteria for robust model
selection. We demonstrate our methodology on three online user activity
datasets and show that network model selection for the appropriate network task
vs. an alternate task increases performance by an order of magnitude in our
experiments.
Speech-Based Visual Question Answering
Authors: Ted Zhang, Dengxin Dai, Tinne Tuytelaars, Marie-Francine Moens, Luc van Gool
Date: 2017-05-01 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Question Answering, speech-recognition, Speech Recognition, Visual Question Answering, Visual Question Answering (VQA)

This paper introduces speech-based visual question answering (VQA), the task
of generating an answer given an image and a spoken question. Two methods are
studied: an end-to-end, deep neural network that directly uses audio waveforms
as input versus a pipelined approach that performs ASR (Automatic Speech
Recognition) on the question, followed by text-based visual question answering.
Furthermore, we investigate the robustness of both methods by injecting various
levels of noise into the spoken question and find both methods to be tolerate
noise at similar levels.