pwc-archive/papers-with-abstracts huggingface dataset:
Stability of Topic Modeling via Matrix Factorization
Authors: Mark Belford, Brian Mac Namee, Derek Greene
Date: 2017-02-23 00:00:00
Tasks: Clustering, Ensemble Learning, Topic Models

Topic models can provide us with an insight into the underlying latent
structure of a large corpus of documents. A range of methods have been proposed
in the literature, including probabilistic topic models and techniques based on
matrix factorization. However, in both cases, standard implementations rely on
stochastic elements in their initialization phase, which can potentially lead
to different results being generated on the same corpus when using the same
parameter values. This corresponds to the concept of "instability" which has
previously been studied in the context of $k$-means clustering. In many
applications of topic modeling, this problem of instability is not considered
and topic models are treated as being definitive, even though the results may
change considerably if the initialization process is altered. In this paper we
demonstrate the inherent instability of popular topic modeling approaches,
using a number of new measures to assess stability. To address this issue in
the context of matrix factorization for topic modeling, we propose the use of
ensemble learning strategies. Based on experiments performed on annotated text
corpora, we show that a K-Fold ensemble strategy, combining both ensembles and
structured initialization, can significantly reduce instability, while
simultaneously yielding more accurate topic models.
RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
Authors: Shuochao Yao, Yiran Zhao, Huajie Shao, Aston Zhang, Chao Zhang, Shen Li, Tarek Abdelzaher
Date: 2017-09-09 00:00:00
Tasks: scoring rule

Recent advances in deep learning have led various applications to
unprecedented achievements, which could potentially bring higher intelligence
to a broad spectrum of mobile and ubiquitous applications. Although existing
studies have demonstrated the effectiveness and feasibility of running deep
neural network inference operations on mobile and embedded devices, they
overlooked the reliability of mobile computing models. Reliability measurements
such as predictive uncertainty estimations are key factors for improving the
decision accuracy and user experience. In this work, we propose RDeepSense, the
first deep learning model that provides well-calibrated uncertainty estimations
for resource-constrained mobile and embedded devices. RDeepSense enables the
predictive uncertainty by adopting a tunable proper scoring rule as the
training criterion and dropout as the implicit Bayesian approximation, which
theoretically proves its correctness.To reduce the computational complexity,
RDeepSense employs efficient dropout and predictive distribution estimation
instead of model ensemble or sampling-based method for inference operations. We
evaluate RDeepSense with four mobile sensing applications using Intel Edison
devices. Results show that RDeepSense can reduce around 90% of the energy
consumption while producing superior uncertainty estimations and preserving at
least the same model accuracy compared with other state-of-the-art methods.
Matrix and Graph Operations for Relationship Inference: An Illustration with the Kinship Inference in the China Biographical Database
Authors: Chao-Lin Liu, Hongsu Wang
Date: 2017-09-09 00:00:00

Biographical databases contain diverse information about individuals. Person
names, birth information, career, friends, family and special achievements are
some possible items in the record for an individual. The relationships between
individuals, such as kinship and friendship, provide invaluable insights about
hidden communities which are not directly recorded in databases. We show that
some simple matrix and graph-based operations are effective for inferring
relationships among individuals, and illustrate the main ideas with the China
Biographical Database (CBDB).
Sequential 3D U-Nets for Biologically-Informed Brain Tumor Segmentation
Authors: Andrew Beers, Ken Chang, James Brown, Emmett Sartor, CP Mammen, Elizabeth Gerstner, Bruce Rosen, Jayashree Kalpathy-Cramer
Date: 2017-09-09 00:00:00
Tasks: Brain Tumor Segmentation, Lesion Segmentation, Segmentation, Tumor Segmentation

Deep learning has quickly become the weapon of choice for brain lesion
segmentation. However, few existing algorithms pre-configure any biological
context of their chosen segmentation tissues, and instead rely on the neural
network's optimizer to develop such associations de novo. We present a novel
method for applying deep neural networks to the problem of glioma tissue
segmentation that takes into account the structured nature of gliomas -
edematous tissue surrounding mutually-exclusive regions of enhancing and
non-enhancing tumor. We trained multiple deep neural networks with a 3D U-Net
architecture in a tree structure to create segmentations for edema,
non-enhancing tumor, and enhancing tumor regions. Specifically, training was
configured such that the whole tumor region including edema was predicted
first, and its output segmentation was fed as input into separate models to
predict enhancing and non-enhancing tumor. Our method was trained and evaluated
on the publicly available BraTS dataset, achieving Dice scores of 0.882, 0.732,
and 0.730 for whole tumor, enhancing tumor and tumor core respectively.
Complex spectrogram enhancement by convolutional neural network with multi-metrics learning
Authors: Szu-Wei Fu, Ting-yao Hu, Yu Tsao, Xugang Lu
Date: 2017-04-27 00:00:00
Tasks: Speech Enhancement

This paper aims to address two issues existing in the current speech
enhancement methods: 1) the difficulty of phase estimations; 2) a single
objective function cannot consider multiple metrics simultaneously. To solve
the first problem, we propose a novel convolutional neural network (CNN) model
for complex spectrogram enhancement, namely estimating clean real and imaginary
(RI) spectrograms from noisy ones. The reconstructed RI spectrograms are
directly used to synthesize enhanced speech waveforms. In addition, since
log-power spectrogram (LPS) can be represented as a function of RI
spectrograms, its reconstruction is also considered as another target. Thus a
unified objective function, which combines these two targets (reconstruction of
RI spectrograms and LPS), is equivalent to simultaneously optimizing two
commonly used objective metrics: segmental signal-to-noise ratio (SSNR) and
logspectral distortion (LSD). Therefore, the learning process is called
multi-metrics learning (MML). Experimental results confirm the effectiveness of
the proposed CNN with RI spectrograms and MML in terms of improved standardized
evaluation metrics on a speech enhancement task.
Deep Residual Networks and Weight Initialization
Authors: Masato Taki
Date: 2017-09-09 00:00:00

Residual Network (ResNet) is the state-of-the-art architecture that realizes
successful training of really deep neural network. It is also known that good
weight initialization of neural network avoids problem of vanishing/exploding
gradients. In this paper, simplified models of ResNets are analyzed. We argue
that goodness of ResNet is correlated with the fact that ResNets are relatively
insensitive to choice of initial weights. We also demonstrate how batch
normalization improves backpropagation of deep ResNets without tuning initial
values of weights.
Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging
Authors: Zachary DeVito, Michael Mara, Michael Zollhöfer, Gilbert Bernstein, Jonathan Ragan-Kelley, Christian Theobalt, Pat Hanrahan, Matthew Fisher, Matthias Nießner
Date: 2016-04-22 00:00:00
Tasks: GPU

Many graphics and vision problems can be expressed as non-linear least
squares optimizations of objective functions over visual data, such as images
and meshes. The mathematical descriptions of these functions are extremely
concise, but their implementation in real code is tedious, especially when
optimized for real-time performance on modern GPUs in interactive applications.
In this work, we propose a new language, Opt (available under
http://optlang.org), for writing these objective functions over image- or
graph-structured unknowns concisely and at a high level. Our compiler
automatically transforms these specifications into state-of-the-art GPU solvers
based on Gauss-Newton or Levenberg-Marquardt methods. Opt can generate
different variations of the solver, so users can easily explore tradeoffs in
numerical precision, matrix-free methods, and solver approaches. In our
results, we implement a variety of real-world graphics and vision applications.
Their energy functions are expressible in tens of lines of code, and produce
highly-optimized GPU solver implementations. These solver have performance
competitive with the best published hand-tuned, application-specific GPU
solvers, and orders of magnitude beyond a general-purpose auto-generated
solver.
How to Train Triplet Networks with 100K Identities?
Authors: Chong Wang, Xue Zhang, Xipeng Lan
Date: 2017-09-09 00:00:00
Tasks: Face Recognition, Image Retrieval, Retrieval, Triplet

Training triplet networks with large-scale data is challenging in face
recognition. Due to the number of possible triplets explodes with the number of
samples, previous studies adopt the online hard negative mining(OHNM) to handle
it. However, as the number of identities becomes extremely large, the training
will suffer from bad local minima because effective hard triplets are difficult
to be found. To solve the problem, in this paper, we propose training triplet
networks with subspace learning, which splits the space of all identities into
subspaces consisting of only similar identities. Combined with the batch OHNM,
hard triplets can be found much easier. Experiments on the large-scale
MS-Celeb-1M challenge with 100K identities demonstrate that the proposed method
can largely improve the performance. In addition, to deal with heavy noise and
large-scale retrieval, we also make some efforts on robust noise removing and
efficient image retrieval, which are used jointly with the subspace learning to
obtain the state-of-the-art performance on the MS-Celeb-1M competition (without
external data in Challenge1).
Comparison of Multiple Features and Modeling Methods for Text-dependent Speaker Verification
Authors: Yi Liu, Liang He, Yao Tian, Zhuzi Chen, Jia Liu, Michael T. Johnson
Date: 2017-07-14 00:00:00
Tasks: Speaker Identification, Speaker Recognition, Speaker Verification, Text-Dependent Speaker Verification, Text-Independent Speaker Verification

Text-dependent speaker verification is becoming popular in the speaker
recognition society. However, the conventional i-vector framework which has
been successful for speaker identification and other similar tasks works
relatively poorly in this task. Researchers have proposed several new methods
to improve performance, but it is still unclear that which model is the best
choice, especially when the pass-phrases are prompted during enrollment and
test. In this paper, we introduce four modeling methods and compare their
performance on the newly published RedDots dataset. To further explore the
influence of different frame alignments, Viterbi and forward-backward
algorithms are both used in the HMM-based models. Several bottleneck features
are also investigated. Our experiments show that, by explicitly modeling the
lexical content, the HMM-based modeling achieves good results in the
fixed-phrase condition. In the prompted-phrase condition, GMM-HMM and
i-vector/HMM are not as successful. In both conditions, the forward-backward
algorithm brings more benefits to the i-vector/HMM system. Additionally, we
also find that even though bottleneck features perform well for
text-independent speaker verification, they do not outperform MFCCs on the most
challenging Imposter-Correct trials on RedDots.
Less Is More: A Comprehensive Framework for the Number of Components of Ensemble Classifiers
Authors: Hamed Bonab, Fazli Can
Date: 2017-09-09 00:00:00

The number of component classifiers chosen for an ensemble greatly impacts
the prediction ability. In this paper, we use a geometric framework for a
priori determining the ensemble size, which is applicable to most of existing
batch and online ensemble classifiers. There are only a limited number of
studies on the ensemble size examining Majority Voting (MV) and Weighted
Majority Voting (WMV). Almost all of them are designed for batch-mode, hardly
addressing online environments. Big data dimensions and resource limitations,
in terms of time and memory, make determination of ensemble size crucial,
especially for online environments. For the MV aggregation rule, our framework
proves that the more strong components we add to the ensemble, the more
accurate predictions we can achieve. For the WMV aggregation rule, our
framework proves the existence of an ideal number of components, which is equal
to the number of class labels, with the premise that components are completely
independent of each other and strong enough. While giving the exact definition
for a strong and independent classifier in the context of an ensemble is a
challenging task, our proposed geometric framework provides a theoretical
explanation of diversity and its impact on the accuracy of predictions. We
conduct a series of experimental evaluations to show the practical value of our
theorems and existing challenges.
Graph Scaling Cut with L1-Norm for Classification of Hyperspectral Images
Authors: Ramanarayan Mohanty, S. L. Happy, Aurobinda Routray
Date: 2017-09-09 00:00:00
Tasks: Classification Of Hyperspectral Images, Dimensionality Reduction, General Classification

In this paper, we propose an L1 normalized graph based dimensionality
reduction method for Hyperspectral images, called as L1-Scaling Cut (L1-SC).
The underlying idea of this method is to generate the optimal projection matrix
by retaining the original distribution of the data. Though L2-norm is generally
preferred for computation, it is sensitive to noise and outliers. However,
L1-norm is robust to them. Therefore, we obtain the optimal projection matrix
by maximizing the ratio of between-class dispersion to within-class dispersion
using L1-norm. Furthermore, an iterative algorithm is described to solve the
optimization problem. The experimental results of the HSI classification
confirm the effectiveness of the proposed L1-SC method on both noisy and
noiseless data.
Communication-efficient Algorithm for Distributed Sparse Learning via Two-way Truncation
Authors: Jineng Ren, Jarvis Haupt
Date: 2017-09-02 00:00:00
Tasks: Sparse Learning, Vocal Bursts Valence Prediction

We propose a communicationally and computationally efficient algorithm for
high-dimensional distributed sparse learning. At each iteration, local machines
compute the gradient on local data and the master machine solves one shifted
$l_1$ regularized minimization problem. The communication cost is reduced from
constant times of the dimension number for the state-of-the-art algorithm to
constant times of the sparsity number via Two-way Truncation procedure.
Theoretically, we prove that the estimation error of the proposed algorithm
decreases exponentially and matches that of the centralized method under mild
assumptions. Extensive experiments on both simulated data and real data verify
that the proposed algorithm is efficient and has performance comparable with
the centralized method on solving high-dimensional sparse learning problems.
Semi-Supervised Instance Population of an Ontology using Word Vector Embeddings
Authors: Vindula Jayawardana, Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Keet Sugathadasa, Buddhi Ayesha, Madhavi Perera
Date: 2017-09-09 00:00:00
Tasks: Management, Word Embeddings

In many modern day systems such as information extraction and knowledge
management agents, ontologies play a vital role in maintaining the concept
hierarchies of the selected domain. However, ontology population has become a
problematic process due to its nature of heavy coupling with manual human
intervention. With the use of word embeddings in the field of natural language
processing, it became a popular topic due to its ability to cope up with
semantic sensitivity. Hence, in this study, we propose a novel way of
semi-supervised ontology population through word embeddings as the basis. We
built several models including traditional benchmark models and new types of
models which are based on word embeddings. Finally, we ensemble them together
to come up with a synergistic model with better accuracy. We demonstrate that
our ensemble model can outperform the individual models.
A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer
Authors: Tianbao Yang, Zhe Li, Lijun Zhang
Date: 2017-09-09 00:00:00

In this paper, we present a simple analysis of {\bf fast rates} with {\it
high probability} of {\bf empirical minimization} for {\it stochastic composite
optimization} over a finite-dimensional bounded convex set with exponential
concave loss functions and an arbitrary convex regularization. To the best of
our knowledge, this result is the first of its kind. As a byproduct, we can
directly obtain the fast rate with {\it high probability} for exponential
concave empirical risk minimization with and without any convex regularization,
which not only extends existing results of empirical risk minimization but also
provides a unified framework for analyzing exponential concave empirical risk
minimization with and without {\it any} convex regularization. Our proof is
very simple only exploiting the covering number of a finite-dimensional bounded
set and a concentration inequality of random vectors.
Image Processing Operations Identification via Convolutional Neural Network
Authors: Bolin Chen, Haodong Li, Weiqi Luo
Date: 2017-09-09 00:00:00
Tasks: Image Forensics, Steganalysis

In recent years, image forensics has attracted more and more attention, and
many forensic methods have been proposed for identifying image processing
operations. Up to now, most existing methods are based on hand crafted
features, and just one specific operation is considered in their methods. In
many forensic scenarios, however, multiple classification for various image
processing operations is more practical. Besides, it is difficult to obtain
effective features by hand for some image processing operations. In this paper,
therefore, we propose a new convolutional neural network (CNN) based method to
adaptively learn discriminative features for identifying typical image
processing operations. We carefully design the high pass filter bank to get the
image residuals of the input image, the channel expansion layer to mix up the
resulting residuals, the pooling layers, and the activation functions employed
in our method. The extensive results show that the proposed method can
outperform the currently best method based on hand crafted features and three
related methods based on CNN for image steganalysis and/or forensics, achieving
the state-of-the-art results. Furthermore, we provide more supplementary
results to show the rationality and robustness of the proposed model.
Simultaneously Learning Neighborship and Projection Matrix for Supervised Dimensionality Reduction
Authors: Yanwei Pang, Bo Zhou, Feiping Nie
Date: 2017-09-09 00:00:00
Tasks: Dimensionality Reduction, Supervised dimensionality reduction

Explicitly or implicitly, most of dimensionality reduction methods need to
determine which samples are neighbors and the similarity between the neighbors
in the original highdimensional space. The projection matrix is then learned on
the assumption that the neighborhood information (e.g., the similarity) is
known and fixed prior to learning. However, it is difficult to precisely
measure the intrinsic similarity of samples in high-dimensional space because
of the curse of dimensionality. Consequently, the neighbors selected according
to such similarity might and the projection matrix obtained according to such
similarity and neighbors are not optimal in the sense of classification and
generalization. To overcome the drawbacks, in this paper we propose to let the
similarity and neighbors be variables and model them in low-dimensional space.
Both the optimal similarity and projection matrix are obtained by minimizing a
unified objective function. Nonnegative and sum-to-one constraints on the
similarity are adopted. Instead of empirically setting the regularization
parameter, we treat it as a variable to be optimized. It is interesting that
the optimal regularization parameter is adaptive to the neighbors in
low-dimensional space and has intuitive meaning. Experimental results on the
YALE B, COIL-100, and MNIST datasets demonstrate the effectiveness of the
proposed method.
Optimization assisted MCMC
Authors: Ricky Fok, Aijun An, Xiaogang Wang
Date: 2017-09-09 00:00:00
Tasks: global-optimization

Markov Chain Monte Carlo (MCMC) sampling methods are widely used but often
encounter either slow convergence or biased sampling when applied to multimodal
high dimensional distributions. In this paper, we present a general framework
of improving classical MCMC samplers by employing a global optimization method.
The global optimization method first reduces a high dimensional search to an
one dimensional geodesic to find a starting point close to a local mode. The
search is accelerated and completed by using a local search method such as
BFGS. We modify the target distribution by extracting a local Gaussian
distribution aound the found mode. The process is repeated to find all the
modes during sampling on the fly. We integrate the optimization algorithm into
the Wormhole Hamiltonian Monte Carlo (WHMC) method. Experimental results show
that, when applied to high dimensional, multimodal Gaussian mixture models and
the network sensor localization problem, the proposed method achieves much
faster convergence, with relative error from the mean improved by about an
order of magnitude than WHMC in some cases.
TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow
Authors: Danijar Hafner, James Davidson, Vincent Vanhoucke
Date: 2017-09-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce TensorFlow Agents, an efficient infrastructure paradigm for
building parallel reinforcement learning algorithms in TensorFlow. We simulate
multiple environments in parallel, and group them to perform the neural network
computation on a batch rather than individual observations. This allows the
TensorFlow execution engine to parallelize computation, without the need for
manual synchronization. Environments are stepped in separate Python processes
to progress them in parallel without interference of the global interpreter
lock. As part of this project, we introduce BatchPPO, an efficient
implementation of the proximal policy optimization algorithm. By open sourcing
TensorFlow Agents, we hope to provide a flexible starting point for future
projects that accelerates future research in the field.