pwc-archive/papers-with-abstracts huggingface dataset:
Online Adaptive Principal Component Analysis and Its extensions
Authors: Jianjun Yuan, Andrew Lamperski
Date: 2019-01-23 00:00:00

We propose algorithms for online principal component analysis (PCA) and variance minimization for adaptive settings. Previous literature has focused on upper bounding the static adversarial regret, whose comparator is the optimal fixed action in hindsight. However, static regret is not an appropriate metric when the underlying environment is changing. Instead, we adopt the adaptive regret metric from the previous literature and propose online adaptive algorithms for PCA and variance minimization, that have sub-linear adaptive regret guarantees. We demonstrate both theoretically and experimentally that the proposed algorithms can adapt to the changing environments.
Class Activation Map Generation by Representative Class Selection and Multi-Layer Feature Fusion
Authors: Fanman Meng, Kaixu Huang, Hongliang Li, Qingbo Wu
Date: 2019-01-23 00:00:00
Tasks: Binary Classification, Clustering, General Classification

Existing method generates class activation map (CAM) by a set of fixed
classes (i.e., using all the classes), while the discriminative cues between
class pairs are not considered. Note that activation maps by considering
different class pair are complementary, and therefore can provide more
discriminative cues to overcome the shortcoming of the existing CAM generation
that the highlighted regions are usually local part regions rather than global
object regions due to the lack of object cues. In this paper, we generate CAM
by using a few of representative classes, with aim of extracting more
discriminative cues by considering each class pair to obtain CAM more globally.
The advantages are twofold. Firstly, the representative classes are able to
obtain activation regions that are complementary to each other, and therefore
leads to generating activation map more accurately. Secondly, we only need to
consider a small number of representative classes, making the CAM generation
suitable for small networks. We propose a clustering based method to select the
representative classes. Multiple binary classification models rather than a
multiple class classification model are used to generate the CAM. Moreover, we
propose a multi-layer fusion based CAM generation method to simultaneously
combine high-level semantic features and low-level detail features. We validate
the proposed method on the PASCAL VOC and COCO database in terms of
segmentation groundtruth. Various networks such as classical network
(Resnet-50, Resent-101 and Resnet-152) and small network (VGG-19, Resnet-18 and
Mobilenet) are considered. Experimental results show that the proposed method
improves the CAM generation obviously.
A Top-down Approach to Articulated Human Pose Estimation and Tracking
Authors: Guanghan Ning, Ping Liu, Xiaochuan Fan, Chi Zhang
Date: 2019-01-23 00:00:00
Tasks: Pose Estimation, Pose Tracking

Both the tasks of multi-person human pose estimation and pose tracking in
videos are quite challenging. Existing methods can be categorized into two
groups: top-down and bottom-up approaches. In this paper, following the
top-down approach, we aim to build a strong baseline system with three modules:
human candidate detector, single-person pose estimator and human pose tracker.
Firstly, we choose a generic object detector among state-of-the-art methods to
detect human candidates. Then, the cascaded pyramid network is used to estimate
the corresponding human pose. Finally, we use a flow-based pose tracker to
render keypoint-association across frames, i.e., assigning each human candidate
a unique and temporally-consistent id, for the multi-target pose tracking
purpose. We conduct extensive ablative experiments to validate various choices
of models and configurations. We take part in two ECCV 18 PoseTrack challenges:
pose estimation and pose tracking.
Composition and decomposition of GANs
Authors: Yeu-Chern Harn, Zhenghao Chen, Vladimir Jojic
Date: 2019-01-23 00:00:00

In this work, we propose a composition/decomposition framework for
adversarially training generative models on composed data - data where each
sample can be thought of as being constructed from a fixed number of
components. In our framework, samples are generated by sampling components from
component generators and feeding these components to a composition function
which combines them into a "composed sample". This compositional training
approach improves the modularity, extensibility and interpretability of
Generative Adversarial Networks (GANs) - providing a principled way to
incrementally construct complex models out of simpler component models, and
allowing for explicit "division of responsibility" between these components.
Using this framework, we define a family of learning tasks and evaluate their
feasibility on two datasets in two different data modalities (image and text).
Lastly, we derive sufficient conditions such that these compositional
generative models are identifiable. Our work provides a principled approach to
building on pre-trained generative models or for exploiting the compositional
nature of data distributions to train extensible and interpretable models.
Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning
Authors: Chi-Sing Ho, Neal Jean, Catherine A. Hogan, Lena Blackmon, Stefanie S. Jeffrey, Mark Holodniy, Niaz Banaei, Amr A. E. Saleh, Stefano Ermon, Jennifer Dionne
Date: 2019-01-23 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

Rapid identification of bacteria is essential to prevent the spread of infectious disease, help combat antimicrobial resistance, and improve patient outcomes. Raman optical spectroscopy promises to combine bacterial detection, identification, and antibiotic susceptibility testing in a single step. However, achieving clinically relevant speeds and accuracies remains challenging due to the weak Raman signal from bacterial cells and the large number of bacterial species and phenotypes. By amassing the largest known dataset of bacterial Raman spectra, we are able to apply state-of-the-art deep learning approaches to identify 30 of the most common bacterial pathogens from noisy Raman spectra, achieving antibiotic treatment identification accuracies of 99.0$\pm$0.1%. This novel approach distinguishes between methicillin-resistant and -susceptible isolates of Staphylococcus aureus (MRSA and MSSA) as well as a pair of isogenic MRSA and MSSA that are genetically identical apart from deletion of the mecA resistance gene, indicating the potential for culture-free detection of antibiotic resistance. Results from initial clinical validation are promising: using just 10 bacterial spectra from each of 25 isolates, we achieve 99.0$\pm$1.9% species identification accuracy. Our combined Raman-deep learning system represents an important proof-of-concept for rapid, culture-free identification of bacterial isolates and antibiotic resistance and could be readily extended for diagnostics on blood, urine, and sputum.
Low-Complexity Nonparametric Bayesian Online Prediction with Universal Guarantees
Authors: Alix Lhéritier, Frédéric Cazals
Date: 2019-01-23 00:00:00

We propose a novel nonparametric online predictor for discrete labels conditioned on multivariate continuous features. The predictor is based on a feature space discretization induced by a full-fledged k-d tree with randomly picked directions and a recursive Bayesian distribution, which allows to automatically learn the most relevant feature scales characterizing the conditional distribution. We prove its pointwise universality, i.e., it achieves a normalized log loss performance asymptotically as good as the true conditional entropy of the labels given the features. The time complexity to process the $n$-th sample point is $O(\log n)$ in probability with respect to the distribution generating the data points, whereas other exact nonparametric methods require to process all past observations. Experiments on challenging datasets show the computational and statistical efficiency of our algorithm in comparison to standard and state-of-the-art methods.
Attenuating Bias in Word Vectors
Authors: Sunipa Dev, Jeff Phillips
Date: 2019-01-23 00:00:00

Word vector representations are well developed tools for various NLP and
Machine Learning tasks and are known to retain significant semantic and
syntactic structure of languages. But they are prone to carrying and amplifying
bias which can perpetrate discrimination in various applications. In this work,
we explore new simple ways to detect the most stereotypically gendered words in
an embedding and remove the bias from them. We verify how names are masked
carriers of gender bias and then use that as a tool to attenuate bias in
embeddings. Further, we extend this property of names to show how names can be
used to detect other types of bias in the embeddings such as bias based on
race, ethnicity, and age.
Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings
Authors: Hwiyeol Jo, Ceyda Cinarel
Date: 2019-01-22 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: General Classification, Semi-Supervised Text Classification, Sentiment Analysis, Text Classification

We propose a novel and simple method for semi-supervised text classification. The method stems from the hypothesis that a classifier with pretrained word embeddings always outperforms the same classifier with randomly initialized word embeddings, as empirically observed in NLP tasks. Our method first builds two sets of classifiers as a form of model ensemble, and then initializes their word embeddings differently: one using random, the other using pretrained word embeddings. We focus on different predictions between the two classifiers on unlabeled data while following the self-training framework. We also use early-stopping in meta-epoch to improve the performance of our method. Our method, Delta-training, outperforms the self-training and the co-training framework in 4 different text classification datasets, showing robustness against error accumulation.
Finite-Sum Smooth Optimization with SARAH
Authors: Lam M. Nguyen, Marten van Dijk, Dzung T. Phan, Phuong Ha Nguyen, Tsui-Wei Weng, Jayant R. Kalagnanam
Date: 2019-01-22 00:00:00

The total complexity (measured as the total number of gradient computations)
of a stochastic first-order optimization algorithm that finds a first-order
stationary point of a finite-sum smooth nonconvex objective function
$F(w)=\frac{1}{n} \sum_{i=1}^n f_i(w)$ has been proven to be at least
$\Omega(\sqrt{n}/\epsilon)$ for $n \leq \mathcal{O}(\epsilon^{-2})$ where
$\epsilon$ denotes the attained accuracy $\mathbb{E}[ \|\nabla
F(\tilde{w})\|^2] \leq \epsilon$ for the outputted approximation $\tilde{w}$
(Fang et al., 2018). In this paper, we provide a convergence analysis for a
slightly modified version of the SARAH algorithm (Nguyen et al., 2017a;b) and
achieve total complexity that matches the lower-bound worst case complexity in
(Fang et al., 2018) up to a constant factor when $n \leq
\mathcal{O}(\epsilon^{-2})$ for nonconvex problems. For convex optimization, we
propose SARAH++ with sublinear convergence for general convex and linear
convergence for strongly convex problems; and we provide a practical version
for which numerical experiments on various datasets show an improved
performance.
Understanding Geometry of Encoder-Decoder CNNs
Authors: Jong Chul Ye, Woon Kyoung Sung
Date: 2019-01-22 00:00:00
Tasks: Decoder

Encoder-decoder networks using convolutional neural network (CNN) architecture have been extensively used in deep learning literatures thanks to its excellent performance for various inverse problems. However, it is still difficult to obtain coherent geometric view why such an architecture gives the desired performance. Inspired by recent theoretical understanding on generalizability, expressivity and optimization landscape of neural networks, as well as the theory of convolutional framelets, here we provide a unified theoretical framework that leads to a better understanding of geometry of encoder-decoder CNNs. Our unified mathematical framework shows that encoder-decoder CNN architecture is closely related to nonlinear basis representation using combinatorial convolution frames, whose expressibility increases exponentially with the network depth. We also demonstrate the importance of skipped connection in terms of expressibility, and optimization landscape.
Learning Configuration Space Belief Model from Collision Checks for Motion Planning
Authors: Sumit Kumar, Shushman Choudhary, Siddhartha Srinivasa
Date: 2019-01-22 00:00:00
Tasks: Motion Planning

For motion planning in high dimensional configuration spaces, a significant
computational bottleneck is collision detection. Our aim is to reduce the
expected number of collision checks by creating a belief model of the
configuration space using results from collision tests. We assume the robot's
configuration space to be a continuous ambient space whereby neighbouring
points tend to share the same collision state. This enables us to formulate a
probabilistic model that assigns to unevaluated configurations a belief
estimate of being collision-free. We have presented a detailed comparative
analysis of various kNN methods and distance metrics used to evaluate C-space
belief. We have also proposed a weighting matrix in C-space to improve the
performance of kNN methods. Moreover, we have proposed a topological method
that exploits the higher order structure of the C-space to generate a belief
model. Our results indicate that our proposed topological method outperforms
kNN methods by achieving higher model accuracy while being computationally
efficient.
Solving All Regression Models For Learning Gaussian Networks Using Givens Rotations
Authors: Borzou Alipourfard, Jean X. Gao
Date: 2019-01-22 00:00:00
Tasks: All, regression

Score based learning (SBL) is a promising approach for learning Bayesian networks. The initial step in the majority of the SBL algorithms consists of computing the scores of all possible child and parent-set combinations for the variables. For Bayesian networks with continuous variables, a particular score is usually calculated as a function of the regression of the child over the variables in the parent-set. The sheer number of regressions models to be solved necessitates the design of efficient numerical algorithms. In this paper, we propose an algorithm for an efficient and exact calculation of regressions for all child and parent-set combinations. In the proposed algorithm, we use QR decompositions (QRDs) to capture the dependencies between the regressions for different families and Givens rotations to efficiently traverse through the space of QRDs such that all the regression models are accounted for in the shortest path possible. We compare the complexity of the suggested method with different algorithms, mainly those arising in all subset regression problems, and show that our algorithm has the smallest algorithmic complexity. We also explain how to parallelize the proposed method so as to decrease the runtime by a factor proportional to the number of processors utilized.
DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD
Authors: Lam M. Nguyen, Phuong Ha Nguyen, Dzung T. Phan, Jayant R. Kalagnanam, Marten van Dijk
Date: 2019-01-22 00:00:00
Tasks: LEMMA, valid

This paper has some inconsistent results, i.e., we made some failed claims
because we did some mistakes for using the test criterion for a series.
Precisely, our claims on the convergence rate of $\mathcal{O}(1/t)$ of SGD
presented in Theorem 1, Corollary 1, Theorem 2 and Corollary 2 are wrongly
derived because they are based on Lemma 5. In Lemma 5, we do not correctly use
the test criterion for a series. Hence, the result of Lemma 5 is not valid. We
would like to thank the community for pointing out this mistake!
Neutron drip line in the Ca region from Bayesian model averaging
Authors: Léo Neufcourt, Yuchen Cao, Witold Nazarewicz, Erik Olsen, Frederi Viens
Date: 2019-01-22 00:00:00

The region of heavy calcium isotopes forms the frontier of experimental and theoretical nuclear structure research where the basic concepts of nuclear physics are put to stringent test. The recent discovery of the extremely neutron-rich nuclei around $^{60}$Ca [Tarasov, 2018] and the experimental determination of masses for $^{55-57}$Ca (Michimasa, 2018] provide unique information about the binding energy surface in this region. To assess the impact of these experimental discoveries on the nuclear landscape's extent, we use global mass models and statistical machine learning to make predictions, with quantified levels of certainty, for bound nuclides between Si and Ti. Using a Bayesian model averaging analysis based on Gaussian-process-based extrapolations we introduce the posterior probability $p_{ex}$ for each nucleus to be bound to neutron emission. We find that extrapolations for drip-line locations, at which the nuclear binding ends, are consistent across the global mass models used, in spite of significant variations between their raw predictions. In particular, considering the current experimental information and current global mass models, we predict that $^{68}$Ca has an average posterior probability ${p_{ex}\approx76}$% to be bound to two-neutron emission while the nucleus $^{61}$Ca is likely to decay by emitting a neutron (${p_{ex}\approx 46}$ %).
Single Deep Counterfactual Regret Minimization
Authors: Eric Steinberger
Date: 2019-01-22 00:00:00
Tasks: counterfactual

Counterfactual Regret Minimization (CFR) is the most successful algorithm for finding approximate Nash equilibria in imperfect information games. However, CFR's reliance on full game-tree traversals limits its scalability. For this reason, the game's state- and action-space is often abstracted (i.e. simplified) for CFR, and the resulting strategy is then translated back to the full game, which requires extensive expert-knowledge and often converges to highly exploitable policies. A recently proposed method, Deep CFR, applies deep learning directly to CFR, allowing the agent to intrinsically abstract and generalize over the state-space from samples, without requiring expert knowledge. In this paper, we introduce Single Deep CFR (SD-CFR), a simplified variant of Deep CFR that has a lower overall approximation error by avoiding the training of an average strategy network. We show that SD-CFR is more attractive from a theoretical perspective and empirically outperforms Deep CFR with respect to exploitability and one-on-one play in poker.
On orthogonal projections for dimension reduction and applications in augmented target loss functions for learning problems
Authors: Anna Breger, Jose Ignacio Orlando, Pavol Harar, Monika Dörfler, Sophie Klimscha, Christoph Grechenig, Bianca S. Gerendas, Ursula Schmidt-Erfurth, Martin Ehler
Date: 2019-01-22 00:00:00
Tasks: Dimensionality Reduction, General Classification, Image Segmentation, Semantic Segmentation

The use of orthogonal projections on high-dimensional input and target data in learning frameworks is studied. First, we investigate the relations between two standard objectives in dimension reduction, preservation of variance and of pairwise relative distances. Investigations of their asymptotic correlation as well as numerical experiments show that a projection does usually not satisfy both objectives at once. In a standard classification problem we determine projections on the input data that balance the objectives and compare subsequent results. Next, we extend our application of orthogonal projections to deep learning tasks and introduce a general framework of augmented target loss functions. These loss functions integrate additional information via transformations and projections of the target data. In two supervised learning problems, clinical image segmentation and music information classification, the application of our proposed augmented target loss functions increase the accuracy.
Aggregated Pairwise Classification of Statistical Shapes
Authors: Min Ho Cho, Sebastian Kurtek, Steven N. MacEachern
Date: 2019-01-22 00:00:00
Tasks: Classification, General Classification

The classification of shapes is of great interest in diverse areas ranging
from medical imaging to computer vision and beyond. While many statistical
frameworks have been developed for the classification problem, most are
strongly tied to early formulations of the problem - with an object to be
classified described as a vector in a relatively low-dimensional Euclidean
space. Statistical shape data have two main properties that suggest a need for
a novel approach: (i) shapes are inherently infinite dimensional with strong
dependence among the positions of nearby points, and (ii) shape space is not
Euclidean, but is fundamentally curved. To accommodate these features of the
data, we work with the square-root velocity function of the curves to provide a
useful formal description of the shape, pass to tangent spaces of the manifold
of shapes at different projection points which effectively separate shapes for
pairwise classification in the training data, and use principal components
within these tangent spaces to reduce dimensionality. We illustrate the impact
of the projection point and choice of subspace on the misclassification rate
with a novel method of combining pairwise classifiers.
What Can Machine Learning Teach Us about Communications?
Authors: Mengke Lian, Christian Häger, Henry D. Pfister
Date: 2019-01-22 00:00:00
Tasks: BIG-bench Machine Learning

Rapid improvements in machine learning over the past decade are beginning to
have far-reaching effects. For communications, engineers with limited domain
expertise can now use off-the-shelf learning packages to design
high-performance systems based on simulations. Prior to the current revolution
in machine learning, the majority of communication engineers were quite aware
that system parameters (such as filter coefficients) could be learned using
stochastic gradient descent. It was not at all clear, however, that more
complicated parts of the system architecture could be learned as well. In this
paper, we discuss the application of machine-learning techniques to two
communications problems and focus on what can be learned from the resulting
systems. We were pleasantly surprised that the observed gains in one example
have a simple explanation that only became clear in hindsight. In essence, deep
learning discovered a simple and effective strategy that had not been
considered earlier.
Striking the Right Balance with Uncertainty
Authors: Salman Khan, Munawar Hayat, Waqas Zamir, Jianbing Shen, Ling Shao
Date: 2019-01-22 00:00:00
Proceeding: CVPR 2019 6
Tasks: Attribute, Classification, Face Verification, General Classification, Lesion Detection

Learning unbiased models on imbalanced datasets is a significant challenge.
Rare classes tend to get a concentrated representation in the classification
space which hampers the generalization of learned boundaries to new test
examples. In this paper, we demonstrate that the Bayesian uncertainty estimates
directly correlate with the rarity of classes and the difficulty level of
individual samples. Subsequently, we present a novel framework for uncertainty
based class imbalance learning that follows two key insights: First,
classification boundaries should be extended further away from a more uncertain
(rare) class to avoid overfitting and enhance its generalization. Second, each
sample should be modeled as a multi-variate Gaussian distribution with a mean
vector and a covariance matrix defined by the sample's uncertainty. The learned
boundaries should respect not only the individual samples but also their
distribution in the feature space. Our proposed approach efficiently utilizes
sample and class uncertainty information to learn robust features and more
generalizable classifiers. We systematically study the class imbalance problem
and derive a novel loss formulation for max-margin learning based on Bayesian
uncertainty measure. The proposed method shows significant performance
improvements on six benchmark datasets for face verification, attribute
prediction, digit/object classification and skin lesion detection.