pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for Tumor Classification in Imaging Mass Spectrometry
Authors: Jens Behrmann, Christian Etmann, Tobias Boskamp, Rita Casadonte, Jörg Kriegsmann, Peter Maass
Date: 2017-05-02 00:00:00
Tasks: Classification, Deep Learning, General Classification, Sensitivity

Motivation: Tumor classification using Imaging Mass Spectrometry (IMS) data
has a high potential for future applications in pathology. Due to the
complexity and size of the data, automated feature extraction and
classification steps are required to fully process the data. Deep learning
offers an approach to learn feature extraction and classification combined in a
single model. Commonly these steps are handled separately in IMS data analysis,
hence deep learning offers an alternative strategy worthwhile to explore.
Results: Methodologically, we propose an adapted architecture based on deep
convolutional networks to handle the characteristics of mass spectrometry data,
as well as a strategy to interpret the learned model in the spectral domain
based on a sensitivity analysis. The proposed methods are evaluated on two
challenging tumor classification tasks and compared to a baseline approach.
Competitiveness of the proposed methods are shown on both tasks by studying the
performance via cross-validation. Moreover, the learned models are analyzed by
the proposed sensitivity analysis revealing biologically plausible effects as
well as confounding factors of the considered task. Thus, this study may serve
as a starting point for further development of deep learning approaches in IMS
classification tasks.
Learning Generative Models with Sinkhorn Divergences
Authors: Aude Genevay, Gabriel Peyré, Marco Cuturi
Date: 2017-06-01 00:00:00
Tasks: GPU

The ability to compare two degenerate probability distributions (i.e. two
probability distributions supported on two distinct low-dimensional manifolds
living in a much higher-dimensional space) is a crucial problem arising in the
estimation of generative models for high-dimensional observations such as those
arising in computer vision or natural language. It is known that optimal
transport metrics can represent a cure for this problem, since they were
specifically designed as an alternative to information divergences to handle
such problematic scenarios. Unfortunately, training generative machines using
OT raises formidable computational and statistical challenges, because of (i)
the computational burden of evaluating OT losses, (ii) the instability and lack
of smoothness of these losses, (iii) the difficulty to estimate robustly these
losses and their gradients in high dimension. This paper presents the first
tractable computational method to train large scale generative models using an
optimal transport loss, and tackles these three issues by relying on two key
ideas: (a) entropic smoothing, which turns the original OT loss into one that
can be computed using Sinkhorn fixed point iterations; (b) algorithmic
(automatic) differentiation of these iterations. These two approximations
result in a robust and differentiable approximation of the OT loss with
streamlined GPU execution. Entropic smoothing generates a family of losses
interpolating between Wasserstein (OT) and Maximum Mean Discrepancy (MMD), thus
allowing to find a sweet spot leveraging the geometry of OT and the favorable
high-dimensional sample complexity of MMD which comes with unbiased gradient
estimates. The resulting computational architecture complements nicely standard
deep network generative models by a stack of extra layers implementing the loss
function.
From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning
Authors: Jingkuan Song, Yuyu Guo, Lianli Gao, Xuelong. Li, Alan Hanjalic, Heng Tao Shen
Date: 2017-08-08 00:00:00
Tasks: Decoder, Video Captioning

Video captioning in essential is a complex natural process, which is affected
by various uncertainties stemming from video content, subjective judgment, etc.
In this paper we build on the recent progress in using encoder-decoder
framework for video captioning and address what we find to be a critical
deficiency of the existing methods, that most of the decoders propagate
deterministic hidden states. Such complex uncertainty cannot be modeled
efficiently by the deterministic models. In this paper, we propose a generative
approach, referred to as multi-modal stochastic RNNs networks (MS-RNN), which
models the uncertainty observed in the data using latent stochastic variables.
Therefore, MS-RNN can improve the performance of video captioning, and generate
multiple sentences to describe a video considering different random factors.
Specifically, a multi-modal LSTM (M-LSTM) is first proposed to interact with
both visual and textual features to capture a high-level representation. Then,
a backward stochastic LSTM (S-LSTM) is proposed to support uncertainty
propagation by introducing latent variables. Experimental results on the
challenging datasets MSVD and MSR-VTT show that our proposed MS-RNN approach
outperforms the state-of-the-art video captioning benchmarks.
Learning Wasserstein Embeddings
Authors: Nicolas Courty, Rémi Flamary, Mélanie Ducoffe
Date: 2017-10-20 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder, Dimensionality Reduction, Domain Adaptation

The Wasserstein distance received a lot of attention recently in the
community of machine learning, especially for its principled way of comparing
distributions. It has found numerous applications in several hard problems,
such as domain adaptation, dimensionality reduction or generative models.
However, its use is still limited by a heavy computational cost. Our goal is to
alleviate this problem by providing an approximation mechanism that allows to
break its inherent complexity. It relies on the search of an embedding where
the Euclidean distance mimics the Wasserstein distance. We show that such an
embedding can be found with a siamese architecture associated with a decoder
network that allows to move from the embedding space back to the original input
space. Once this embedding has been found, computing optimization problems in
the Wasserstein space (e.g. barycenters, principal directions or even
archetypes) can be conducted extremely fast. Numerical experiments supporting
this idea are conducted on image datasets, and show the wide potential benefits
of our method.
Generalized Zero-Shot Learning for Action Recognition with Web-Scale Video Data
Authors: Kun Liu, Wu Liu, Huadong Ma, Wenbing Huang, Xiongxiong Dong
Date: 2017-10-20 00:00:00
Tasks: Action Recognition, Generalized Zero-Shot Learning, Temporal Action Localization, Zero-Shot Learning

Action recognition in surveillance video makes our life safer by detecting
the criminal events or predicting violent emergencies. However, efficient
action recognition is not free of difficulty. First, there are so many action
classes in daily life that we cannot pre-define all possible action classes
beforehand. Moreover, it is very hard to collect real-word videos for certain
particular actions such as steal and street fight due to legal restrictions and
privacy protection. These challenges make existing data-driven recognition
methods insufficient to attain desired performance. Zero-shot learning is
potential to be applied to solve these issues since it can perform
classification without positive example. Nevertheless, current zero-shot
learning algorithms have been studied under the unreasonable setting where seen
classes are absent during the testing phase. Motivated by this, we study the
task of action recognition in surveillance video under a more realistic
\emph{generalized zero-shot setting}, where testing data contains both seen and
unseen classes. To our best knowledge, this is the first work to study video
action recognition under the generalized zero-shot setting. We firstly perform
extensive empirical studies on several existing zero-shot leaning approaches
under this new setting on a web-scale video data. Our experimental results
demonstrate that, under the generalize setting, typical zero-shot learning
methods are no longer effective for the dataset we applied. Then, we propose a
method for action recognition by deploying generalized zero-shot learning,
which transfers the knowledge of web video to detect the anomalous actions in
surveillance videos. To verify the effectiveness of our proposed method, we
further construct a new surveillance video dataset consisting of nine action
classes related to the public safety situation.
Finite-dimensional Gaussian approximation with linear inequality constraints
Authors: Andrés F. López-Lopera, François Bachoc, Nicolas Durrande, Olivier Roustant
Date: 2017-10-20 00:00:00
Tasks: parameter estimation, Uncertainty Quantification

Introducing inequality constraints in Gaussian process (GP) models can lead
to more realistic uncertainties in learning a great variety of real-world
problems. We consider the finite-dimensional Gaussian approach from Maatouk and
Bay (2017) which can satisfy inequality conditions everywhere (either
boundedness, monotonicity or convexity). Our contributions are threefold.
First, we extend their approach in order to deal with general sets of linear
inequalities. Second, we explore several Markov Chain Monte Carlo (MCMC)
techniques to approximate the posterior distribution. Third, we investigate
theoretical and numerical properties of the constrained likelihood for
covariance parameter estimation. According to experiments on both artificial
and real data, our full framework together with a Hamiltonian Monte Carlo-based
sampler provides efficient results on both data fitting and uncertainty
quantification.
Multi-Objective Approaches to Markov Decision Processes with Uncertain Transition Parameters
Authors: Dimitri Scheftelowitsch, Peter Buchholz, Vahid Hashemi, Holger Hermanns
Date: 2017-10-20 00:00:00
Tasks: Decision Making

Markov decision processes (MDPs) are a popular model for performance analysis
and optimization of stochastic systems. The parameters of stochastic behavior
of MDPs are estimates from empirical observations of a system; their values are
not known precisely. Different types of MDPs with uncertain, imprecise or
bounded transition rates or probabilities and rewards exist in the literature.
  Commonly, analysis of models with uncertainties amounts to searching for the
most robust policy which means that the goal is to generate a policy with the
greatest lower bound on performance (or, symmetrically, the lowest upper bound
on costs). However, hedging against an unlikely worst case may lead to losses
in other situations. In general, one is interested in policies that behave well
in all situations which results in a multi-objective view on decision making.
  In this paper, we consider policies for the expected discounted reward
measure of MDPs with uncertain parameters. In particular, the approach is
defined for bounded-parameter MDPs (BMDPs) [8]. In this setting the worst, best
and average case performances of a policy are analyzed simultaneously, which
yields a multi-scenario multi-objective optimization problem. The paper
presents and evaluates approaches to compute the pure Pareto optimal policies
in the value vector space.
A Semantically Motivated Approach to Compute ROUGE Scores
Authors: Elaheh ShafieiBavani, Mohammad Ebrahimi, Raymond Wong, Fang Chen
Date: 2017-10-20 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity, Text Summarization

ROUGE is one of the first and most widely used evaluation metrics for text
summarization. However, its assessment merely relies on surface similarities
between peer and model summaries. Consequently, ROUGE is unable to fairly
evaluate abstractive summaries including lexical variations and paraphrasing.
Exploring the effectiveness of lexical resource-based models to address this
issue, we adopt a graph-based algorithm into ROUGE to capture the semantic
similarities between peer and model summaries. Our semantically motivated
approach computes ROUGE scores based on both lexical and semantic similarities.
Experiment results over TAC AESOP datasets indicate that exploiting the
lexico-semantic similarity of the words used in summaries would significantly
help ROUGE correlate better with human judgments.
Unified Backpropagation for Multi-Objective Deep Learning
Authors: Arash Shahriari
Date: 2017-10-20 00:00:00
Tasks: Classification, Deep Learning, General Classification

A common practice in most of deep convolutional neural architectures is to
employ fully-connected layers followed by Softmax activation to minimize
cross-entropy loss for the sake of classification. Recent studies show that
substitution or addition of the Softmax objective to the cost functions of
support vector machines or linear discriminant analysis is highly beneficial to
improve the classification performance in hybrid neural networks. We propose a
novel paradigm to link the optimization of several hybrid objectives through
unified backpropagation. This highly alleviates the burden of extensive
boosting for independent objective functions or complex formulation of
multiobjective gradients. Hybrid loss functions are linked by basic probability
assignment from evidence theory. We conduct our experiments for a variety of
scenarios and standard datasets to evaluate the advantage of our proposed
unification approach to deliver consistent improvements into the classification
performance of deep convolutional neural networks.
Distributed Deep Transfer Learning by Basic Probability Assignment
Authors: Arash Shahriari
Date: 2017-10-20 00:00:00
Tasks: Transfer Learning

Transfer learning is a popular practice in deep neural networks, but
fine-tuning of large number of parameters is a hard task due to the complex
wiring of neurons between splitting layers and imbalance distributions of data
in pretrained and transferred domains. The reconstruction of the original
wiring for the target domain is a heavy burden due to the size of
interconnections across neurons. We propose a distributed scheme that tunes the
convolutional filters individually while backpropagates them jointly by means
of basic probability assignment. Some of the most recent advances in evidence
theory show that in a vast variety of the imbalanced regimes, optimizing of
some proper objective functions derived from contingency matrices prevents
biases towards high-prior class distributions. Therefore, the original filters
get gradually transferred based on individual contributions to overall
performance of the target domain. This largely reduces the expected complexity
of transfer learning whilst highly improves precision. Our experiments on
standard benchmarks and scenarios confirm the consistent improvement of our
distributed deep transfer learning strategy.
Multipartite Pooling for Deep Convolutional Neural Networks
Authors: Arash Shahriari, Fatih Porikli
Date: 2017-10-20 00:00:00

We propose a novel pooling strategy that learns how to adaptively rank deep
convolutional features for selecting more informative representations. To this
end, we exploit discriminative analysis to project the features onto a space
spanned by the number of classes in the dataset under study. This maps the
notion of labels in the feature space into instances in the projected space. We
employ these projected distances as a measure to rank the existing features
with respect to their specific discriminant power for each individual class. We
then apply multipartite ranking to score the separability of the instances and
aggregate one-versus-all scores to compute an overall distinction score for
each feature. For the pooling, we pick features with the highest scores in a
pooling window instead of maximum, average or stochastic random assignments.
Our experiments on various benchmarks confirm that the proposed strategy of
multipartite pooling is highly beneficial to consistently improve the
performance of deep convolutional networks via better generalization of the
trained models for the test-time data.
Light-weight place recognition and loop detection using road markings
Authors: Oleksandr Bailo, Francois Rameau, In So Kweon
Date: 2017-10-20 00:00:00

In this paper, we propose an efficient algorithm for robust place recognition
and loop detection using camera information only. Our pipeline purely relies on
spatial localization and semantic information of road markings. The creation of
the database of road markings sequences is performed online, which makes the
method applicable for real-time loop closure for visual SLAM techniques.
Furthermore, our algorithm is robust to various weather conditions, occlusions
from vehicles, and shadows. We have performed an extensive number of
experiments which highlight the effectiveness and scalability of the proposed
method.
Differentially Private Empirical Risk Minimization with Input Perturbation
Authors: Kazuto Fukuchi, Quang Khai Tran, Jun Sakuma
Date: 2017-10-20 00:00:00

We propose a novel framework for the differentially private ERM, input
perturbation. Existing differentially private ERM implicitly assumed that the
data contributors submit their private data to a database expecting that the
database invokes a differentially private mechanism for publication of the
learned model. In input perturbation, each data contributor independently
randomizes her/his data by itself and submits the perturbed data to the
database. We show that the input perturbation framework theoretically
guarantees that the model learned with the randomized data eventually satisfies
differential privacy with the prescribed privacy parameters. At the same time,
input perturbation guarantees that local differential privacy is guaranteed to
the server. We also show that the excess risk bound of the model learned with
input perturbation is $O(1/n)$ under a certain condition, where $n$ is the
sample size. This is the same as the excess risk bound of the state-of-the-art.
Graph Convolution: A High-Order and Adaptive Approach
Authors: Zhenpeng Zhou, Xiaocheng Li
Date: 2017-06-29 00:00:00
Tasks: General Classification, Node Classification, Property Prediction, Vocal Bursts Intensity Prediction

In this paper, we presented a novel convolutional neural network framework
for graph modeling, with the introduction of two new modules specially designed
for graph-structured data: the $k$-th order convolution operator and the
adaptive filtering module. Importantly, our framework of High-order and
Adaptive Graph Convolutional Network (HA-GCN) is a general-purposed
architecture that fits various applications on both node and graph centrics, as
well as graph generative models. We conducted extensive experiments on
demonstrating the advantages of our framework. Particularly, our HA-GCN
outperforms the state-of-the-art models on node classification and molecule
property prediction tasks. It also generates 32% more real molecules on the
molecule generation task, both of which will significantly benefit real-world
applications such as material design and drug screening.
First-order Methods Almost Always Avoid Saddle Points
Authors: Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael. I. Jordan, Benjamin Recht
Date: 2017-10-20 00:00:00

We establish that first-order methods avoid saddle points for almost all
initializations. Our results apply to a wide variety of first-order methods,
including gradient descent, block coordinate descent, mirror descent and
variants thereof. The connecting thread is that such algorithms can be studied
from a dynamical systems perspective in which appropriate instantiations of the
Stable Manifold Theorem allow for a global stability analysis. Thus, neither
access to second-order derivative information nor randomness beyond
initialization is necessary to provably avoid saddle points.
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
Authors: Zhehuai Chen, Jasha Droppo, Jinyu Li, Wayne Xiong
Date: 2017-07-21 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, Speech Separation, Transfer Learning

Unsupervised single-channel overlapped speech recognition is one of the
hardest problems in automatic speech recognition (ASR). Permutation invariant
training (PIT) is a state of the art model-based approach, which applies a
single neural network to solve this single-input, multiple-output modeling
problem. We propose to advance the current state of the art by imposing a
modular structure on the neural network, applying a progressive pretraining
regimen, and improving the objective function with transfer learning and a
discriminative training criterion. The modular structure splits the problem
into three sub-tasks: frame-wise interpreting, utterance-level speaker tracing,
and speech recognition. The pretraining regimen uses these modules to solve
progressively harder tasks. Transfer learning leverages parallel clean speech
to improve the training targets for the network. Our discriminative training
formulation is a modification of standard formulations, that also penalizes
competing outputs of the system. Experiments are conducted on the artificial
overlapped Switchboard and hub5e-swb dataset. The proposed framework achieves
over 30% relative improvement of WER over both a strong jointly trained system,
PIT for ASR, and a separately optimized system, PIT for speech separation with
clean speech ASR model. The improvement comes from better model generalization,
training efficiency and the sequence level linguistic knowledge integration.
Ligand Pose Optimization with Atomic Grid-Based Convolutional Neural Networks
Authors: Matthew Ragoza, Lillian Turner, David Ryan Koes
Date: 2017-10-20 00:00:00
Tasks: Drug Discovery

Docking is an important tool in computational drug discovery that aims to
predict the binding pose of a ligand to a target protein through a combination
of pose scoring and optimization. A scoring function that is differentiable
with respect to atom positions can be used for both scoring and gradient-based
optimization of poses for docking. Using a differentiable grid-based atomic
representation as input, we demonstrate that a scoring function learned by
training a convolutional neural network (CNN) to identify binding poses can
also be applied to pose optimization. We also show that an iteratively-trained
CNN that includes poses optimized by the first CNN in its training set performs
even better at optimizing randomly initialized poses than either the first CNN
scoring function or AutoDock Vina.
Unsupervised Sentence Representations as Word Information Series: Revisiting TF--IDF
Authors: Ignacio Arroyo-Fernández, Carlos-Francisco Méndez-Cruz, Gerardo Sierra, Juan-Manuel Torres-Moreno, Grigori Sidorov
Date: 2017-10-17 00:00:00
Tasks: Open-Ended Question Answering, Semantic Textual Similarity, Sentence, STS, Word Embeddings

Sentence representation at the semantic level is a challenging task for
Natural Language Processing and Artificial Intelligence. Despite the advances
in word embeddings (i.e. word vector representations), capturing sentence
meaning is an open question due to complexities of semantic interactions among
words. In this paper, we present an embedding method, which is aimed at
learning unsupervised sentence representations from unlabeled text. We propose
an unsupervised method that models a sentence as a weighted series of word
embeddings. The weights of the word embeddings are fitted by using Shannon's
word entropies provided by the Term Frequency--Inverse Document Frequency
(TF--IDF) transform. The hyperparameters of the model can be selected according
to the properties of data (e.g. sentence length and textual gender).
Hyperparameter selection involves word embedding methods and dimensionalities,
as well as weighting schemata. Our method offers advantages over existing
methods: identifiable modules, short-term training, online inference of
(unseen) sentence representations, as well as independence from domain,
external knowledge and language resources. Results showed that our model
outperformed the state of the art in well-known Semantic Textual Similarity
(STS) benchmarks. Moreover, our model reached state-of-the-art performance when
compared to supervised and knowledge-based STS systems.