pwc-archive/papers-with-abstracts huggingface dataset:
Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks
Authors: Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, Jing Gao
Date: 2017-06-19 00:00:00
Tasks: Prediction

Predicting the future health information of patients from the historical
Electronic Health Records (EHR) is a core research task in the development of
personalized healthcare. Patient EHR data consist of sequences of visits over
time, where each visit contains multiple medical codes, including diagnosis,
medication, and procedure codes. The most important challenges for this task
are to model the temporality and high dimensionality of sequential EHR data and
to interpret the prediction results. Existing work solves this problem by
employing recurrent neural networks (RNNs) to model EHR data and utilizing
simple attention mechanism to interpret the results. However, RNN-based
approaches suffer from the problem that the performance of RNNs drops when the
length of sequences is large, and the relationships between subsequent visits
are ignored by current RNN-based approaches. To address these issues, we
propose {\sf Dipole}, an end-to-end, simple and robust model for predicting
patients' future health information. Dipole employs bidirectional recurrent
neural networks to remember all the information of both the past visits and the
future visits, and it introduces three attention mechanisms to measure the
relationships of different visits for the prediction. With the attention
mechanisms, Dipole can interpret the prediction results effectively. Dipole
also allows us to interpret the learned medical code representations which are
confirmed positively by medical experts. Experimental results on two real world
EHR datasets show that the proposed Dipole can significantly improve the
prediction accuracy compared with the state-of-the-art diagnosis prediction
approaches and provide clinically meaningful interpretation.
On Pairwise Clustering with Side Information
Authors: Stephen Pasteris, Fabio Vitale, Claudio Gentile, Mark Herbster
Date: 2017-06-19 00:00:00
Tasks: Clustering, Inductive Bias

Pairwise clustering, in general, partitions a set of items via a known
similarity function. In our treatment, clustering is modeled as a transductive
prediction problem. Thus rather than beginning with a known similarity
function, the function instead is hidden and the learner only receives a random
sample consisting of a subset of the pairwise similarities. An additional set
of pairwise side-information may be given to the learner, which then determines
the inductive bias of our algorithms. We measure performance not based on the
recovery of the hidden similarity function, but instead on how well we classify
each item. We give tight bounds on the number of misclassifications. We provide
two algorithms. The first algorithm SACA is a simple agglomerative clustering
algorithm which runs in near linear time, and which serves as a baseline for
our analyses. Whereas the second algorithm, RGCA, enables the incorporation of
side-information which may lead to improved bounds at the cost of a longer
running time.
Using Entropy Estimates for DAG-Based Ontologies
Authors: Andrew Warren, Joao Setubal
Date: 2014-03-19 00:00:00
Tasks: Information Retrieval, Language Modeling, Language Modelling, Retrieval, Semantic Similarity, Semantic Textual Similarity

Motivation: Entropy measurements on hierarchical structures have been used in
methods for information retrieval and natural language modeling. Here we
explore its application to semantic similarity. By finding shared ontology
terms, semantic similarity can be established between annotated genes. A common
procedure for establishing semantic similarity is to calculate the
descriptiveness (information content) of ontology terms and use these values to
determine the similarity of annotations. Most often information content is
calculated for an ontology term by analyzing its frequency in an annotation
corpus. The inherent problems in using these values to model functional
similarity motivates our work. Summary: We present a novel calculation for
establishing the entropy of a DAG-based ontology, which can be used in an
alternative method for establishing the information content of its terms. We
also compare our IC metric to two others using semantic and sequence
similarity.
Analysis of a Natural Gradient Algorithm on Monotonic Convex-Quadratic-Composite Functions
Authors: Youhei Akimoto
Date: 2012-04-18 00:00:00

In this paper we investigate the convergence properties of a variant of the
Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Our study is based on
the recent theoretical foundation that the pure rank-mu update CMA-ES performs
the natural gradient descent on the parameter space of Gaussian distributions.
We derive a novel variant of the natural gradient method where the parameters
of the Gaussian distribution are updated along the natural gradient to improve
a newly defined function on the parameter space. We study this algorithm on
composites of a monotone function with a convex quadratic function. We prove
that our algorithm adapts the covariance matrix so that it becomes proportional
to the inverse of the Hessian of the original objective function. We also show
the speed of covariance matrix adaptation and the speed of convergence of the
parameters. We introduce a stochastic algorithm that approximates the natural
gradient with finite samples and present some simulated results to evaluate how
precisely the stochastic algorithm approximates the deterministic, ideal one
under finite samples and to see how similarly our algorithm and the CMA-ES
perform.
A Kind of Affine Weighted Moment Invariants
Authors: Hanlin Mo, You Hao, Shirui Li, Hua Li
Date: 2017-06-05 00:00:00
Tasks: Image Retrieval, Retrieval

A new kind of geometric invariants is proposed in this paper, which is called
affine weighted moment invariant (AWMI). By combination of local affine
differential invariants and a framework of global integral, they can more
effectively extract features of images and help to increase the number of
low-order invariants and to decrease the calculating cost. The experimental
results show that AWMIs have good stability and distinguishability and achieve
better results in image retrieval than traditional moment invariants. An
extension to 3D is straightforward.
Dex: Incremental Learning for Complex Environments in Deep Reinforcement Learning
Authors: Nick Erickson, Qi Zhao
Date: 2017-06-19 00:00:00
Tasks: Continual Learning, Deep Reinforcement Learning, General Reinforcement Learning, Incremental Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper introduces Dex, a reinforcement learning environment toolkit
specialized for training and evaluation of continual learning methods as well
as general reinforcement learning problems. We also present the novel continual
learning method of incremental learning, where a challenging environment is
solved using optimal weight initialization learned from first solving a similar
easier environment. We show that incremental learning can produce vastly
superior results than standard methods by providing a strong baseline method
across ten Dex environments. We finally develop a saliency method for
qualitative analysis of reinforcement learning, which shows the impact
incremental learning has on network attention.
Computing Web-scale Topic Models using an Asynchronous Parameter Server
Authors: Rolf Jagerman, Carsten Eickhoff, Maarten de Rijke
Date: 2016-05-24 00:00:00
Tasks: Information Retrieval, Retrieval, Topic Models

Topic models such as Latent Dirichlet Allocation (LDA) have been widely used
in information retrieval for tasks ranging from smoothing and feedback methods
to tools for exploratory search and discovery. However, classical methods for
inferring topic models do not scale up to the massive size of today's publicly
available Web-scale data sets. The state-of-the-art approaches rely on custom
strategies, implementations and hardware to facilitate their asynchronous,
communication-intensive workloads.
  We present APS-LDA, which integrates state-of-the-art topic modeling with
cluster computing frameworks such as Spark using a novel asynchronous parameter
server. Advantages of this integration include convenient usage of existing
data processing pipelines and eliminating the need for disk writes as data can
be kept in memory from start to finish. Our goal is not to outperform highly
customized implementations, but to propose a general high-performance topic
modeling framework that can easily be used in today's data processing
pipelines. We compare APS-LDA to the existing Spark LDA implementations and
show that our system can, on a 480-core cluster, process up to 135 times more
data and 10 times more topics without sacrificing model quality.
Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data
Authors: Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher
Date: 2017-06-18 00:00:00
Proceeding: NeurIPS 2017 12

Several important applications, such as streaming PCA and semidefinite
programming, involve a large-scale positive-semidefinite (psd) matrix that is
presented as a sequence of linear updates. Because of storage limitations, it
may only be possible to retain a sketch of the psd matrix. This paper develops
a new algorithm for fixed-rank psd approximation from a sketch. The approach
combines the Nystrom approximation with a novel mechanism for rank truncation.
Theoretical analysis establishes that the proposed method can achieve any
prescribed relative error in the Schatten 1-norm and that it exploits the
spectral decay of the input matrix. Computer experiments show that the proposed
method dominates alternative techniques for fixed-rank psd matrix approximation
across a wide range of examples.
Data set operations to hide decision tree rules
Authors: Dimitris Kalles, Vassilios S. Verykios, Georgios Feretzakis, Athanasios Papagelis
Date: 2017-06-18 00:00:00
Tasks: General Classification

This paper focuses on preserving the privacy of sensitive patterns when
inducing decision trees. We adopt a record augmentation approach for hiding
sensitive classification rules in binary datasets. Such a hiding methodology is
preferred over other heuristic solutions like output perturbation or
cryptographic techniques - which restrict the usability of the data - since the
raw data itself is readily available for public use. We show some key lemmas
which are related to the hiding process and we also demonstrate the methodology
with an example and an indicative experiment using a prototype hiding tool.
Addressing Item-Cold Start Problem in Recommendation Systems using Model Based Approach and Deep Learning
Authors: Ivica Obadić, Gjorgji Madjarov, Ivica Dimitrovski, Dejan Gjorgjevikj
Date: 2017-06-18 00:00:00
Tasks: Recommendation Systems

Traditional recommendation systems rely on past usage data in order to
generate new recommendations. Those approaches fail to generate sensible
recommendations for new users and items into the system due to missing
information about their past interactions. In this paper, we propose a solution
for successfully addressing item-cold start problem which uses model-based
approach and recent advances in deep learning. In particular, we use latent
factor model for recommendation, and predict the latent factors from item's
descriptions using convolutional neural network when they cannot be obtained
from usage data. Latent factors obtained by applying matrix factorization to
the available usage data are used as ground truth to train the convolutional
neural network. To create latent factor representations for the new items, the
convolutional neural network uses their textual description. The results from
the experiments reveal that the proposed approach significantly outperforms
several baseline estimators.
Using Deep Networks for Drone Detection
Authors: Cemal Aker, Sinan Kalkan
Date: 2017-06-18 00:00:00
Tasks: object-detection, Object Detection

Drone detection is the problem of finding the smallest rectangle that
encloses the drone(s) in a video sequence. In this study, we propose a solution
using an end-to-end object detection model based on convolutional neural
networks. To solve the scarce data problem for training the network, we propose
an algorithm for creating an extensive artificial dataset by combining
background-subtracted real images. With this approach, we can achieve precision
and recall values both of which are high at the same time.
Detecting Large Concept Extensions for Conceptual Analysis
Authors: Louis Chartrand, Jackie C. K. Cheung, Mohamed Bouguessa
Date: 2017-06-18 00:00:00

When performing a conceptual analysis of a concept, philosophers are
interested in all forms of expression of a concept in a text---be it direct or
indirect, explicit or implicit. In this paper, we experiment with topic-based
methods of automating the detection of concept expressions in order to
facilitate philosophical conceptual analysis. We propose six methods based on
LDA, and evaluate them on a new corpus of court decision that we had annotated
by experts and non-experts. Our results indicate that these methods can yield
important improvements over the keyword heuristic, which is often used as a
concept detection heuristic in many contexts. While more work remains to be
done, this indicates that detecting concepts through topics can serve as a
general-purpose method for at least some forms of concept expression that are
not captured using naive keyword approaches.
Tversky loss function for image segmentation using 3D fully convolutional deep networks
Authors: Seyed Sadegh Mohseni Salehi, Deniz Erdogmus, Ali Gholipour
Date: 2017-06-18 00:00:00
Tasks: Image Segmentation, Lesion Segmentation, Segmentation, Semantic Segmentation

Fully convolutional deep neural networks carry out excellent potential for
fast and accurate image segmentation. One of the main challenges in training
these networks is data imbalance, which is particularly problematic in medical
imaging applications such as lesion segmentation where the number of lesion
voxels is often much lower than the number of non-lesion voxels. Training with
unbalanced data can lead to predictions that are severely biased towards high
precision but low recall (sensitivity), which is undesired especially in
medical applications where false negatives are much less tolerable than false
positives. Several methods have been proposed to deal with this problem
including balanced sampling, two step training, sample re-weighting, and
similarity loss functions. In this paper, we propose a generalized loss
function based on the Tversky index to address the issue of data imbalance and
achieve much better trade-off between precision and recall in training 3D fully
convolutional deep neural networks. Experimental results in multiple sclerosis
lesion segmentation on magnetic resonance images show improved F2 score, Dice
coefficient, and the area under the precision-recall curve in test data. Based
on these results we suggest Tversky loss function as a generalized framework to
effectively train deep neural networks.
Towards the Improvement of Automated Scientific Document Categorization by Deep Learning
Authors: Thomas Krause
Date: 2017-06-18 00:00:00
Tasks: Deep Learning, General Classification

This master thesis describes an algorithm for automated categorization of
scientific documents using deep learning techniques and compares the results to
the results of existing classification algorithms. As an additional goal a
reusable API is to be developed allowing the automation of classification tasks
in existing software. A design will be proposed using a convolutional neural
network as a classifier and integrating this into a REST based API. This is
then used as the basis for an actual proof of concept implementation presented
as well in this thesis. It will be shown that the deep learning classifier
provides very good result in the context of multi-class document categorization
and that it is feasible to integrate such classifiers into a larger ecosystem
using REST based services.
Environmental Modeling Framework using Stacked Gaussian Processes
Authors: Kareem Abdelfatah, Junshu Bao, Gabriel Terejanu
Date: 2016-12-09 00:00:00
Tasks: Gaussian Processes

A network of independently trained Gaussian processes (StackedGP) is
introduced to obtain predictions of quantities of interest with quantified
uncertainties. The main applications of the StackedGP framework are to
integrate different datasets through model composition, enhance predictions of
quantities of interest through a cascade of intermediate predictions, and to
propagate uncertainties through emulated dynamical systems driven by uncertain
forcing variables. By using analytical first and second-order moments of a
Gaussian process with uncertain inputs using squared exponential and polynomial
kernels, approximated expectations of quantities of interests that require an
arbitrary composition of functions can be obtained. The StackedGP model is
extended to any number of layers and nodes per layer, and it provides
flexibility in kernel selection for the input nodes. The proposed nonparametric
stacked model is validated using synthetic datasets, and its performance in
model composition and cascading predictions is measured in two applications
using real data.
Discovering Reliable Approximate Functional Dependencies
Authors: Panagiotis Mandros, Mario Boley, Jilles Vreeken
Date: 2017-05-25 00:00:00
Tasks: Attribute

Given a database and a target attribute of interest, how can we tell whether
there exists a functional, or approximately functional dependence of the target
on any set of other attributes in the data? How can we reliably, without bias
to sample size or dimensionality, measure the strength of such a dependence?
And, how can we efficiently discover the optimal or $\alpha$-approximate
top-$k$ dependencies? These are exactly the questions we answer in this paper.
  As we want to be agnostic on the form of the dependence, we adopt an
information-theoretic approach, and construct a reliable, bias correcting score
that can be efficiently computed. Moreover, we give an effective optimistic
estimator of this score, by which for the first time we can mine the
approximate functional dependencies from data with guarantees of optimality.
Empirical evaluation shows that the derived score achieves a good bias for
variance trade-off, can be used within an efficient discovery algorithm, and
indeed discovers meaningful dependencies. Most important, it remains reliable
in the face of data sparsity.
Sparse Neural Networks Topologies
Authors: Alfred Bourely, John Patrick Boueri, Krzysztof Choromonski
Date: 2017-06-18 00:00:00

We propose Sparse Neural Network architectures that are based on random or
structured bipartite graph topologies. Sparse architectures provide compression
of the models learned and speed-ups of computations, they can also surpass
their unstructured or fully connected counterparts. As we show, even more
compact topologies of the so-called SNN (Sparse Neural Network) can be achieved
with the use of structured graphs of connections between consecutive layers of
neurons. In this paper, we investigate how the accuracy and training speed of
the models depend on the topology and sparsity of the neural network. Previous
approaches using sparcity are all based on fully connected neural network
models and create sparcity during training phase, instead we explicitly define
a sparse architectures of connections before the training. Building compact
neural network models is coherent with empirical observations showing that
there is much redundancy in learned neural network models. We show
experimentally that the accuracy of the models learned with neural networks
depends on expander-like properties of the underlying topologies such as the
spectral gap and algebraic connectivity rather than the density of the graphs
of connections.
On the convergence of mirror descent beyond stochastic convex programming
Authors: Zhengyuan Zhou, Panayotis Mertikopoulos, Nicholas Bambos, Stephen Boyd, Peter Glynn
Date: 2017-06-18 00:00:00
Tasks: Stochastic Optimization

In this paper, we examine the convergence of mirror descent in a class of
stochastic optimization problems that are not necessarily convex (or even
quasi-convex), and which we call variationally coherent. Since the standard
technique of "ergodic averaging" offers no tangible benefits beyond convex
programming, we focus directly on the algorithm's last generated sample (its
"last iterate"), and we show that it converges with probabiility $1$ if the
underlying problem is coherent. We further consider a localized version of
variational coherence which ensures local convergence of stochastic mirror
descent (SMD) with high probability. These results contribute to the landscape
of non-convex stochastic optimization by showing that (quasi-)convexity is not
essential for convergence to a global minimum: rather, variational coherence, a
much weaker requirement, suffices. Finally, building on the above, we reveal an
interesting insight regarding the convergence speed of SMD: in problems with
sharp minima (such as generic linear programs or concave minimization
problems), SMD reaches a minimum point in a finite number of steps (a.s.), even
in the presence of persistent gradient noise. This result is to be contrasted
with existing black-box convergence rate estimates that are only asymptotic.
FreezeOut: Accelerate Training by Progressively Freezing Layers
Authors: Andrew Brock, Theodore Lim, J. M. Ritchie, Nick Weston
Date: 2017-06-15 00:00:00

The early layers of a deep neural net have the fewest parameters, but take up
the most computation. In this extended abstract, we propose to only train the
hidden layers for a set portion of the training run, freezing them out
one-by-one and excluding them from the backward pass. Through experiments on
CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20%
wall-clock time during training with 3% loss in accuracy for DenseNets, a 20%
speedup without loss of accuracy for ResNets, and no improvement for VGG
networks. Our code is publicly available at
https://github.com/ajbrock/FreezeOut
Image Matching via Loopy RNN
Authors: Donghao Luo, Bingbing Ni, Yichao Yan, Xiaokang Yang
Date: 2017-06-10 00:00:00

Most existing matching algorithms are one-off algorithms, i.e., they usually
measure the distance between the two image feature representation vectors for
only one time. In contrast, human's vision system achieves this task, i.e.,
image matching, by recursively looking at specific/related parts of both images
and then making the final judgement. Towards this end, we propose a novel loopy
recurrent neural network (Loopy RNN), which is capable of aggregating
relationship information of two input images in a progressive/iterative manner
and outputting the consolidated matching score in the final iteration. A Loopy
RNN features two uniqueness. First, built on conventional long short-term
memory (LSTM) nodes, it links the output gate of the tail node to the input
gate of the head node, thus it brings up symmetry property required for
matching. Second, a monotonous loss designed for the proposed network
guarantees increasing confidence during the recursive matching process.
Extensive experiments on several image matching benchmarks demonstrate the
great potential of the proposed method.