pwc-archive/papers-with-abstracts huggingface dataset:
Training of spiking neural networks based on information theoretic costs
Authors: Oleg Y. Sinyavskiy
Date: 2016-02-15 00:00:00
Tasks: Reinforcement Learning

Spiking neural network is a type of artificial neural network in which
neurons communicate between each other with spikes. Spikes are identical
Boolean events characterized by the time of their arrival. A spiking neuron has
internal dynamics and responds to the history of inputs as opposed to the
current inputs only. Because of such properties a spiking neural network has
rich intrinsic capabilities to process spatiotemporal data. However, because
the spikes are discontinuous 'yes or no' events, it is not trivial to apply
traditional training procedures such as gradient descend to the spiking
neurons. In this thesis we propose to use stochastic spiking neuron models in
which probability of a spiking output is a continuous function of parameters.
We formulate several learning tasks as minimization of certain
information-theoretic cost functions that use spiking output probability
distributions. We develop a generalized description of the stochastic spiking
neuron and a new spiking neuron model that allows to flexibly process rich
spatiotemporal data. We formulate and derive learning rules for the following
tasks:
  - a supervised learning task of detecting a spatiotemporal pattern as a
minimization of the negative log-likelihood (the surprisal) of the neuron's
output
  - an unsupervised learning task of increasing the stability of neurons output
as a minimization of the entropy
  - a reinforcement learning task of controlling an agent as a modulated
optimization of filtered surprisal of the neuron's output.
  We test the derived learning rules in several experiments such as
spatiotemporal pattern detection, spatiotemporal data storing and recall with
autoassociative memory, combination of supervised and unsupervised learning to
speed up the learning process, adaptive control of simple virtual agents in
changing environments.
A Model of Selective Advantage for the Efficient Inference of Cancer Clonal Evolution
Authors: Daniele Ramazzotti
Date: 2016-02-15 00:00:00
Tasks: Learning Theory

Recently, there has been a resurgence of interest in rigorous algorithms for
the inference of cancer progression from genomic data. The motivations are
manifold: (i) growing NGS and single cell data from cancer patients, (ii) need
for novel Data Science and Machine Learning algorithms to infer models of
cancer progression, and (iii) a desire to understand the temporal and
heterogeneous structure of tumor to tame its progression by efficacious
therapeutic intervention. This thesis presents a multi-disciplinary effort to
model tumor progression involving successive accumulation of genetic
alterations, each resulting populations manifesting themselves in a cancer
phenotype. The framework presented in this work along with algorithms derived
from it, represents a novel approach for inferring cancer progression, whose
accuracy and convergence rates surpass the existing techniques. The approach
derives its power from several fields including algorithms in machine learning,
theory of causality and cancer biology. Furthermore, a modular pipeline to
extract ensemble-level progression models from sequenced cancer genomes is
proposed. The pipeline combines state-of-the-art techniques for sample
stratification, driver selection, identification of fitness-equivalent
exclusive alterations and progression model inference. Furthermore, the results
are validated by synthetic data with realistic generative models, and
empirically interpreted in the context of real cancer datasets; in the later
case, biologically significant conclusions are also highlighted. Specifically,
it demonstrates the pipeline's ability to reproduce much of the knowledge on
colorectal cancer, as well as to suggest novel hypotheses. Lastly, it also
proves that the proposed framework can be applied to reconstruct the
evolutionary history of cancer clones in single patients, as illustrated by an
example from clear cell renal carcinomas.
Efficient Representation of Low-Dimensional Manifolds using Deep Networks
Authors: Ronen Basri, David Jacobs
Date: 2016-02-15 00:00:00

We consider the ability of deep neural networks to represent data that lies
near a low-dimensional manifold in a high-dimensional space. We show that deep
networks can efficiently extract the intrinsic, low-dimensional coordinates of
such data. We first show that the first two layers of a deep network can
exactly embed points lying on a monotonic chain, a special type of piecewise
linear manifold, mapping them to a low-dimensional Euclidean space. Remarkably,
the network can do this using an almost optimal number of parameters. We also
show that this network projects nearby points onto the manifold and then embeds
them with little error. We then extend these results to more general manifolds.
A Test of Relative Similarity For Model Selection in Generative Models
Authors: Wacha Bounliphone, Eugene Belilovsky, Matthew B. Blaschko, Ioannis Antonoglou, Arthur Gretton
Date: 2015-11-14 00:00:00
Tasks: Model Selection

Probabilistic generative models provide a powerful framework for representing
data that avoids the expense of manual annotation typically needed by
discriminative approaches. Model selection in this generative setting can be
challenging, however, particularly when likelihoods are not easily accessible.
To address this issue, we introduce a statistical test of relative similarity,
which is used to determine which of two models generates samples that are
significantly closer to a real-world reference dataset of interest. We use as
our test statistic the difference in maximum mean discrepancies (MMDs) between
the reference dataset and each model dataset, and derive a powerful,
low-variance test based on the joint asymptotic distribution of the MMDs
between each reference-model pair. In experiments on deep generative models,
including the variational auto-encoder and generative moment matching network,
the tests provide a meaningful ranking of model performance as a function of
parameter and training settings.
Maximin Action Identification: A New Bandit Framework for Games
Authors: Aurélien Garivier, Emilie Kaufmann, Wouter Koolen
Date: 2016-02-15 00:00:00

We study an original problem of pure exploration in a strategic bandit model
motivated by Monte Carlo Tree Search. It consists in identifying the best
action in a game, when the player may sample random outcomes of sequentially
chosen pairs of actions. We propose two strategies for the fixed-confidence
setting: Maximin-LUCB, based on lower-and upper-confidence bounds; and
Maximin-Racing, which operates by successively eliminating the sub-optimal
actions. We discuss the sample complexity of both methods and compare their
performance empirically. We sketch a lower bound analysis, and possible
connections to an optimal algorithm.
Towards reducing the multidimensionality of OLAP cubes using the Evolutionary Algorithms and Factor Analysis Methods
Authors: Sami Naouali, Semeh Ben Salem
Date: 2016-02-15 00:00:00
Tasks: Evolutionary Algorithms

Data Warehouses are structures with large amount of data collected from
heterogeneous sources to be used in a decision support system. Data Warehouses
analysis identifies hidden patterns initially unexpected which analysis
requires great memory and computation cost. Data reduction methods were
proposed to make this analysis easier. In this paper, we present a hybrid
approach based on Genetic Algorithms (GA) as Evolutionary Algorithms and the
Multiple Correspondence Analysis (MCA) as Analysis Factor Methods to conduct
this reduction. Our approach identifies reduced subset of dimensions from the
initial subset p where p'<p where it is proposed to find the profile fact that
is the closest to reference. GAs identify the possible subsets and the Khi
formula of the ACM evaluates the quality of each subset. The study is based on
a distance measurement between the reference and n facts profile extracted from
the Warehouses.
Generalization and Exploration via Randomized Value Functions
Authors: Ian Osband, Benjamin Van Roy, Zheng Wen
Date: 2014-02-04 00:00:00
Tasks: Efficient Exploration, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose randomized least-squares value iteration (RLSVI) -- a new
reinforcement learning algorithm designed to explore and generalize efficiently
via linearly parameterized value functions. We explain why versions of
least-squares value iteration that use Boltzmann or epsilon-greedy exploration
can be highly inefficient, and we present computational results that
demonstrate dramatic efficiency gains enjoyed by RLSVI. Further, we establish
an upper bound on the expected regret of RLSVI that demonstrates
near-optimality in a tabula rasa learning context. More broadly, our results
suggest that randomized value functions offer a promising approach to tackling
a critical challenge in reinforcement learning: synthesizing efficient
exploration and effective generalization.
Edge Detection for Pattern Recognition: A Survey
Authors: Alex Pappachen James
Date: 2016-02-15 00:00:00
Tasks: Edge Detection, Survey

This review provides an overview of the literature on the edge detection
methods for pattern recognition that inspire from the understanding of human
vision. We note that edge detection is one of the most fundamental process
within the low level vision and provides the basis for the higher level visual
intelligence in primates. The recognition of the patterns within the images
relate closely to the spatiotemporal processes of edge formations, and its
implementation needs a crossdisciplanry approach in neuroscience, computing and
pattern recognition. In this review, the edge detectors are grouped in as edge
features, gradients and sketch models, and some example applications are
provided for reference. We note a significant increase in the amount of
published research in the last decade that utilizes edge features in a wide
range of problems in computer vision and image understanding having a direct
implication to pattern recognition with images.
An Efficient Algorithm for Video Super-Resolution Based On a Sequential Model
Authors: Patrick Héas, Angélique Drémeau, Cédric Herzet
Date: 2015-06-01 00:00:00
Tasks: Super-Resolution, Video Super-Resolution

In this work, we propose a novel procedure for video super-resolution, that
is the recovery of a sequence of high-resolution images from its low-resolution
counterpart. Our approach is based on a "sequential" model (i.e., each
high-resolution frame is supposed to be a displaced version of the preceding
one) and considers the use of sparsity-enforcing priors. Both the recovery of
the high-resolution images and the motion fields relating them is tackled. This
leads to a large-dimensional, non-convex and non-smooth problem. We propose an
algorithmic framework to address the latter. Our approach relies on fast
gradient evaluation methods and modern optimization techniques for
non-differentiable/non-convex problems. Unlike some other previous works, we
show that there exists a provably-convergent method with a complexity linear in
the problem dimensions. We assess the proposed optimization method on {several
video benchmarks and emphasize its good performance with respect to the state
of the art.}
Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization
Authors: Toshiyuki Takada, Hiroyuki Hanada, Yoshiji Yamada, Jun Sakuma, Ichiro Takeuchi
Date: 2016-02-15 00:00:00
Tasks: Privacy Preserving

Privacy concern has been increasingly important in many machine learning (ML)
problems. We study empirical risk minimization (ERM) problems under secure
multi-party computation (MPC) frameworks. Main technical tools for MPC have
been developed based on cryptography. One of limitations in current
cryptographically private ML is that it is computationally intractable to
evaluate non-linear functions such as logarithmic functions or exponential
functions. Therefore, for a class of ERM problems such as logistic regression
in which non-linear function evaluations are required, one can only obtain
approximate solutions. In this paper, we introduce a novel cryptographically
private tool called secure approximation guarantee (SAG) method. The key
property of SAG method is that, given an arbitrary approximate solution, it can
provide a non-probabilistic assumption-free bound on the approximation quality
under cryptographically secure computation framework. We demonstrate the
benefit of the SAG method by applying it to several problems including a
practical privacy-preserving data analysis task on genomic and clinical
information.
Personalized Expertise Search at LinkedIn
Authors: Viet Ha-Thuc, Ganesh Venkataraman, Mario Rodriguez, Shakti Sinha, Senthil Sundaram, Lin Guo
Date: 2016-02-15 00:00:00
Tasks: Collaborative Filtering

LinkedIn is the largest professional network with more than 350 million
members. As the member base increases, searching for experts becomes more and
more challenging. In this paper, we propose an approach to address the problem
of personalized expertise search on LinkedIn, particularly for exploratory
search queries containing {\it skills}. In the offline phase, we introduce a
collaborative filtering approach based on matrix factorization. Our approach
estimates expertise scores for both the skills that members list on their
profiles as well as the skills they are likely to have but do not explicitly
list. In the online phase (at query time) we use expertise scores on these
skills as a feature in combination with other features to rank the results. To
learn the personalized ranking function, we propose a heuristic to extract
training data from search logs while handling position and sample selection
biases. We tested our models on two products - LinkedIn homepage and LinkedIn
recruiter. A/B tests showed significant improvements in click through rates -
31% for CTR@1 for recruiter (18% for homepage) as well as downstream messages
sent from search - 37% for recruiter (20% for homepage). As of writing this
paper, these models serve nearly all live traffic for skills search on LinkedIn
homepage as well as LinkedIn recruiter.
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
Authors: Song Han, Huizi Mao, William J. Dally
Date: 2015-10-01 00:00:00
Tasks: CPU, GPU, Network Pruning, Quantization

Neural networks are both computationally intensive and memory intensive,
making them difficult to deploy on embedded systems with limited hardware
resources. To address this limitation, we introduce "deep compression", a three
stage pipeline: pruning, trained quantization and Huffman coding, that work
together to reduce the storage requirement of neural networks by 35x to 49x
without affecting their accuracy. Our method first prunes the network by
learning only the important connections. Next, we quantize the weights to
enforce weight sharing, finally, we apply Huffman coding. After the first two
steps we retrain the network to fine tune the remaining connections and the
quantized centroids. Pruning, reduces the number of connections by 9x to 13x;
Quantization then reduces the number of bits that represent each connection
from 32 to 5. On the ImageNet dataset, our method reduced the storage required
by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method
reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of
accuracy. This allows fitting the model into on-chip SRAM cache rather than
off-chip DRAM memory. Our compression method also facilitates the use of
complex neural networks in mobile applications where application size and
download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU,
compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy
efficiency.
Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
Authors: Marten Wegkamp, Yue Zhao
Date: 2013-05-28 00:00:00

We study the adaptive estimation of copula correlation matrix $\Sigma$ for
the semi-parametric elliptical copula model. In this context, the correlations
are connected to Kendall's tau through a sine function transformation. Hence, a
natural estimate for $\Sigma$ is the plug-in estimator $\hat{\Sigma}$ with
Kendall's tau statistic. We first obtain a sharp bound on the operator norm of
$\hat{\Sigma}-\Sigma$. Then we study a factor model of $\Sigma$, for which we
propose a refined estimator $\widetilde{\Sigma}$ by fitting a low-rank matrix
plus a diagonal matrix to $\hat{\Sigma}$ using least squares with a nuclear
norm penalty on the low-rank matrix. The bound on the operator norm of
$\hat{\Sigma}-\Sigma$ serves to scale the penalty term, and we obtain finite
sample oracle inequalities for $\widetilde{\Sigma}$. We also consider an
elementary factor copula model of $\Sigma$, for which we propose closed-form
estimators. All of our estimation procedures are entirely data-driven.
Adversarial Top-$K$ Ranking
Authors: Changho Suh, Vincent Y. F. Tan, Renbo Zhao
Date: 2016-02-15 00:00:00
Tasks: Tensor Decomposition

We study the top-$K$ ranking problem where the goal is to recover the set of
top-$K$ ranked items out of a large collection of items based on partially
revealed preferences. We consider an adversarial crowdsourced setting where
there are two population sets, and pairwise comparison samples drawn from one
of the populations follow the standard Bradley-Terry-Luce model (i.e., the
chance of item $i$ beating item $j$ is proportional to the relative score of
item $i$ to item $j$), while in the other population, the corresponding chance
is inversely proportional to the relative score. When the relative size of the
two populations is known, we characterize the minimax limit on the sample size
required (up to a constant) for reliably identifying the top-$K$ items, and
demonstrate how it scales with the relative size. Moreover, by leveraging a
tensor decomposition method for disambiguating mixture distributions, we extend
our result to the more realistic scenario in which the relative population size
is unknown, thus establishing an upper bound on the fundamental limit of the
sample size for recovering the top-$K$ set.
Trust from the past: Bayesian Personalized Ranking based Link Prediction in Knowledge Graphs
Authors: Baichuan Zhang, Sutanay Choudhury, Mohammad Al Hasan, Xia Ning, Khushbu Agarwal, Sumit Purohit, Paola Pesntez Cabrera
Date: 2016-01-14 00:00:00
Tasks: Knowledge Graphs, Link Prediction, Prediction

Link prediction, or predicting the likelihood of a link in a knowledge graph
based on its existing state is a key research task. It differs from a
traditional link prediction task in that the links in a knowledge graph are
categorized into different predicates and the link prediction performance of
different predicates in a knowledge graph generally varies widely. In this
work, we propose a latent feature embedding based link prediction model which
considers the prediction task for each predicate disjointly. To learn the model
parameters it utilizes a Bayesian personalized ranking based optimization
technique. Experimental results on large-scale knowledge bases such as YAGO2
show that our link prediction approach achieves substantially higher
performance than several state-of-art approaches. We also show that for a given
predicate the topological properties of the knowledge graph induced by the
given predicate edges are key indicators of the link prediction performance of
that predicate in the knowledge graph.
Safe Pattern Pruning: An Efficient Approach for Predictive Pattern Mining
Authors: Kazuya Nakagawa, Shinya Suzumura, Masayuki Karasuyama, Koji Tsuda, Ichiro Takeuchi
Date: 2016-02-15 00:00:00
Tasks: Graph Mining

In this paper we study predictive pattern mining problems where the goal is
to construct a predictive model based on a subset of predictive patterns in the
database. Our main contribution is to introduce a novel method called safe
pattern pruning (SPP) for a class of predictive pattern mining problems. The
SPP method allows us to efficiently find a superset of all the predictive
patterns in the database that are needed for the optimal predictive model. The
advantage of the SPP method over existing boosting-type method is that the
former can find the superset by a single search over the database, while the
latter requires multiple searches. The SPP method is inspired by recent
development of safe feature screening. In order to extend the idea of safe
feature screening into predictive pattern mining, we derive a novel pruning
rule called safe pattern pruning (SPP) rule that can be used for searching over
the tree defined among patterns in the database. The SPP rule has a property
that, if a node corresponding to a pattern in the database is pruned out by the
SPP rule, then it is guaranteed that all the patterns corresponding to its
descendant nodes are never needed for the optimal predictive model. We apply
the SPP method to graph mining and item-set mining problems, and demonstrate
its computational advantage.
Validity and reliability of free software for bidimensional gait analysis
Authors: Ana Paula Quixadá, Andrea Naomi Onodera, Norberto Peña, José Garcia Vivas Miranda, Katia Nunes Sá
Date: 2016-02-14 00:00:00

Despite the evaluation systems of human movement that have been advancing in
recent decades, their use are not feasible for clinical practice because it has
a high cost and scarcity of trained operators to interpret their results. An
ideal videogrammetry system should be easy to use, low cost, with minimal
equipment, and fast realization. The CvMob is a free tool for dynamic
evaluation of human movements that express measurements in figures, tables, and
graphics. This paper aims to determine if CvMob is a reliable tool for the
evaluation of two dimensional human gait. This is a validity and reliability
study. The sample was composed of 56 healthy individuals who walked on a
9-meterlong walkway and were simultaneously filmed by CvMob and Vicon system
cameras. Linear trajectories and angular measurements were compared to validate
the CvMob system, and inter and intrarater findings of the same measurements
were used to determine reliability. A strong correlation (rs mean = 0.988) of
the linear trajectories between systems and inter and intrarater analysis were
found. According to the Bland-Altman method, the angles that had good agreement
between systems were maximum flexion and extension (stance and swing) of the
knee and dorsiflexion range of motion and stride length. The CvMob is a
reliable tool for analysis of linear motion and lengths in two-dimensional
evaluations of human gait. The angular measurements demonstrate high agreement
for the knee joint; however, the hip and ankle measurements were limited by
differences between systems.
Stacked What-Where Auto-encoders
Authors: Junbo Zhao, Michael Mathieu, Ross Goroshin, Yann Lecun
Date: 2015-06-08 00:00:00
Tasks: Decoder, Image Classification, Semi-Supervised Image Classification

We present a novel architecture, the "stacked what-where auto-encoders"
(SWWAE), which integrates discriminative and generative pathways and provides a
unified approach to supervised, semi-supervised and unsupervised learning
without relying on sampling during training. An instantiation of SWWAE uses a
convolutional net (Convnet) (LeCun et al. (1998)) to encode the input, and
employs a deconvolutional net (Deconvnet) (Zeiler et al. (2010)) to produce the
reconstruction. The objective function includes reconstruction terms that
induce the hidden states in the Deconvnet to be similar to those of the
Convnet. Each pooling layer produces two sets of variables: the "what" which
are fed to the next layer, and its complementary variable "where" that are fed
to the corresponding layer in the generative decoder.