pwc-archive/papers-with-abstracts huggingface dataset:
GPatt: Fast Multidimensional Pattern Extrapolation with Gaussian Processes
Authors: Andrew Gordon Wilson, Elad Gilboa, Arye Nehorai, John P. Cunningham
Date: 2013-10-20 00:00:00
Tasks: Gaussian Processes

Gaussian processes are typically used for smoothing and interpolation on
small datasets. We introduce a new Bayesian nonparametric framework -- GPatt --
enabling automatic pattern extrapolation with Gaussian processes on large
multidimensional datasets. GPatt unifies and extends highly expressive kernels
and fast exact inference techniques. Without human intervention -- no hand
crafting of kernel features, and no sophisticated initialisation procedures --
we show that GPatt can solve large scale pattern extrapolation, inpainting, and
kernel discovery problems, including a problem with 383400 training points. We
find that GPatt significantly outperforms popular alternative scalable Gaussian
process methods in speed and accuracy. Moreover, we discover profound
differences between each of these methods, suggesting expressive kernels,
nonparametric representations, and exact inference are useful for modelling
large scale multidimensional patterns.
Inference of Network Summary Statistics Through Network Denoising
Authors: Prakash Balachandran, Edoardo Airoldi, Eric Kolaczyk
Date: 2013-10-01 00:00:00
Tasks: Denoising, Functional Connectivity

Consider observing an undirected network that is `noisy' in the sense that
there are Type I and Type II errors in the observation of edges. Such errors
can arise, for example, in the context of inferring gene regulatory networks in
genomics or functional connectivity networks in neuroscience. Given a single
observed network then, to what extent are summary statistics for that network
representative of their analogues for the true underlying network? Can we infer
such statistics more accurately by taking into account the noise in the
observed network edges?
  In this paper, we answer both of these questions. In particular, we develop a
spectral-based methodology using the adjacency matrix to `denoise' the observed
network data and produce more accurate inference of the summary statistics of
the true network. We characterize performance of our methodology through bounds
on appropriate notions of risk in the $L^2$ sense, and conclude by illustrating
the practical impact of this work on synthetic and real-world data.
System Analysis And Design For Multimedia Retrieval Systems
Authors: Avinash N Bhute, B. B. Meshram
Date: 2013-12-31 00:00:00
Tasks: Retrieval, Video Retrieval

Due to the extensive use of information technology and the recent
developments in multimedia systems, the amount of multimedia data available to
users has increased exponentially. Video is an example of multimedia data as it
contains several kinds of data such as text, image, meta-data, visual and
audio. Content based video retrieval is an approach for facilitating the
searching and browsing of large multimedia collections over WWW. In order to
create an effective video retrieval system, visual perception must be taken
into account. We conjectured that a technique which employs multiple features
for indexing and retrieval would be more effective in the discrimination and
search tasks of videos. In order to validate this, content based indexing and
retrieval systems were implemented using color histogram, Texture feature
(GLCM), edge density and motion..
Black Box Variational Inference
Authors: Rajesh Ranganath, Sean Gerrish, David M. Blei
Date: 2013-12-31 00:00:00
Tasks: Stochastic Optimization, Variational Inference

Variational inference has become a widely used method to approximate
posteriors in complex latent variables models. However, deriving a variational
inference algorithm generally requires significant model-specific analysis, and
these efforts can hinder and deter us from quickly developing and exploring a
variety of models for a problem at hand. In this paper, we present a "black
box" variational inference algorithm, one that can be quickly applied to many
models with little additional derivation. Our method is based on a stochastic
optimization of the variational objective where the noisy gradient is computed
from Monte Carlo samples from the variational distribution. We develop a number
of methods to reduce the variance of the gradient, always maintaining the
criterion that we want to avoid difficult model-based derivations. We evaluate
our method against the corresponding black box sampling based methods. We find
that our method reaches better predictive likelihoods much faster than sampling
methods. Finally, we demonstrate that Black Box Variational Inference lets us
easily explore a wide space of models by quickly constructing and evaluating
several models of longitudinal healthcare data.
Controlled Sparsity Kernel Learning
Authors: Dinesh Govindaraj, Raman Sankaran, Sreedal Menon, Chiranjib Bhattacharyya
Date: 2013-12-31 00:00:00
Tasks: Object Categorization

Multiple Kernel Learning(MKL) on Support Vector Machines(SVMs) has been a
popular front of research in recent times due to its success in application
problems like Object Categorization. This success is due to the fact that MKL
has the ability to choose from a variety of feature kernels to identify the
optimal kernel combination. But the initial formulation of MKL was only able to
select the best of the features and misses out many other informative kernels
presented. To overcome this, the Lp norm based formulation was proposed by
Kloft et. al. This formulation is capable of choosing a non-sparse set of
kernels through a control parameter p. Unfortunately, the parameter p does not
have a direct meaning to the number of kernels selected. We have observed that
stricter control over the number of kernels selected gives us an edge over
these techniques in terms of accuracy of classification and also helps us to
fine tune the algorithms to the time requirements at hand. In this work, we
propose a Controlled Sparsity Kernel Learning (CSKL) formulation that can
strictly control the number of kernels which we wish to select. The CSKL
formulation introduces a parameter t which directly corresponds to the number
of kernels selected. It is important to note that a search in t space is finite
and fast as compared to p. We have also provided an efficient Reduced Gradient
Descent based algorithm to solve the CSKL formulation, which is proven to
converge. Through our experiments on the Caltech101 Object Categorization
dataset, we have also shown that one can achieve better accuracies than the
previous formulations through the right choice of t.
PSO-MISMO Modeling Strategy for Multi-Step-Ahead Time Series Prediction
Authors: Yukun Bao, Tao Xiong, Zhongyi Hu
Date: 2013-12-31 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis, Time Series Prediction

Multi-step-ahead time series prediction is one of the most challenging
research topics in the field of time series modeling and prediction, and is
continually under research. Recently, the multiple-input several
multiple-outputs (MISMO) modeling strategy has been proposed as a promising
alternative for multi-step-ahead time series prediction, exhibiting advantages
compared with the two currently dominating strategies, the iterated and the
direct strategies. Built on the established MISMO strategy, this study proposes
a particle swarm optimization (PSO)-based MISMO modeling strategy, which is
capable of determining the number of sub-models in a self-adaptive mode, with
varying prediction horizons. Rather than deriving crisp divides with equal-size
s prediction horizons from the established MISMO, the proposed PSO-MISMO
strategy, implemented with neural networks, employs a heuristic to create
flexible divides with varying sizes of prediction horizons and to generate
corresponding sub-models, providing considerable flexibility in model
construction, which has been validated with simulated and real datasets.
A Novel Approach For Generating Face Template Using Bda
Authors: Shraddha S. Shinde, Prof. Anagha P. Khedkar
Date: 2013-12-31 00:00:00
Tasks: Management

In identity management system, commonly used biometric recognition system needs attention towards issue of biometric template protection as far as more reliable solution is concerned. In view of this biometric template protection algorithm should satisfy security, discriminability and cancelability. As no single template protection method is capable of satisfying the basic requirements, a novel technique for face template generation and protection is proposed. The novel approach is proposed to provide security and accuracy in new user enrollment as well as authentication process. This novel technique takes advantage of both the hybrid approach and the binary discriminant analysis algorithm. This algorithm is designed on the basis of random projection, binary discriminant analysis and fuzzy commitment scheme. Three publicly available benchmark face databases are used for evaluation. The proposed novel technique enhances the discriminability and recognition accuracy by 80% in terms of matching score of the face images and provides high security.
Fused Multiple Graphical Lasso
Authors: Sen Yang, Zhaosong Lu, Xiaotong Shen, Peter Wonka, Jieping Ye
Date: 2012-09-10 00:00:00

In this paper, we consider the problem of estimating multiple graphical
models simultaneously using the fused lasso penalty, which encourages adjacent
graphs to share similar structures. A motivating example is the analysis of
brain networks of Alzheimer's disease using neuroimaging data. Specifically, we
may wish to estimate a brain network for the normal controls (NC), a brain
network for the patients with mild cognitive impairment (MCI), and a brain
network for Alzheimer's patients (AD). We expect the two brain networks for NC
and MCI to share common structures but not to be identical to each other;
similarly for the two brain networks for MCI and AD. The proposed formulation
can be solved using a second-order method. Our key technical contribution is to
establish the necessary and sufficient condition for the graphs to be
decomposable. Based on this key property, a simple screening rule is presented,
which decomposes the large graphs into small subgraphs and allows an efficient
estimation of multiple independent (small) subgraphs, dramatically reducing the
computational cost. We perform experiments on both synthetic and real data; our
results demonstrate the effectiveness and efficiency of the proposed approach.
Approximating the Bethe partition function
Authors: Adrian Weller, Tony Jebara
Date: 2013-12-30 00:00:00

When belief propagation (BP) converges, it does so to a stationary point of
the Bethe free energy $F$, and is often strikingly accurate. However, it may
converge only to a local optimum or may not converge at all. An algorithm was
recently introduced for attractive binary pairwise MRFs which is guaranteed to
return an $\epsilon$-approximation to the global minimum of $F$ in polynomial
time provided the maximum degree $\Delta=O(\log n)$, where $n$ is the number of
variables. Here we significantly improve this algorithm and derive several
results including a new approach based on analyzing first derivatives of $F$,
which leads to performance that is typically far superior and yields a fully
polynomial-time approximation scheme (FPTAS) for attractive models without any
degree restriction. Further, the method applies to general (non-attractive)
models, though with no polynomial time guarantee in this case, leading to the
important result that approximating $\log$ of the Bethe partition function,
$\log Z_B=-\min F$, for a general model to additive $\epsilon$-accuracy may be
reduced to a discrete MAP inference problem. We explore an application to
predicting equipment failure on an urban power network and demonstrate that the
Bethe approximation can perform well even when BP fails to converge.
Evolutionary Design of Numerical Methods: Generating Finite Difference and Integration Schemes by Differential Evolution
Authors: C. D. Erdbrink, V. V. Krzhizhanovskaya, P. M. A. Sloot
Date: 2013-12-30 00:00:00

Classical and new numerical schemes are generated using evolutionary
computing. Differential Evolution is used to find the coefficients of finite
difference approximations of function derivatives, and of single and multi-step
integration methods. The coefficients are reverse engineered based on samples
from a target function and its derivative used for training. The Runge-Kutta
schemes are trained using the order condition equations. An appealing feature
of the evolutionary method is the low number of model parameters. The
population size, termination criterion and number of training points are
determined in a sensitivity analysis. Computational results show good agreement
between evolved and analytical coefficients. In particular, a new fifth-order
Runge-Kutta scheme is computed which adheres to the order conditions with a sum
of absolute errors of order 10^-14. Execution of the evolved schemes proved the
intended orders of accuracy. The outcome of this study is valuable for future
developments in the design of complex numerical methods that are out of reach
by conventional means.
A Fused Elastic Net Logistic Regression Model for Multi-Task Binary Classification
Authors: Venelin Mitov, Manfred Claassen
Date: 2013-12-30 00:00:00
Tasks: Binary Classification, General Classification, Multi-Task Learning, regression

Multi-task learning has shown to significantly enhance the performance of
multiple related learning tasks in a variety of situations. We present the
fused logistic regression, a sparse multi-task learning approach for binary
classification. Specifically, we introduce sparsity inducing penalties over
parameter differences of related logistic regression models to encode
similarity across related tasks. The resulting joint learning task is cast into
a form that lends itself to be efficiently optimized with a recursive variant
of the alternating direction method of multipliers. We show results on
synthetic data and describe the regime of settings where our multi-task
approach achieves significant improvements over the single task learning
approach and discuss the implications on applying the fused logistic regression
in different real world settings.
Assessment of Customer Credit through Combined Clustering of Artificial Neural Networks, Genetics Algorithm and Bayesian Probabilities
Authors: Reza Mortezapour, Mehdi Afzali
Date: 2013-12-30 00:00:00
Tasks: Clustering, General Classification

Today, with respect to the increasing growth of demand to get credit from the
customers of banks and finance and credit institutions, using an effective and
efficient method to decrease the risk of non-repayment of credit given is very
necessary. Assessment of customers' credit is one of the most important and the
most essential duties of banks and institutions, and if an error occurs in this
field, it would leads to the great losses for banks and institutions. Thus,
using the predicting computer systems has been significantly progressed in
recent decades. The data that are provided to the credit institutions' managers
help them to make a straight decision for giving the credit or not-giving it.
In this paper, we will assess the customer credit through a combined
classification using artificial neural networks, genetics algorithm and
Bayesian probabilities simultaneously, and the results obtained from three
methods mentioned above would be used to achieve an appropriate and final
result. We use the K_folds cross validation test in order to assess the method
and finally, we compare the proposed method with the methods such as
Clustering-Launched Classification (CLC), Support Vector Machine (SVM) as well
as GA+SVM where the genetics algorithm has been used to improve them.
Total variation regularization for manifold-valued data
Authors: Andreas Weinmann, Laurent Demaret, Martin Storath
Date: 2013-12-30 00:00:00
Tasks: Denoising

We consider total variation minimization for manifold valued data. We propose
a cyclic proximal point algorithm and a parallel proximal point algorithm to
minimize TV functionals with $\ell^p$-type data terms in the manifold case.
These algorithms are based on iterative geodesic averaging which makes them
easily applicable to a large class of data manifolds. As an application, we
consider denoising images which take their values in a manifold. We apply our
algorithms to diffusion tensor images, interferometric SAR images as well as
sphere and cylinder valued images. For the class of Cartan-Hadamard manifolds
(which includes the data space in diffusion tensor imaging) we show the
convergence of the proposed TV minimizing algorithms to a global minimizer.
A Multi-Orientation Analysis Approach to Retinal Vessel Tracking
Authors: Erik Bekkers, Remco Duits, Tos Berendschot, Bart ter Haar Romeny
Date: 2012-12-14 00:00:00

This paper presents a method for retinal vasculature extraction based on
biologically inspired multi-orientation analysis. We apply multi-orientation
analysis via so-called invertible orientation scores, modeling the cortical
columns in the visual system of higher mammals. This allows us to generically
deal with many hitherto complex problems inherent to vessel tracking, such as
crossings, bifurcations, parallel vessels, vessels of varying widths and
vessels with high curvature. Our approach applies tracking in invertible
orientation scores via a novel geometrical principle for curve optimization in
the Euclidean motion group SE(2). The method runs fully automatically and
provides a detailed model of the retinal vasculature, which is crucial as a
sound basis for further quantitative analysis of the retina, especially in
screening applications.
Towards a New Science of a Clinical Data Intelligence
Authors: Volker Tresp, Sonja Zillner, Maria J. Costa, Yi Huang, Alexander Cavallaro, Peter A. Fasching, Andre Reis, Martin Sedlmayr, Thomas Ganslandt, Klemens Budde, Carl Hinrichs, Danilo Schmidt, Philipp Daumke, Daniel Sonntag, Thomas Wittenberg, Patricia G. Oppelt, Denis Krompass
Date: 2013-11-17 00:00:00

In this paper we define Clinical Data Intelligence as the analysis of data
generated in the clinical routine with the goal of improving patient care. We
define a science of a Clinical Data Intelligence as a data analysis that
permits the derivation of scientific, i.e., generalizable and reliable results.
We argue that a science of a Clinical Data Intelligence is sensible in the
context of a Big Data analysis, i.e., with data from many patients and with
complete patient information. We discuss that Clinical Data Intelligence
requires the joint efforts of knowledge engineering, information extraction
(from textual and other unstructured data), and statistics and statistical
machine learning. We describe some of our main results as conjectures and
relate them to a recently funded research project involving two major German
university hospitals.
Feature vector regularization in machine learning
Authors: Yue Fan, Louise Raphael, Mark Kon
Date: 2012-12-19 00:00:00
Tasks: BIG-bench Machine Learning, Cancer Classification, Denoising, General Classification, regression

Problems in machine learning (ML) can involve noisy input data, and ML
classification methods have reached limiting accuracies when based on standard
ML data sets consisting of feature vectors and their classes. Greater accuracy
will require incorporation of prior structural information on data into
learning. We study methods to regularize feature vectors (unsupervised
regularization methods), analogous to supervised regularization for estimating
functions in ML. We study regularization (denoising) of ML feature vectors
using Tikhonov and other regularization methods for functions on ${\bf R}^n$. A
feature vector ${\bf x}=(x_1,\ldots,x_n)=\{x_q\}_{q=1}^n$ is viewed as a
function of its index $q$, and smoothed using prior information on its
structure. This can involve a penalty functional on feature vectors analogous
to those in statistical learning, or use of proximity (e.g. graph) structure on
the set of indices. Such feature vector regularization inherits a property from
function denoising on ${\bf R}^n$, in that accuracy is non-monotonic in the
denoising (regularization) parameter $\alpha$. Under some assumptions about the
noise level and the data structure, we show that the best reconstruction
accuracy also occurs at a finite positive $\alpha$ in index spaces with graph
structures. We adapt two standard function denoising methods used on ${\bf
R}^n$, local averaging and kernel regression. In general the index space can be
any discrete set with a notion of proximity, e.g. a metric space, a subset of
${\bf R}^n$, or a graph/network, with feature vectors as functions with some
notion of continuity. We show this improves feature vector recovery, and thus
the subsequent classification or regression done on them. We give an example in
gene expression analysis for cancer classification with the genome as an index
space and network structure based protein-protein interactions.
Response-Based Approachability and its Application to Generalized No-Regret Algorithms
Authors: Andrey Bernstein, Nahum Shimkin
Date: 2013-12-30 00:00:00

Approachability theory, introduced by Blackwell (1956), provides fundamental
results on repeated games with vector-valued payoffs, and has been usefully
applied since in the theory of learning in games and to learning algorithms in
the online adversarial setup. Given a repeated game with vector payoffs, a
target set $S$ is approachable by a certain player (the agent) if he can ensure
that the average payoff vector converges to that set no matter what his
adversary opponent does. Blackwell provided two equivalent sets of conditions
for a convex set to be approachable. The first (primary) condition is a
geometric separation condition, while the second (dual) condition requires that
the set be {\em non-excludable}, namely that for every mixed action of the
opponent there exists a mixed action of the agent (a {\em response}) such that
the resulting payoff vector belongs to $S$. Existing approachability algorithms
rely on the primal condition and essentially require to compute at each stage a
projection direction from a given point to $S$. In this paper, we introduce an
approachability algorithm that relies on Blackwell's {\em dual} condition.
Thus, rather than projection, the algorithm relies on computation of the
response to a certain action of the opponent at each stage. The utility of the
proposed algorithm is demonstrated by applying it to certain generalizations of
the classical regret minimization problem, which include regret minimization
with side constraints and regret minimization for global cost functions. In
these problems, computation of the required projections is generally complex
but a response is readily obtainable.