pwc-archive/papers-with-abstracts huggingface dataset:
Sparse Deep Stacking Network for Image Classification
Authors: Jun Li, Heyou Chang, Jian Yang
Date: 2015-01-05 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Sparse coding can learn good robust representation to noise and model more
higher-order representation for image classification. However, the inference
algorithm is computationally expensive even though the supervised signals are
used to learn compact and discriminative dictionaries in sparse coding
techniques. Luckily, a simplified neural network module (SNNM) has been
proposed to directly learn the discriminative dictionaries for avoiding the
expensive inference. But the SNNM module ignores the sparse representations.
Therefore, we propose a sparse SNNM module by adding the mixed-norm
regularization (l1/l2 norm). The sparse SNNM modules are further stacked to
build a sparse deep stacking network (S-DSN). In the experiments, we evaluate
S-DSN with four databases, including Extended YaleB, AR, 15 scene and
Caltech101. Experimental results show that our model outperforms related
classification methods with only a linear classifier. It is worth noting that
we reach 98.8% recognition accuracy on 15 scene.
Hashing with binary autoencoders
Authors: Miguel Á. Carreira-Perpiñán, Ramin Raziperchikolaei
Date: 2015-01-05 00:00:00
Proceeding: CVPR 2015 6
Tasks: Decoder, Image Retrieval, Retrieval

An attractive approach for fast search in image databases is binary hashing,
where each high-dimensional, real-valued image is mapped onto a
low-dimensional, binary vector and the search is done in this binary space.
Finding the optimal hash function is difficult because it involves binary
constraints, and most approaches approximate the optimization by relaxing the
constraints and then binarizing the result. Here, we focus on the binary
autoencoder model, which seeks to reconstruct an image from the binary code
produced by the hash function. We show that the optimization can be simplified
with the method of auxiliary coordinates. This reformulates the optimization as
alternating two easier steps: one that learns the encoder and decoder
separately, and one that optimizes the code for each image. Image retrieval
experiments, using precision/recall and a measure of code utilization, show the
resulting hash function outperforms or is competitive with state-of-the-art
methods for binary hashing.
Concave Penalized Estimation of Sparse Gaussian Bayesian Networks
Authors: Bryon Aragam, Qing Zhou
Date: 2014-01-04 00:00:00

We develop a penalized likelihood estimation framework to estimate the
structure of Gaussian Bayesian networks from observational data. In contrast to
recent methods which accelerate the learning problem by restricting the search
space, our main contribution is a fast algorithm for score-based structure
learning which does not restrict the search space in any way and works on
high-dimensional datasets with thousands of variables. Our use of concave
regularization, as opposed to the more popular $\ell_0$ (e.g. BIC) penalty, is
new. Moreover, we provide theoretical guarantees which generalize existing
asymptotic results when the underlying distribution is Gaussian. Most notably,
our framework does not require the existence of a so-called faithful DAG
representation, and as a result the theory must handle the inherent
nonidentifiability of the estimation problem in a novel way. Finally, as a
matter of independent interest, we provide a comprehensive comparison of our
approach to several standard structure learning methods using open-source
packages developed for the R language. Based on these experiments, we show that
our algorithm is significantly faster than other competing methods while
obtaining higher sensitivity with comparable false discovery rates for
high-dimensional data. In particular, the total runtime for our method to
generate a solution path of 20 estimates for DAGs with 8000 nodes is around one
hour.
Differential Search Algorithm-based Parametric Optimization of Fuzzy Generalized Eigenvalue Proximal Support Vector Machine
Authors: M. H. Marghny, Rasha M. Abd El-Aziz, Ahmed I. Taloba
Date: 2015-01-04 00:00:00
Tasks: General Classification

Support Vector Machine (SVM) is an effective model for many classification
problems. However, SVM needs the solution of a quadratic program which require
specialized code. In addition, SVM has many parameters, which affects the
performance of SVM classifier. Recently, the Generalized Eigenvalue Proximal
SVM (GEPSVM) has been presented to solve the SVM complexity. In real world
applications data may affected by error or noise, working with this data is a
challenging problem. In this paper, an approach has been proposed to overcome
this problem. This method is called DSA-GEPSVM. The main improvements are
carried out based on the following: 1) a novel fuzzy values in the linear case.
2) A new Kernel function in the nonlinear case. 3) Differential Search
Algorithm (DSA) is reformulated to find near optimal values of the GEPSVM
parameters and its kernel parameters. The experimental results show that the
proposed approach is able to find the suitable parameter values, and has higher
classification accuracy compared with some other algorithms.
Sparse and low-rank multivariate Hawkes processes
Authors: Emmanuel Bacry, Martin Bompaire, Stéphane Gaïffas, Jean-François Muzy
Date: 2015-01-04 00:00:00

We consider the problem of unveiling the implicit network structure of node interactions (such as user interactions in a social network), based only on high-frequency timestamps. Our inference is based on the minimization of the least-squares loss associated with a multivariate Hawkes model, penalized by $\ell_1$ and trace norm of the interaction tensor. We provide a first theoretical analysis for this problem, that includes sparsity and low-rank inducing penalizations. This result involves a new data-driven concentration inequality for matrix martingales in continuous time with observable variance, which is a result of independent interest and a broad range of possible applications since it extends to matrix martingales former results restricted to the scalar case. A consequence of our analysis is the construction of sharply tuned $\ell_1$ and trace-norm penalizations, that leads to a data-driven scaling of the variability of information available for each users. Numerical experiments illustrate the significant improvements achieved by the use of such data-driven penalizations.
Protein Secondary Structure Prediction with Long Short Term Memory Networks
Authors: Søren Kaae Sønderby, Ole Winther
Date: 2014-12-25 00:00:00
Tasks: Protein Secondary Structure Prediction

Prediction of protein secondary structure from the amino acid sequence is a
classical bioinformatics problem. Common methods use feed forward neural
networks or SVMs combined with a sliding window, as these models does not
naturally handle sequential data. Recurrent neural networks are an
generalization of the feed forward neural network that naturally handle
sequential data. We use a bidirectional recurrent neural network with long
short term memory cells for prediction of secondary structure and evaluate
using the CB513 dataset. On the secondary structure 8-class problem we report
better performance (0.674) than state of the art (0.664). Our model includes
feed forward networks between the long short term memory cells, a path that can
be further explored.
On Enhancing The Performance Of Nearest Neighbour Classifiers Using Hassanat Distance Metric
Authors: Mouhammd Al-kasassbeh, Ghada A. Altarawneh, Ahmad B. A. Hassanat
Date: 2015-01-04 00:00:00

We showed in this work how the Hassanat distance metric enhances the
performance of the nearest neighbour classifiers. The results demonstrate the
superiority of this distance metric over the traditional and most-used
distances, such as Manhattan distance and Euclidian distance. Moreover, we
proved that the Hassanat distance metric is invariant to data scale, noise and
outliers. Throughout this work, it is clearly notable that both ENN and IINC
performed very well with the distance investigated, as their accuracy increased
significantly by 3.3% and 3.1% respectively, with no significant advantage of
the ENN over the IINC in terms of accuracy. Correspondingly, it can be noted
from our results that there is no optimal algorithm that can solve all
real-life problems perfectly; this is supported by the no-free-lunch theorem
A New Method for Signal and Image Analysis: The Square Wave Method
Authors: Osvaldo Skliar, Ricardo E. Monge, Sherry Gapper
Date: 2015-01-04 00:00:00

A brief review is provided of the use of the Square Wave Method (SWM) in the
field of signal and image analysis and it is specified how results thus
obtained are expressed using the Square Wave Transform (SWT), in the frequency
domain. To illustrate the new approach introduced in this field, the results of
two cases are analyzed: a) a sequence of samples (that is, measured values) of
an electromyographic recording; and b) the classic image of Lenna.
Hostile Intent Identification by Movement Pattern Analysis: Using Artificial Neural Networks
Authors: Souham Biswas, Manisha J. Nene
Date: 2015-01-04 00:00:00

In the recent years, the problem of identifying suspicious behavior has
gained importance and identifying this behavior using computational systems and
autonomous algorithms is highly desirable in a tactical scenario. So far, the
solutions have been primarily manual which elicit human observation of entities
to discern the hostility of the situation. To cater to this problem statement,
a number of fully automated and partially automated solutions exist. But, these
solutions lack the capability of learning from experiences and work in
conjunction with human supervision which is extremely prone to error. In this
paper, a generalized methodology to predict the hostility of a given object
based on its movement patterns is proposed which has the ability to learn and
is based upon the mechanism of humans of learning from experiences. The
methodology so proposed has been implemented in a computer simulation. The
results show that the posited methodology has the potential to be applied in
real world tactical scenarios.
Microbial community pattern detection in human body habitats via ensemble clustering framework
Authors: Peng Yang, Xiaoquan Su, Le Ou-Yang, Hon-Nian Chua, Xiao-Li Li, Kang Ning
Date: 2014-12-21 00:00:00
Tasks: Clustering

The human habitat is a host where microbial species evolve, function, and
continue to evolve. Elucidating how microbial communities respond to human
habitats is a fundamental and critical task, as establishing baselines of human
microbiome is essential in understanding its role in human disease and health.
However, current studies usually overlook a complex and interconnected
landscape of human microbiome and limit the ability in particular body habitats
with learning models of specific criterion. Therefore, these methods could not
capture the real-world underlying microbial patterns effectively. To obtain a
comprehensive view, we propose a novel ensemble clustering framework to mine
the structure of microbial community pattern on large-scale metagenomic data.
Particularly, we first build a microbial similarity network via integrating
1920 metagenomic samples from three body habitats of healthy adults. Then a
novel symmetric Nonnegative Matrix Factorization (NMF) based ensemble model is
proposed and applied onto the network to detect clustering pattern. Extensive
experiments are conducted to evaluate the effectiveness of our model on
deriving microbial community with respect to body habitat and host gender. From
clustering results, we observed that body habitat exhibits a strong bound but
non-unique microbial structural patterns. Meanwhile, human microbiome reveals
different degree of structural variations over body habitat and host gender. In
summary, our ensemble clustering framework could efficiently explore integrated
clustering results to accurately identify microbial communities, and provide a
comprehensive view for a set of microbial communities. Such trends depict an
integrated biography of microbial communities, which offer a new insight
towards uncovering pathogenic model of human microbiome.
Understanding Trajectory Behavior: A Motion Pattern Approach
Authors: Mahdi M. Kalayeh, Stephen Mussmann, Alla Petrakova, Niels da Vitoria Lobo, Mubarak Shah
Date: 2015-01-04 00:00:00
Tasks: Clustering, Trajectory Clustering

Mining the underlying patterns in gigantic and complex data is of great
importance to data analysts. In this paper, we propose a motion pattern
approach to mine frequent behaviors in trajectory data. Motion patterns,
defined by a set of highly similar flow vector groups in a spatial locality,
have been shown to be very effective in extracting dominant motion behaviors in
video sequences. Inspired by applications and properties of motion patterns, we
have designed a framework that successfully solves the general task of
trajectory clustering. Our proposed algorithm consists of four phases: flow
vector computation, motion component extraction, motion component's
reachability set creation, and motion pattern formation. For the first phase,
we break down trajectories into flow vectors that indicate instantaneous
movements. In the second phase, via a Kmeans clustering approach, we create
motion components by clustering the flow vectors with respect to their location
and velocity. Next, we create motion components' reachability set in terms of
spatial proximity and motion similarity. Finally, for the fourth phase, we
cluster motion components using agglomerative clustering with the weighted
Jaccard distance between the motion components' signatures, a set created using
path reachability. We have evaluated the effectiveness of our proposed method
in an extensive set of experiments on diverse datasets. Further, we have shown
how our proposed method handles difficulties in the general task of trajectory
clustering that challenge the existing state-of-the-art methods.
Evaluation of Predictive Data Mining Algorithms in Erythemato-Squamous Disease Diagnosis
Authors: Kwetishe Danjuma, Adenike O. Osofisan
Date: 2015-01-03 00:00:00

A lot of time is spent searching for the most performing data mining
algorithms applied in clinical diagnosis. The study set out to identify the
most performing predictive data mining algorithms applied in the diagnosis of
Erythemato-squamous diseases. The study used Naive Bayes, Multilayer Perceptron
and J48 decision tree induction to build predictive data mining models on 366
instances of Erythemato-squamous diseases datasets. Also, 10-fold
cross-validation and sets of performance metrics were used to evaluate the
baseline predictive performance of the classifiers. The comparative analysis
shows that the Naive Bayes performed best with accuracy of 97.4%, Multilayer
Perceptron came out second with accuracy of 96.6%, and J48 came out the worst
with accuracy of 93.5%. The evaluation of these classifiers on clinical
datasets, gave an insight into the predictive ability of different data mining
algorithms applicable in clinical diagnosis especially in the diagnosis of
Erythemato-squamous diseases.
A Taxonomy of Big Data for Optimal Predictive Machine Learning and Data Mining
Authors: Ernest Fokoue
Date: 2015-01-03 00:00:00
Tasks: BIG-bench Machine Learning, Imputation

Big data comes in various ways, types, shapes, forms and sizes. Indeed,
almost all areas of science, technology, medicine, public health, economics,
business, linguistics and social science are bombarded by ever increasing flows
of data begging to analyzed efficiently and effectively. In this paper, we
propose a rough idea of a possible taxonomy of big data, along with some of the
most commonly used tools for handling each particular category of bigness. The
dimensionality p of the input space and the sample size n are usually the main
ingredients in the characterization of data bigness. The specific statistical
machine learning technique used to handle a particular big data set will depend
on which category it falls in within the bigness taxonomy. Large p small n data
sets for instance require a different set of tools from the large n small p
variety. Among other tools, we discuss Preprocessing, Standardization,
Imputation, Projection, Regularization, Penalization, Compression, Reduction,
Selection, Kernelization, Hybridization, Parallelization, Aggregation,
Randomization, Replication, Sequentialization. Indeed, it is important to
emphasize right away that the so-called no free lunch theorem applies here, in
the sense that there is no universally superior method that outperforms all
other methods on all categories of bigness. It is also important to stress the
fact that simplicity in the sense of Ockham's razor non plurality principle of
parsimony tends to reign supreme when it comes to massive data. We conclude
with a comparison of the predictive performance of some of the most commonly
used methods on a few data sets.
The Learnability of Unknown Quantum Measurements
Authors: Hao-Chung Cheng, Min-Hsiu Hsieh, Ping-Cheng Yeh
Date: 2015-01-03 00:00:00
Tasks: BIG-bench Machine Learning, Learning Theory, Quantum Machine Learning

Quantum machine learning has received significant attention in recent years,
and promising progress has been made in the development of quantum algorithms
to speed up traditional machine learning tasks. In this work, however, we focus
on investigating the information-theoretic upper bounds of sample complexity -
how many training samples are sufficient to predict the future behaviour of an
unknown target function. This kind of problem is, arguably, one of the most
fundamental problems in statistical learning theory and the bounds for
practical settings can be completely characterised by a simple measure of
complexity.
  Our main result in the paper is that, for learning an unknown quantum
measurement, the upper bound, given by the fat-shattering dimension, is
linearly proportional to the dimension of the underlying Hilbert space.
Learning an unknown quantum state becomes a dual problem to ours, and as a
byproduct, we can recover Aaronson's famous result [Proc. R. Soc. A
463:3089-3144 (2007)] solely using a classical machine learning technique. In
addition, other famous complexity measures like covering numbers and Rademacher
complexities are derived explicitly. We are able to connect measures of sample
complexity with various areas in quantum information science, e.g. quantum
state/measurement tomography, quantum state discrimination and quantum random
access codes, which may be of independent interest. Lastly, with the assistance
of general Bloch-sphere representation, we show that learning quantum
measurements/states can be mathematically formulated as a neural network.
Consequently, classical ML algorithms can be applied to efficiently accomplish
the two quantum learning tasks.
An Empirical Study of the L2-Boost technique with Echo State Networks
Authors: Sebastián Basterrech
Date: 2015-01-02 00:00:00
Tasks: Time Series, Time Series Analysis

A particular case of Recurrent Neural Network (RNN) was introduced at the
beginning of the 2000s under the name of Echo State Networks (ESNs). The ESN
model overcomes the limitations during the training of the RNNs while
introducing no significant disadvantages. Although the model presents some
well-identified drawbacks when the parameters are not well initialised. The
performance of an ESN is highly dependent on its internal parameters and
pattern of connectivity of the hidden-hidden weights Often, the tuning of the
network parameters can be hard and can impact in the accuracy of the models.
  In this work, we investigate the performance of a specific boosting technique
(called L2-Boost) with ESNs as single predictors. The L2-Boost technique has
been shown to be an effective tool to combine "weak" predictors in regression
problems. In this study, we use an ensemble of random initialized ESNs (without
control their parameters) as "weak" predictors of the boosting procedure. We
evaluate our approach on five well-know time-series benchmark problems.
Additionally, we compare this technique with a baseline approach that consists
of averaging the prediction of an ensemble of ESNs.
Feature Augmentation via Nonparametrics and Selection (FANS) in High Dimensional Classification
Authors: Jianqing Fan, Yang Feng, Jiancheng Jiang, Xin Tong
Date: 2013-12-31 00:00:00
Tasks: Additive models, General Classification, Vocal Bursts Intensity Prediction

We propose a high dimensional classification method that involves
nonparametric feature augmentation. Knowing that marginal density ratios are
the most powerful univariate classifiers, we use the ratio estimates to
transform the original feature measurements. Subsequently, penalized logistic
regression is invoked, taking as input the newly transformed or augmented
features. This procedure trains models equipped with local complexity and
global simplicity, thereby avoiding the curse of dimensionality while creating
a flexible nonlinear decision boundary. The resulting method is called Feature
Augmentation via Nonparametrics and Selection (FANS). We motivate FANS by
generalizing the Naive Bayes model, writing the log ratio of joint densities as
a linear combination of those of marginal densities. It is related to
generalized additive models, but has better interpretability and computability.
Risk bounds are developed for FANS. In numerical analysis, FANS is compared
with competing methods, so as to provide a guideline on its best application
domain. Real data analysis demonstrates that FANS performs very competitively
on benchmark email spam and gene expression data sets. Moreover, FANS is
implemented by an extremely fast algorithm through parallel computing.
Computational Feasibility of Clustering under Clusterability Assumptions
Authors: Shai Ben-David
Date: 2015-01-02 00:00:00
Tasks: Clustering

It is well known that most of the common clustering objectives are NP-hard to
optimize. In practice, however, clustering is being routinely carried out. One
approach for providing theoretical understanding of this seeming discrepancy is
to come up with notions of clusterability that distinguish realistically
interesting input data from worst-case data sets. The hope is that there will
be clustering algorithms that are provably efficient on such 'clusterable'
instances. In other words, hope that "Clustering is difficult only when it does
not matter" (CDNM thesis, for short).
  We believe that to some extent this may indeed be the case. This paper
provides a survey of recent papers along this line of research and a critical
evaluation their results. Our bottom line conclusion is that that CDNM thesis
is still far from being formally substantiated. We start by discussing which
requirements should be met in order to provide formal support the validity of
the CDNM thesis. In particular, we list some implied requirements for notions
of clusterability. We then examine existing results in view of those
requirements and outline some research challenges and open questions.