pwc-archive/papers-with-abstracts huggingface dataset:
Learning the Number of Autoregressive Mixtures in Time Series Using the Gap Statistics
Authors: Jie Ding, Mohammad Noshad, Vahid Tarokh
Date: 2015-09-11 00:00:00
Tasks: Model Selection, Time Series, Time Series Analysis

Using a proper model to characterize a time series is crucial in making
accurate predictions. In this work we use time-varying autoregressive process
(TVAR) to describe non-stationary time series and model it as a mixture of
multiple stable autoregressive (AR) processes. We introduce a new model
selection technique based on Gap statistics to learn the appropriate number of
AR filters needed to model a time series. We define a new distance measure
between stable AR filters and draw a reference curve that is used to measure
how much adding a new AR filter improves the performance of the model, and then
choose the number of AR filters that has the maximum gap with the reference
curve. To that end, we propose a new method in order to generate uniform random
stable AR filters in root domain. Numerical results are provided demonstrating
the performance of the proposed approach.
A DEEP analysis of the META-DES framework for dynamic selection of ensemble of classifiers
Authors: Rafael M. O. Cruz, Robert Sabourin, George D. C. Cavalcanti
Date: 2015-09-02 00:00:00
Tasks: Meta-Learning

Dynamic ensemble selection (DES) techniques work by estimating the level of
competence of each classifier from a pool of classifiers. Only the most
competent ones are selected to classify a given test sample. Hence, the key
issue in DES is the criterion used to estimate the level of competence of the
classifiers in predicting the label of a given test sample. In order to perform
a more robust ensemble selection, we proposed the META-DES framework using
meta-learning, where multiple criteria are encoded as meta-features and are
passed down to a meta-classifier that is trained to estimate the competence
level of a given classifier. In this technical report, we present a
step-by-step analysis of each phase of the framework during training and test.
We show how each set of meta-features is extracted as well as their impact on
the estimation of the competence level of the base classifier. Moreover, an
analysis of the impact of several factors in the system performance, such as
the number of classifiers in the pool, the use of different linear base
classifiers, as well as the size of the validation data. We show that using the
dynamic selection of linear classifiers through the META-DES framework, we can
solve complex non-linear classification problems where other combination
techniques such as AdaBoost cannot.
Efficient Convolutional Neural Networks for Pixelwise Classification on Heterogeneous Hardware Systems
Authors: Fabian Tschopp
Date: 2015-09-11 00:00:00
Tasks: General Classification

This work presents and analyzes three convolutional neural network (CNN)
models for efficient pixelwise classification of images. When using
convolutional neural networks to classify single pixels in patches of a whole
image, a lot of redundant computations are carried out when using sliding
window networks. This set of new architectures solve this issue by either
removing redundant computations or using fully convolutional architectures that
inherently predict many pixels at once.
  The implementations of the three models are accessible through a new utility
on top of the Caffe library. The utility provides support for a wide range of
image input and output formats, pre-processing parameters and methods to
equalize the label histogram during training. The Caffe library has been
extended by new layers and a new backend for availability on a wider range of
hardware such as CPUs and GPUs through OpenCL.
  On AMD GPUs, speedups of $54\times$ (SK-Net), $437\times$ (U-Net) and
$320\times$ (USK-Net) have been observed, taking the SK equivalent SW (sliding
window) network as the baseline. The label throughput is up to one megapixel
per second.
  The analyzed neural networks have distinctive characteristics that apply
during training or processing, and not every data set is suitable to every
architecture. The quality of the predictions is assessed on two neural tissue
data sets, of which one is the ISBI 2012 challenge data set. Two different loss
functions, Malis loss and Softmax loss, were used during training.
  The whole pipeline, consisting of models, interface and modified Caffe
library, is available as Open Source software under the working title Project
Greentea.
Performance Bounds for Pairwise Entity Resolution
Authors: Matt Barnes, Kyle Miller, Artur Dubrawski
Date: 2015-09-10 00:00:00
Tasks: BIG-bench Machine Learning, Entity Resolution

One significant challenge to scaling entity resolution algorithms to massive
datasets is understanding how performance changes after moving beyond the realm
of small, manually labeled reference datasets. Unlike traditional machine
learning tasks, when an entity resolution algorithm performs well on small
hold-out datasets, there is no guarantee this performance holds on larger
hold-out datasets. We prove simple bounding properties between the performance
of a match function on a small validation set and the performance of a pairwise
entity resolution algorithm on arbitrarily sized datasets. Thus, our approach
enables optimization of pairwise entity resolution algorithms for large
datasets, using a small set of labeled data.
Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests
Authors: Lucas Mentch, Giles Hooker
Date: 2014-04-25 00:00:00

This work develops formal statistical inference procedures for machine
learning ensemble methods. Ensemble methods based on bootstrapping, such as
bagging and random forests, have improved the predictive accuracy of individual
trees, but fail to provide a framework in which distributional results can be
easily determined. Instead of aggregating full bootstrap samples, we consider
predicting by averaging over trees built on subsamples of the training set and
demonstrate that the resulting estimator takes the form of a U-statistic. As
such, predictions for individual feature vectors are asymptotically normal,
allowing for confidence intervals to accompany predictions. In practice, a
subset of subsamples is used for computational speed; here our estimators take
the form of incomplete U-statistics and equivalent results are derived. We
further demonstrate that this setup provides a framework for testing the
significance of features. Moreover, the internal estimation method we develop
allows us to estimate the variance parameters and perform these inference
procedures at no additional computational cost. Simulations and illustrations
on a real dataset are provided.
Classifying Tweet Level Judgements of Rumours in Social Media
Authors: Michal Lukasik, Trevor Cohn, Kalina Bontcheva
Date: 2015-06-01 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Domain Adaptation, Multi-Task Learning, Rumour Detection, Unsupervised Domain Adaptation

Social media is a rich source of rumours and corresponding community
reactions. Rumours reflect different characteristics, some shared and some
individual. We formulate the problem of classifying tweet level judgements of
rumours as a supervised learning task. Both supervised and unsupervised domain
adaptation are considered, in which tweets from a rumour are classified on the
basis of other annotated rumours. We demonstrate how multi-task learning helps
achieve good results on rumours from the 2011 England riots.
A deep matrix factorization method for learning attribute representations
Authors: George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, Bjoern W. Schuller
Date: 2015-09-10 00:00:00
Tasks: Attribute, Clustering

Semi-Non-negative Matrix Factorization is a technique that learns a
low-dimensional representation of a dataset that lends itself to a clustering
interpretation. It is possible that the mapping between this new representation
and our original data matrix contains rather complex hierarchical information
with implicit lower-level hidden attributes, that classical one level
clustering methodologies can not interpret. In this work we propose a novel
model, Deep Semi-NMF, that is able to learn such hidden representations that
allow themselves to an interpretation of clustering according to different,
unknown attributes of a given dataset. We also present a semi-supervised
version of the algorithm, named Deep WSF, that allows the use of (partial)
prior information for each of the known attributes of a dataset, that allows
the model to be used on datasets with mixed attribute knowledge. Finally, we
show that our models are able to learn low-dimensional representations that are
better suited for clustering, but also classification, outperforming
Semi-Non-negative Matrix Factorization, but also other state-of-the-art
methodologies variants.
An Epsilon Hierarchical Fuzzy Twin Support Vector Regression
Authors: Arindam Chaudhuri
Date: 2015-09-10 00:00:00
Tasks: regression

The research presents epsilon hierarchical fuzzy twin support vector
regression based on epsilon fuzzy twin support vector regression and epsilon
twin support vector regression. Epsilon FTSVR is achieved by incorporating
trapezoidal fuzzy numbers to epsilon TSVR which takes care of uncertainty
existing in forecasting problems. Epsilon FTSVR determines a pair of epsilon
insensitive proximal functions by solving two related quadratic programming
problems. The structural risk minimization principle is implemented by
introducing regularization term in primal problems of epsilon FTSVR. This
yields dual stable positive definite problems which improves regression
performance. Epsilon FTSVR is then reformulated as epsilon HFTSVR consisting of
a set of hierarchical layers each containing epsilon FTSVR. Experimental
results on both synthetic and real datasets reveal that epsilon HFTSVR has
remarkable generalization performance with minimum training time.
Gibbs Sampling Strategies for Semantic Perception of Streaming Video Data
Authors: Yogesh Girdhar, Gregory Dudek
Date: 2015-09-10 00:00:00

Topic modeling of streaming sensor data can be used for high level perception
of the environment by a mobile robot. In this paper we compare various Gibbs
sampling strategies for topic modeling of streaming spatiotemporal data, such
as video captured by a mobile robot. Compared to previous work on online topic
modeling, such as o-LDA and incremental LDA, we show that the proposed
technique results in lower online and final perplexity, given the realtime
constraints.
Scalable Kernel Methods via Doubly Stochastic Gradients
Authors: Bo Dai, Bo Xie, Niao He, YIngyu Liang, Anant Raj, Maria-Florina Balcan, Le Song
Date: 2014-07-21 00:00:00
Proceeding: NeurIPS 2014 12

The general perception is that kernel methods are not scalable, and neural
nets are the methods of choice for nonlinear learning problems. Or have we
simply not tried hard enough for kernel methods? Here we propose an approach
that scales up kernel methods using a novel concept called "doubly stochastic
functional gradients". Our approach relies on the fact that many kernel methods
can be expressed as convex optimization problems, and we solve the problems by
making two unbiased stochastic approximations to the functional gradient, one
using random training points and another using random functions associated with
the kernel, and then descending using this noisy functional gradient. We show
that a function produced by this procedure after $t$ iterations converges to
the optimal function in the reproducing kernel Hilbert space in rate $O(1/t)$,
and achieves a generalization performance of $O(1/\sqrt{t})$. This doubly
stochasticity also allows us to avoid keeping the support vectors and to
implement the algorithm in a small memory footprint, which is linear in number
of iterations and independent of data dimension. Our approach can readily scale
kernel methods up to the regimes which are dominated by neural nets. We show
that our method can achieve competitive performance to neural nets in datasets
such as 8 million handwritten digits from MNIST, 2.3 million energy materials
from MolecularSpace, and 1 million photos from ImageNet.
Learning to Linearize Under Uncertainty
Authors: Ross Goroshin, Michael Mathieu, Yann Lecun
Date: 2015-06-09 00:00:00
Proceeding: NeurIPS 2015 12

Training deep feature hierarchies to solve supervised learning tasks has
achieved state of the art performance on many problems in computer vision.
However, a principled way in which to train such hierarchies in the
unsupervised setting has remained elusive. In this work we suggest a new
architecture and loss for training deep feature hierarchies that linearize the
transformations observed in unlabeled natural video sequences. This is done by
training a generative model to predict video frames. We also address the
problem of inherent uncertainty in prediction by introducing latent variables
that are non-deterministic functions of the input into the network
architecture.
Use it or Lose it: Selective Memory and Forgetting in a Perpetual Learning Machine
Authors: Andrew J. R. Simpson
Date: 2015-09-10 00:00:00

In a recent article we described a new type of deep neural network - a
Perpetual Learning Machine (PLM) - which is capable of learning 'on the fly'
like a brain by existing in a state of Perpetual Stochastic Gradient Descent
(PSGD). Here, by simulating the process of practice, we demonstrate both
selective memory and selective forgetting when we introduce statistical recall
biases during PSGD. Frequently recalled memories are remembered, whilst
memories recalled rarely are forgotten. This results in a 'use it or lose it'
stimulus driven memory process that is similar to human memory.
OneMax in Black-Box Models with Several Restrictions
Authors: Carola Doerr, Johannes Lengler
Date: 2015-04-10 00:00:00
Tasks: Evolutionary Algorithms

Black-box complexity studies lower bounds for the efficiency of
general-purpose black-box optimization algorithms such as evolutionary
algorithms and other search heuristics. Different models exist, each one being
designed to analyze a different aspect of typical heuristics such as the memory
size or the variation operators in use. While most of the previous works focus
on one particular such aspect, we consider in this work how the combination of
several algorithmic restrictions influence the black-box complexity. Our
testbed are so-called OneMax functions, a classical set of test functions that
is intimately related to classic coin-weighing problems and to the board game
Mastermind.
  We analyze in particular the combined memory-restricted ranking-based
black-box complexity of OneMax for different memory sizes. While its isolated
memory-restricted as well as its ranking-based black-box complexity for bit
strings of length $n$ is only of order $n/\log n$, the combined model does not
allow for algorithms being faster than linear in $n$, as can be seen by
standard information-theoretic considerations. We show that this linear bound
is indeed asymptotically tight. Similar results are obtained for other memory-
and offspring-sizes. Our results also apply to the (Monte Carlo) complexity of
OneMax in the recently introduced elitist model, in which only the best-so-far
solution can be kept in the memory. Finally, we also provide improved lower
bounds for the complexity of OneMax in the regarded models.
  Our result enlivens the quest for natural evolutionary algorithms optimizing
OneMax in $o(n \log n)$ iterations.
Nonparametric regression using needlet kernels for spherical data
Authors: Shaobo Lin
Date: 2015-02-14 00:00:00
Tasks: regression

Needlets have been recognized as state-of-the-art tools to tackle spherical
data, due to their excellent localization properties in both spacial and
frequency domains.
  This paper considers developing kernel methods associated with the needlet
kernel for nonparametric regression problems whose predictor variables are
defined on a sphere. Due to the localization property in the frequency domain,
we prove that the regularization parameter of the kernel ridge regression
associated with the needlet kernel can decrease arbitrarily fast. A natural
consequence is that the regularization term for the kernel ridge regression is
not necessary in the sense of rate optimality. Based on the excellent
localization property in the spacial domain further, we also prove that all the
$l^{q}$ $(01\leq q < \infty)$ kernel regularization estimates associated with
the needlet kernel, including the kernel lasso estimate and the kernel bridge
estimate, possess almost the same generalization capability for a large range
of regularization parameters in the sense of rate optimality.
  This finding tentatively reveals that, if the needlet kernel is utilized,
then the choice of $q$ might not have a strong impact in terms of the
generalization capability in some modeling contexts. From this perspective, $q$
can be arbitrarily specified, or specified merely by other no generalization
criteria like smoothness, computational complexity, sparsity, etc..
The World of Combinatorial Fuzzy Problems and the Efficiency of Fuzzy Approximation Algorithms
Authors: Tomoyuki Yamakami
Date: 2015-09-10 00:00:00

We re-examine a practical aspect of combinatorial fuzzy problems of various
types, including search, counting, optimization, and decision problems. We are
focused only on those fuzzy problems that take series of fuzzy input objects
and produce fuzzy values. To solve such problems efficiently, we design fast
fuzzy algorithms, which are modeled by polynomial-time deterministic fuzzy
Turing machines equipped with read-only auxiliary tapes and write-only output
tapes and also modeled by polynomial-size fuzzy circuits composed of fuzzy
gates. We also introduce fuzzy proof verification systems to model the
fuzzification of nondeterminism. Those models help us identify four complexity
classes: Fuzzy-FPA of fuzzy functions, Fuzzy-PA and Fuzzy-NPA of fuzzy decision
problems, and Fuzzy-NPAO of fuzzy optimization problems. Based on a relative
approximation scheme targeting fuzzy membership degree, we formulate two
notions of "reducibility" in order to compare the computational complexity of
two fuzzy problems. These reducibility notions make it possible to locate the
most difficult fuzzy problems in Fuzzy-NPA and in Fuzzy-NPAO.
Integrate Document Ranking Information into Confidence Measure Calculation for Spoken Term Detection
Authors: Quan Liu, Wu Guo, Zhen-Hua Ling
Date: 2015-09-07 00:00:00
Tasks: Document Ranking

This paper proposes an algorithm to improve the calculation of confidence
measure for spoken term detection (STD). Given an input query term, the
algorithm first calculates a measurement named document ranking weight for each
document in the speech database to reflect its relevance with the query term by
summing all the confidence measures of the hypothesized term occurrences in
this document. The confidence measure of each term occurrence is then
re-estimated through linear interpolation with the calculated document ranking
weight to improve its reliability by integrating document-level information.
Experiments are conducted on three standard STD tasks for Tamil, Vietnamese and
English respectively. The experimental results all demonstrate that the
proposed algorithm achieves consistent improvements over the state-of-the-art
method for confidence measure calculation. Furthermore, this algorithm is still
effective even if a high accuracy speech recognizer is not available, which
makes it applicable for the languages with limited speech resources.
Weighted Classification Cascades for Optimizing Discovery Significance in the HiggsML Challenge
Authors: Lester Mackey, Jordan Bryan, Man Yue Mo
Date: 2014-09-09 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, Classification, General Classification

We introduce a minorization-maximization approach to optimizing common
measures of discovery significance in high energy physics. The approach
alternates between solving a weighted binary classification problem and
updating class weights in a simple, closed-form manner. Moreover, an argument
based on convex duality shows that an improvement in weighted classification
error on any round yields a commensurate improvement in discovery significance.
We complement our derivation with experimental results from the 2014 Higgs
boson machine learning challenge.
Identifying Emotion from Natural Walking
Authors: Liqing Cui, Shun Li, Wan Zhang, Zhan Zhang, Tingshao Zhu
Date: 2015-08-03 00:00:00
Tasks: Classification, General Classification

Emotion identification from gait aims to automatically determine persons
affective state, it has attracted a great deal of interests and offered immense
potential value in action tendency, health care, psychological detection and
human-computer(robot) interaction.In this paper, we propose a new method of
identifying emotion from natural walking, and analyze the relevance between the
traits of walking and affective states. After obtaining the pure acceleration
data of wrist and ankle, we set a moving average filter window with different
sizes w, then extract 114 features including time-domain, frequency-domain,
power and distribution features from each data slice, and run principal
component analysis (PCA) to reduce dimension. In experiments, we train SVM,
Decision Tree, multilayerperception, Random Tree and Random Forest
classification models, and compare the classification accuracy on data of wrist
and ankle with respect to different w. The performance of emotion
identification on acceleration data of ankle is better than wrist.Comparing
different classification models' results, SVM has best accuracy of identifying
anger and happy could achieve 90:31% and 89:76% respectively, and
identification ratio of anger-happy is 87:10%.The anger-neutral-happy
classification reaches 85%-78%-78%.The results show that it is capable of
identifying personal emotional states through the gait of walking.
Making Risk Minimization Tolerant to Label Noise
Authors: Aritra Ghosh, Naresh Manwani, P. S. Sastry
Date: 2014-03-14 00:00:00

In many applications, the training data, from which one needs to learn a
classifier, is corrupted with label noise. Many standard algorithms such as SVM
perform poorly in presence of label noise. In this paper we investigate the
robustness of risk minimization to label noise. We prove a sufficient condition
on a loss function for the risk minimization under that loss to be tolerant to
uniform label noise. We show that the $0-1$ loss, sigmoid loss, ramp loss and
probit loss satisfy this condition though none of the standard convex loss
functions satisfy it. We also prove that, by choosing a sufficiently large
value of a parameter in the loss function, the sigmoid loss, ramp loss and
probit loss can be made tolerant to non-uniform label noise also if we can
assume the classes to be separable under noise-free data distribution. Through
extensive empirical studies, we show that risk minimization under the $0-1$
loss, the sigmoid loss and the ramp loss has much better robustness to label
noise when compared to the SVM algorithm.