pwc-archive/papers-with-abstracts huggingface dataset:
Learning Opposites with Evolving Rules
Authors: Hamid. R. Tizhoosh, Shahryar Rahnamayan
Date: 2015-04-21 00:00:00
Tasks: Evolutionary Algorithms

The idea of opposition-based learning was introduced 10 years ago. Since then
a noteworthy group of researchers has used some notions of oppositeness to
improve existing optimization and learning algorithms. Among others,
evolutionary algorithms, reinforcement agents, and neural networks have been
reportedly extended into their opposition-based version to become faster and/or
more accurate. However, most works still use a simple notion of opposites,
namely linear (or type- I) opposition, that for each $x\in[a,b]$ assigns its
opposite as $\breve{x}_I=a+b-x$. This, of course, is a very naive estimate of
the actual or true (non-linear) opposite $\breve{x}_{II}$, which has been
called type-II opposite in literature. In absence of any knowledge about a
function $y=f(\mathbf{x})$ that we need to approximate, there seems to be no
alternative to the naivety of type-I opposition if one intents to utilize
oppositional concepts. But the question is if we can receive some level of
accuracy increase and time savings by using the naive opposite estimate
$\breve{x}_I$ according to all reports in literature, what would we be able to
gain, in terms of even higher accuracies and more reduction in computational
complexity, if we would generate and employ true opposites? This work
introduces an approach to approximate type-II opposites using evolving fuzzy
rules when we first perform opposition mining. We show with multiple examples
that learning true opposites is possible when we mine the opposites from the
training data to subsequently approximate $\breve{x}_{II}=f(\mathbf{x},y)$.
Temporal-Difference Networks
Authors: Richard S. Sutton, Brian Tanner
Date: 2015-04-21 00:00:00
Proceeding: NeurIPS 2004
Tasks: World Knowledge

We introduce a generalization of temporal-difference (TD) learning to
networks of interrelated predictions. Rather than relating a single prediction
to itself at a later time, as in conventional TD methods, a TD network relates
each prediction in a set of predictions to other predictions in the set at a
later time. TD networks can represent and apply TD learning to a much wider
class of predictions than has previously been possible. Using a random-walk
example, we show that these networks can be used to learn to predict by a fixed
interval, which is not possible with conventional TD methods. Secondly, we show
that if the inter-predictive relationships are made conditional on action, then
the usual learning-efficiency advantage of TD methods over Monte Carlo
(supervised learning) methods becomes particularly pronounced. Thirdly, we
demonstrate that TD networks can learn predictive state representations that
enable exact solution of a non-Markov problem. A very broad range of
inter-predictive temporal relationships can be expressed in these networks.
Overall we argue that TD networks represent a substantial extension of the
abilities of TD methods and bring us closer to the goal of representing world
knowledge in entirely predictive, grounded terms.
A robust and efficient video representation for action recognition
Authors: Heng Wang, Dan Oneata, Jakob Verbeek, Cordelia Schmid
Date: 2015-04-21 00:00:00
Tasks: Action Recognition, Homography Estimation, Motion Estimation, Optical Flow Estimation, Temporal Action Localization, Video Recognition

This paper introduces a state-of-the-art video representation and applies it
to efficient action recognition and detection. We first propose to improve the
popular dense trajectory features by explicit camera motion estimation. More
specifically, we extract feature point matches between frames using SURF
descriptors and dense optical flow. The matches are used to estimate a
homography with RANSAC. To improve the robustness of homography estimation, a
human detector is employed to remove outlier matches from the human body as
human motion is not constrained by the camera. Trajectories consistent with the
homography are considered as due to camera motion, and thus removed. We also
use the homography to cancel out camera motion from the optical flow. This
results in significant improvement on motion-based HOF and MBH descriptors. We
further explore the recent Fisher vector as an alternative feature encoding
approach to the standard bag-of-words histogram, and consider different ways to
include spatial layout information in these encodings. We present a large and
varied set of evaluations, considering (i) classification of short basic
actions on six datasets, (ii) localization of such actions in feature-length
movies, and (iii) large-scale recognition of complex events. We find that our
improved trajectory features significantly outperform previous dense
trajectories, and that Fisher vectors are superior to bag-of-words encodings
for video recognition tasks. In all three tasks, we show substantial
improvements over the state-of-the-art results.
Online Learning Algorithm for Time Series Forecasting Suitable for Low Cost Wireless Sensor Networks Nodes
Authors: Juan Pardo, Francisco Zamora-Martinez, Paloma Botella-Rocamora
Date: 2015-04-21 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Time series forecasting is an important predictive methodology which can be
applied to a wide range of problems. Particularly, forecasting the indoor
temperature permits an improved utilization of the HVAC (Heating, Ventilating
and Air Conditioning) systems in a home and thus a better energy efficiency.
With such purpose the paper describes how to implement an Artificial Neural
Network (ANN) algorithm in a low cost system-on-chip to develop an autonomous
intelligent wireless sensor network. The present paper uses a Wireless Sensor
Networks (WSN) to monitor and forecast the indoor temperature in a smart home,
based on low resources and cost microcontroller technology as the 8051MCU. An
on-line learning approach, based on Back-Propagation (BP) algorithm for ANNs,
has been developed for real-time time series learning. It performs the model
training with every new data that arrive to the system, without saving enormous
quantities of data to create a historical database as usual, i.e., without
previous knowledge. Consequently to validate the approach a simulation study
through a Bayesian baseline model have been tested in order to compare with a
database of a real application aiming to see the performance and accuracy. The
core of the paper is a new algorithm, based on the BP one, which has been
described in detail, and the challenge was how to implement a computational
demanding algorithm in a simple architecture with very few hardware resources.
Deep Convolutional Neural Networks Based on Semi-Discrete Frames
Authors: Thomas Wiatowski, Helmut Bölcskei
Date: 2015-04-21 00:00:00
Tasks: Translation

Deep convolutional neural networks have led to breakthrough results in
practical feature extraction applications. The mathematical analysis of these
networks was pioneered by Mallat, 2012. Specifically, Mallat considered
so-called scattering networks based on identical semi-discrete wavelet frames
in each network layer, and proved translation-invariance as well as deformation
stability of the resulting feature extractor. The purpose of this paper is to
develop Mallat's theory further by allowing for different and, most
importantly, general semi-discrete frames (such as, e.g., Gabor frames,
wavelets, curvelets, shearlets, ridgelets) in distinct network layers. This
allows to extract wider classes of features than point singularities resolved
by the wavelet transform. Our generalized feature extractor is proven to be
translation-invariant, and we develop deformation stability results for a
larger class of deformations than those considered by Mallat. For Mallat's
wavelet-based feature extractor, we get rid of a number of technical
conditions. The mathematical engine behind our results is continuous frame
theory, which allows us to completely detach the invariance and deformation
stability proofs from the particular algebraic structure of the underlying
frames.
Can FCA-based Recommender System Suggest a Proper Classifier?
Authors: Yury Kashnitsky, Dmitry I. Ignatov
Date: 2015-04-21 00:00:00
Tasks: Classification, General Classification, Object, Recommendation Systems

The paper briefly introduces multiple classifier systems and describes a new
algorithm, which improves classification accuracy by means of recommendation of
a proper algorithm to an object classification. This recommendation is done
assuming that a classifier is likely to predict the label of the object
correctly if it has correctly classified its neighbors. The process of
assigning a classifier to each object is based on Formal Concept Analysis. We
explain the idea of the algorithm with a toy example and describe our first
experiments with real-world datasets.
Visual analytics in FCA-based clustering
Authors: Yury Kashnitsky
Date: 2015-04-21 00:00:00
Tasks: Clustering, Decision Making, Recommendation Systems

Visual analytics is a subdomain of data analysis which combines both human
and machine analytical abilities and is applied mostly in decision-making and
data mining tasks. Triclustering, based on Formal Concept Analysis (FCA), was
developed to detect groups of objects with similar properties under similar
conditions. It is used in Social Network Analysis (SNA) and is a basis for
certain types of recommender systems. The problem of triclustering algorithms
is that they do not always produce meaningful clusters. This article describes
a specific triclustering algorithm and a prototype of a visual analytics
platform for working with obtained clusters. This tool is designed as a testing
frameworkis and is intended to help an analyst to grasp the results of
triclustering and recommender algorithms, and to make decisions on
meaningfulness of certain triclusters and recommendations.
Detecting Epileptic Seizures from EEG Data using Neural Networks
Authors: Siddharth Pramod, Adam Page, Tinoosh Mohsenin, Tim Oates
Date: 2014-12-19 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Sensitivity, Specificity

We explore the use of neural networks trained with dropout in predicting
epileptic seizures from electroencephalographic data (scalp EEG). The input to
the neural network is a 126 feature vector containing 9 features for each of
the 14 EEG channels obtained over 1-second, non-overlapping windows. The models
in our experiments achieved high sensitivity and specificity on patient records
not used in the training process. This is demonstrated using
leave-one-out-cross-validation across patient records, where we hold out one
patient's record as the test set and use all other patients' records for
training; repeating this procedure for all patients in the database.
A local approach to estimation in discrete loglinear models
Authors: Helene Massam, Nanwei Wang
Date: 2015-04-21 00:00:00

We consider two connected aspects of maximum likelihood estimation of the
parameter for high-dimensional discrete graphical models: the existence of the
maximum likelihood estimate (mle) and its computation.
  When the data is sparse, there are many zeros in the contingency table and
the maximum likelihood estimate of the parameter may not exist. Fienberg and
Rinaldo (2012) have shown that the mle does not exists iff the data vector
belongs to a face of the so-called marginal cone spanned by the rows of the
design matrix of the model. Identifying these faces in high-dimension is
challenging. In this paper, we take a local approach : we show that one such
face, albeit possibly not the smallest one, can be identified by looking at a
collection of marginal graphical models generated by induced subgraphs
$G_i,i=1,\ldots,k$ of $G$. This is our first contribution.
  Our second contribution concerns the composite maximum likelihood estimate.
When the dimension of the problem is large, estimating the parameters of a
given graphical model through maximum likelihood is challenging, if not
impossible. The traditional approach to this problem has been local with the
use of composite likelihood based on local conditional likelihoods.
  A more recent development is to have the components of the composite
likelihood be marginal likelihoods centred around each $v$. We first show that
the estimates obtained by consensus through local conditional and marginal
likelihoods are identical. We then study the asymptotic properties of the
composite maximum likelihood estimate when both the dimension of the model and
the sample size $N$ go to infinity.
Reasoning about Unmodelled Concepts - Incorporating Class Taxonomies in Probabilistic Relational Models
Authors: Daniel Nyga, Michael Beetz
Date: 2015-04-21 00:00:00

A key problem in the application of first-order probabilistic methods is the
enormous size of graphical models they imply. The size results from the
possible worlds that can be generated by a domain of objects and relations. One
of the reasons for this explosion is that so far the approaches do not
sufficiently exploit the structure and similarity of possible worlds in order
to encode the models more compactly. We propose fuzzy inference in Markov logic
networks, which enables the use of taxonomic knowledge as a source of imposing
structure onto possible worlds. We show that by exploiting this structure,
probability distributions can be represented more compactly and that the
reasoning systems become capable of reasoning about concepts not contained in
the probabilistic knowledge base.
Effective Discriminative Feature Selection with Non-trivial Solutions
Authors: Hong Tao, Chenping Hou, Feiping Nie, Yuanyuan Jiao, Dongyun Yi
Date: 2015-04-21 00:00:00
Tasks: Dimensionality Reduction, feature selection

Feature selection and feature transformation, the two main ways to reduce
dimensionality, are often presented separately. In this paper, a feature
selection method is proposed by combining the popular transformation based
dimensionality reduction method Linear Discriminant Analysis (LDA) and sparsity
regularization. We impose row sparsity on the transformation matrix of LDA
through ${\ell}_{2,1}$-norm regularization to achieve feature selection, and
the resultant formulation optimizes for selecting the most discriminative
features and removing the redundant ones simultaneously. The formulation is
extended to the ${\ell}_{2,p}$-norm regularized case: which is more likely to
offer better sparsity when $0<p<1$. Thus the formulation is a better
approximation to the feature selection problem. An efficient algorithm is
developed to solve the ${\ell}_{2,p}$-norm based optimization problem and it is
proved that the algorithm converges when $0<p\le 2$. Systematical experiments
are conducted to understand the work of the proposed method. Promising
experimental results on various types of real-world data sets demonstrate the
effectiveness of our algorithm.
Nonparametric Testing for Heterogeneous Correlation
Authors: Stephen Bamattre, Rex Hu, Joseph S. Verducci
Date: 2015-04-21 00:00:00

In the presence of weak overall correlation, it may be useful to investigate
if the correlation is significantly and substantially more pronounced over a
subpopulation. Two different testing procedures are compared. Both are based on
the rankings of the values of two variables from a data set with a large number
n of observations. The first maintains its level against Gaussian copulas; the
second adapts to general alternatives in the sense that that the number of
parameters used in the test grows with n. An analysis of wine quality
illustrates how the methods detect heterogeneity of association between
chemical properties of the wine, which are attributable to a mix of different
cultivars.
Computing Horn Rewritings of Description Logics Ontologies
Authors: Mark Kaminski, Bernardo Cuenca Grau
Date: 2015-04-20 00:00:00

We study the problem of rewriting an ontology O1 expressed in a DL L1 into an
ontology O2 in a Horn DL L2 such that O1 and O2 are equisatisfiable when
extended with an arbitrary dataset. Ontologies that admit such rewritings are
amenable to reasoning techniques ensuring tractability in data complexity.
After showing undecidability whenever L1 extends ALCF, we focus on devising
efficiently checkable conditions that ensure existence of a Horn rewriting. By
lifting existing techniques for rewriting Disjunctive Datalog programs into
plain Datalog to the case of arbitrary first-order programs with function
symbols, we identify a class of ontologies that admit Horn rewritings of
polynomial size. Our experiments indicate that many real-world ontologies
satisfy our sufficient conditions and thus admit polynomial Horn rewritings.
Key-Pose Prediction in Cyclic Human Motion
Authors: Dan Zecha, Rainer Lienhart
Date: 2015-04-21 00:00:00
Tasks: Pose Prediction, Prediction

In this paper we study the problem of estimating innercyclic time intervals
within repetitive motion sequences of top-class swimmers in a swimming channel.
Interval limits are given by temporal occurrences of key-poses, i.e.
distinctive postures of the body. A key-pose is defined by means of only one or
two specific features of the complete posture. It is often difficult to detect
such subtle features directly. We therefore propose the following method: Given
that we observe the swimmer from the side, we build a pictorial structure of
poselets to robustly identify random support poses within the regular motion of
a swimmer. We formulate a maximum likelihood model which predicts a key-pose
given the occurrences of multiple support poses within one stroke. The maximum
likelihood can be extended with prior knowledge about the temporal location of
a key-pose in order to improve the prediction recall. We experimentally show
that our models reliably and robustly detect key-poses with a high precision
and that their performance can be improved by extending the framework with
additional camera views.
Learning Activation Functions to Improve Deep Neural Networks
Authors: Forest Agostinelli, Matthew Hoffman, Peter Sadowski, Pierre Baldi
Date: 2014-12-21 00:00:00
Tasks: Image Classification

Artificial neural networks typically have a fixed, non-linear activation
function at each neuron. We have designed a novel form of piecewise linear
activation function that is learned independently for each neuron using
gradient descent. With this adaptive activation function, we are able to
improve upon deep neural network architectures composed of static rectified
linear units, achieving state-of-the-art performance on CIFAR-10 (7.51%),
CIFAR-100 (30.83%), and a benchmark from high-energy physics involving Higgs
boson decay modes.
A Group Theoretic Perspective on Unsupervised Deep Learning
Authors: Arnab Paul, Suresh Venkatasubramanian
Date: 2015-04-08 00:00:00
Tasks: Deep Learning

Why does Deep Learning work? What representations does it capture? How do
higher-order representations emerge? We study these questions from the
perspective of group theory, thereby opening a new approach towards a theory of
Deep learning.
  One factor behind the recent resurgence of the subject is a key algorithmic
step called {\em pretraining}: first search for a good generative model for the
input samples, and repeat the process one layer at a time. We show deeper
implications of this simple principle, by establishing a connection with the
interplay of orbits and stabilizers of group actions. Although the neural
networks themselves may not form groups, we show the existence of {\em shadow}
groups whose elements serve as close approximations.
  Over the shadow groups, the pre-training step, originally introduced as a
mechanism to better initialize a network, becomes equivalent to a search for
features with minimal orbits. Intuitively, these features are in a way the {\em
simplest}. Which explains why a deep learning network learns simple features
first. Next, we show how the same principle, when repeated in the deeper
layers, can capture higher order representations, and why representation
complexity increases as the layers get deeper.
Streaming Variational Inference for Bayesian Nonparametric Mixture Models
Authors: Alex Tank, Nicholas J. Foti, Emily B. Fox
Date: 2014-12-01 00:00:00
Tasks: Clustering, Variational Inference

In theory, Bayesian nonparametric (BNP) models are well suited to streaming
data scenarios due to their ability to adapt model complexity with the observed
data. Unfortunately, such benefits have not been fully realized in practice;
existing inference algorithms are either not applicable to streaming
applications or not extensible to BNP models. For the special case of Dirichlet
processes, streaming inference has been considered. However, there is growing
interest in more flexible BNP models building on the class of normalized random
measures (NRMs). We work within this general framework and present a streaming
variational inference algorithm for NRM mixture models. Our algorithm is based
on assumed density filtering (ADF), leading straightforwardly to expectation
propagation (EP) for large-scale batch inference as well. We demonstrate the
efficacy of the algorithm on clustering documents in large, streaming text
corpora.
Automatic Face Recognition from Video
Authors: Ognjen Arandjelovic
Date: 2015-04-21 00:00:00
Tasks: Clustering, Face Recognition

The objective of this work is to automatically recognize faces from video
sequences in a realistic, unconstrained setup in which illumination conditions
are extreme and greatly changing, viewpoint and user motion pattern have a wide
variability, and video input is of low quality. At the centre of focus are face
appearance manifolds: this thesis presents a significant advance of their
understanding and application in the sphere of face recognition. The two main
contributions are the Generic Shape-Illumination Manifold recognition algorithm
and the Anisotropic Manifold Space clustering. The Generic Shape-Illumination
Manifold is evaluated on a large data corpus acquired in real-world conditions
and its performance is shown to greatly exceed that of state-of-the-art methods
in the literature and the best performing commercial software. Empirical
evaluation of the Anisotropic Manifold Space clustering on a popular situation
comedy is also described with excellent preliminary results.
The adaptable buffer algorithm for high quantile estimation in non-stationary data streams
Authors: Ognjen Arandjelovic, Duc-Son Pham, Svetha Venkatesh
Date: 2015-04-21 00:00:00

The need to estimate a particular quantile of a distribution is an important
problem which frequently arises in many computer vision and signal processing
applications. For example, our work was motivated by the requirements of many
semi-automatic surveillance analytics systems which detect abnormalities in
close-circuit television (CCTV) footage using statistical models of low-level
motion features. In this paper we specifically address the problem of
estimating the running quantile of a data stream with non-stationary
stochasticity when the memory for storing observations is limited. We make
several major contributions: (i) we derive an important theoretical result
which shows that the change in the quantile of a stream is constrained
regardless of the stochastic properties of data, (ii) we describe a set of
high-level design goals for an effective estimation algorithm that emerge as a
consequence of our theoretical findings, (iii) we introduce a novel algorithm
which implements the aforementioned design goals by retaining a sample of data
values in a manner adaptive to changes in the distribution of data and
progressively narrowing down its focus in the periods of quasi-stationary
stochasticity, and (iv) we present a comprehensive evaluation of the proposed
algorithm and compare it with the existing methods in the literature on both
synthetic data sets and three large `real-world' streams acquired in the course
of operation of an existing commercial surveillance system. Our findings
convincingly demonstrate that the proposed method is highly successful and
vastly outperforms the existing alternatives, especially when the target
quantile is high valued and the available buffer capacity severely limited.