pwc-archive/papers-with-abstracts huggingface dataset:
A Telescopic Binary Learning Machine for Training Neural Networks
Authors: Mauro Brunato, Roberto Battiti
Date: 2015-09-01 00:00:00

This paper proposes a new algorithm based on multi-scale stochastic local
search with binary representation for training neural networks.
  In particular, we study the effects of neighborhood evaluation strategies,
the effect of the number of bits per weight and that of the maximum weight
range used for mapping binary strings to real values. Following this
preliminary investigation, we propose a telescopic multi-scale version of local
search where the number of bits is increased in an adaptive manner, leading to
a faster search and to local minima of better quality. An analysis related to
adapting the number of bits in a dynamic way is also presented. The control on
the number of bits, which happens in a natural manner in the proposed method,
is effective to increase the generalization performance. Benchmark tasks
include a highly non-linear artificial problem, a control problem requiring
either feed-forward or recurrent architectures for feedback control, and
challenging real-world tasks in different application domains.
  The results demonstrate the effectiveness of the proposed method.
Online Supervised Subspace Tracking
Authors: Yao Xie, Ruiyang Song, Hanjun Dai, Qingbin Li, Le Song
Date: 2015-09-01 00:00:00
Tasks: Dimensionality Reduction, regression, Time Series, Time Series Analysis

We present a framework for supervised subspace tracking, when there are two
time series $x_t$ and $y_t$, one being the high-dimensional predictors and the
other being the response variables and the subspace tracking needs to take into
consideration of both sequences. It extends the classic online subspace
tracking work which can be viewed as tracking of $x_t$ only. Our online
sufficient dimensionality reduction (OSDR) is a meta-algorithm that can be
applied to various cases including linear regression, logistic regression,
multiple linear regression, multinomial logistic regression, support vector
machine, the random dot product model and the multi-scale union-of-subspace
model. OSDR reduces data-dimensionality on-the-fly with low-computational
complexity and it can also handle missing data and dynamic data. OSDR uses an
alternating minimization scheme and updates the subspace via gradient descent
on the Grassmannian manifold. The subspace update can be performed efficiently
utilizing the fact that the Grassmannian gradient with respect to the subspace
in many settings is rank-one (or low-rank in certain cases). The optimization
problem for OSDR is non-convex and hard to analyze in general; we provide
convergence analysis of OSDR in a simple linear regression setting. The good
performance of OSDR compared with the conventional unsupervised subspace
tracking are demonstrated via numerical examples on simulated and real data.
Sequential Information Guided Sensing
Authors: Ruiyang Song, Yao Xie, Sebastian Pokutta
Date: 2015-09-01 00:00:00
Tasks: compressed sensing

We study the value of information in sequential compressed sensing by
characterizing the performance of sequential information guided sensing in
practical scenarios when information is inaccurate. In particular, we assume
the signal distribution is parameterized through Gaussian or Gaussian mixtures
with estimated mean and covariance matrices, and we can measure compressively
through a noisy linear projection or using one-sparse vectors, i.e., observing
one entry of the signal each time. We establish a set of performance bounds for
the bias and variance of the signal estimator via posterior mean, by capturing
the conditional entropy (which is also related to the size of the uncertainty),
and the additional power required due to inaccurate information to reach a
desired precision. Based on this, we further study how to estimate covariance
based on direct samples or covariance sketching. Numerical examples also
demonstrate the superior performance of Info-Greedy Sensing algorithms compared
with their random and non-adaptive counterparts.
Evolving Unipolar Memristor Spiking Neural Networks
Authors: David Howard, Larry Bull, Ben De Lacy Costello
Date: 2015-09-01 00:00:00

Neuromorphic computing --- brainlike computing in hardware --- typically
requires myriad CMOS spiking neurons interconnected by a dense mesh of
nanoscale plastic synapses. Memristors are frequently citepd as strong synapse
candidates due to their statefulness and potential for low-power
implementations. To date, plentiful research has focused on the bipolar
memristor synapse, which is capable of incremental weight alterations and can
provide adaptive self-organisation under a Hebbian learning scheme. In this
paper we consider the Unipolar memristor synapse --- a device capable of
non-Hebbian switching between only two states (conductive and resistive)
through application of a suitable input voltage --- and discuss its suitability
for neuromorphic systems. A self-adaptive evolutionary process is used to
autonomously find highly fit network configurations. Experimentation on a two
robotics tasks shows that unipolar memristor networks evolve task-solving
controllers faster than both bipolar memristor networks and networks containing
constant nonplastic connections whilst performing at least comparably.
Metastatic liver tumour segmentation from discriminant Grassmannian manifolds
Authors: Samuel Kadoury, Eugene Vorontsov, An Tang
Date: 2015-08-31 00:00:00
Tasks: Segmentation

The early detection, diagnosis and monitoring of liver cancer progression can
be achieved with the precise delineation of metastatic tumours. However,
accurate automated segmentation remains challenging due to the presence of
noise, inhomogeneity and the high appearance variability of malignant tissue.
In this paper, we propose an unsupervised metastatic liver tumour segmentation
framework using a machine learning approach based on discriminant Grassmannian
manifolds which learns the appearance of tumours with respect to normal tissue.
First, the framework learns within-class and between-class similarity
distributions from a training set of images to discover the optimal manifold
discrimination between normal and pathological tissue in the liver. Second, a
conditional optimisation scheme computes nonlocal pairwise as well as
pattern-based clique potentials from the manifold subspace to recognise regions
with similar labelings and to incorporate global consistency in the
segmentation process. The proposed framework was validated on a clinical
database of 43 CT images from patients with metastatic liver cancer. Compared
to state-of-the-art methods, our method achieves a better performance on two
separate datasets of metastatic liver tumours from different clinical sites,
yielding an overall mean Dice similarity coefficient of 90.7 +/- 2.4 in over 50
tumours with an average volume of 27.3 mm3.
Value function approximation via low-rank models
Authors: Hao Yi Ong
Date: 2015-08-31 00:00:00

We propose a novel value function approximation technique for Markov decision
processes. We consider the problem of compactly representing the state-action
value function using a low-rank and sparse matrix model. The problem is to
decompose a matrix that encodes the true value function into low-rank and
sparse components, and we achieve this using Robust Principal Component
Analysis (PCA). Under minimal assumptions, this Robust PCA problem can be
solved exactly via the Principal Component Pursuit convex optimization problem.
We experiment the procedure on several examples and demonstrate that our method
yields approximations essentially identical to the true function.
Pure and Hybrid Evolutionary Computing in Global Optimization of Chemical Structures: from Atoms and Molecules to Clusters and Crystals
Authors: Kanchan Sarkar, S. P. Bhattacharyya
Date: 2015-08-31 00:00:00
Tasks: global-optimization

The growth of evolutionary computing (EC) methods in the exploration of
complex potential energy landscapes of atomic and molecular clusters, as well
as crystals over the last decade or so is reviewed. The trend of growth
indicates that pure as well as hybrid evolutionary computing techniques in
conjunction of DFT has been emerging as a powerful tool, although work on
molecular clusters has been rather limited so far. Some attempts to solve the
atomic/molecular Schrodinger Equation (SE) directly by genetic algorithms (GA)
are available in literature. At the Born-Oppenheimer level of approximation
GA-density methods appear to be a viable tool which could be more extensively
explored in the coming years, specially in the context of designing molecules
and materials with targeted properties.
Approximate Nearest Neighbor Fields in Video
Authors: Nir Ben-Zrihem, Lihi Zelnik-Manor
Date: 2015-08-31 00:00:00
Proceeding: CVPR 2015 6
Tasks: Colorization, Denoising

We introduce RIANN (Ring Intersection Approximate Nearest Neighbor search),
an algorithm for matching patches of a video to a set of reference patches in
real-time. For each query, RIANN finds potential matches by intersecting rings
around key points in appearance space. Its search complexity is reversely
correlated to the amount of temporal change, making it a good fit for videos,
where typically most patches change slowly with time. Experiments show that
RIANN is up to two orders of magnitude faster than previous ANN methods, and is
the only solution that operates in real-time. We further demonstrate how RIANN
can be used for real-time video processing and provide examples for a range of
real-time video applications, including colorization, denoising, and several
artistic effects.
Relax but stay in control: from value to algorithms for online Markov decision processes
Authors: Peng Guan, Maxim Raginsky, Rebecca Willett
Date: 2013-10-28 00:00:00

Online learning algorithms are designed to perform in non-stationary
environments, but generally there is no notion of a dynamic state to model
constraints on current and future actions as a function of past actions.
State-based models are common in stochastic control settings, but commonly used
frameworks such as Markov Decision Processes (MDPs) assume a known stationary
environment. In recent years, there has been a growing interest in combining
the above two frameworks and considering an MDP setting in which the cost
function is allowed to change arbitrarily after each time step. However, most
of the work in this area has been algorithmic: given a problem, one would
develop an algorithm almost from scratch. Moreover, the presence of the state
and the assumption of an arbitrarily varying environment complicate both the
theoretical analysis and the development of computationally efficient methods.
This paper describes a broad extension of the ideas proposed by Rakhlin et al.
to give a general framework for deriving algorithms in an MDP setting with
arbitrarily changing costs. This framework leads to a unifying view of existing
methods and provides a general procedure for constructing new ones. Several new
methods are presented, and one of them is shown to have important advantages
over a similar method developed from scratch via an online version of
approximate dynamic programming.
Multi-Projector Color Structured-Light Vision
Authors: Changsoo Je, Kwang Hee Lee, Sang Wook Lee
Date: 2015-08-31 00:00:00

Research interest in rapid structured-light imaging has grown increasingly
for the modeling of moving objects, and a number of methods have been suggested
for the range capture in a single video frame. The imaging area of a 3D object
using a single projector is restricted since the structured light is projected
only onto a limited area of the object surface. Employing additional projectors
to broaden the imaging area is a challenging problem since simultaneous
projection of multiple patterns results in their superposition in the
light-intersected areas and the recognition of original patterns is by no means
trivial. This paper presents a novel method of multi-projector color
structured-light vision based on projector-camera triangulation. By analyzing
the behavior of superposed-light colors in a chromaticity domain, we show that
the original light colors cannot be properly extracted by the conventional
direct estimation. We disambiguate multiple projectors by multiplexing the
orientations of projector patterns so that the superposed patterns can be
separated by explicit derivative computations. Experimental studies are carried
out to demonstrate the validity of the presented method. The proposed method
increases the efficiency of range acquisition compared to conventional active
stereo using multiple projectors.
Model Guided Sampling Optimization for Low-dimensional Problems
Authors: Lukas Bajer, Martin Holena
Date: 2015-08-31 00:00:00
Tasks: model

Optimization of very expensive black-box functions requires utilization of
maximum information gathered by the process of optimization. Model Guided
Sampling Optimization (MGSO) forms a more robust alternative to Jones'
Gaussian-process-based EGO algorithm. Instead of EGO's maximizing expected
improvement, the MGSO uses sampling the probability of improvement which is
shown to be helpful against trapping in local minima. Further, the MGSO can
reach close-to-optimum solutions faster than standard optimization algorithms
on low dimensional or smooth problems.
Learning Modulo Theories for preference elicitation in hybrid domains
Authors: Paolo Campigotto, Roberto Battiti, Andrea Passerini
Date: 2015-08-18 00:00:00
Tasks: Learning-To-Rank

This paper introduces CLEO, a novel preference elicitation algorithm capable
of recommending complex objects in hybrid domains, characterized by both
discrete and continuous attributes and constraints defined over them. The
algorithm assumes minimal initial information, i.e., a set of catalog
attributes, and defines decisional features as logic formulae combining Boolean
and algebraic constraints over the attributes. The (unknown) utility of the
decision maker (DM) is modelled as a weighted combination of features. CLEO
iteratively alternates a preference elicitation step, where pairs of candidate
solutions are selected based on the current utility model, and a refinement
step where the utility is refined by incorporating the feedback received. The
elicitation step leverages a Max-SMT solver to return optimal hybrid solutions
according to the current utility model. The refinement step is implemented as
learning to rank, and a sparsifying norm is used to favour the selection of few
informative features in the combinatorial space of candidate decisional
features.
  CLEO is the first preference elicitation algorithm capable of dealing with
hybrid domains, thanks to the use of Max-SMT technology, while retaining
uncertainty in the DM utility and noisy feedback. Experimental results on
complex recommendation tasks show the ability of CLEO to quickly focus towards
optimal solutions, as well as its capacity to recover from suboptimal initial
choices. While no competitors exist in the hybrid setting, CLEO outperforms a
state-of-the-art Bayesian preference elicitation algorithm when applied to a
purely discrete task.
A Cognitive Architecture Based on a Learning Classifier System with Spiking Classifiers
Authors: David Howard, Larry Bull, Pier-Luca Lanzi
Date: 2015-08-31 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Learning Classifier Systems (LCS) are population-based reinforcement learners
that were originally designed to model various cognitive phenomena. This paper
presents an explicitly cognitive LCS by using spiking neural networks as
classifiers, providing each classifier with a measure of temporal dynamism. We
employ a constructivist model of growth of both neurons and synaptic
connections, which permits a Genetic Algorithm (GA) to automatically evolve
sufficiently-complex neural structures. The spiking classifiers are coupled
with a temporally-sensitive reinforcement learning algorithm, which allows the
system to perform temporal state decomposition by appropriately rewarding
"macro-actions," created by chaining together multiple atomic actions. The
combination of temporal reinforcement learning and neural information
processing is shown to outperform benchmark neural classifier systems, and
successfully solve a robotic navigation task.
Domain Generalization for Object Recognition with Multi-task Autoencoders
Authors: Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang, David Balduzzi
Date: 2015-08-31 00:00:00
Proceeding: ICCV 2015 12
Tasks: Denoising, Domain Generalization, Object, Object Recognition

The problem of domain generalization is to take knowledge acquired from a
number of related domains where training data is available, and to then
successfully apply it to previously unseen domains. We propose a new feature
learning algorithm, Multi-Task Autoencoder (MTAE), that provides good
generalization performance for cross-domain object recognition.
  Our algorithm extends the standard denoising autoencoder framework by
substituting artificially induced corruption with naturally occurring
inter-domain variability in the appearance of objects. Instead of
reconstructing images from noisy versions, MTAE learns to transform the
original image into analogs in multiple related domains. It thereby learns
features that are robust to variations across domains. The learnt features are
then used as inputs to a classifier.
  We evaluated the performance of the algorithm on benchmark image recognition
datasets, where the task is to learn features from multiple datasets and to
then predict the image label from unseen datasets. We found that (denoising)
MTAE outperforms alternative autoencoder-based models as well as the current
state-of-the-art algorithms for domain generalization.
Online Model Evaluation in a Large-Scale Computational Advertising Platform
Authors: Shahriar Shariat, Burkay Orten, Ali Dasdan
Date: 2015-08-31 00:00:00
Tasks: Marketing

Online media provides opportunities for marketers through which they can
deliver effective brand messages to a wide range of audiences. Advertising
technology platforms enable advertisers to reach their target audience by
delivering ad impressions to online users in real time. In order to identify
the best marketing message for a user and to purchase impressions at the right
price, we rely heavily on bid prediction and optimization models. Even though
the bid prediction models are well studied in the literature, the equally
important subject of model evaluation is usually overlooked. Effective and
reliable evaluation of an online bidding model is crucial for making faster
model improvements as well as for utilizing the marketing budgets more
efficiently. In this paper, we present an experimentation framework for bid
prediction models where our focus is on the practical aspects of model
evaluation. Specifically, we outline the unique challenges we encounter in our
platform due to a variety of factors such as heterogeneous goal definitions,
varying budget requirements across different campaigns, high seasonality and
the auction-based environment for inventory purchasing. Then, we introduce
return on investment (ROI) as a unified model performance (i.e., success)
metric and explain its merits over more traditional metrics such as
click-through rate (CTR) or conversion rate (CVR). Most importantly, we discuss
commonly used evaluation and metric summarization approaches in detail and
propose a more accurate method for online evaluation of new experimental models
against the baseline. Our meta-analysis-based approach addresses various
shortcomings of other methods and yields statistically robust conclusions that
allow us to conclude experiments more quickly in a reliable manner. We
demonstrate the effectiveness of our evaluation strategy on real campaign data
through some experiments.
Action Recognition by Hierarchical Mid-level Action Elements
Authors: Tian Lan, Yuke Zhu, Amir Roshan Zamir, Silvio Savarese
Date: 2015-08-31 00:00:00
Proceeding: ICCV 2015 12
Tasks: Action Parsing, Action Recognition, Clustering, Temporal Action Localization

Realistic videos of human actions exhibit rich spatiotemporal structures at
multiple levels of granularity: an action can always be decomposed into
multiple finer-grained elements in both space and time. To capture this
intuition, we propose to represent videos by a hierarchy of mid-level action
elements (MAEs), where each MAE corresponds to an action-related spatiotemporal
segment in the video. We introduce an unsupervised method to generate this
representation from videos. Our method is capable of distinguishing
action-related segments from background segments and representing actions at
multiple spatiotemporal resolutions. Given a set of spatiotemporal segments
generated from the training data, we introduce a discriminative clustering
algorithm that automatically discovers MAEs at multiple levels of granularity.
We develop structured models that capture a rich set of spatial, temporal and
hierarchical relations among the segments, where the action label and multiple
levels of MAE labels are jointly inferred. The proposed model achieves
state-of-the-art performance in multiple action recognition benchmarks.
Moreover, we demonstrate the effectiveness of our model in real-world
applications such as action recognition in large-scale untrimmed videos and
action parsing.
Dictionary Learning for Blind One Bit Compressed Sensing
Authors: Hadi Zayyani, Mehdi Korki, Farrokh Marvasti
Date: 2015-08-30 00:00:00
Tasks: compressed sensing, Dictionary Learning

This letter proposes a dictionary learning algorithm for blind one bit
compressed sensing. In the blind one bit compressed sensing framework, the
original signal to be reconstructed from one bit linear random measurements is
sparse in an unknown domain. In this context, the multiplication of measurement
matrix $\Ab$ and sparse domain matrix $\Phi$, \ie $\Db=\Ab\Phi$, should be
learned. Hence, we use dictionary learning to train this matrix. Towards that
end, an appropriate continuous convex cost function is suggested for one bit
compressed sensing and a simple steepest-descent method is exploited to learn
the rows of the matrix $\Db$. Experimental results show the effectiveness of
the proposed algorithm against the case of no dictionary learning, specially
with increasing the number of training signals and the number of sign
measurements.
Stabilized Nearest Neighbor Classifier and Its Statistical Properties
Authors: Wei Sun, Xingye Qiao, Guang Cheng
Date: 2014-05-26 00:00:00
Tasks: Classification, General Classification

The stability of statistical analysis is an important indicator for
reproducibility, which is one main principle of scientific method. It entails
that similar statistical conclusions can be reached based on independent
samples from the same underlying population. In this paper, we introduce a
general measure of classification instability (CIS) to quantify the sampling
variability of the prediction made by a classification method. Interestingly,
the asymptotic CIS of any weighted nearest neighbor classifier turns out to be
proportional to the Euclidean norm of its weight vector. Based on this concise
form, we propose a stabilized nearest neighbor (SNN) classifier, which
distinguishes itself from other nearest neighbor classifiers, by taking the
stability into consideration. In theory, we prove that SNN attains the minimax
optimal convergence rate in risk, and a sharp convergence rate in CIS. The
latter rate result is established for general plug-in classifiers under a
low-noise condition. Extensive simulated and real examples demonstrate that SNN
achieves a considerable improvement in CIS over existing nearest neighbor
classifiers, with comparable classification accuracy. We implement the
algorithm in a publicly available R package snn.