pwc-archive/papers-with-abstracts huggingface dataset:
Early stopping and non-parametric regression: An optimal data-dependent stopping rule
Authors: Garvesh Raskutti, Martin J. Wainwright, Bin Yu
Date: 2013-06-15 00:00:00
Tasks: regression

The strategy of early stopping is a regularization technique based on
choosing a stopping time for an iterative algorithm. Focusing on non-parametric
regression in a reproducing kernel Hilbert space, we analyze the early stopping
strategy for a form of gradient-descent applied to the least-squares loss
function. We propose a data-dependent stopping rule that does not involve
hold-out or cross-validation data, and we prove upper bounds on the squared
error of the resulting function estimate, measured in either the $L^2(P)$ and
$L^2(P_n)$ norm. These upper bounds lead to minimax-optimal rates for various
kernel classes, including Sobolev smoothness classes and other forms of
reproducing kernel Hilbert spaces. We show through simulation that our stopping
rule compares favorably to two other stopping rules, one based on hold-out data
and the other based on Stein's unbiased risk estimate. We also establish a
tight connection between our early stopping strategy and the solution path of a
kernel ridge regression estimator.
iCub World: Friendly Robots Help Building Good Vision Data-Sets
Authors: Sean Ryan Fanello, Carlo Ciliberto, Matteo Santoro, Lorenzo Natale, Giorgio Metta, Lorenzo Rosasco, Francesca Odone
Date: 2013-06-15 00:00:00
Tasks: Object Recognition

In this paper we present and start analyzing the iCub World data-set, an
object recognition data-set, we acquired using a Human-Robot Interaction (HRI)
scheme and the iCub humanoid robot platform. Our set up allows for rapid
acquisition and annotation of data with corresponding ground truth. While more
constrained in its scopes -- the iCub world is essentially a robotics research
lab -- we demonstrate how the proposed data-set poses challenges to current
recognition systems. The iCubWorld data-set is publicly available. The data-set
can be downloaded from: http://www.iit.it/en/projects/data-sets.html.
Outlying Property Detection with Numerical Attributes
Authors: Fabrizio Angiulli, Fabio Fassetti, Luigi Palopoli, Giuseppe Manco
Date: 2013-06-15 00:00:00
Tasks: Object

The outlying property detection problem is the problem of discovering the
properties distinguishing a given object, known in advance to be an outlier in
a database, from the other database objects. In this paper, we analyze the
problem within a context where numerical attributes are taken into account,
which represents a relevant case left open in the literature. We introduce a
measure to quantify the degree the outlierness of an object, which is
associated with the relative likelihood of the value, compared to the to the
relative likelihood of other objects in the database. As a major contribution,
we present an efficient algorithm to compute the outlierness relative to
significant subsets of the data. The latter subsets are characterized in a
"rule-based" fashion, and hence the basis for the underlying explanation of the
outlierness.
Constructive Setting of the Density Ratio Estimation Problem and its Rigorous Solution
Authors: Vladimir Vapnik, Igor Braga, Rauf Izmailov
Date: 2013-06-03 00:00:00
Tasks: Density Ratio Estimation

We introduce a general constructive setting of the density ratio estimation
problem as a solution of a (multidimensional) integral equation. In this
equation, not only its right hand side is known approximately, but also the
integral operator is defined approximately. We show that this ill-posed problem
has a rigorous solution and obtain the solution in a closed form. The key
element of this solution is the novel V-matrix, which captures the geometry of
the observed samples. We compare our method with three well-known previously
proposed ones. Our experimental results demonstrate the good potential of the
new approach.
A framework for (under)specifying dependency syntax without overloading annotators
Authors: Nathan Schneider, Brendan O'Connor, Naomi Saphra, David Bamman, Manaal Faruqui, Noah A. Smith, Chris Dyer, Jason Baldridge
Date: 2013-06-10 00:00:00
Proceeding: WS 2013 8

We introduce a framework for lightweight dependency syntax annotation. Our
formalism builds upon the typical representation for unlabeled dependencies,
permitting a simple notation and annotation workflow. Moreover, the formalism
encourages annotators to underspecify parts of the syntax if doing so would
streamline the annotation process. We demonstrate the efficacy of this
annotation on three languages and develop algorithms to evaluate and compare
underspecified annotations.
Hyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a "Null" Model be?
Authors: James Bergstra, David D. Cox
Date: 2013-06-14 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER), Hyperparameter Optimization, image-classification, Image Classification

One of the goals of the ICML workshop on representation and learning is to
establish benchmark scores for a new data set of labeled facial expressions.
This paper presents the performance of a "Null" model consisting of
convolutions with random weights, PCA, pooling, normalization, and a linear
readout. Our approach focused on hyperparameter optimization rather than novel
model components. On the Facial Expression Recognition Challenge held by the
Kaggle website, our hyperparameter optimization approach achieved a score of
60% accuracy on the test data. This paper also introduces a new ensemble
construction variant that combines hyperparameter optimization with the
construction of ensembles. This algorithm constructed an ensemble of four
models that scored 65.5% accuracy. These scores rank 12th and 5th respectively
among the 56 challenge participants. It is worth noting that our approach was
developed prior to the release of the data set, and applied without
modification; our strong competition performance suggests that the TPE
hyperparameter optimization algorithm and domain expertise encoded in our Null
model can generalize to new image classification data sets.
Classifying Single-Trial EEG during Motor Imagery with a Small Training Set
Authors: Yijun Wang
Date: 2013-06-14 00:00:00
Tasks: BIG-bench Machine Learning, Brain Computer Interface, EEG, Electroencephalogram (EEG), Motor Imagery

Before the operation of a motor imagery based brain-computer interface (BCI)
adopting machine learning techniques, a cumbersome training procedure is
unavoidable. The development of a practical BCI posed the challenge of
classifying single-trial EEG with a small training set. In this letter, we
addressed this problem by employing a series of signal processing and machine
learning approaches to alleviate overfitting and obtained test accuracy similar
to training accuracy on the datasets from BCI Competition III and our own
experiments.
Live-wire 3D medical images segmentation
Authors: Ognjen Arandjelovic
Date: 2013-06-14 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

This report describes the design, implementation, evaluation and original
enhancements to the Live-Wire method for 2D and 3D image segmentation.
Live-Wire 2D employs a semi-automatic paradigm; the user is asked to select a
few boundary points of the object to segment, to steer the process in the right
direction, while the result is displayed in real time. In our implementation
segmentation is extended to three dimensions by performing this process on a
slice-by-slice basis. User's time and involvement is further reduced by
allowing him to specify object contours in planes orthogonal to the slices. If
these planes are chosen strategically, Live-Wire 3D can perform 2D segmentation
in the plane of each slice automatically. This report also proposes two
improvements to the original method, path heating and a new graph edge feature
function based on variance of path properties along the boundary. We show that
these improvements lead up to a 33% reduction in interaction with the user, and
improved delineation in presence of strong interfering edges.
Constrained fractional set programs and their application in local clustering and community detection
Authors: Thomas Bühler, Syama Sundar Rangapuram, Simon Setzer, Matthias Hein
Date: 2013-06-14 00:00:00
Tasks: Clustering, Community Detection

The (constrained) minimization of a ratio of set functions is a problem
frequently occurring in clustering and community detection. As these
optimization problems are typically NP-hard, one uses convex or spectral
relaxations in practice. While these relaxations can be solved globally
optimally, they are often too loose and thus lead to results far away from the
optimum. In this paper we show that every constrained minimization problem of a
ratio of non-negative set functions allows a tight relaxation into an
unconstrained continuous optimization problem. This result leads to a flexible
framework for solving constrained problems in network analysis. While a
globally optimal solution for the resulting non-convex problem cannot be
guaranteed, we outperform the loose convex or spectral relaxations by a large
margin on constrained local clustering problems.
h-approximation: History-Based Approximation of Possible World Semantics as ASP
Authors: Manfred Eppe, Mehul Bhatt, Frank Dylla
Date: 2013-04-17 00:00:00

We propose an approximation of the Possible Worlds Semantics (PWS) for action
planning. A corresponding planning system is implemented by a transformation of
the action specification to an Answer-Set Program. A novelty is support for
postdiction wrt. (a) the plan existence problem in our framework can be solved
in NP, as compared to $\Sigma_2^P$ for non-approximated PWS of Baral(2000); and
(b) the planner generates optimal plans wrt. a minimal number of actions in
$\Delta_2^P$. We demo the planning system with standard problems, and
illustrate its integration in a larger software framework for robot control in
a smart home.
Sparse Recovery of Streaming Signals Using L1-Homotopy
Authors: M. Salman Asif, Justin Romberg
Date: 2013-06-14 00:00:00

Most of the existing methods for sparse signal recovery assume a static
system: the unknown signal is a finite-length vector for which a fixed set of
linear measurements and a sparse representation basis are available and an
L1-norm minimization program is solved for the reconstruction. However, the
same representation and reconstruction framework is not readily applicable in a
streaming system: the unknown signal changes over time, and it is measured and
reconstructed sequentially over small time intervals.
  In this paper, we discuss two such streaming systems and a homotopy-based
algorithm for quickly solving the associated L1-norm minimization programs: 1)
Recovery of a smooth, time-varying signal for which, instead of using block
transforms, we use lapped orthogonal transforms for sparse representation. 2)
Recovery of a sparse, time-varying signal that follows a linear dynamic model.
For both the systems, we iteratively process measurements over a sliding
interval and estimate sparse coefficients by solving a weighted L1-norm
minimization program. Instead of solving a new L1 program from scratch at every
iteration, we use an available signal estimate as a starting point in a
homotopy formulation. Starting with a warm-start vector, our homotopy algorithm
updates the solution in a small number of computationally inexpensive steps as
the system changes. The homotopy algorithm presented in this paper is highly
versatile as it can update the solution for the L1 problem in a number of
dynamical settings. We demonstrate with numerical experiments that our proposed
streaming recovery framework outperforms the methods that represent and
reconstruct a signal as independent, disjoint blocks, in terms of quality of
reconstruction, and that our proposed homotopy-based updating scheme
outperforms current state-of-the-art solvers in terms of the computation time
and complexity.
Matching objects across the textured-smooth continuum
Authors: Ognjen Arandjelovic
Date: 2013-06-14 00:00:00
Tasks: 3D Object Recognition, Object, Object Recognition

The problem of 3D object recognition is of immense practical importance, with
the last decade witnessing a number of breakthroughs in the state of the art.
Most of the previous work has focused on the matching of textured objects using
local appearance descriptors extracted around salient image points. The
recently proposed bag of boundaries method was the first to address directly
the problem of matching smooth objects using boundary features. However, no
previous work has attempted to achieve a holistic treatment of the problem by
jointly using textural and shape features which is what we describe herein. Due
to the complementarity of the two modalities, we fuse the corresponding
matching scores and learn their relative weighting in a data specific manner by
optimizing discriminative performance on synthetically distorted data. For the
textural description of an object we adopt a representation in the form of a
histogram of SIFT based visual words. Similarly the apparent shape of an object
is represented by a histogram of discretized features capturing local shape. On
a large public database of a diverse set of objects, the proposed method is
shown to outperform significantly both purely textural and purely shape based
approaches for matching across viewpoint variation.
Feature Learning by Multidimensional Scaling and its Applications in Object Recognition
Authors: Quan Wang, Kim L. Boyer
Date: 2013-06-14 00:00:00
Tasks: Object Recognition

We present the MDS feature learning framework, in which multidimensional
scaling (MDS) is applied on high-level pairwise image distances to learn
fixed-length vector representations of images. The aspects of the images that
are captured by the learned features, which we call MDS features, completely
depend on what kind of image distance measurement is employed. With properly
selected semantics-sensitive image distances, the MDS features provide rich
semantic information about the images that is not captured by other feature
extraction techniques. In our work, we introduce the iterated
Levenberg-Marquardt algorithm for solving MDS, and study the MDS feature
learning with IMage Euclidean Distance (IMED) and Spatial Pyramid Matching
(SPM) distance. We present experiments on both synthetic data and real images
--- the publicly accessible UIUC car image dataset. The MDS features based on
SPM distance achieve exceptional performance for the car recognition task.
Sparse Inverse Covariance Matrix Estimation Using Quadratic Approximation
Authors: Cho-Jui Hsieh, Matyas A. Sustik, Inderjit S. Dhillon, Pradeep Ravikumar
Date: 2013-06-13 00:00:00
Proceeding: NeurIPS 2011 12

The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown
to have strong statistical guarantees in recovering a sparse inverse covariance
matrix, or alternatively the underlying graph structure of a Gaussian Markov
Random Field, from very limited samples. We propose a novel algorithm for
solving the resulting optimization problem which is a regularized
log-determinant program. In contrast to recent state-of-the-art methods that
largely use first order gradient information, our algorithm is based on
Newton's method and employs a quadratic approximation, but with some
modifications that leverage the structure of the sparse Gaussian MLE problem.
We show that our method is superlinearly convergent, and present experimental
results using synthetic and real-world application data that demonstrate the
considerable improvements in performance of our method when compared to other
state-of-the-art methods.
Segmentation et Interprétation de Nuages de Points pour la Modélisation d'Environnements Urbains
Authors: Jorge Hernandez, Beatriz Marcotegui
Date: 2013-06-13 00:00:00
Tasks: General Classification, Model Selection

Dans cet article, nous pr\'esentons une m\'ethode pour la d\'etection et la
classification d'artefacts au niveau du sol, comme phase de filtrage
pr\'ealable \`a la mod\'elisation d'environnements urbains. La m\'ethode de
d\'etection est r\'ealis\'ee sur l'image profondeur, une projection de nuage de
points sur un plan image o\`u la valeur du pixel correspond \`a la distance du
point au plan. En faisant l'hypoth\`ese que les artefacts sont situ\'es au sol,
ils sont d\'etect\'es par une transformation de chapeau haut de forme par
remplissage de trous sur l'image de profondeur. Les composantes connexes ainsi
obtenues, sont ensuite caract\'eris\'ees et une analyse des variables est
utilis\'ee pour la s\'election des caract\'eristiques les plus discriminantes.
Les composantes connexes sont donc classifi\'ees en quatre cat\'egories
(lampadaires, pi\'etons, voitures et "Reste") \`a l'aide d'un algorithme
d'apprentissage supervis\'e. La m\'ethode a \'et\'e test\'ee sur des nuages de
points de la ville de Paris, en montrant de bons r\'esultats de d\'etection et
de classification dans l'ensemble de donn\'ees.---In this article, we present a
method for detection and classification of artifacts at the street level, in
order to filter cloud point, facilitating the urban modeling process. Our
approach exploits 3D information by using range image, a projection of 3D
points onto an image plane where the pixel intensity is a function of the
measured distance between 3D points and the plane. By assuming that the
artifacts are on the ground, they are detected using a Top-Hat of the hole
filling algorithm of range images. Then, several features are extracted from
the detected connected components and a stepwise forward variable/model
selection by using the Wilk's Lambda criterion is performed. Afterward, CCs are
classified in four categories (lampposts, pedestrians, cars and others) by
using a supervised machine learning method. The proposed method was tested on
cloud points of Paris, and have shown satisfactory results on the whole
dataset.
Physeter catodon localization by sparse coding
Authors: Sébastien Paris, Yann Doh, Hervé Glotin, Xanadu Halkias, Joseph Razik
Date: 2013-06-13 00:00:00
Tasks: BIG-bench Machine Learning, Position

This paper presents a spermwhale' localization architecture using jointly a
bag-of-features (BoF) approach and machine learning framework. BoF methods are
known, especially in computer vision, to produce from a collection of local
features a global representation invariant to principal signal transformations.
Our idea is to regress supervisely from these local features two rough
estimates of the distance and azimuth thanks to some datasets where both
acoustic events and ground-truth position are now available. Furthermore, these
estimates can feed a particle filter system in order to obtain a precise
spermwhale' position even in mono-hydrophone configuration. Anti-collision
system and whale watching are considered applications of this work.
The Ripple Pond: Enabling Spiking Networks to See
Authors: Saeed Afshar, Gregory Cohen, Runchun Wang, Andre van Schaik, Jonathan Tapson, Torsten Lehmann, Tara Julia Hamilton
Date: 2013-06-13 00:00:00
Tasks: Object Recognition

In this paper we present the biologically inspired Ripple Pond Network (RPN),
a simply connected spiking neural network that, operating together with
recently proposed PolyChronous Networks (PCN), enables rapid, unsupervised,
scale and rotation invariant object recognition using efficient spatio-temporal
spike coding. The RPN has been developed as a hardware solution linking
previously implemented neuromorphic vision and memory structures capable of
delivering end-to-end high-speed, low-power and low-resolution recognition for
mobile and autonomous applications where slow, highly sophisticated and power
hungry signal processing solutions are ineffective. Key aspects in the proposed
approach include utilising the spatial properties of physically embedded neural
networks and propagating waves of activity therein for information processing,
using dimensional collapse of imagery information into amenable temporal
patterns and the use of asynchronous frames for information binding.
A Face-like Structure Detection on Planet and Satellite Surfaces using Image Processing
Authors: Kazutaka Kurihara, Masakazu Takasu, Kazuhiro Sasao, Hal Seki, Takayuki Narabu, Mitsuo Yamamoto, Satoshi Iida, Hiroyuki Yamamoto
Date: 2013-06-13 00:00:00

This paper demonstrates that face-like structures are everywhere, and can be
de-tected automatically even with computers. Huge amount of satellite images of
the Earth, the Moon, the Mars are explored and many interesting face-like
structure are detected. Throughout this fact, we believe that science and
technologies can alert people not to easily become an occultist.
Second Order Swarm Intelligence
Authors: Vitorino Ramos, David M. S. Rodrigues, Jorge Louçã
Date: 2013-06-13 00:00:00
Tasks: Combinatorial Optimization

An artificial Ant Colony System (ACS) algorithm to solve general-purpose
combinatorial Optimization Problems (COP) that extends previous AC models [21]
by the inclusion of a negative pheromone, is here described. Several Travelling
Salesman Problem (TSP) were used as benchmark. We show that by using two
different sets of pheromones, a second-order co-evolved compromise between
positive and negative feedbacks achieves better results than single positive
feedback systems. The algorithm was tested against known NP-complete
combinatorial Optimization Problems, running on symmetrical TSP's. We show that
the new algorithm compares favourably against these benchmarks, accordingly to
recent biological findings by Robinson [26,27], and Gruter [28] where "No
entry" signals and negative feedback allows a colony to quickly reallocate the
majority of its foragers to superior food patches. This is the first time an
extended ACS algorithm is implemented with these successful characteristics.
Non-parametric Power-law Data Clustering
Authors: Xuhui Fan, Yiling Zeng, Longbing Cao
Date: 2013-06-13 00:00:00
Tasks: Clustering, Variational Inference

It has always been a great challenge for clustering algorithms to
automatically determine the cluster numbers according to the distribution of
datasets. Several approaches have been proposed to address this issue,
including the recent promising work which incorporate Bayesian Nonparametrics
into the $k$-means clustering procedure. This approach shows simplicity in
implementation and solidity in theory, while it also provides a feasible way to
inference in large scale datasets. However, several problems remains unsolved
in this pioneering work, including the power-law data applicability, mechanism
to merge centers to avoid the over-fitting problem, clustering order problem,
e.t.c.. To address these issues, the Pitman-Yor Process based k-means (namely
\emph{pyp-means}) is proposed in this paper. Taking advantage of the Pitman-Yor
Process, \emph{pyp-means} treats clusters differently by dynamically and
adaptively changing the threshold to guarantee the generation of power-law
clustering results. Also, one center agglomeration procedure is integrated into
the implementation to be able to merge small but close clusters and then
adaptively determine the cluster number. With more discussion on the clustering
order, the convergence proof, complexity analysis and extension to spectral
clustering, our approach is compared with traditional clustering algorithm and
variational inference methods. The advantages and properties of pyp-means are
validated by experiments on both synthetic datasets and real world datasets.