pwc-archive/papers-with-abstracts huggingface dataset:
fMRI-Based Inter-Subject Cortical Alignment Using Functional Connectivity
Authors: Bryan Conroy, Ben Singer, James Haxby, Peter J. Ramadge
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Functional Connectivity

The inter-subject alignment of functional MRI (fMRI) data is important for improving the statistical power of fMRI group analyses. In contrast to existing anatomically-based methods, we propose a novel multi-subject algorithm that derives a functional correspondence by aligning spatial patterns of functional connectivity across a set of subjects. We test our method on fMRI data collected during a movie viewing experiment. By cross-validating the results of our algorithm, we show that the correspondence successfully generalizes to a secondary movie dataset not used to derive the alignment.
Sensitivity analysis in HMMs with application to likelihood maximization
Authors: Pierre-Arnaud Coquelin, Romain Deguest, Rémi Munos
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Sensitivity

This paper considers a sensitivity analysis in Hidden Markov Models with continuous state and observation spaces. We propose an Infinitesimal Perturbation Analysis (IPA) on the filtering distribution with respect to some parameters of the model. We describe a methodology for using any algorithm that estimates the filtering density, such as Sequential Monte Carlo methods, to design an algorithm that estimates its gradient. The resulting IPA estimator is proven to be asymptotically unbiased, consistent and has computational complexity linear in the number of particles. We consider an application of this analysis to the problem of identifying unknown parameters of the model given a sequence of observations. We derive an IPA estimator for the gradient of the log-likelihood, which may be used in a gradient method for the purpose of likelihood maximization. We illustrate the method with several numerical experiments.
Learning Non-Linear Combinations of Kernels
Authors: Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: regression

This paper studies the general problem of learning kernels based on a polynomial combination of base kernels. It analyzes this problem in the case of regression and the kernel ridge regression algorithm. It examines the corresponding learning kernel optimization problem, shows how that minimax problem can be reduced to a simpler minimization problem, and proves that the global solution of this problem always lies on the boundary. We give a projection-based gradient descent algorithm for solving the optimization problem, shown empirically to converge in few iterations. Finally, we report the results of extensive experiments with this algorithm using several publicly available datasets demonstrating the effectiveness of our technique.
An Infinite Factor Model Hierarchy Via a Noisy-Or Mechanism
Authors: Douglas Eck, Yoshua Bengio, Aaron C. Courville
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: TAG

The Indian Buffet Process is a Bayesian nonparametric approach that models objects as arising from an infinite number of latent factors. Here we extend the latent factor model framework to two or more unbounded layers of latent factors. From a generative perspective, each layer defines a conditional \emph{factorial} prior distribution over the binary latent variables of the layer below via a noisy-or mechanism. We explore the properties of the model with two empirical studies, one digit recognition task and one music tag data experiment.
Adaptive Regularization of Weight Vectors
Authors: Koby Crammer, Alex Kulesza, Mark Dredze
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

We present AROW, a new online learning algorithm that combines several properties of successful : large margin training, confidence weighting, and the capacity to handle non-separable data. AROW performs adaptive regularization of the prediction function upon seeing each new instance, allowing it to perform especially well in the presence of label noise.  We derive a mistake bound, similar in form to the second order perceptron bound, which does not assume separability. We also relate our algorithm to recent confidence-weighted online learning techniques and empirically show that AROW achieves state-of-the-art performance and notable robustness in the case of non-separable data.
Learning transport operators for image manifolds
Authors: Benjamin Culpepper, Bruno A. Olshausen
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

We describe a method for learning a group of continuous transformation operators  to traverse smooth nonlinear manifolds. The method is applied to model how  natural images change over time and scale. The group of continuous transform  operators is represented by a basis that is adapted to the statistics of the data so  that the inﬁnitesimal generator for a measurement orbit can be produced by a  linear combination of a few basis elements. We illustrate how the method can be  used to efﬁciently code time-varying images by describing changes across time  and scale in terms of the learned operators.
White Functionals for Anomaly Detection in Dynamical Systems
Authors: Marco Cuturi, Jean-Philippe Vert, Alexandre d'Aspremont
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Anomaly Detection

We propose new methodologies to detect anomalies in discrete-time processes taking values in a set. The method is based on the inference of functionals whose evaluations on successive states visited by the process have low autocorrelations. Deviations from this behavior are used to flag anomalies. The candidate functionals are estimated in a subset of a reproducing kernel Hilbert space associated with the set where the process takes values. We provide experimental results which show that these techniques compare favorably with other algorithms.
L_1-Penalized Robust Estimation for a Class of Inverse Problems Arising in Multiview Geometry
Authors: Arnak Dalalyan, Renaud Keriven
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

We propose a new approach to the problem of robust estimation in multiview geometry. Inspired by recent  advances in the sparse recovery problem of statistics, our estimator is defined as a Bayesian maximum a posteriori  with multivariate Laplace prior on the vector describing the outliers. This leads to an estimator in which  the fidelity to the data is measured by the $L_\infty$-norm while the regularization is done by the $L_1$-norm.  The proposed procedure is fairly fast since the outlier removal is done by solving one linear program (LP).  An important difference compared to existing algorithms is that for our estimator it is not necessary  to specify neither the number nor the proportion of the outliers. The theoretical results, as well as  the numerical example reported in this work, confirm the efficiency of the proposed approach.
Distribution-Calibrated Hierarchical Classification
Authors: Ofer Dekel
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Classification, General Classification

While many advances have already been made on the topic of hierarchical classi-  ﬁcation learning, we take a step back and examine how a hierarchical classiﬁca-  tion problem should be formally deﬁned. We pay particular attention to the fact  that many arbitrary decisions go into the design of the the label taxonomy that  is provided with the training data, and that this taxonomy is often unbalanced.  We correct this problem by using the data distribution to calibrate the hierarchical  classiﬁcation loss function. This distribution-based correction must be done with  care, to avoid introducing unmanagable statstical dependencies into the learning  problem. This leads us off the beaten path of binomial-type estimation and into  the uncharted waters of geometric-type estimation. We present a new calibrated  deﬁnition of statistical risk for hierarchical classiﬁcation, an unbiased geometric  estimator for this risk, and a new algorithmic reduction from hierarchical classiﬁ-  cation to cost-sensitive classiﬁcation.
A Smoothed Approximate Linear Program
Authors: Vijay Desai, Vivek Farias, Ciamac C. Moallemi
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

We present a novel linear program for the approximation of the dynamic programming cost-to-go function in high-dimensional stochastic control problems. LP approaches to approximate DP naturally restrict attention to approximations that are lower bounds to the optimal cost-to-go function. Our program -- the `smoothed approximate linear program -- relaxes this restriction in an appropriate fashion while remaining computationally tractable. Doing so appears to have several advantages: First, we demonstrate superior bounds on the quality of approximation to the optimal cost-to-go function afforded by our approach. Second, experiments with our approach on a challenging problem (the game of Tetris) show that the approach outperforms the existing LP approach (which has previously been shown to be competitive with several ADP algorithms) by an order of magnitude.
Localizing Bugs in Program Executions with Graphical Models
Authors: Laura Dietz, Valentin Dallmeier, Andreas Zeller, Tobias Scheffer
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Bayesian Inference

We devise a graphical model that supports the process of debugging software by guiding developers to code that is likely to contain defects. The model is trained using execution traces of passing test runs; it reflects the distribution over transitional patterns of code positions. Given a failing test case, the model determines the least likely transitional pattern in the execution trace. The model is designed such that Bayesian inference has a closed-form solution. We evaluate the  Bernoulli graph model on data of the software projects AspectJ and Rhino.
The Infinite Partially Observable Markov Decision Process
Authors: Finale Doshi-Velez
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Reinforcement Learning

The Partially Observable Markov Decision Process (POMDP) framework has proven useful in planning domains that require balancing actions that increase an agents knowledge and actions that increase an agents reward.  Unfortunately, most POMDPs are complex structures with a large number of parameters.  In many realworld problems, both the structure and the parameters are difficult to specify from domain knowledge alone.  Recent work in Bayesian reinforcement learning has made headway in learning POMDP models; however, this work has largely focused on learning the parameters of the POMDP model.  We define an infinite POMDP (iPOMDP) model that does not require knowledge of the size of the state space; instead, it assumes that the number of visited states will grow as the agent explores its world and explicitly models only visited states.  We demonstrate the iPOMDP utility on several standard problems.
A Bayesian Model for Simultaneous Image Clustering, Annotation and Object Segmentation
Authors: Lan Du, Lu Ren, Lawrence Carin, David B. Dunson
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Clustering, Image Clustering, Object, Semantic Segmentation

A non-parametric Bayesian model is proposed for processing multiple images. The analysis employs image features and, when present, the words associated with accompanying annotations. The model clusters the images into classes, and each image is segmented into a set of objects, also allowing the opportunity to assign a word to each object (localized labeling). Each object is assumed to be represented as a heterogeneous mix of components, with this realized via mixture models linking image features to object types. The number of image classes, number of object types, and the characteristics of the object-feature mixture models are inferred non-parametrically. To constitute spatially contiguous objects, a new logistic stick-breaking process is developed. Inference is performed efficiently via variational Bayesian analysis, with example results presented on two image databases.
Efficient Learning using Forward-Backward Splitting
Authors: Yoram Singer, John C. Duchi
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

We describe, analyze, and experiment with a new framework for empirical loss minimization with regularization. Our algorithmic framework alternates between two phases. On each iteration we first perform an {\em unconstrained} gradient descent step. We then cast and solve an instantaneous optimization problem that trades off minimization of a regularization term while keeping close proximity to the result of the first phase. This yields a simple yet effective algorithm for both batch penalized risk minimization and online learning. Furthermore, the two phase approach enables sparse solutions when used in conjunction with regularization functions that promote sparsity, such as $\ell_1$. We derive concrete and very simple algorithms for minimization of loss functions with $\ell_1$, $\ell_2$, $\ell_2^2$, and $\ell_\infty$ regularization. We also show how to construct efficient algorithms for mixed-norm $\ell_1/\ell_q$ regularization. We further extend the algorithms and give efficient implementations for very high-dimensional data with sparsity. We demonstrate the potential of the proposed framework in experiments with synthetic and natural datasets.
A Data-Driven Approach to Modeling Choice
Authors: Vivek Farias, Srikanth Jagabathula, Devavrat Shah
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Econometrics, Marketing

We visit the following fundamental problem: For a `generic model of consumer choice (namely, distributions over preference lists) and a limited amount of data on how consumers actually make decisions (such as marginal preference information), how may one predict revenues from offering a particular assortment of choices? This problem is central to areas within operations research, marketing and econometrics. We present a framework to answer such questions and design a number of tractable algorithms (from a data and computational standpoint) for the same.
Subject independent EEG-based BCI decoding
Authors: Siamac Fazli, Cristian Grozea, Marton Danoczy, Benjamin Blankertz, Florin Popescu, Klaus-Robert Müller
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: EEG, Electroencephalogram (EEG)

In the quest to make Brain Computer Interfacing (BCI) more usable, dry electrodes have emerged that get rid of the initial 30 minutes required for placing an electrode cap.  Another time consuming step is the required individualized adaptation to the BCI user, which involves another 30 minutes calibration for assessing a subjects brain signature. In this paper we aim to also remove this calibration proceedure from BCI setup time by means of machine learning. In particular, we harvest a large database of EEG BCI motor imagination recordings (83 subjects) for constructing a library of subject-specific spatio-temporal filters and derive a subject independent BCI classifier. Our offline results indicate that BCI-na\{i}ve users could start real-time BCI use with no prior calibration at only a very moderate performance loss."
Semi-Supervised Learning in Gigantic Image Collections
Authors: Rob Fergus, Yair Weiss, Antonio Torralba
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12

With the advent of the Internet it is now possible to collect   hundreds of millions of images. These images come with varying   degrees of label information. ``Clean labels can be manually obtained on a   small fraction, ``noisy labels may be extracted automatically from  surrounding text, while for most images there are no labels at all. Semi-supervised learning is a principled framework for combining these different label sources. However, it scales polynomially with the  number of images, making it impractical for use on gigantic  collections with hundreds of millions of images and thousands of classes.   In this paper we show how to utilize recent results in machine   learning to obtain highly efficient approximations for   semi-supervised learning that are linear in the number of images.  Specifically, we use the convergence of the eigenvectors of the normalized graph Laplacian to eigenfunctions of weighted Laplace-Beltrami operators. We combine this with a label sharing framework obtained from Wordnet to propagate label information to classes lacking manual annotations. Our algorithm enables us to apply semi-supervised learning to a  database of 80 million images with 74 thousand classes.
Evaluating multi-class learning strategies in a generative hierarchical framework for object detection
Authors: Sanja Fidler, Marko Boben, Ales Leonardis
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Object, object-detection, Object Detection

Multiple object class learning and detection is a challenging problem due to the large number of object classes and their high visual variability. Specialized detectors usually excel in performance, while joint representations optimize sharing and reduce inference time --- but are complex to train. Conveniently, sequential learning of categories cuts down training time by transferring existing knowledge to novel classes, but cannot fully exploit the richness of shareability and might depend on ordering in learning. In hierarchical frameworks these issues have been little explored. In this paper, we show how different types of multi-class learning can be done within one generative hierarchical framework and provide a rigorous experimental analysis of various object class learning strategies as the number of classes grows. Specifically, we propose, evaluate and compare three important types of multi-class learning: 1.) independent training of individual categories, 2.) joint training of classes, 3.) sequential learning of classes. We explore and compare their computational behavior (space and time) and detection performance as a function of the number of learned classes on several recognition data sets.
Orthogonal Matching Pursuit From Noisy Random Measurements: A New Analysis
Authors: Sundeep Rangan, Alyson K. Fletcher
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: 2k, 4k

Orthogonal matching pursuit (OMP) is a widely used greedy algorithm for recovering sparse vectors from linear measurements.  A well-known analysis of Tropp and Gilbert shows that OMP can recover a k-sparse n-dimensional real vector from m = 4k log(n) noise-free random linear measurements with a probability that goes to one as n goes to infinity. This work shows strengthens this result by showing that a lower number of measurements, m = 2k log(n-k), is in fact sufficient for asymptotic recovery. Moreover, this number of measurements is also sufficient for detection of the sparsity pattern (support) of the vector with measurement errors provided the signal-to-noise ratio (SNR) scales to infinity. The scaling m = 2k log(n-k) exactly matches the number of measurements required by the more complex lasso for signal recovery.
Sharing Features among Dynamical Systems with Beta Processes
Authors: Emily Fox, Michael. I. Jordan, Erik B. Sudderth, Alan S. Willsky
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Time Series, Time Series Analysis

We propose a Bayesian nonparametric approach to relating multiple time series via a set of latent, dynamical behaviors.  Using a beta process prior, we allow data-driven selection of the size of this set, as well as the pattern with which behaviors are shared among time series.  Via the Indian buffet process representation of the beta process predictive distributions, we develop an exact Markov chain Monte Carlo inference method.  In particular, our approach uses the sum-product algorithm to efficiently compute Metropolis-Hastings acceptance probabilities, and explores new dynamical behaviors via birth/death proposals.  We validate our sampling algorithm using several synthetic datasets, and also demonstrate promising unsupervised segmentation of visual motion capture data.
An Additive Latent Feature Model for Transparent Object Recognition
Authors: Mario Fritz, Gary Bradski, Sergey Karayev, Trevor Darrell, Michael J. Black
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Object, Object Recognition, Quantization, Transparent objects

Existing methods for recognition of object instances and categories based on quantized local features can perform poorly when local features exist on transparent surfaces, such as glass or plastic objects. There are characteristic patterns to the local appearance of transparent objects, but they may not be well captured by distances to individual examples or by a local pattern codebook obtained by vector quantization.  The appearance of a transparent patch is determined in part by the refraction of a background pattern through a transparent medium: the energy from the background usually dominates the patch appearance.  We model transparent local patch appearance using an additive  model of latent factors: background factors due to scene content,  and factors which capture a local edge energy distribution characteristic of the refraction.  We implement our method using a novel LDA-SIFT formulation which performs LDA prior to any vector quantization step; we discover latent topics which are characteristic of particular transparent patches and quantize the SIFT space into transparent visual words according to the latent topic dimensions. No knowledge of the background scene is required at test time; we show examples recognizing transparent glasses in a domestic environment.
Estimating image bases for visual image reconstruction from human brain activity
Authors: Yusuke Fujiwara, Yoichi Miyawaki, Yukiyasu Kamitani
Date: 2009-12-01 00:00:00
Proceeding: NeurIPS 2009 12
Tasks: Image Reconstruction

Image representation based on image bases provides a framework for understanding neural representation of visual perception. A recent fMRI study has shown that arbitrary contrast-defined visual images can be reconstructed from fMRI activity patterns using a combination of multi-scale local image bases. In the reconstruction model, the mapping from an fMRI activity pattern to the contrasts of the image bases was learned from measured fMRI responses to visual images. But the shapes of the images bases were fixed, and thus may not be optimal for reconstruction. Here, we propose a method to build a reconstruction model in which image bases are automatically extracted from the measured data. We constructed a probabilistic model that relates the fMRI activity space to the visual image space via a set of latent variables. The mapping from the latent variables to the visual image space can be regarded as a set of image bases. We found that spatially localized, multi-scale image bases were estimated near the fovea, and that the model using the estimated image bases was able to accurately reconstruct novel visual images. The proposed method provides a means to discover a novel functional mapping between stimuli and brain activity patterns.