pwc-archive/papers-with-abstracts huggingface dataset:
Boosting with Maximum Adaptive Sampling
Authors: Charles Dubout, Francois Fleuret
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Object Recognition

Classical Boosting algorithms, such as AdaBoost, build a strong classifier without concern about the computational cost.  Some applications, in particular in computer vision, may involve up to millions of training examples and features.  In such contexts, the training time may become prohibitive.  Several methods exist to accelerate training, typically either by sampling the features, or the examples, used to train the weak learners.  Even if those methods can precisely quantify the speed improvement they deliver, they offer no guarantee of being more efficient than any other, given the same amount of time.    This paper aims at shading some light on this problem, i.e. given a fixed amount of time, for a particular problem, which strategy is optimal in order to reduce the training loss the most.  We apply this analysis to the design of new algorithms which estimate on the fly at every iteration the optimal trade-off between the number of samples and the number of features to look at in order to maximize the expected loss reduction.  Experiments in object recognition with two standard computer vision data-sets show that the adaptive methods we propose outperform basic sampling and state-of-the-art bandit methods.
Gaussian Process Training with Input Noise
Authors: Andrew Mchutchon, Carl E. Rasmussen
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: regression

In standard Gaussian Process regression input locations are assumed to be noise free. We present a simple yet effective GP model for training on input points corrupted by i.i.d. Gaussian noise. To make computations tractable we use a local linear expansion about each input point. This allows the input noise to be recast as output noise proportional to the squared gradient of the GP posterior mean. The input noise variances are inferred from the data as extra hyperparameters. They are trained alongside other hyperparameters by the usual method of maximisation of the marginal likelihood. Training uses an iterative scheme, which alternates between optimising the hyperparameters and calculating the posterior gradient. Analytic predictive moments can then be found for Gaussian distributed test points. We compare our model to others over a range of different regression problems and show that it improves over current methods.
Empirical models of spiking in neural populations
Authors: Jakob H. Macke, Lars Buesing, John P. Cunningham, Byron M. Yu, Krishna V. Shenoy, Maneesh Sahani
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

Neurons in the neocortex code and compute as part of a locally interconnected population. Large-scale multi-electrode recording makes it possible to access these population processes empirically by fitting statistical models to unaveraged data. What statistical structure best describes the concurrent spiking of cells within  a local network? We argue that in the cortex, where firing exhibits extensive correlations in both time and space and where a typical sample of neurons still reflects only a very small fraction of the local population, the most appropriate model captures shared variability by a low-dimensional latent process evolving with smooth dynamics, rather than by putative direct coupling. We test this claim by comparing  a latent dynamical model with realistic spiking observations to coupled generalised  linear spike-response models (GLMs) using cortical recordings. We find that the latent dynamical approach outperforms the GLM in terms of goodness-of-fit, and reproduces the temporal correlations in the data more accurately. We also compare models whose observations models are either derived from a Gaussian or point-process models, finding that the non-Gaussian model provides slightly  better goodness-of-fit and more realistic population spike counts.
Learning Probabilistic Non-Linear Latent Variable Models for Tracking Complex Activities
Authors: Angela Yao, Juergen Gall, Luc V. Gool, Raquel Urtasun
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: 3D Human Pose Estimation, Pose Estimation

A common approach for handling the complexity and inherent ambiguities of 3D human pose estimation is to use pose priors learned from  training data. Existing approaches however, are either too simplistic (linear), too complex to learn, or  can only learn latent spaces from "simple data", i.e., single activities such as walking or running. In this paper, we present an efficient stochastic gradient descent algorithm that is able to learn probabilistic non-linear latent  spaces composed of multiple  activities. Furthermore, we derive an incremental algorithm for the online setting which can update the latent space without extensive relearning. We demonstrate the  effectiveness of our approach on the  task of monocular and multi-view tracking and show that our approach  outperforms the state-of-the-art.
Bayesian Partitioning of Large-Scale Distance Data
Authors: David Adametz, Volker Roth
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Translation

A Bayesian approach to partitioning distance matrices is presented. It is inspired by the 'Translation-Invariant Wishart-Dirichlet' process (TIWD) in (Vogt et al., 2010) and shares a number of advantageous properties like the fully probabilistic nature of the inference model, automatic selection of the number of clusters and applicability in semi-supervised settings. In addition, our method (which we call 'fastTIWD') overcomes the main shortcoming of the original TIWD, namely its high computational costs. The fastTIWD reduces the workload in each iteration of a Gibbs sampler from O(n^3) in the TIWD to O(n^2). Our experiments show that this cost reduction does not compromise the quality of the inferred partitions. With this new method it is now possible to 'mine' large relational datasets with a probabilistic model, thereby automatically detecting new and potentially interesting clusters.
From Stochastic Nonlinear Integrate-and-Fire to Generalized Linear Models
Authors: Skander Mensi, Richard Naud, Wulfram Gerstner
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

Variability in single neuron models is typically implemented either by a stochastic Leaky-Integrate-and-Fire model or by a model of the Generalized Linear Model (GLM) family. We use analytical and numerical methods to relate state-of-the-art models from both schools of thought. First we find the analytical expressions relating the subthreshold voltage from the Adaptive Exponential Integrate-and-Fire model (AdEx) to the Spike-Response Model with escape noise (SRM as an example of a GLM). Then we calculate numerically the link-function that provides the firing probability given a deterministic membrane potential. We find a mathematical expression for this link-function and test the ability of the GLM to predict the firing probability of a neuron receiving complex stimulation. Comparing the prediction performance of various link-functions, we find that a GLM with an exponential link-function provides an excellent approximation to the Adaptive Exponential Integrate-and-Fire with colored-noise input. These results help to understand the relationship between the different approaches to stochastic neuron models.
On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference
Authors: Guy Broeck
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

Probabilistic logics are receiving a lot of attention today because of their expressive power for knowledge representation and learning. However, this expressivity is detrimental to the tractability of inference, when done at the propositional level.  To solve this problem, various lifted inference algorithms have been proposed that reason at the first-order level, about groups of objects as a whole. Despite the existence of various lifted inference approaches, there are currently no completeness results about these algorithms.  The key contribution of this paper is that we introduce a formal definition of lifted inference that allows us to reason about the completeness of lifted inference algorithms relative to a particular class of probabilistic models.  We then show how to obtain a completeness result using a first-order knowledge compilation approach for theories of formulae containing up to two logical variables.
Hierarchical Topic Modeling for Analysis of Time-Evolving Personal Choices
Authors: Xianxing Zhang, Lawrence Carin, David B. Dunson
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

The nested Chinese restaurant process is extended to design a nonparametric  topic-model tree for representation of human choices. Each tree branch corresponds  to a type of person, and each node (topic) has a corresponding probability  vector over items that may be selected. The observed data are assumed to have  associated temporal covariates (corresponding to the time at which choices are  made), and we wish to impose that with increasing time it is more probable that  topics deeper in the tree are utilized. This structure is imposed by developing  a new “change point" stick-breaking model that is coupled with a Poisson and  product-of-gammas construction. To share topics across the tree nodes, topic distributions  are drawn from a Dirichlet process. As a demonstration of this concept,  we analyze real data on course selections of undergraduate students at Duke University,  with the goal of uncovering and concisely representing structure in the  curriculum and in the characteristics of the student body.
An Exact Algorithm for F-Measure Maximization
Authors: Krzysztof J. Dembczynski, Willem Waegeman, Weiwei Cheng, Eyke Hüllermeier
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Binary Classification, Classification, General Classification, Information Retrieval, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Retrieval

The F-measure, originally introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction. Optimizing this measure remains a statistically and computationally challenging problem, since no closed-form maximizer exists. Current algorithms are approximate and typically rely on additional assumptions regarding the statistical distribution of the binary response variables. In this paper, we present an algorithm which is not only computationally efficient but also exact, regardless of the underlying distribution. The algorithm requires only a quadratic number of parameters of the joint distribution  (with respect to the number of binary responses). We illustrate its practical performance by means of experimental results for multi-label classification.
Co-regularized Multi-view Spectral Clustering
Authors: Abhishek Kumar, Piyush Rai, Hal Daume
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Clustering

In many clustering problems, we have access to multiple views of the data each  of which could be individually used for clustering. Exploiting information from  multiple views, one can hope to find a clustering that is more accurate than the  ones obtained using the individual views. Since the true clustering would assign  a point to the same cluster irrespective of the view, we can approach this problem  by looking for clusterings that are consistent across the views, i.e., corresponding  data points in each view should have same cluster membership. We propose a  spectral clustering framework that achieves this goal by co-regularizing the clustering  hypotheses, and propose two co-regularization schemes to accomplish this.  Experimental comparisons with a number of baselines on two synthetic and three  real-world datasets establish the efficacy of our proposed approaches.
Sequence learning with hidden units in spiking neural networks
Authors: Johanni Brea, Walter Senn, Jean-Pascal Pfister
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

We consider a statistical framework in which recurrent networks of spiking neurons learn to generate spatio-temporal spike patterns. Given biologically realistic stochastic neuronal dynamics we derive a tractable learning rule for the synaptic weights towards hidden and visible neurons that leads to optimal recall of the training sequences. We show that learning synaptic weights towards hidden neurons significantly improves the storing capacity of the network. Furthermore, we derive an approximate online learning rule and show that our learning rule is consistent with Spike-Timing Dependent Plasticity in that if a presynaptic spike shortly precedes a postynaptic spike, potentiation is induced and otherwise depression is elicited.
Identifying Alzheimer's Disease-Related Brain Regions from Multi-Modality Neuroimaging Data using Sparse Composite Linear Discrimination Analysis
Authors: Shuai Huang, Jing Li, Jieping Ye, Teresa Wu, Kewei Chen, Adam Fleisher, Eric Reiman
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: feature selection, Specificity

Diagnosis of Alzheimer's disease (AD) at the early stage of the disease development is of great clinical importance. Current clinical assessment that relies primarily on cognitive measures proves low sensitivity and specificity. The fast growing neuroimaging techniques hold great promise. Research so far has focused on single neuroimaging modalities. However, as different modalities provide complementary measures for the same disease pathology, fusion of multi-modality data may increase the statistical power in identification of disease-related brain regions. This is especially true for early AD, at which stage the disease-related regions are most likely to be weak-effect regions that are difficult to be detected from a single modality alone. We propose a sparse composite linear discriminant analysis model (SCLDA) for identification of disease-related brain regions of early AD from multi-modality data. SCLDA uses a novel formulation that decomposes each LDA parameter into a product of a common parameter shared by all the modalities and a parameter specific to each modality, which enables joint analysis of all the modalities and borrowing strength from one another. We prove that this formulation is equivalent to a penalized likelihood with non-convex regularization, which can be solved by the DC ((difference of convex functions) programming. We show that in using the DC programming, the property of the non-convex regularization in terms of preserving weak-effect features can be nicely revealed. We perform extensive simulations to show that SCLDA outperforms existing competing algorithms on feature selection, especially on the ability for identifying weak-effect features. We apply SCLDA to the Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) images of 49 AD patients and 67 normal controls (NC). Our study identifies disease-related brain regions consistent with findings in the AD literature.
A blind sparse deconvolution method for neural spike identification
Authors: Chaitanya Ekanadham, Daniel Tranchina, Eero P. Simoncelli
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Clustering

We consider the problem of estimating neural spikes from extracellular voltage recordings. Most current methods are based on clustering, which requires substantial human supervision and produces systematic errors by failing to properly handle temporally overlapping spikes. We formulate the problem as one of statistical inference, in which the recorded voltage is a noisy sum of the spike trains of each neuron convolved with its associated spike waveform.  Joint maximum-a-posteriori (MAP) estimation of the waveforms and spikes is then a blind deconvolution problem in which the coefficients are sparse. We develop a block-coordinate descent method for approximating the MAP solution.  We validate our method on data simulated according to the generative model, as well as on real data for which ground truth is available via simultaneous intracellular recordings. In both cases, our method substantially reduces the number of missed spikes and false positives when compared to a standard clustering algorithm, primarily by recovering temporally overlapping spikes.  The method offers a fully automated alternative to clustering methods that is less susceptible to systematic errors.
How Do Humans Teach: On Curriculum Learning and Teaching Dimension
Authors: Faisal Khan, Bilge Mutlu, Jerry Zhu
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

We study the empirical strategies that humans follow as they teach a target concept with a simple 1D threshold to a robot.  Previous studies of computational teaching, particularly the teaching dimension model and the curriculum learning principle, offer contradictory predictions on what optimal strategy the teacher should follow in this teaching task. We show through behavioral studies that humans employ three distinct teaching strategies, one of which is consistent with the curriculum learning principle, and propose a novel theoretical framework as a potential explanation for this strategy. This framework, which assumes a teaching goal of minimizing the learner's expected generalization error at each iteration, extends the standard teaching dimension model and offers a theoretical justification for curriculum learning.
Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization
Authors: Mark Schmidt, Nicolas L. Roux, Francis R. Bach
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using proximal-gradient methods, where an error is present in the calculation of the gradient of the smooth term or in the proximity operator with respect to the second term. We show that the basic proximal-gradient method, the basic proximal-gradient method with a strong convexity assumption, and the accelerated proximal-gradient method achieve the same convergence rates as in the error-free case, provided the errors decrease at an appropriate rate.  Our experimental results on a structured sparsity problem indicate that sequences of errors with these appealing theoretical properties can lead to practical performance improvements.
Joint 3D Estimation of Objects and Scene Layout
Authors: Andreas Geiger, Christian Wojek, Raquel Urtasun
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

We propose a novel generative model that is able to reason jointly about the 3D scene layout as well as the 3D location and orientation of objects in the scene. In particular, we infer the scene topology, geometry as well as traffic activities from a short video sequence acquired with a single camera mounted on a moving car. Our generative model takes advantage of dynamic information in the form of vehicle tracklets as well as static information coming from semantic labels and geometry  (i.e., vanishing points). Experiments show that our approach outperforms a discriminative baseline based on multiple kernel learning (MKL) which has access to the same image information. Furthermore, as we reason about objects in 3D, we are able to significantly increase the performance of state-of-the-art object detectors in their ability to estimate  object orientation.
Spatial distance dependent Chinese restaurant processes for image segmentation
Authors: Soumya Ghosh, Andrei B. Ungureanu, Erik B. Sudderth, David M. Blei
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Unsupervised Image Segmentation

The distance dependent Chinese restaurant process (ddCRP) was recently introduced to accommodate random partitions of non-exchangeable data.  The ddCRP clusters data in a biased way: each data point is more likely to be clustered with other data that are near it in an external sense.  This paper examines the ddCRP in a spatial setting with the goal of natural image segmentation.  We explore the biases of the spatial ddCRP model and propose a novel hierarchical extension better suited for producing "human-like" segmentations. We then study the sensitivity of the models to various distance and appearance hyperparameters, and provide the first rigorous comparison of nonparametric Bayesian models in the image segmentation domain. On unsupervised image segmentation, we demonstrate that similar performance to existing nonparametric Bayesian models is possible with substantially simpler models and algorithms.
Pylon Model for Semantic Segmentation
Authors: Victor Lempitsky, Andrea Vedaldi, Andrew Zisserman
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Image Segmentation, model, Segmentation, Semantic Segmentation

Graph cut optimization is one of the standard workhorses of image segmentation since for binary random field representations of the  image, it gives globally optimal results and there are efficient  polynomial time implementations.  Often, the random field is applied  over a flat partitioning of the image into non-intersecting elements,  such as pixels or super-pixels.    In the paper we show that if, instead of a flat partitioning, the image is represented by a hierarchical segmentation tree, then the resulting energy combining unary and boundary terms can still be optimized using graph cut (with all the corresponding benefits of global optimality and efficiency). As a result of such inference, the image gets partitioned into a set of segments that may come from different layers of the tree.    We apply this formulation, which we call the pylon model, to the task of semantic segmentation where the goal is to separate an image into areas belonging to different semantic classes. The experiments highlight the advantage of inference on a segmentation tree (over a flat partitioning) and demonstrate that the optimization in the pylon model is able to flexibly choose the level of segmentation across the image. Overall, the proposed system has superior segmentation accuracy on several datasets (Graz-02, Stanford background) compared to previously suggested approaches.
t-divergence Based Approximate Inference
Authors: Nan Ding, Yuan Qi, S. V. N. Vishwanathan
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12

Approximate inference is an important technique for dealing with large, intractable graphical models based on the exponential family of distributions. We extend the idea of approximate inference to the t-exponential family by defining a new t-divergence. This divergence measure is obtained via convex duality between the log-partition function of the t-exponential family and a new t-entropy. We illustrate our approach on the Bayes Point Machine with a Student's t-prior.
Learning person-object interactions for action recognition in still images
Authors: Vincent Delaitre, Josef Sivic, Ivan Laptev
Date: 2011-12-01 00:00:00
Proceeding: NeurIPS 2011 12
Tasks: Action Recognition, Action Recognition In Still Images, Object, Temporal Action Localization

We investigate a discriminatively trained model of person-object interactions for recognizing common human actions in still images. We build on the locally order-less spatial pyramid bag-of-features model, which was shown to perform extremely well on a range of object, scene and human action recognition tasks. We introduce three principal contributions. First, we replace the standard quantized local HOG/SIFT features with stronger discriminatively trained body part and object detectors. Second, we introduce new person-object interaction features based on spatial co-occurrences of individual body parts and objects. Third, we address the combinatorial problem of a large number of possible interaction pairs and propose a discriminative selection procedure using a linear support vector machine (SVM) with a sparsity inducing regularizer. Learning of action-specific body part and object interactions bypasses the difficult problem of estimating the complete human body pose configuration. Benefits of the proposed model are shown on human action recognition in consumer photographs, outperforming the strong bag-of-features baseline.