pwc-archive/papers-with-abstracts huggingface dataset:
Convex Multi-view Subspace Learning
Authors: Martha White, Xinhua Zhang, Dale Schuurmans, Yao-Liang Yu
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Subspace learning seeks a low dimensional representation of data that enables accurate reconstruction.  However, in many applications, data is obtained from multiple sources rather than a single source (e.g. an object might be viewed by cameras at different angles, or a document might consist of text and images).  The conditional independence of separate sources imposes constraints on their shared latent representation, which, if respected, can improve the quality of the learned low dimensional representation.  In this paper, we present a convex formulation of multi-view subspace learning that enforces conditional independence while reducing dimensionality.  For this formulation, we develop an efficient algorithm that recovers an optimal data reconstruction by exploiting an implicit convex regularizer, then recovers the corresponding latent representation and reconstruction model, jointly and optimally.  Experiments illustrate that the proposed method produces high quality results.
Spectral learning of linear dynamics from generalised-linear observations with application to neural population data
Authors: Lars Buesing, Jakob H. Macke, Maneesh Sahani
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Latent linear dynamical systems with generalised-linear observation models arise in a variety of applications, for example when modelling the spiking activity of populations of neurons.  Here, we show how  spectral learning methods for linear systems with Gaussian observations   (usually called subspace identification in this context) can be extended to estimate the parameters of dynamical system models observed through non-Gaussian noise models. We use this approach to obtain estimates of parameters for a dynamical model of neural population data, where the observed spike-counts are Poisson-distributed with log-rates determined by the latent dynamical process, possibly driven by external inputs. We show that the extended system identification algorithm is consistent and accurately recovers the correct parameters on large simulated data sets with much smaller computational cost than approximate expectation-maximisation (EM) due to the non-iterative nature of subspace identification. Even on smaller data sets, it provides an effective initialization for EM, leading to more robust performance and faster convergence. These benefits are shown to extend to real neural data.
Mixability in Statistical Learning
Authors: Tim V. Erven, Peter Grünwald, Mark D. Reid, Robert C. Williamson
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Bayesian Inference, Prediction

Statistical learning and sequential prediction are two different but related formalisms to study the quality of predictions. Mapping out their relations and transferring ideas is an active area of investigation. We provide another piece of the puzzle by showing that an important concept in sequential prediction, the mixability of a loss, has a natural counterpart in the statistical setting, which we call stochastic mixability. Just as ordinary mixability characterizes fast rates for the worst-case regret in sequential prediction, stochastic mixability characterizes fast rates in statistical learning. We show that, in the special case of log-loss, stochastic mixability reduces to a well-known (but usually unnamed) martingale condition, which is used in existing convergence theorems for minimum description length and Bayesian inference. In the case of 0/1-loss, it reduces to the margin condition of Mammen and Tsybakov, and in the case that the model under consideration contains all possible predictors, it is equivalent to ordinary mixability.
Waveform Driven Plasticity in BiFeO3 Memristive Devices: Model and Implementation
Authors: Christian Mayr, Paul Stärke, Johannes Partzsch, Love Cederstroem, Rene Schüffny, Yao Shuai, Nan Du, Heidemarie Schmidt
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Triplet

Memristive devices have recently been proposed as efficient implementations of plastic synapses in neuromorphic systems. The plasticity in these memristive devices, i.e. their resistance change, is defined by the applied waveforms. This behavior resembles biological synapses, whose plasticity is also triggered by mechanisms that are determined by local waveforms. However, learning in memristive devices has so far been approached mostly on a pragmatic technological level. The focus seems to be on finding any waveform that achieves spike-timing-dependent plasticity (STDP), without regard to the biological veracity of said waveforms or to further important forms of plasticity. Bridging this gap, we make use of a plasticity model driven by neuron waveforms that explains a large number of experimental observations and adapt it to the characteristics of the recently introduced BiFeO$_3$ memristive material. Based on this approach, we show STDP for the first time for this material, with learning window replication superior to previous memristor-based STDP implementations. We also demonstrate in measurements that it is possible to overlay short and long term plasticity at a memristive device in the form of the well-known triplet plasticity. To the best of our knowledge, this is the first implementations of triplet plasticity on any physical memristive device.
A lattice filter model of the visual pathway
Authors: Karol Gregor, Dmitri B. Chklovskii
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: model

Early stages of visual processing are thought to decorrelate, or whiten, the incoming temporally varying signals. Because the typical correlation time of natural stimuli, as well as the extent of temporal receptive fields of lateral geniculate nucleus (LGN) neurons, is much greater than neuronal time constants, such decorrelation must be done in stages combining contributions of multiple neurons. We propose to model temporal decorrelation in the visual pathway with the lattice filter, a signal processing device for stage-wise decorrelation of temporal signals. The stage-wise architecture of the lattice filter maps naturally onto the visual pathway (photoreceptors -> bipolar cells -> retinal ganglion cells -> LGN) and its filter weights can be learned using Hebbian rules in a stage-wise sequential manner. Moreover, predictions of neural activity from the lattice filter model are consistent with physiological measurements in LGN neurons and fruit fly second-order visual neurons. Therefore, the lattice filter model is a useful abstraction that may help unravel visual system function.
Semantic Kernel Forests from Multiple Taxonomies
Authors: Sung Ju Hwang, Kristen Grauman, Fei Sha
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Object, Object Recognition

When learning features for complex visual recognition problems, labeled image exemplars alone can be insufficient.  While an \emph{object taxonomy} specifying the categories' semantic relationships could bolster the learning process, not all relationships are relevant to a given visual classification task, nor does a single taxonomy capture all ties that \emph{are} relevant.  In light of these issues, we propose a discriminative feature learning approach that leverages \emph{multiple} hierarchical taxonomies representing different semantic views of the object categories (e.g., for animal classes, one taxonomy could reflect their phylogenic ties, while another could reflect their habitats).  For each taxonomy, we first learn a tree of semantic kernels, where each node has a Mahalanobis kernel optimized to distinguish between the classes in its children nodes.  Then, using the resulting \emph{semantic kernel forest}, we learn class-specific kernel combinations to select only those relationships relevant to recognize each object class.  To learn the weights, we introduce a novel hierarchical regularization term that further exploits the taxonomies' structure.  We demonstrate our method on challenging object recognition datasets, and show that interleaving multiple taxonomic views yields significant accuracy improvements.
Causal discovery with scale-mixture model for spatiotemporal variance dependencies
Authors: Zhitang Chen, Kun Zhang, Laiwan Chan
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Causal Discovery

In conventional causal discovery, structural equation models (SEM) are directly applied to the observed variables, meaning that the causal effect can be represented as a function of the direct causes themselves. However, in many real world problems, there are significant dependencies in the variances or energies, which indicates that causality may possibly take place at the level of variances or energies. In this paper, we propose a probabilistic causal scale-mixture model with spatiotemporal variance dependencies to represent a specific type of generating mechanism of the observations. In particular, the causal mechanism including contemporaneous and temporal causal relations in variances or energies is represented by a Structural Vector AutoRegressive model (SVAR). We prove the identifiability of this model under the non-Gaussian assumption on the innovation processes. We also propose algorithms to estimate the involved parameters and discover the contemporaneous causal structure. Experiments on synthesis and real world data are conducted to show the applicability of the proposed model and algorithms.
Natural Images, Gaussian Mixtures and Dead Leaves
Authors: Daniel Zoran, Yair Weiss
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Denoising

Simple Gaussian Mixture Models (GMMs) learned from pixels of natural image patches have been recently shown to be surprisingly strong performers in modeling the statistics of natural images. Here we provide an in depth analysis of this simple yet rich model. We show that such a GMM model is able to compete with even the most successful models of natural images in log likelihood scores, denoising performance and sample quality. We provide an analysis of what such a model learns from natural images as a function of number of mixture components --- including covariance structure, contrast variation and intricate structures such as textures, boundaries and more. Finally, we show that the salient properties of the GMM learned from natural images can be derived from a simplified Dead Leaves model which explicitly models occlusion, explaining its surprising success relative to other models.
Dual-Space Analysis of the Sparse Linear Model
Authors: Yi Wu, David P. Wipf
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: model, Vocal Bursts Type Prediction

Sparse linear (or generalized linear) models combine a standard likelihood function with a sparse prior on the unknown coefficients.  These priors can conveniently be expressed as a maximization over zero-mean Gaussians with different variance hyperparameters.  Standard MAP estimation (Type I) involves maximizing over both the hyperparameters and coefficients, while an empirical Bayesian alternative (Type II) first marginalizes the coefficients and then maximizes over the hyperparameters, leading to a tractable posterior approximation.  The underlying cost functions can be related via a dual-space framework from Wipf et al. (2011), which allows both the Type I or Type II objectives to be expressed in either coefficient or hyperparmeter space.  This perspective is useful because some analyses or extensions are more conducive to development in one space or the other.  Herein we consider the estimation of a trade-off parameter balancing sparsity and data fit.  As this parameter is effectively a variance, natural estimators exist by assessing the problem in hyperparameter (variance) space, transitioning natural ideas from Type II to solve what is much less intuitive for Type I.  In contrast, for analyses of update rules and sparsity properties of local and global solutions, as well as extensions to more general likelihood models, we can leverage coefficient-space techniques developed for Type I and apply them to Type II.  For example, this allows us to prove that Type II-inspired techniques can be successful recovering sparse coefficients when unfavorable restricted isometry properties (RIP) lead to failure of popular L1 reconstructions. It also facilitates the analysis of Type II when non-Gaussian likelihood models lead to intractable integrations.
Active Comparison of Prediction Models
Authors: Christoph Sawade, Niels Landwehr, Tobias Scheffer
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Model Selection, Prediction

We address the problem of comparing the risks of two given predictive models - for instance, a baseline model and a challenger - as confidently as possible on a fixed labeling budget. This problem occurs whenever models cannot be compared on held-out training data, possibly because the training data are unavailable or do not reflect the desired test distribution. In this case, new test instances have to be drawn and labeled at a cost. We devise an active comparison method that selects instances according to an instrumental sampling distribution. We derive the sampling distribution that maximizes the power of a statistical test applied to the observed empirical risks, and thereby minimizes the likelihood of choosing the inferior model. Empirically, we investigate model selection problems on several classification and regression tasks and study the accuracy of the resulting p-values.
Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning
Authors: Jinfeng Yi, Rong Jin, Shaili Jain, Tianbao Yang, Anil K. Jain
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Clustering, Computational Efficiency, Matrix Completion, Metric Learning

One of the main challenges in data clustering is to define an appropriate similarity measure between two objects. Crowdclustering addresses this challenge by defining the pairwise similarity based on the manual annotations obtained through crowdsourcing. Despite its encouraging results, a key limitation of crowdclustering is that it can only cluster objects when their manual annotations are available. To address this limitation, we propose a new approach for clustering, called \textit{semi-crowdsourced clustering} that effectively combines the low-level features of objects with the manual annotations of a subset of the objects obtained via crowdsourcing. The key idea is to learn an appropriate similarity measure, based on the low-level features of objects, from the manual annotations of only a small portion of the data to be clustered. One difficulty in learning the pairwise similarity measure is that there is a significant amount of noise and inter-worker variations in the manual annotations obtained via crowdsourcing. We address this difficulty by developing a metric learning algorithm based on the matrix completion method. Our empirical study with two real-world image data sets shows that the proposed algorithm outperforms state-of-the-art distance metric learning algorithms in both clustering accuracy and computational efficiency.
Learning curves for multi-task Gaussian process regression
Authors: Peter Sollich, Simon Ashton
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Multi-Task Learning, regression

We study the average case performance of multi-task Gaussian process (GP)   regression as captured in the learning curve, i.e.\ the average Bayes error   for a chosen task versus the total number of examples $n$ for all   tasks. For GP covariances that are the product of an   input-dependent covariance function and a free-form inter-task   covariance matrix, we   show that accurate approximations for the learning curve can be   obtained for an arbitrary number of tasks $T$.  We use   these to study the asymptotic learning behaviour for large   $n$. Surprisingly, multi-task learning can be asymptotically essentially   useless: examples from other tasks only help when the   degree of inter-task correlation, $\rho$, is near its maximal value   $\rho=1$. This effect is most extreme for learning of smooth target   functions as described by e.g.\ squared exponential kernels. We also   demonstrate that when learning {\em many} tasks, the learning curves   separate into an initial phase, where the Bayes error on each task   is reduced down to a plateau value by ``collective learning''    even though most tasks have not seen examples,   and a final decay that occurs only once the number of examples is   proportional to the number of tasks.
Kernel Hyperalignment
Authors: Alexander Lorbert, Peter J. Ramadge
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

We offer a regularized, kernel extension of the multi-set, orthogonal Procrustes problem, or hyperalignment. Our new method, called Kernel Hyperalignment, expands the scope of hyperalignment to include nonlinear measures of similarity and enables the alignment of multiple datasets with a large number of base features. With direct application to fMRI data analysis, kernel hyperalignment is well-suited for multi-subject alignment of large ROIs, including the entire cortex. We conducted experiments using real-world, multi-subject fMRI data.
Multiple Choice Learning: Learning to Produce Multiple Structured Outputs
Authors: Abner Guzmán-Rivera, Dhruv Batra, Pushmeet Kohli
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Multiple-choice, Prediction

The paper addresses the problem of generating multiple hypotheses for prediction tasks that involve interaction with users or successive components in a cascade. Given a set of multiple hypotheses, such components/users have the ability to automatically rank the results and thus retrieve the best one. The standard approach for handling this scenario is to learn a single model and then produce M-best Maximum a Posteriori (MAP) hypotheses from this model. In contrast, we formulate this multiple {\em choice} learning task as a multiple-output structured-output prediction problem with a loss function that captures the natural setup of the problem. We present a max-margin formulation  that minimizes an upper-bound on this loss-function. Experimental results on the problems of image co-segmentation and protein side-chain prediction show that our method outperforms conventional approaches used for this  scenario and leads to substantial improvements in prediction accuracy.
Mixing Properties of Conditional Markov Chains with Unbounded Feature Functions
Authors: Mathieu Sinn, Bei Chen
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Conditional Markov Chains (also known as Linear-Chain Conditional Random Fields  in the literature) are a versatile class of discriminative models for the distribution of a sequence of hidden states conditional on a sequence of observable variables. Large-sample properties of Conditional Markov Chains have been first studied by Sinn and Poupart [1]. The paper extends this work in two directions: first, mixing properties of models with unbounded feature functions are being established; second, necessary conditions for model identifiability and the uniqueness of maximum likelihood estimates are being given.
Persistent Homology for Learning Densities with Bounded Support
Authors: Florian T. Pokorny, Hedvig Kjellström, Danica Kragic, Carl Ek
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Density Estimation

We present a novel method for learning densities with bounded support which enables us to incorporate `hard' topological constraints. In particular, we show how emerging techniques from computational algebraic topology and the notion of Persistent Homology can be combined with kernel based  methods from Machine Learning for the purpose of density estimation. The proposed formalism facilitates learning of models with bounded support in a principled way, and -- by incorporating Persistent Homology techniques in our approach -- we are able to encode algebraic-topological constraints which are not addressed in current state-of the art probabilistic models. We study the behaviour of our method on two synthetic examples for various sample sizes and exemplify the benefits of the proposed approach on a real-world data-set by learning a motion model for a racecar. We show how to learn a model which respects the underlying topological structure of the racetrack, constraining the trajectories of the car.
On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes
Authors: Bruno Scherrer, Boris Lesner
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

We consider infinite-horizon stationary $\gamma$-discounted Markov   Decision Processes, for which it is known that there exists a   stationary optimal policy. Using Value and Policy Iteration with   some error $\epsilon$ at each iteration, it is well-known that one   can compute stationary policies that are $\frac{2\gamma{(1-\gamma)^2}\epsilon$-optimal. After arguing that this   guarantee is tight, we develop variations of Value and Policy   Iteration for computing non-stationary policies that can be up to   $\frac{2\gamma}{1-\gamma}\epsilon$-optimal, which constitutes a significant   improvement in the usual situation when $\gamma$ is close to   $1$. Surprisingly, this shows that the problem of ``computing near-optimal non-stationary policies'' is much simpler than that   of ``computing near-optimal stationary policies''.
Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model
Authors: Sander M. Bohte
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Neural adaptation underlies the ability of neurons to maximize encoded information over a wide dynamic range of input stimuli. While adaptation is an intrinsic feature of neuronal models like the Hodgkin-Huxley model, the challenge is to integrate adaptation in models of neural computation.  Recent computational models like the Adaptive Spike Response Model implement adaptation as spike-based addition of fixed-size fast spike-triggered threshold dynamics and slow spike-triggered currents. Such adaptation has been shown to accurately model neural spiking behavior over a limited dynamic range. Taking a cue from kinetic models of adaptation, we propose a multiplicative Adaptive Spike Response Model where the spike-triggered adaptation dynamics are scaled multiplicatively by the adaptation state at the time of spiking. We show that unlike the additive adaptation model, the firing rate in the multiplicative adaptation model saturates to a maximum spike-rate. When simulating variance switching experiments, the model also quantitatively fits the experimental data over a wide dynamic range. Furthermore, dynamic threshold models of adaptation suggest a straightforward interpretation of neural activity in terms of dynamic signal encoding with shifted and weighted exponential kernels. We show that when thus encoding rectified filtered stimulus signals, the multiplicative Adaptive Spike Response Model achieves a high coding efficiency and maintains this efficiency over changes in the dynamic signal range of several orders of magnitude, without changing model parameters.