pwc-archive/papers-with-abstracts huggingface dataset:
Semi-Supervised Domain Adaptation with Non-Parametric Copulas
Authors: David Lopez-Paz, Jose M. Hernández-Lobato, Bernhard Schölkopf
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Domain Adaptation, regression, Semi-supervised Domain Adaptation

A new framework based on the theory of copulas is proposed to address semi-supervised domain adaptation problems. The presented method factorizes any multivariate density into a product of marginal distributions and bivariate copula functions. Therefore, changes in each of these factors can be detected and corrected to adapt a density model across different learning domains. Importantly, we introduce a novel vine copula model, which allows for this factorization in a non-parametric manner. Experimental results on regression problems with real-world data illustrate the efficacy of the proposed approach when compared to state-of-the-art techniques.
Identification of Recurrent Patterns in the Activation of Brain Networks
Authors: Firdaus Janoos, Weichang Li, Niranjan Subrahmanya, Istvan Morocz, William Wells
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Clustering, Time Series Analysis

Identifying patterns from the neuroimaging recordings of brain activity  related to the unobservable psychological or mental state of an individual can be treated as a unsupervised pattern recognition problem. The main challenges, however, for such an analysis of fMRI data are: a) defining a physiologically meaningful feature-space for representing the spatial patterns across time; b) dealing with the high-dimensionality of the data; and c) robustness to the various artifacts and confounds in the fMRI time-series.  In this paper, we present a network-aware feature-space to represent the states of a general network, that enables  comparing and clustering such states in a manner that is a) meaningful in terms of the network connectivity structure; b)computationally efficient; c) low-dimensional; and d) relatively robust to structured and random noise artifacts. This feature-space is obtained from a spherical relaxation of the transportation distance metric which measures the cost of transporting ``mass'' over the network to transform one function into another. Through theoretical and empirical assessments, we demonstrate the accuracy and efficiency of the approximation, especially for large problems.  While the application presented here is for identifying distinct brain activity patterns from fMRI, this feature-space can be applied to the problem of identifying recurring patterns and detecting outliers in measurements on many different types of networks, including sensor, control and social networks.
Density-Difference Estimation
Authors: Masashi Sugiyama, Takafumi Kanamori, Taiji Suzuki, Marthinus D. Plessis, Song Liu, Ichiro Takeuchi
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Change Point Detection

We address the problem of estimating the difference between two probability densities. A naive approach  is a two-step procedure of first estimating two densities separately and then computing their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small estimation error incurred in the first stage can cause a big error in the second stage. In this paper, we propose a single-shot procedure  for directly estimating the density difference without separately estimating two densities. We derive a non-parametric finite-sample error bound for the proposed single-shot density-difference estimator and show that it achieves the optimal convergence rate. We then show how the proposed density-difference estimator can be utilized in L2-distance approximation. Finally, we experimentally demonstrate the usefulness of the proposed method in robust distribution comparison such as class-prior estimation and change-point detection.
Variational Inference for Crowdsourcing
Authors: Qiang Liu, Jian Peng, Alexander T. Ihler
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Variational Inference

Crowdsourcing has become a popular paradigm for labeling large datasets. However, it has given rise to the computational task of aggregating the crowdsourced labels provided by a collection of unreliable annotators. We approach this problem by transforming it into a standard inference problem in graphical models, and applying approximate variational methods, including belief propagation (BP) and mean field (MF). We show that our BP algorithm generalizes both majority voting and a recent algorithm by Karger et al, while our MF method is closely related to a commonly used EM algorithm. In both cases, we find that the performance of the algorithms critically depends on the choice of a prior distribution on the workers' reliability; by choosing the prior properly, both BP and MF (and EM) perform surprisingly well on both simulated and real-world datasets, competitive with state-of-the-art algorithms based on more complicated modeling assumptions.
MCMC for continuous-time discrete-state systems
Authors: Vinayak Rao, Yee W. Teh
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

We propose a simple and novel framework for MCMC inference in continuous-time discrete-state systems with pure jump trajectories. We construct an exact MCMC sampler for such systems by alternately sampling a random discretization of time given a trajectory of the system, and then a new trajectory given the discretization.  The first step can be performed efficiently using properties of the Poisson process, while the second step can avail of discrete-time MCMC techniques based on the forward-backward algorithm. We compare our approach to particle MCMC and a uniformization-based sampler, and show its advantages.
A P300 BCI for the Masses: Prior Information Enables Instant Unsupervised Spelling
Authors: Pieter-Jan Kindermans, Hannes Verschore, David Verstraeten, Benjamin Schrauwen
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Transfer Learning

The usability of Brain Computer Interfaces (BCI) based on the P300 speller is severely hindered by the need for long training times and many repetitions of the same stimulus. In this contribution we introduce a set of unsupervised hierarchical probabilistic models that tackle both problems simultaneously by incorporating prior knowledge from two sources: information from other training subjects (through transfer learning) and information about the words being spelled (through language models). We show, that due to this prior knowledge, the performance of the unsupervised models parallels and in some cases even surpasses that of supervised models, while eliminating the tedious training session.
Learning about Canonical Views from Internet Image Collections
Authors: Elad Mezuman, Yair Weiss
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Object Recognition

Although human object recognition is supposedly robust to viewpoint, much research on human perception indicates that there is a preferred or “canonical” view of objects. This phenomenon was discovered more than 30 years ago but the canonical view of only a small number of categories has been validated experimentally. Moreover, the explanation for why humans prefer the canonical view over other views remains elusive. In this paper we ask: Can we use Internet image collections to learn more about canonical views? We start by manually finding the most common view in the results returned by Internet search engines when queried with the objects used in psychophysical experiments. Our results clearly show that the most likely view in the search engine corresponds to the same view preferred by human subjects in experiments. We also present a simple method to find the most likely view in an image collection and apply it to hundreds of categories. Using the new data we have collected we present strong evidence against the two most prominent formal theories of canonical views and provide novel constraints for new theories.
Learning High-Density Regions for a Generalized Kolmogorov-Smirnov Test in High-Dimensional Data
Authors: Assaf Glazer, Michael Lindenbaum, Shaul Markovitch
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Vocal Bursts Intensity Prediction

We propose an efficient, generalized, nonparametric, statistical Kolmogorov-Smirnov test for detecting distributional change in high-dimensional data. To implement the test, we introduce a novel, hierarchical, minimum-volume sets estimator to represent the distributions to be tested. Our work is motivated by the need to detect changes in data streams, and the test is especially efficient in this context. We provide the theoretical foundations of our test and show its superiority over existing methods.
Multiresolution Gaussian Processes
Authors: Emily Fox, David B. Dunson
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Gaussian Processes

We propose a multiresolution Gaussian process to capture long-range, non-Markovian dependencies while allowing for abrupt changes.  The multiresolution GP hierarchically couples a collection of smooth GPs, each defined over an element of a random nested partition.  Long-range dependencies are captured by the top-level GP while the partition points define the abrupt changes.  Due to the inherent conjugacy of the GPs, one can analytically marginalize the GPs and compute the conditional likelihood of the observations given the partition tree.  This allows for efficient inference of the partition itself, for which we employ graph-theoretic techniques.  We apply the multiresolution GP to the analysis of Magnetoencephalography (MEG) recordings of brain activity.
Localizing 3D cuboids in single-view images
Authors: Jianxiong Xiao, Bryan Russell, Antonio Torralba
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

In this paper we seek to detect rectangular cuboids and localize their corners in uncalibrated single-view images depicting everyday scenes. In contrast to recent approaches that rely on detecting vanishing points of the scene and grouping line segments to form cuboids, we build a discriminative parts-based detector that models the appearance of the cuboid corners and internal edges while enforcing consistency to a 3D cuboid model. Our model is invariant to the different 3D viewpoints and aspect ratios and is able to detect cuboids across many different object categories. We introduce a database of images with cuboid annotations that spans a variety of indoor and outdoor scenes and show qualitative and quantitative results on our collected database. Our model out-performs baseline detectors that use 2D constraints alone on the task of localizing cuboid corners.
Newton-Like Methods for Sparse Inverse Covariance Estimation
Authors: Figen Oztoprak, Jorge Nocedal, Steven Rennie, Peder A. Olsen
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

We propose two classes of second-order optimization methods for solving the sparse inverse covariance estimation problem. The first approach, which we call the Newton-LASSO method, minimizes a piecewise quadratic model of the objective function at every iteration to generate a step. We employ the fast iterative shrinkage thresholding method (FISTA) to solve this subproblem. The second approach, which we call the Orthant-Based Newton method, is a two-phase algorithm that first identifies an orthant face and then minimizes a smooth quadratic approximation of the objective function using the conjugate gradient method.  These methods exploit the structure of the Hessian to efficiently compute the search direction and to avoid explicitly storing the Hessian.  We show that quasi-Newton methods are also effective in this context, and describe a limited memory BFGS variant of the orthant-based Newton method.  We present numerical results that suggest that all the techniques described in this paper have attractive properties and constitute useful tools for solving the sparse inverse covariance estimation problem. Comparisons with the method implemented in the QUIC software package are presented.
Learning to Align from Scratch
Authors: Gary Huang, Marwan Mattar, Honglak Lee, Erik G. Learned-Miller
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Face Verification

Unsupervised joint alignment of images has been demonstrated to   improve performance on recognition tasks such as face verification.   Such alignment reduces undesired variability due to factors such as   pose, while only requiring weak supervision in the form of poorly   aligned examples.  However, prior work on unsupervised alignment of   complex, real world images has required the careful selection of   feature representation based on hand-crafted image descriptors, in   order to achieve an appropriate, smooth optimization landscape.    In this paper, we instead propose a novel combination of   unsupervised joint alignment with unsupervised feature learning.   Specifically, we incorporate deep learning into the {\em congealing}   alignment framework.  Through deep learning, we obtain features that   can represent the image at differing resolutions based on network   depth, and that are tuned to the statistics of the specific data   being aligned.  In addition, we modify the learning algorithm for   the restricted Boltzmann machine by incorporating a group sparsity   penalty, leading to a topographic organization on the learned   filters and improving subsequent alignment results.    We apply our method to the Labeled Faces in the Wild database   (LFW). Using the aligned images produced by our proposed   unsupervised algorithm, we achieve a significantly higher accuracy   in face verification than obtained using the original face images,   prior work in unsupervised alignment, and prior work in supervised   alignment.  We also match the accuracy for the best available, but   unpublished method.
Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints
Authors: Stefan Habenschuss, Johannes Bill, Bernhard Nessler
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Bayesian Inference, Variational Inference

Recent spiking network models of Bayesian inference and unsupervised learning frequently assume either inputs to arrive in a special format or employ complex computations in neuronal activation functions and synaptic plasticity rules. Here we show in a rigorous mathematical treatment how homeostatic processes, which have previously received little attention in this context, can overcome common theoretical limitations and facilitate the neural implementation and performance of existing models. In particular, we show that homeostatic plasticity can be understood as the enforcement of a 'balancing' posterior constraint during probabilistic inference and learning with Expectation Maximization. We link homeostatic dynamics to the theory of variational inference, and show that nontrivial terms, which typically appear during probabilistic inference in a large class of models, drop out. We demonstrate the feasibility of our approach in a spiking Winner-Take-All architecture of Bayesian inference and learning. Finally, we sketch how the mathematical framework can be extended to richer recurrent network architectures. Altogether, our theory provides a novel perspective on the interplay of homeostatic processes and synaptic plasticity in cortical microcircuits, and points to an essential role of homeostasis during inference and learning in spiking networks.
Clustering Aggregation as Maximum-Weight Independent Set
Authors: Nan Li, Longin J. Latecki
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Clustering

We formulate clustering aggregation as a special instance of Maximum-Weight Independent Set (MWIS) problem. For a given dataset, an attributed graph is constructed from the union of the input clusterings generated by different underlying clustering algorithms with different parameters. The vertices, which represent the distinct clusters, are weighted by an internal index measuring both cohesion and separation. The edges connect the vertices whose corresponding clusters overlap. Intuitively, an optimal aggregated clustering can be obtained by selecting an optimal subset of non-overlapping clusters partitioning the dataset together. We formalize this intuition as the MWIS problem on the attributed graph, i.e., finding the heaviest subset of mutually non-adjacent vertices.  This MWIS problem exhibits a special structure. Since the clusters of each input clustering form a partition of the dataset, the vertices corresponding to each clustering form a maximal independent set (MIS) in the attributed graph. We propose a variant of simulated annealing method that takes advantage of this special structure. Our algorithm starts from each MIS, which is close to a distinct local optimum of the MWIS problem, and utilizes a local search heuristic to explore its neighborhood in order to find the MWIS. Extensive experiments on many challenging datasets show that: 1. our approach to clustering aggregation automatically decides the optimal number of clusters; 2. it does not require any parameter tuning for the underlying clustering algorithms; 3. it can combine the advantages of different underlying clustering algorithms to achieve superior performance; 4. it is robust against moderate or even bad input clusterings.
Topology Constraints in Graphical Models
Authors: Marcelo Fiori, Pablo Musé, Guillermo Sapiro
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Graphical models are a very useful tool to describe and understand natural phenomena, from gene expression to climate change and social interactions. The topological structure of these graphs/networks is a fundamental part of the analysis, and in many cases the main goal of the study. However, little work has been done on incorporating prior topological knowledge onto the estimation of the underlying graphical models from sample data. In this work we propose extensions to the basic joint regression model for network estimation, which explicitly incorporate graph-topological constraints into the corresponding optimization approach. The first proposed extension includes an eigenvector centrality constraint, thereby promoting this important prior topological property. The second developed extension promotes the formation of certain motifs, triangle-shaped ones in particular, which are known to exist for example in genetic regulatory networks. The presentation of the underlying formulations, which serve as examples of the introduction of topological constraints in network estimation, is complemented with examples in diverse datasets demonstrating the importance of incorporating such critical prior knowledge.
Kernel Latent SVM for Visual Recognition
Authors: Weilong Yang, Yang Wang, Arash Vahdat, Greg Mori
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

Latent SVMs (LSVMs) are a class of powerful tools that have been successfully applied to many applications in computer vision. However, a limitation of LSVMs is that they rely on linear models. For many computer vision tasks, linear models are suboptimal and nonlinear models learned with kernels typically perform much better. Therefore it is desirable to develop the kernel version of LSVM. In this paper, we propose kernel latent SVM (KLSVM) -- a new learning framework that combines latent SVMs and kernel methods. We develop an iterative training algorithm to learn the model parameters. We demonstrate the effectiveness of KLSVM using three different applications in visual recognition. Our KLSVM formulation is very general and can be applied to solve a wide range of applications in computer vision and machine learning.
Learning Partially Observable Models Using Temporally Abstract Decision Trees
Authors: Erik Talvitie
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

This paper introduces timeline trees, which are partial models of partially observable environments. Timeline trees are given some specific predictions to make and learn a decision tree over history. The main idea of timeline trees is to use temporally abstract features to identify and split on features of key events, spread arbitrarily far apart in the past (whereas previous decision-tree-based methods have been limited to a finite suffix of history). Experiments demonstrate that timeline trees can learn to make high quality predictions in complex, partially observable environments with high-dimensional observations (e.g. an arcade game).
Multi-criteria Anomaly Detection using Pareto Depth Analysis
Authors: Ko-Jen Hsiao, Kevin Xu, Jeff Calder, Alfred O. Hero
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Anomaly Detection

We consider the problem of identifying patterns in a data set that exhibit anomalous behavior, often referred to as anomaly detection. In most anomaly detection algorithms, the dissimilarity between data samples is calculated by a single criterion, such as Euclidean distance. However, in many cases there may not exist a single dissimilarity measure that captures all possible anomalous patterns. In such a case, multiple criteria can be defined, and one can test for anomalies by scalarizing the multiple criteria by taking some linear combination of them. If the importance of the different criteria are not known in advance, the algorithm may need to be executed multiple times with different choices of weights in the linear combination. In this paper, we introduce a novel non-parametric multi-criteria anomaly detection method using Pareto depth analysis (PDA). PDA uses the concept of Pareto optimality to detect anomalies under multiple criteria without having to run an algorithm multiple times with different choices of weights. The proposed PDA approach scales linearly in the number of criteria and is provably better than linear combinations of the criteria.
Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes
Authors: Jake Bouvrie, Jean-Jeacques Slotine
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12

To learn reliable rules that can generalize to novel situations, the brain must be capable of imposing some form of regularization. Here we suggest, through theoretical and computational arguments, that the combination of noise with synchronization provides a plausible mechanism for regularization in the nervous system. The functional role of regularization is considered in a general context in which coupled computational systems receive inputs corrupted by correlated noise. Noise on the inputs is shown to impose regularization, and when synchronization upstream induces time-varying correlations across noise variables, the degree of regularization can be calibrated over time. The resulting qualitative behavior matches experimental data from visual cortex.
Calibrated Elastic Regularization in Matrix Completion
Authors: Tingni Sun, Cun-Hui Zhang
Date: 2012-12-01 00:00:00
Proceeding: NeurIPS 2012 12
Tasks: Matrix Completion

This paper concerns the problem of matrix completion, which is to estimate a matrix from observations in a small subset of indices. We propose a calibrated spectrum elastic net method with a sum of the nuclear and Frobenius penalties and develop an iterative algorithm to solve the convex minimization problem. The iterative algorithm alternates between imputing the missing entries in the incomplete matrix by the current guess and estimating the matrix by a scaled soft-thresholding singular value decomposition of the imputed matrix until the resulting matrix converges. A calibration step follows to correct the bias caused by the Frobenius penalty. Under proper coherence conditions and for suitable penalties levels, we prove that the proposed estimator achieves an error bound of nearly optimal order and in proportion to the noise level. This provides a unified analysis of the noisy and noiseless matrix completion problems. Simulation results are presented to compare our proposal with previous ones.