pwc-archive/papers-with-abstracts huggingface dataset:
Policy Shaping: Integrating Human Feedback with Reinforcement Learning
Authors: Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L. Isbell, Andrea L. Thomaz
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A long term goal of Interactive Reinforcement Learning is to incorporate non-expert human feedback to solve complex tasks. State-of-the-art methods have approached this problem by mapping human information to reward and value signals to indicate preferences and then iterating over them to compute the necessary control policy. In this paper we argue for an alternate, more effective characterization of human feedback: Policy Shaping. We introduce Advise, a Bayesian approach that attempts to maximize the information gained from human feedback by utilizing it as direct labels on the policy. We compare Advise to state-of-the-art approaches and highlight scenarios where it outperforms them and importantly is robust to infrequent and inconsistent human feedback.
Multilinear Dynamical Systems for Tensor Time Series
Authors: Mark Rogers, Lei LI, Stuart J. Russell
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Time Series, Time Series Analysis

Many scientific data occur as sequences of multidimensional arrays called tensors.  How can hidden, evolving trends in such data be extracted while preserving the tensor structure?  The model that is traditionally used is the linear dynamical system (LDS), which treats the observation at each time slice as a vector.  In this paper, we propose the multilinear dynamical system (MLDS) for modeling tensor time series and an expectation-maximization (EM) algorithm to estimate the parameters.  The MLDS models each time slice of the tensor time series as the multilinear projection of a corresponding member of a sequence of latent, low-dimensional tensors.  Compared to the LDS with an equal number of parameters, the MLDS achieves higher prediction accuracy and marginal likelihood for both simulated and real datasets.
Deep content-based music recommendation
Authors: Aaron Van Den Oord, Sander Dieleman, Benjamin Schrauwen
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Collaborative Filtering, Music Recommendation, Recommendation Systems

Automatic music recommendation has become an increasingly relevant problem in recent years, since a lot of music is now sold and consumed digitally. Most recommender systems rely on collaborative filtering. However, this approach suffers from the cold start problem: it fails when no usage data is available, so it is not effective for recommending new and unpopular songs. In this paper, we propose to use a latent factor model for recommendation, and predict the latent factors from music audio when they cannot be obtained from usage data. We compare a traditional approach using a bag-of-words representation of the audio signals with deep convolutional neural networks, and evaluate the predictions quantitatively and qualitatively on the Million Song Dataset. We show that using predicted latent factors produces sensible recommendations, despite the fact that there is a large semantic gap between the characteristics of a song that affect user preference and the corresponding audio signal. We also show that recent advances in deep learning translate very well to the music recommendation setting, with deep convolutional neural networks significantly outperforming the traditional approach.
A Stability-based Validation Procedure for Differentially Private Machine Learning
Authors: Kamalika Chaudhuri, Staal A. Vinterbo
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: BIG-bench Machine Learning

Differential privacy is a cryptographically motivated definition of privacy which has gained considerable attention in the algorithms, machine-learning and data-mining communities. While there has been an explosion of work on differentially private machine learning algorithms, a major barrier to achieving end-to-end differential privacy in practical machine learning applications is the lack of an effective procedure for differentially private parameter tuning, or, determining the parameter value, such as a bin size in a histogram, or a regularization parameter, that is suitable for a particular application.   In this paper, we introduce a generic validation procedure for differentially private machine learning algorithms that apply when a certain stability condition holds on the training algorithm and the validation performance metric. The training data size and the privacy budget used for training in our procedure is independent of the number of parameter values searched over. We apply our generic procedure to two fundamental tasks in statistics and machine-learning -- training a regularized linear classifier and building a histogram density estimator that result in end-to-end differentially private solutions for these problems.
Capacity of strong attractor patterns to model behavioural and cognitive prototypes
Authors: Abbas Edalat
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

We solve the mean field equations for a stochastic Hopfield network with temperature (noise) in the presence of strong, i.e., multiply stored patterns, and use this solution to obtain the storage capacity of such a network. Our result provides for the first time a rigorous solution of the mean field equations for the standard Hopfield model and is in contrast to the mathematically unjustifiable replica technique that has been hitherto used for this derivation. We show that the critical temperature for stability of a strong pattern is equal to its degree or multiplicity, when sum of the cubes of degrees of all stored patterns is negligible compared to the network size. In the case of a single strong pattern in the presence of simple patterns, when the ratio of the number of all stored patterns and the network size is a positive constant, we obtain the distribution of the overlaps of the patterns with the mean field and deduce that the storage capacity for retrieving a strong pattern exceeds that for retrieving a simple pattern by a multiplicative factor equal to the square of the degree of the strong pattern. This square law property provides justification for using strong patterns to model attachment types and behavioural prototypes in psychology and psychotherapy.
Fantope Projection and Selection: A near-optimal convex relaxation of sparse PCA
Authors: Vincent Q. Vu, Juhee Cho, Jing Lei, Karl Rohe
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

We propose a novel convex relaxation of sparse principal subspace estimation based on the convex hull of rank-$d$ projection matrices (the Fantope). The convex problem can be solved efficiently using alternating direction method of multipliers (ADMM). We establish a near-optimal convergence rate, in terms of the sparsity, ambient dimension, and sample size, for estimation of the principal subspace of a general covariance matrix without assuming the spiked covariance model. In the special case of $d=1$, our result implies the near- optimality of DSPCA even when the solution is not rank 1. We also provide a general theoretical framework for analyzing the statistical  properties of the method for arbitrary input matrices that extends the  applicability and provable guarantees to a wide array of settings.  We  demonstrate this with an application to Kendall's tau correlation matrices  and transelliptical component analysis.
Bayesian inference for low rank spatiotemporal neural receptive fields
Authors: Mijung Park, Jonathan W. Pillow
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Bayesian Inference, Computational Efficiency

The receptive field (RF) of a sensory neuron describes how the   neuron integrates sensory stimuli over time and space. In typical   experiments with naturalistic or flickering spatiotemporal stimuli,   RFs are very high-dimensional, due to the large number of   coefficients needed to specify an integration profile across time   and space.  Estimating these coefficients from small amounts of data   poses a variety of challenging statistical and computational   problems.  Here we address these challenges by developing Bayesian   reduced rank regression methods for RF estimation. This corresponds   to modeling the RF as a sum of several space-time separable (i.e.,   rank-1) filters, which proves accurate even for neurons with   strongly oriented space-time RFs.  This approach substantially   reduces the number of parameters needed to specify the RF, from   1K-100K down to mere 100s in the examples we consider, and confers   substantial benefits in statistical power and computational   efficiency.  In particular, we introduce a novel prior over low-rank   RFs using the restriction of a matrix normal prior to the manifold   of low-rank matrices. We then use a localized'' prior over row and   column covariances to obtain sparse, smooth, localized estimates of   the spatial and temporal RF components.  We develop two methods for   inference in the resulting hierarchical model: (1) a fully Bayesian   method using blocked-Gibbs sampling; and (2) a fast, approximate   method that employs alternating coordinate ascent of the conditional   marginal likelihood.  We develop these methods under Gaussian and   Poisson noise models, and show that low-rank estimates substantially   outperform full rank estimates in accuracy and speed using neural   data from retina and V1."
Adaptive Submodular Maximization in Bandit Setting
Authors: Victor Gabillon, Branislav Kveton, Zheng Wen, Brian Eriksson, S. Muthukrishnan
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

Maximization of submodular functions has wide applications in machine learning and artificial intelligence. Adaptive submodular maximization has been traditionally studied under the assumption that the model of the world, the expected gain of choosing an item given previously selected items and their states, is known. In this paper, we study the scenario where the expected gain is initially unknown and it is learned by interacting repeatedly with the optimized function. We propose an efficient algorithm for solving our problem and prove that its expected cumulative regret increases logarithmically with time. Our regret bound captures the inherent property of submodular maximization, earlier mistakes are more costly than later ones. We refer to our approach as Optimistic Adaptive Submodular Maximization (OASM) because it trades off exploration and exploitation based on the optimism in the face of uncertainty principle. We evaluate our method on a preference elicitation problem and show that non-trivial K-step policies can be learned from just a few hundred interactions with the problem.
Generalized Method-of-Moments for Rank Aggregation
Authors: Hossein Azari Soufiani, William Chen, David C. Parkes, Lirong Xia
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

In this paper we propose a class of efficient Generalized Method-of-Moments(GMM) algorithms for computing parameters of the Plackett-Luce model, where the data consists of full rankings over alternatives. Our technique is based on breaking the full rankings into pairwise comparisons, and then computing parameters that satisfy a set of generalized moment conditions. We identify conditions for the output of GMM to be unique, and identify a general class of consistent and inconsistent breakings. We then show by theory and experiments that our algorithms run significantly faster than the classical Minorize-Maximization (MM) algorithm, while achieving competitive statistical efficiency.
Analyzing Hogwild Parallel Gaussian Gibbs Sampling
Authors: Matthew Johnson, James Saunderson, Alan Willsky
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

Sampling inference methods are computationally difficult to scale for many models in part because global dependencies can reduce opportunities for parallel computation.  Without strict conditional independence structure among variables, standard Gibbs sampling theory requires sample updates to be performed sequentially, even if dependence between most variables is not strong. Empirical work has shown that some models can be sampled effectively by going Hogwild'' and simply running Gibbs updates in parallel with only periodic global communication, but the successes and limitations of such a strategy are not well understood.  As a step towards such an understanding, we study the Hogwild Gibbs sampling strategy in the context of Gaussian distributions. We develop a framework which provides convergence conditions and error bounds along with simple proofs and connections to methods in numerical linear algebra.  In particular, we show that if the Gaussian precision matrix is generalized diagonally dominant, then any Hogwild Gibbs sampler, with any update schedule or allocation of variables to processors, yields a stable sampling process with the correct sample mean. "
Minimax Optimal Algorithms for Unconstrained Linear Optimization
Authors: Brendan Mcmahan, Jacob Abernethy
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

We design and analyze minimax-optimal algorithms for online linear   optimization games where the player's choice is unconstrained.  The   player strives to minimize regret, the difference between his loss   and the loss of a post-hoc benchmark strategy.  The standard   benchmark is the loss of the best strategy chosen from a bounded   comparator set, whereas we consider a broad range of benchmark   functions. We consider the problem as a sequential multi-stage   zero-sum game, and we give a thorough analysis of the minimax   behavior of the game, providing characterizations for the value of   the game, as well as both the player's and the adversary's optimal   strategy.  We show how these objects can be computed efficiently   under certain circumstances, and by selecting an appropriate   benchmark, we construct a novel hedging strategy for an   unconstrained betting game.
(Nearly) Optimal Algorithms for Private Online Learning in Full-information and Bandit Settings
Authors: Abhradeep Guha Thakurta, Adam Smith
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

We provide a general technique for making online learning algorithms differentially private, in both the full information and bandit settings. Our technique applies to algorithms that aim to minimize a \emph{convex} loss function which is a sum of smaller convex loss terms, one for each data point. We modify the popular \emph{mirror descent} approach, or rather a variant called \emph{follow the approximate leader}.    The technique leads to the first nonprivate algorithms for private online learning in the bandit setting. In the full information setting, our algorithms improve over the regret bounds of previous work.  In many cases, our algorithms (in both settings) matching the dependence on the input length, $T$, of the \emph{optimal nonprivate} regret bounds up to logarithmic factors in $T$. Our algorithms require logarithmic space and update time.
Σ-Optimality for Active Learning on Gaussian Random Fields
Authors: Yifei Ma, Roman Garnett, Jeff Schneider
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Active Learning, General Classification

A common classifier for unlabeled nodes on undirected graphs uses label propagation from the labeled nodes, equivalent to the harmonic predictor on Gaussian random fields (GRFs). For active learning on GRFs, the commonly used V-optimality criterion queries nodes that reduce the L2 (regression) loss. V-optimality satisfies a submodularity property showing that greedy reduction produces a (1 − 1/e) globally optimal solution. However, L2 loss may not characterise the true nature of 0/1 loss in classification problems and thus may not be the best choice for active learning. We consider a new criterion we call Σ-optimality, which queries the node that minimizes the sum of the elements in the predictive covariance. Σ-optimality directly optimizes the risk of the surveying problem, which is to determine the proportion of nodes belonging to one class. In this paper we extend submodularity guarantees from V-optimality to Σ-optimality using properties specific to GRFs. We further show that GRFs satisfy the suppressor-free condition in addition to the conditional independence inherited from Markov random fields. We test Σ-optimality on real-world graphs with both synthetic and real data and show that it outperforms V-optimality and other related methods on classification.
Learning Kernels Using Local Rademacher Complexity
Authors: Corinna Cortes, Marius Kloft, Mehryar Mohri
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: General Classification, Multi-class Classification

We use the notion of local Rademacher complexity to design new algorithms for learning kernels. Our algorithms thereby benefit from the sharper learning bounds based on that notion which, under certain general conditions, guarantee a faster convergence rate.  We devise two new learning kernel algorithms: one based on a convex optimization problem for which we give an efficient solution using existing learning kernel techniques, and another one that can be formulated as a DC-programming problem for which we describe a solution in detail. We also report the results of experiments with both algorithms in both binary and multi-class classification tasks.
Annealing between distributions by averaging moments
Authors: Roger B. Grosse, Chris J. Maddison, Ruslan R. Salakhutdinov
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

Many powerful Monte Carlo techniques for estimating partition functions, such as annealed importance sampling (AIS), are based on sampling from a sequence of intermediate distributions which interpolate between a tractable initial distribution and an intractable target distribution. The near-universal practice is to use geometric averages of the initial and target distributions, but alternative paths can perform substantially better. We present a novel sequence of intermediate distributions for exponential families: averaging the moments of the initial and target distributions. We derive an asymptotically optimal piecewise linear schedule for the moments path and show that it performs at least as well as geometric averages with a linear schedule.  Moment averaging performs well empirically at estimating partition functions of restricted Boltzmann machines (RBMs), which form the building blocks of many deep learning models, including Deep Belief Networks and Deep Boltzmann Machines.
Optimizing Instructional Policies
Authors: Robert V. Lindsey, Michael C. Mozer, William J. Huggins, Harold Pashler
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

Psychologists are interested in developing instructional policies that boost student learning. An instructional policy specifies the manner and content of instruction. For example, in the domain of concept learning, a policy might specify the nature of exemplars chosen over a training sequence.  Traditional psychological studies compare several hand-selected policies, e.g., contrasting a policy that selects only difficult-to-classify exemplars with a policy that gradually progresses over the training sequence from easy exemplars to more difficult (known as {\em fading}). We propose an alternative to the traditional methodology in which we define a  parameterized space of policies and search this space to identify the optimum policy. For example, in concept learning, policies might be described by a fading function that specifies exemplar difficulty over time.  We propose an experimental technique for searching policy spaces using Gaussian process surrogate-based optimization and a generative model of student performance. Instead of evaluating a few experimental conditions each with many human subjects, as the traditional methodology does, our technique evaluates many experimental conditions each with a few subjects.  Even though individual subjects provide only a noisy estimate of the population mean, the optimization method allows us to determine the shape of the policy space and identify the global optimum, and is as efficient in its subject budget as a traditional A-B comparison.  We evaluate the method via two behavioral studies, and suggest that the method has broad applicability to optimization problems involving humans in domains beyond the educational arena.
Translating Embeddings for Modeling Multi-relational Data
Authors: Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Link Prediction

We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose, TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples.
Real-Time Inference for a Gamma Process Model of Neural Spiking
Authors: David E. Carlson, Vinayak Rao, Joshua T. Vogelstein, Lawrence Carin
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

With simultaneous measurements from ever increasing populations of neurons, there is a growing need for sophisticated tools to recover signals from individual neurons. In electrophysiology experiments, this classically proceeds in a two-step process: (i) threshold the waveforms to detect putative spikes and (ii) cluster the waveforms into single units (neurons). We extend previous Bayesian nonparamet- ric models of neural spiking to jointly detect and cluster neurons using a Gamma process model. Importantly, we develop an online approximate inference scheme enabling real-time analysis, with performance exceeding the previous state-of-the- art. Via exploratory data analysis—using data with partial ground truth as well as two novel data sets—we find several features of our model collectively contribute to our improved performance including: (i) accounting for colored noise, (ii) de- tecting overlapping spikes, (iii) tracking waveform dynamics, and (iv) using mul- tiple channels. We hope to enable novel experiments simultaneously measuring many thousands of neurons and possibly adapting stimuli dynamically to probe ever deeper into the mysteries of the brain.
Understanding Dropout
Authors: Pierre Baldi, Peter J. Sadowski
Date: 2013-12-01 00:00:00
Proceeding: NeurIPS 2013 12

Dropout is a relatively new algorithm for training neural networks which relies on stochastically dropping out'' neurons during training in order to avoid the co-adaptation of feature detectors. We introduce a general formalism for studying dropout on either units or connections, with arbitrary probability values, and use it to analyze the averaging and regularizing properties of dropout in both linear and non-linear networks. For deep neural networks, the averaging properties of dropout are characterized by three recursive equations, including the approximation of expectations by normalized weighted geometric means. We provide estimates and bounds for these approximations and corroborate the results with simulations. We also show in simple cases how dropout performs stochastic gradient descent on a regularized error function."