mtybilly/Instruct2DS huggingface dataset:
Analyzing Auditory Neurons by Learning Distance Functions
Authors: Tomer Hertz, Israel Nelken, Daphna Weinshall Inna Weiner
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present a novel approach to the characterization of complex sensory neurons. One of the main goals of characterizing sensory neurons is to characterize dimensions in stimulus space to which the neurons are highly sensitive (causing large gradients in the neural responses) or al- ternatively dimensions in stimulus space to which the neuronal response are invariant (deﬁning iso-response manifolds). We formulate this prob- lem as that of learning a geometry on stimulus space that is compatible with the neural responses: the distance between stimuli should be large when the responses they evoke are very different, and small when the re- sponses they evoke are similar. Here we show how to successfully train such distance functions using rather limited amount of information. The data consisted of the responses of neurons in primary auditory cortex (A1) of anesthetized cats to 32 stimuli derived from natural sounds. For each neuron, a subset of all pairs of stimuli was selected such that the responses of the two stimuli in a pair were either very similar or very dissimilar. The distance function was trained to ﬁt these constraints. The resulting distance functions generalized to predict the distances between the responses of a test stimulus and the trained stimuli.
Cyclic Equilibria in Markov Games
Authors: Amy Greenwald, Michael L. Littman Martin Zinkevich
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Although variants of value iteration have been proposed for ﬁnding Nash or correlated equilibria in general-sum Markov games, these variants have not been shown to be effective in general. In this paper, we demon- strate by construction that existing variants of value iteration cannot ﬁnd stationary equilibrium policies in arbitrary general-sum Markov games. Instead, we propose an alternative interpretation of the output of value it- eration based on a new (non-stationary) equilibrium concept that we call “cyclic equilibria.” We prove that value iteration identiﬁes cyclic equi- libria in a class of games in which it fails to ﬁnd stationary equilibria. We also demonstrate empirically that value iteration ﬁnds cyclic equilibria in nearly all examples drawn from a random distribution of Markov games.
The Information-Form Data Association Filter
Authors: Sebastian Thrun, Gary Bradski, Kunle Olukotun Brad Schumitsch
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper presents a new ﬁlter for online data association problems in high-dimensional spaces. The key innovation is a representation of the data association posterior in information form, in which the “proxim- ity” of objects and tracks are expressed by numerical links. Updating these links requires linear time, compared to exponential time required for computing the exact posterior probabilities. The paper derives the algorithm formally and provides comparative results using data obtained by a real-world camera array and by a large-scale sensor network simu- lation.
Consistency of one-class SVM and related algorithms
Authors: Jean-philippe Vert Régis Vert
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We determine the asymptotic limit of the function computed by support vector machines (SVM) and related algorithms that minimize a regularized empirical convex loss function in the reproducing kernel Hilbert space of the Gaussian RBF kernel, in the situation where the number of examples tends to infinity, the bandwidth of the Gaussian kernel tends to 0, and the regularization parameter is held fixed. Non-asymptotic convergence bounds to this limit in the L2 sense are provided, together with upper bounds on the classification error that is shown to converge to the Bayes risk, therefore proving the Bayes-consistency of a variety of methods although the regularization term does not vanish. These results are particularly relevant to the one-class SVM, for which the regularization can not vanish by construction, and which is shown for the first time to be a consistent density level set estimator.
Nested sampling for Potts models
Authors: David MacKay, Zoubin Ghahramani, John Skilling Iain Murray
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Nested sampling is a new Monte Carlo method by Skilling [1] intended for general Bayesian computation. Nested sampling provides a robust alternative to annealing-based methods for computing normalizing constants. It can also generate estimates of other quantities such as posterior expectations. The key technical requirement is an ability to draw samples uniformly from the prior sub ject to a constraint on the likelihood. We provide a demonstration with the Potts model, an undirected graphical model.
Correlated Topic Models
Authors: David M. Blei John D. Lafferty
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Topic models, such as latent Dirichlet allocation (LDA), can be useful tools for the statistical analysis of document collections and other discrete data. The LDA model assumes that the words of each document arise from a mixture of topics, each of which is a distribution over the vocabulary. A limitation of LDA is the inability to model topic correlation even though, for example, a document about genetics is more likely to also be about disease than x-ray astronomy. This limitation stems from the use of the Dirichlet distribution to model the variability among the topic proportions. In this paper we develop the correlated topic model (CTM), where the topic proportions exhibit correlation via the logistic normal distribution [1]. We derive a mean-field variational inference algorithm for approximate posterior inference in this model, which is complicated by the fact that the logistic normal is not conjugate to the multinomial. The CTM gives a better fit than LDA on a collection of OCRed articles from the journal Science. Furthermore, the CTM provides a natural way of visualizing and exploring this and other unstructured data sets.
Learning in Silicon: Timing is Everything
Authors: Kwabena Boahen John V. Arthur
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We describe a neuromorphic chip that uses binary synapses with spike timing-dependent plasticity (STDP) to learn stimulated patterns of activ- ity and to compensate for variability in excitability. Speciﬁcally, STDP preferentially potentiates (turns on) synapses that project from excitable neurons, which spike early, to lethargic neurons, which spike late. The additional excitatory synaptic current makes lethargic neurons spike ear- lier, thereby causing neurons that belong to the same pattern to spike in synchrony. Once learned, an entire pattern can be recalled by stimulating a subset.
1 Variability in Neural Systems
Evidence suggests precise spike timing is important in neural coding, speciﬁcally, in the hippocampus. The hippocampus uses timing in the spike activity of place cells (in addition to rate) to encode location in space [1]. Place cells employ a phase code: the timing at which a neuron spikes relative to the phase of the inhibitory theta rhythm (5-12Hz) conveys information. As an animal approaches a place cell’s preferred location, the place cell not only increases its spike rate, but also spikes at earlier phases in the theta cycle.
To implement a phase code, the theta rhythm is thought to prevent spiking until the input synaptic current exceeds the sum of the neuron threshold and the decreasing inhibition on the downward phase of the cycle [2]. However, even with identical inputs and common theta inhibition, neurons do not spike in synchrony. Variability in excitability spreads the activity in phase. Lethargic neurons (such as those with high thresholds) spike late in the theta cycle, since their input exceeds the sum of the neuron threshold and theta inhibition only after the theta inhibition has had time to decrease. Conversely, excitable neurons (such as those with low thresholds) spike early in the theta cycle. Consequently, variability in excitability translates into variability in timing.
We hypothesize that the hippocampus achieves its precise spike timing (about 10ms) through plasticity enhanced phase-coding (PEP). The source of hippocampal timing preci- sion in the presence of variability (and noise) remains unexplained. Synaptic plasticity can compensate for variability in excitability if it increases excitatory synaptic input to neurons in inverse proportion to their excitabilities. Recasting this in a phase-coding framework, we desire a learning rule that increases excitatory synaptic input to neurons directly related to their phases. Neurons that lag require additional synaptic input, whereas neurons that lead
Correcting sample selection bias in maximum entropy density estimation
Authors: Steven J. Phillips, Robert E. Schapire Miroslav Dudík
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We study the problem of maximum entropy density estimation in the presence of known sample selection bias. We propose three bias cor- rection approaches. The ﬁrst one takes advantage of unbiased sufﬁcient statistics which can be obtained from biased samples. The second one es- timates the biased distribution and then factors the bias out. The third one approximates the second by only using samples from the sampling distri- bution. We provide guarantees for the ﬁrst two approaches and evaluate the performance of all three approaches in synthetic experiments and on real data from species habitat modeling, where maxent has been success- fully applied and where sample selection bias is a signiﬁcant problem.
Rate Distortion Codes in Sensor Networks: A System-level Analysis
Authors: Peter Davis Tatsuto Murayama
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper provides a system-level analysis of a scalable distributed sens- ing model for networked sensors. In our system model, a data center ac- quires data from a bunch of L sensors which each independently encode their noisy observations of an original binary sequence, and transmit their encoded data sequences to the data center at a combined rate R, which is limited. Supposing that the sensors use independent LDGM rate dis- tortion codes, we show that the system performance can be evaluated for any given ﬁnite R when the number of sensors L goes to inﬁnity. The analysis shows how the optimal strategy for the distributed sensing prob- lem changes at critical values of the data rate R or the noise level.
Beyond Pair-Based STDP: a Phenomenological Rule for Spike Triplet and Frequency Effects
Authors: Wulfram Gerstner Jean-pascal Pfister
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: While classical experiments on spike-timing dependent plasticity analyzed synaptic changes as a function of the timing of pairs of pre- and postsynaptic spikes, more recent experiments also point to the effect of spike triplets. Here we develop a mathematical framework that allows us to characterize timing based learning rules. Moreover, we identify a candidate learning rule with five variables (and 5 free parameters) that captures a variety of experimental data, including the dependence of potentiation and depression upon pre- and postsynaptic firing frequencies. The relation to the Bienenstock-Cooper-Munro rule as well as to some timing-based rules is discussed.
Is Early Vision Optimized for Extracting Higher-order Dependencies?
Authors: Michael S. Lewicki Yan Karklin
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Linear implementations of the efficient coding hypothesis, such as independent component analysis (ICA) and sparse coding models, have provided functional explanations for properties of simple cells in V1 [1, 2]. These models, however, ignore the non-linear behavior of neurons and fail to match individual and population properties of neural receptive fields in subtle but important ways. Hierarchical models, including Gaussian Scale Mixtures [3, 4] and other generative statistical models [5, 6], can capture higher-order regularities in natural images and explain nonlinear aspects of neural processing such as normalization and context effects [6, 7]. Previously, it had been assumed that the lower level representation is independent of the hierarchy, and had been fixed when training these models. Here we examine the optimal lower-level representations derived in the context of a hierarchical model and find that the resulting representations are strikingly different from those based on linear models. Unlike the the basis functions and filters learned by ICA or sparse coding, these functions individually more closely resemble simple cell receptive fields and collectively span a broad range of spatial scales. Our work unifies several related approaches and observations about natural image structure and suggests that hierarchical models might yield better representations of image structure throughout the hierarchy.
An Approximate Inference Approach for the PCA Reconstruction Error
Authors: Manfred Opper
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: The problem of computing a resample estimate for the reconstruction error in PCA is reformulated as an inference problem with the help of the replica method. Using the expectation consistent (EC) approximation, the intractable inference problem can be solved efficiently using only two variational parameters. A perturbative correction to the result is computed and an alternative simplified derivation is also presented.
Active Bidirectional Coupling in a Cochlear Chip
Authors: Kwabena A. Boahen Bo Wen
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present a novel cochlear model implemented in analog very large scale integration (VLSI) technology that emulates nonlinear active cochlear behavior. This silicon cochlea includes outer hair cell (OHC) electromotility through active bidirectional coupling (ABC), a mech- anism we proposed in which OHC motile forces, through the mi- croanatomical organization of the organ of Corti, realize the cochlear ampliﬁer. Our chip measurements demonstrate that frequency responses become larger and more sharply tuned when ABC is turned on; the de- gree of the enhancement decreases with input intensity as ABC includes saturation of OHC forces.
1 Silicon Cochleae
Cochlear models, mathematical and physical, with the shared goal of emulating nonlinear active cochlear behavior, shed light on how the cochlea works if based on cochlear mi- cromechanics. Among the modeling efforts, silicon cochleae have promise in meeting the need for real-time performance and low power consumption. Lyon and Mead developed the ﬁrst analog electronic cochlea [1], which employed a cascade of second-order ﬁlters with exponentially decreasing resonant frequencies. However, the cascade structure suf- fers from delay and noise accumulation and lacks fault-tolerance. Modeling the cochlea more faithfully, Watts built a two-dimensional (2D) passive cochlea that addressed these shortcomings by incorporating the cochlear ﬂuid using a resistive network [2]. This par- allel structure, however, has its own problem: response gain is diminished by interference among the second-order sections’ outputs due to the large phase change at resonance [3].
Listening more to biology, our silicon cochlea aims to overcome the shortcomings of exist- ing architectures by mimicking the cochlear micromechanics while including outer hair cell (OHC) electromotility. Although how exactly OHC motile forces boost the basilar mem- brane’s (BM) vibration remains a mystery, cochlear microanatomy provides clues. Based on these clues, we previously proposed a novel mechanism, active bidirectional coupling (ABC), for the cochlear ampliﬁer [4]. Here, we report an analog VLSI chip that implements this mechanism. In essence, our implementation is the ﬁrst silicon cochlea that employs stimulus enhancement (i.e., active behavior) instead of undamping (i.e., high ﬁlter Q [5]).
The paper is organized as follows. In Section 2, we present the hypothesized mechanism (ABC), ﬁrst described in [4]. In Section 3, we provide a mathematical formulation of the
Affine Structure From Sound
Authors: Sebastian Thrun
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We consider the problem of localizing a set of microphones together with a set of external acoustic events (e.g., hand claps), emitted at un- known times and unknown locations. We propose a solution that ap- proximates this problem under a far ﬁeld approximation deﬁned in the calculus of afﬁne geometry, and that relies on singular value decompo- sition (SVD) to recover the afﬁne structure of the problem. We then deﬁne low-dimensional optimization techniques for embedding the solu- tion into Euclidean geometry, and further techniques for recovering the locations and emission times of the acoustic events. The approach is use- ful for the calibration of ad-hoc microphone arrays and sensor networks.
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Authors: John Blitzer, Lawrence K. Saul Kilian Q. Weinberger
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We show how to learn a Mahanalobis distance metric for k -nearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the k -nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification--for example, achieving a test error rate of 1.3% on the MNIST handwritten digits. As in support vector machines (SVMs), the learning problem reduces to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our framework requires no modification or extension for problems in multiway (as opposed to binary) classification.
Estimating the wrong Markov random field: Benefits in the computation-limited setting
Authors: Martin J. Wainwright
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Consider the problem of joint parameter estimation and prediction in a Markov random field: i.e., the model parameters are estimated on the basis of an initial set of data, and then the fitted model is used to perform prediction (e.g., smoothing, denoising, interpolation) on a new noisy observation. Working in the computation-limited setting, we analyze a joint method in which the same convex variational relaxation is used to construct an M-estimator for fitting parameters, and to perform approximate marginalization for the prediction step. The key result of this paper is that in the computation-limited setting, using an inconsistent parameter estimator (i.e., an estimator that returns the "wrong" model even in the infinite data limit) is provably beneficial, since the resulting errors can partially compensate for errors made by using an approximate prediction technique. En route to this result, we analyze the asymptotic properties of M-estimators based on convex variational relaxations, and establish a Lipschitz stability property that holds for a broad class of variational methods. We show that joint estimation/prediction based on the reweighted sum-product algorithm substantially outperforms a commonly used heuristic based on ordinary sum-product. 1 Keywords: Markov random fields; variational method; message-passing algorithms; sum-product; belief propagation; parameter estimation; learning.
Gaussian Processes for Multiuser Detection in CDMA receivers
Authors: Sebastian Caro, Fernando Pérez-Cruz Juan J. Murillo-fuentes
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: In this paper we propose a new receiver for digital communications. We focus on the application of Gaussian Processes (GPs) to the multiuser detection (MUD) in code division multiple access (CDMA) systems to solve the near-far problem. Hence, we aim to reduce the interference from other users sharing the same frequency band. While usual approaches minimize the mean square error (MMSE) to linearly retrieve the user of interest, we exploit the same criteria but in the design of a nonlinear MUD. Since the optimal solution is known to be nonlinear, the performance of this novel method clearly improves that of the MMSE detectors. Furthermore, the GP based MUD achieves excellent interference suppression even for short training sequences. We also include some experiments to illustrate that other nonlinear detectors such as those based on Support Vector Machines (SVMs) exhibit a worse performance.
Walk-Sum Interpretation and Analysis of Gaussian Belief Propagation
Authors: Alan S. Willsky, Jason K. Johnson Dmitry Malioutov
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper presents a new framework based on walks in a graph for analysis and inference in Gaussian graphical models. The key idea is to decompose correlations between variables as a sum over all walks between those variables in the graph. The weight of each walk is given by a product of edgewise partial correlations. We provide a walk-sum interpretation of Gaussian belief propagation in trees and of the approximate method of loopy belief propagation in graphs with cycles. This perspective leads to a better understanding of Gaussian belief propagation and of its convergence in loopy graphs.
Analyzing Coupled Brain Sources: Distinguishing True from Spurious Interaction
Authors: Andreas Ziehe, Frank Meinecke, Klaus-Robert Müller Guido Nolte
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: When trying to understand the brain, it is of fundamental importance to analyse (e.g. from EEG/MEG measurements) what parts of the cortex interact with each other in order to infer more accurate models of brain activity. Common techniques like Blind Source Separation (BSS) can estimate brain sources and single out artifacts by using the underlying assumption of source signal independence. However, physiologically interesting brain sources typically interact, so BSS will--by construction-- fail to characterize them properly. Noting that there are truly interacting sources and signals that only seemingly interact due to effects of volume conduction, this work aims to contribute by distinguishing these effects. For this a new BSS technique is proposed that uses anti-symmetrized cross-correlation matrices and subsequent diagonalization. The resulting decomposition consists of the truly interacting brain sources and suppresses any spurious interaction stemming from volume conduction. Our new concept of interacting source analysis (ISA) is successfully demonstrated on MEG data.
Group and Topic Discovery from Relations and Their Attributes
Authors: Natasha Mohanty, Andrew McCallum Xuerui Wang
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present a probabilistic generative model of entity relationships and their attributes that simultaneously discovers groups among the entities and topics among the corresponding textual attributes. Block-models of relationship data have been studied in social network analysis for some time. Here we simultaneously cluster in several modalities at once, incor- porating the attributes (here, words) associated with certain relationships. Signiﬁcantly, joint inference allows the discovery of topics to be guided by the emerging groups, and vice-versa. We present experimental results on two large data sets: sixteen years of bills put before the U.S. Sen- ate, comprising their corresponding text and voting records, and thirteen years of similar data from the United Nations. We show that in compari- son with traditional, separate latent-variable models for words, or Block- structures for votes, the Group-Topic model’s joint inference discovers more cohesive groups and improved topics.
Factorial Switching Kalman Filters for Condition Monitoring in Neonatal Intensive Care
Authors: John Quinn, Neil Mcintosh Christopher Williams
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: The observed physiological dynamics of an infant receiving intensive care are affected by many possible factors, including interventions to the baby, the operation of the monitoring equipment and the state of health. The Factorial Switching Kalman Filter can be used to infer the presence of such factors from a sequence of observations, and to estimate the true values where these observations have been corrupted. We apply this model to clinical time series data and show it to be effective in identifying a number of artifactual and physiological patterns.
A Criterion for the Convergence of Learning with Spike Timing Dependent Plasticity
Authors: Wolfgang Maass Robert A. Legenstein
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We investigate under what conditions a neuron can learn by experimen- tally supported rules for spike timing dependent plasticity (STDP) to pre- dict the arrival times of strong “teacher inputs” to the same neuron. It turns out that in contrast to the famous Perceptron Convergence Theo- rem, which predicts convergence of the perceptron learning rule for a simpliﬁed neuron model whenever a stable solution exists, no equally strong convergence guarantee can be given for spiking neurons with STDP. But we derive a criterion on the statistical dependency structure of input spike trains which characterizes exactly when learning with STDP will converge on average for a simple model of a spiking neuron. This criterion is reminiscent of the linear separability criterion of the Percep- tron Convergence Theorem, but it applies here to the rows of a correlation matrix related to the spike inputs. In addition we show through computer simulations for more realistic neuron models that the resulting analyti- cally predicted positive learning results not only hold for the common interpretation of STDP where STDP changes the weights of synapses, but also for a more realistic interpretation suggested by experimental data where STDP modulates the initial release probability of dynamic synapses.
Q-Clustering
Authors: Nebojsa Jojic, Jeff A. Bilmes Mukund Narasimhan
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We show that Queyranne's algorithm for minimizing symmetric submodular functions can be used for clustering with a variety of different objective functions. Two specific criteria that we consider in this paper are the single linkage and the minimum description length criteria. The first criterion tries to maximize the minimum distance between elements of different clusters, and is inherently "discriminative". It is known that optimal clusterings into k clusters for any given k in polynomial time for this criterion can be computed. The second criterion seeks to minimize the description length of the clusters given a probabilistic generative model. We show that the optimal partitioning into 2 clusters, and approximate partitioning (guaranteed to be within a factor of 2 of the the optimal) for more clusters can be computed. To the best of our knowledge, this is the first time that a tractable algorithm for finding the optimal clustering with respect to the MDL criterion for 2 clusters has been given. Besides the optimality result for the MDL criterion, the chief contribution of this paper is to show that the same algorithm can be used to optimize a broad class of criteria, and hence can be used for many application specific criterion for which efficient algorithm are not known.
Generalization Error Bounds for Aggregation by Mirror Descent with Averaging
Authors: Alexander Nazin, Alexandre Tsybakov, Nicolas Vayatis Anatoli Juditsky
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We consider the problem of constructing an aggregated estimator from a ﬁnite class of base functions which approximately minimizes a con- vex risk functional under the ℓ1 constraint. For this purpose, we propose a stochastic procedure, the mirror descent, which performs gradient de- scent in the dual space. The generated estimates are additionally aver- aged in a recursive fashion with speciﬁc weights. Mirror descent algo- rithms have been developed in different contexts and they are known to be particularly efﬁcient in high dimensional problems. Moreover their implementation is adapted to the online setting. The main result of the paper is the upper bound on the convergence rate for the generalization error.
Asymptotics of Gaussian Regularized Least Squares
Authors: Ryan Rifkin Ross Lippert
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We consider regularized least-squares (RLS) with a Gaussian kernel. We prove that if we let the Gaussian bandwidth σ → ∞ while letting the regularization parameter λ → 0, the RLS solution tends to a polynomial whose order is controlled by the rielative rates of decay of 1 σ2 and λ: if λ = σ−(2k+1), then, as σ → ∞, the RLS solution tends to the kth order polynomial with minimal empirical error. We illustrate the result with an example.
Large scale networks fingerprinting and visualization using the k-core decomposition
Authors: Luca Dall'asta, Alain Barrat, Alessandro Vespignani J. I. Alvarez-hamelin
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We use the k-core decomposition to develop algorithms for the analysis of large scale complex networks. This decomposition, based on a re- cursive pruning of the least connected vertices, allows to disentangle the hierarchical structure of networks by progressively focusing on their cen- tral cores. By using this strategy we develop a general visualization algo- rithm that can be used to compare the structural properties of various net- works and highlight their hierarchical structure. The low computational complexity of the algorithm, O(n + e), where n is the size of the net- work, and e is the number of edges, makes it suitable for the visualization of very large sparse networks. We show how the proposed visualization tool allows to ﬁnd speciﬁc structural ﬁngerprints of networks.
Norepinephrine and Neural Interrupts
Authors: Angela J. Yu Peter Dayan
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Angela J. Yu
Center for Brain, Mind & Behavior Green Hall, Princeton University
Princeton, NJ 08540, USA ajyu@princeton.edu
Experimental data indicate that norepinephrine is critically involved in aspects of vigilance and attention. Previously, we considered the func- tion of this neuromodulatory system on a time scale of minutes and longer, and suggested that it signals global uncertainty arising from gross changes in environmental contingencies. However, norepinephrine is also known to be activated phasically by familiar stimuli in well- learned tasks. Here, we extend our uncertainty-based treatment of nore- pinephrine to this phasic mode, proposing that it is involved in the de- tection and reaction to state uncertainty within a task. This role of nore- pinephrine can be understood through the metaphor of neural interrupts.
Consensus Propagation
Authors: Ciamac C. Moallemi Benjamin V. Roy
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We propose consensus propagation, an asynchronous distributed protocol for averaging numbers across a network. We establish convergence, characterize the convergence rate for regular graphs, and demonstrate that the protocol exhibits better scaling properties than pairwise averaging, an alternative that has received much recent attention. Consensus propagation can be viewed as a special case of belief propagation, and our results contribute to the belief propagation literature. In particular, beyond singly-connected graphs, there are very few classes of relevant problems for which belief propagation is known to converge.
CMOL CrossNets: Possible Neuromorphic Nanoelectronic Circuits
Authors: Xiaolong Ma, Konstantin K. Likharev Jung Hoon Lee
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Hybrid  “CMOL”  integrated  circuits,  combining  CMOS  subsystem  with  nanowire  crossbars  and  simple  two-terminal  nanodevices,  promise  to  extend  the  exponential  Moore-Law  development  of  microelectronics  into  the  sub-10-nm  range.  We  are  developing  neuromorphic  network  (“CrossNet”)  architectures  for  this  future  technology, in which neural cell bodies are implemented in CMOS,  nanowires  are  used  as  axons  and  dendrites,  while  nanodevices  (bistable  latching  switches)  are  used  as  elementary  synapses.  We  have  shown  how  CrossNets  may  be  trained  to  perform  pattern  recovery  and  classification  despite  the  limitations  imposed  by  the  CMOL  hardware.    Preliminary  estimates  have  shown  that  CMOL  CrossNets may be extremely dense (~107 cells per cm2) and operate  approximately a million times faster than biological neural networks,  at  manageable  power  consumption.  In  Conclusion,  we  discuss  in  brief possible short-term and long-term applications of the emerging  technology. 
1  Introduction: CMOL Circuits 
Recent  results  [1,  2]  indicate  that  the  current  VLSI  paradigm  based  on  CMOS  technology  can  be  hardly  extended  beyond  the  10-nm  frontier:  in  this  range  the  sensitivity  of  parameters  (most  importantly,  the  gate  voltage  threshold)  of  silicon  field-effect  transistors  to  inevitable  fabrication  spreads  grows  exponentially.  This  sensitivity will probably send the fabrication facilities costs skyrocketing, and may  lead to the end of Moore’s Law some time during the next decade.   There  is  a  growing  consensus  that  the  impending  Moore’s  Law  crisis  may  be  preempted by a radical paradigm shift from the purely CMOS technology to hybrid  CMOS/nanodevice circuits, e.g., those of “CMOL” variety (Fig. 1). Such circuits (see,  e.g., Ref. 3 for their recent review) would combine a level of advanced CMOS devices  fabricated by the lithographic patterning, and two-layer nanowire crossbar formed,  e.g.,  by  nanoimprint,  with  nanowires  connected  by  simple,  similar,  two-terminal  nanodevices at each crosspoint. For such devices, molecular single-electron latching  switches [4] are presently the leading candidates, in particular because they may be  fabricated using the self-assembled monolayer (SAM) technique which already gave  reproducible results for simpler molecular devices [5].
A General and Efficient Multiple Kernel Learning Algorithm
Authors: Gunnar Rätsch, Christin Schäfer Sören Sonnenburg
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: While classical kernel-based learning algorithms are based on a single kernel, in practice it is often desirable to use multiple kernels. Lankriet et al. (2004) considered conic combinations of kernel matrices for classi- ﬁcation, leading to a convex quadratically constraint quadratic program. We show that it can be rewritten as a semi-inﬁnite linear program that can be efﬁciently solved by recycling the standard SVM implementa- tions. Moreover, we generalize the formulation and our method to a larger class of problems, including regression and one-class classiﬁca- tion. Experimental results show that the proposed algorithm helps for automatic model selection, improving the interpretability of the learn- ing result and works for hundred thousands of examples or hundreds of kernels to be combined.
f (x) = sign N Xi=1
αiyik(xi, x) + b! ,
Stimulus Evoked Independent Factor Analysis of MEG Data with Large Background Activity
Authors: Kensuke Sekihara, Hagai T. Attias, Srikantan S. Nagarajan Kenneth Hild
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper presents a novel technique for analyzing electromagnetic imaging data obtained using the stimulus evoked experimental paradigm. The technique is based on a probabilistic graphical model, which describes the data in terms of underlying evoked and interference sources, and explicitly models the stimulus evoked paradigm. A variational Bayesian EM algorithm infers the model from data, suppresses interference sources, and reconstructs the activity of separated individual brain sources. The new algorithm outperforms existing techniques on two real datasets, as well as on simulated data.
Augmented Rescorla-Wagner and Maximum Likelihood Estimation
Authors: Alan L. Yuille
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We show that linear generalizations of Rescorla-Wagner can perform Maximum Likelihood estimation of the parameters of all generative models for causal reasoning. Our approach involves augmenting variables to deal with conjunctions of causes, similar to the agumented model of Rescorla. Our results involve genericity assumptions on the distributions of causes. If these assumptions are violated, for example for the Cheng causal power theory, then we show that a linear Rescorla-Wagner can estimate the parameters of the model up to a nonlinear transformtion. Moreover, a nonlinear Rescorla-Wagner is able to estimate the parameters directly to within arbitrary accuracy. Previous results can be used to determine convergence and to estimate convergence rates.
Laplacian Score for Feature Selection
Authors: Deng Cai, Partha Niyogi Xiaofei He
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: In supervised learning scenarios, feature selection has been studied widely in the literature. Selecting features in unsupervised learning scenarios is a much harder problem, due to the absence of class labels that would guide the search for relevant information. And, almost all of previous unsupervised feature selection methods are "wrapper" techniques that require a learning algorithm to evaluate the candidate feature subsets. In this paper, we propose a "filter" method for feature selection which is independent of any learning algorithm. Our method can be performed in either supervised or unsupervised fashion. The proposed method is based on the observation that, in many real world classification problems, data from the same class are often close to each other. The importance of a feature is evaluated by its power of locality preserving, or, Laplacian Score. We compare our method with data variance (unsupervised) and Fisher score (supervised) on two data sets. Experimental results demonstrate the effectiveness and efficiency of our algorithm.
Variational Bayesian Stochastic Complexity of Mixture Models
Authors: Sumio Watanabe Kazuho Watanabe
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: The Variational Bayesian framework has been widely used to approximate the Bayesian learning. In various applications, it has provided computational tractability and good generalization performance. In this paper, we discuss the Variational Bayesian learning of the mixture of exponential families and provide some additional theoretical support by deriving the asymptotic form of the stochastic complexity. The stochastic complexity, which corresponds to the minimum free energy and a lower bound of the marginal likelihood, is a key quantity for model selection. It also enables us to discuss the effect of hyperparameters and the accuracy of the Variational Bayesian approach as an approximation of the true Bayesian learning.
Hierarchical Linear/Constant Time SLAM Using Particle Filters for Dense Maps
Authors: Ronald Parr Austin I. Eliazar
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present an improvement to the DP-SLAM algorithm for simultane- ous localization and mapping (SLAM) that maintains multiple hypothe- ses about densely populated maps (one full map per particle in a par- ticle ﬁlter) in time that is linear in all signiﬁcant algorithm parameters and takes constant (amortized) time per iteration. This means that the asymptotic complexity of the algorithm is no greater than that of a pure localization algorithm using a single map and the same number of parti- cles. We also present a hierarchical extension of DP-SLAM that uses a two level particle ﬁlter which models drift in the particle ﬁltering process itself. The hierarchical approach enables recovery from the inevitable drift that results from using a ﬁnite number of particles in a particle ﬁlter and permits the use of DP-SLAM in more challenging domains, while maintaining linear time asymptotic complexity.
An exploration-exploitation model based on norepinepherine and dopamine activity
Authors: Mark S. Gilzenrat, Jonathan D. Cohen Samuel M. McClure
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We propose a model by which dopamine (DA) and norepinepherine  (NE) combine to alternate behavior between relatively exploratory  and  exploitative  modes.  The  model  is  developed  for  a  target  detection  task  for  which  there  is  extant  single  neuron  recording  data  available  from  locus  coeruleus  (LC)  NE  neurons.  An  exploration-exploitation trade-off is elicited by regularly switching  which  of  the  two  stimuli  are  rewarded.  DA  functions  within  the  model  to  change  synaptic  weights  according  to  a  reinforcement  learning  algorithm.  Exploration  is  mediated  by  the  state  of  LC  firing,  with  higher  tonic  and  lower  phasic  activity  producing  greater  response  variability.  The  opposite  state  of  LC  function,  with lower baseline firing rate and greater phasic responses, favors  exploitative  behavior.  Changes  in  LC  firing  mode  result  from  combined  measures  of  response  conflict  and  reward  rate,  where  response  conflict  is  monitored  using  models  of  anterior  cingulate  cortex (ACC). Increased long-term response conflict and decreased  reward  rate,  which  occurs  following  reward  contingency  switch,  favors  the  higher  tonic  state  of  LC  function  and  NE  release.  This  increases exploration, and facilitates discovery of the new target.
Kernels for gene regulatory regions
Authors: Robert Thurman, William S. Noble Jean-philippe Vert
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We describe a hierarchy of motif-based kernels for multiple alignments of biological sequences, particularly suitable to process regulatory regions of genes. The kernels incorporate progressively more information, with the most complex kernel accounting for a multiple alignment of orthologous regions, the phylogenetic tree relating the species, and the prior knowledge that relevant sequence patterns occur in conserved motif blocks. These kernels can be used in the presence of a library of known transcription factor binding sites, or de novo by iterating over all k -mers of a given length. In the latter mode, a discriminative classifier built from such a kernel not only recognizes a given class of promoter regions, but as a side effect simultaneously identifies a collection of relevant, discriminative sequence motifs. We demonstrate the utility of the motif-based multiple alignment kernels by using a collection of aligned promoter regions from five yeast species to recognize classes of cell-cycle regulated genes. Supplementary data is available at http://noble.gs.washington.edu/proj/pkernel.
Transfer learning for text classification
Authors: Andrew Y. Ng Chuong B. Do
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Linear text classiﬁcation algorithms work by computing an inner prod- uct between a test document vector and a parameter vector. In many such algorithms, including naive Bayes and most TFIDF variants, the parame- ters are determined by some simple, closed-form, function of training set statistics; we call this mapping mapping from statistics to parameters, the parameter function. Much research in text classiﬁcation over the last few decades has consisted of manual efforts to identify better parameter func- tions. In this paper, we propose an algorithm for automatically learning this function from related classiﬁcation problems. The parameter func- tion found by our algorithm then deﬁnes a new learning algorithm for text classiﬁcation, which we can apply to novel classiﬁcation tasks. We ﬁnd that our learned classiﬁer outperforms existing methods on a variety of multiclass text classiﬁcation tasks.
arg maxk2f1;:::;KgPn
The Forgetron: A Kernel-Based Perceptron on a Fixed Budget
Authors: Shai Shalev-shwartz, Yoram Singer Ofer Dekel
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: The Perceptron algorithm, despite its simplicity, often performs well on online classification tasks. The Perceptron becomes especially effective when it is used in conjunction with kernels. However, a common difficulty encountered when implementing kernel-based online algorithms is the amount of memory required to store the online hypothesis, which may grow unboundedly. In this paper we present and analyze the Forgetron algorithm for kernel-based online learning on a fixed memory budget. To our knowledge, this is the first online learning algorithm which, on one hand, maintains a strict limit on the number of examples it stores while, on the other hand, entertains a relative mistake bound. In addition to the formal results, we also present experiments with real datasets which underscore the merits of our approach.