mtybilly/Instruct2DS huggingface dataset:
Spectro-Temporal Receptive Fields of Subthreshold Responses in Auditory Cortex
Authors: Michael Wehr, Anthony M. Zador Christian K. Machens
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: How do cortical neurons represent the acoustic environment? This ques- tion is often addressed by probing with simple stimuli such as clicks or tone pips. Such stimuli have the advantage of yielding easily interpreted answers, but have the disadvantage that they may fail to uncover complex or higher-order neuronal response properties. Here we adopt an alternative approach, probing neuronal responses with complex acoustic stimuli, including animal vocalizations and music. We have used in vivo whole cell methods in the rat auditory cortex to record subthreshold membrane potential ﬂuctuations elicited by these stimuli. Whole cell recording reveals the total synaptic input to a neuron from all the other neurons in the circuit, instead of just its output—a sparse bi- nary spike train—as in conventional single unit physiological recordings. Whole cell recording thus provides a much richer source of information about the neuron’s response. Many neurons responded robustly and reliably to the complex stimuli in our ensemble. Here we analyze the linear component—the spectro- temporal receptive ﬁeld (STRF)—of the transformation from the sound (as represented by its time-varying spectrogram) to the neuron’s mem- brane potential. We ﬁnd that the STRF has a rich dynamical structure, including excitatory regions positioned in general accord with the predic- tion of the simple tuning curve. We also ﬁnd that in many cases, much of the neuron’s response, although deterministically related to the stimulus, cannot be predicted by the linear component, indicating the presence of as-yet-uncharacterized nonlinear response properties.
Bayesian Models of Inductive Generalization
Authors: Joshua B. Tenenbaum Neville E. Sanjana
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We argue that human inductive generalization is best explained in a Bayesian framework, rather than by traditional models based on simi- larity computations. We go beyond previous work on Bayesian concept learning by introducing an unsupervised method for constructing ﬂex- ible hypothesis spaces, and we propose a version of the Bayesian Oc- cam’s razor that trades off priors and likelihoods to prevent under- or over-generalization in these ﬂexible spaces. We analyze two published data sets on inductive reasoning as well as the results of a new behavioral study that we have carried out.
Dynamical Constraints on Computing with Spike Timing in the Cortex
Authors: Alexandre Pouget Arunava Banerjee
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: If the  cortex uses  spike timing to  compute,  the  timing of the  spikes  must  be  robust to  perturbations.  Based on  a  recent  framework  that  provides  a  simple  criterion  to  determine  whether  a  spike  sequence  produced by a generic network is  sensitive to  initial conditions, and  numerical  simulations  of  a  variety  of  network  architectures,  we  argue  within  the  limits  set  by  our  model  of the  neuron,  that  it  is  unlikely  that  precise  sequences  of  spike  timings  are  used  for  computation under conditions typically found  in the cortex.
Bayesian Estimation of Time-Frequency Coefficients for Audio Signal Enhancement
Authors: Simon J. Godsill Patrick J. Wolfe
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: The Bayesian paradigm provides a natural and effective means of exploit- ing prior knowledge concerning the time-frequency structure of sound signals such as speech and music—something which has often been over- looked in traditional audio signal processing approaches. Here, after con- structing a Bayesian model and prior distributions capable of taking into account the time-frequency characteristics of typical audio waveforms, we apply Markov chain Monte Carlo methods in order to sample from the resultant posterior distribution of interest. We present speech enhance- ment results which compare favourably in objective terms with standard time-varying ﬁltering techniques (and in several cases yield superior per- formance, both objectively and subjectively); moreover, in contrast to such methods, our results are obtained without an assumption of prior knowledge of the noise power.
Real-Time Monitoring of Complex Industrial Processes with Particle Filters
Authors: Nando de Freitas, David Poole Rubén Morales-Menéndez
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: This paper discusses the application of particle ﬁltering algorithms to fault diagnosis in complex industrial processes. We consider two ubiq- uitous processes: an industrial dryer and a level tank. For these appli- cations, we compared three particle ﬁltering variants: standard parti- cle ﬁltering, Rao-Blackwellised particle ﬁltering and a version of Rao- Blackwellised particle ﬁltering that does one-step look-ahead to select good sampling regions. We show that the overhead of the extra process- ing per particle of the more sophisticated methods is more than compen- sated by the decrease in error and variance.
Retinal Processing Emulation in a Programmable 2-Layer Analog Array Processor CMOS Chip
Authors: F. Jiménez-garrido, R. Dominguez-castro, S. Espejo, A. Rodriguez-vázquez R. Carmona
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: A bio-inspired model for an analog programmable array processor (APAP), based on studies on the vertebrate retina, has permitted the realization of complex programmable spatio-temporal dynam- ics in VLSI. This model mimics the way in which images are pro- cessed in the visual pathway, rendering a feasible alternative for the implementation of early vision applications in standard tech- nologies. A prototype chip has been designed and fabricated in a 0.5(cid:22)m standard CMOS process. Computing power per area and power consumption is amongst the highest reported for a single chip. Design challenges, trade-o(cid:11)s and some experimental results are presented in this paper.
Bayesian Image Super-Resolution
Authors: Christopher M. Bishop Michael E. Tipping
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: The  extraction  of a  single  high-quality  image  from  a  set  of  low(cid:173) resolution  images  is  an  important  problem  which  arises  in  fields  such as  remote  sensing,  surveillance,  medical  imaging and the  ex(cid:173) traction of still  images  from  video.  Typical  approaches are  based  on  the  use  of cross-correlation  to  register  the  images  followed  by  the  inversion  of the  transformation  from  the  unknown  high  reso(cid:173) lution image to the observed low  resolution images,  using regular(cid:173) ization to resolve  the  ill-posed  nature of the  inversion  process.  In  this paper we  develop a Bayesian treatment of the super-resolution  problem  in  which  the  likelihood  function  for  the  image  registra(cid:173) tion  parameters  is  based  on  a  marginalization  over  the  unknown  high-resolution  image.  This  approach  allows  us  to  estimate  the  unknown point spread function,  and is  rendered tractable through  the introduction of a  Gaussian  process  prior over images.  Results  indicate a  significant improvement over techniques  based on MAP  (maximum  a-posteriori)  point  optimization  of the  high  resolution  image and associated registration parameters.
A Minimal Intervention Principle for Coordinated Movement
Authors: Michael I. Jordan Emanuel Todorov
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. The resulting behavior exhibits task-constrained variabil- ity, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon.
A Probabilistic Approach to Single Channel Blind Signal Separation
Authors: Te-Won Lee Gil-jin Jang
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We present a new technique for achieving source separation when given only a single channel recording. The main idea is based on exploiting the inherent time structure of sound sources by learning a priori sets of basis ﬁlters in time domain that encode the sources in a statistically efﬁcient manner. We derive a learning algorithm using a maximum likelihood approach given the observed single channel data and sets of basis ﬁlters. For each time point we infer the source signals and their contribution factors. This inference is possible due to the prior knowledge of the basis ﬁlters and the associated coefﬁcient densities. A ﬂexible model for density estimation allows accurate modeling of the observation and our experimental results exhibit a high level of separation performance for mixtures of two music signals as well as the separation of two voice signals.
A Digital Antennal Lobe for Pattern Equalization: Analysis and Design
Authors: Gilles Laurent, Pietro Perona Alex Holub
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Re-mapping  patterns  in  order  to  equalize  their  distribution  may  greatly  simplify  both the  structure  and  the  training of classifiers.  Here,  the  properties  of one  such  map  obtained  by  running  a  few  steps of discrete-time dynamical system are explored.  The system  is  called  'Digital  Antennal  Lobe'  (DAL)  because  it  is  inspired  by  recent studies of the antennallobe, a structure in the olfactory sys(cid:173) tem  of the  grasshopper.  The  pattern-spreading  properties  of the  DAL  as  well  as  its  average behavior  as  a  function  of its  (few)  de(cid:173) sign  parameters are analyzed by extending previous results of Van  Vreeswijk and Sompolinsky.  Furthermore, a technique for  adapting  the  parameters of the  initial  design  in  order  to  obtain  opportune  noise-rejection behavior is suggested.  Our results are demonstrated  with a  number of simulations.
Knowledge-Based Support Vector Machine Classifiers
Authors: Olvi L. Mangasarian, Jude W. Shavlik Glenn M. Fung
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Prior knowledge  in the  form  of multiple  polyhedral sets,  each  be(cid:173) longing to one of two categories, is introduced into a  reformulation  of a  linear support vector machine classifier.  The resulting formu(cid:173) lation leads to a linear program that can be solved efficiently.  Real  world examples, from DNA sequencing and breast cancer prognosis,  demonstrate the effectiveness  of the proposed method.  Numerical  results  show  improvement  in  test  set  accuracy  after  the  incorpo(cid:173) ration of prior knowledge into ordinary, data-based linear support  vector  machine  classifiers.  One  experiment  also  shows  that  a  lin(cid:173) ear classifier,  based solely on prior knowledge,  far  outperforms the  direct  application of prior knowledge  rules  to classify data.  Keywords:  use  and  refinement  of prior  knowledge,  sup(cid:173) port  vector machines,  linear programming
Conditional Models on the Ranking Poset
Authors: John D. Lafferty Guy Lebanon
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: A distance-based conditional model on the ranking poset is presented for use in classiﬁcation and ranking. The model is an extension of the Mallows  model, and generalizes the classiﬁer combination methods used by several ensemble learning algorithms, including error correcting output codes, discrete AdaBoost, logistic regression and cranking. The algebraic structure of the ranking poset leads to a simple Bayesian inter- pretation of the conditional model and its special cases. In addition to a unifying view, the framework suggests a probabilistic interpretation for error correcting output codes and an extension beyond the binary coding scheme.
A Formulation for Minimax Probability Machine Regression
Authors: Gregory Z. Grudic Thomas Strohmann
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We formulate the regression problem as one of maximizing the mini- mum probability, symbolized by (cid:10), that future predicted outputs of the regression model will be within some (cid:6)" bound of the true regression function. Our formulation is unique in that we obtain a direct estimate of this lower probability bound (cid:10). The proposed framework, minimax probability machine regression (MPMR), is based on the recently de- scribed minimax probability machine classiﬁcation algorithm [Lanckriet et al.] and uses Mercer Kernels to obtain nonlinear regression models. MPMR is tested on both toy and real world data, verifying the accuracy of the (cid:10) bound, and the efﬁcacy of the regression models.
Multiclass Learning by Probabilistic Embeddings
Authors: Yoram Singer Ofer Dekel
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We describe a new algorithmic framework for learning multiclass catego- rization problems. In this framework a multiclass predictor is composed of a pair of embeddings that map both instances and labels into a common space. In this space each instance is assigned the label it is nearest to. We outline and analyze an algorithm, termed Bunching, for learning the pair of embeddings from labeled data. A key construction in the analysis of the algorithm is the notion of probabilistic output codes, a generaliza- tion of error correcting output codes (ECOC). Furthermore, the method of multiclass categorization using ECOC is shown to be an instance of Bunching. We demonstrate the advantage of Bunching over ECOC by comparing their performance on numerous categorization problems.
Improving a Page Classifier with Anchor Extraction and Link Analysis
Authors: William W. Cohen
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Most text categorization systems use simple models of documents and document collections. In this paper we describe a technique that im- proves a simple web page classiﬁer’s performance on pages from a new, unseen web site, by exploiting link structure within a site as well as page structure within hub pages. On real-world test cases, this technique signiﬁcantly and substantially improves the accuracy of a bag-of-words classiﬁer, reducing error rate by about half, on average. The system uses a variant of co-training to exploit unlabeled data from a new site. Pages are labeled using the base classiﬁer; the results are used by a restricted wrapper-learner to propose potential “main-category anchor wrappers”; and ﬁnally, these wrappers are used as features by a third learner to ﬁnd a categorization of the site that implies a simple hub structure, but which also largely agrees with the original bag-of-words classiﬁer.
VIBES: A Variational Inference Engine for Bayesian Networks
Authors: David Spiegelhalter, John Winn Christopher M. Bishop
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: In recent years variational methods have become a popular tool for approximate inference and learning in a wide variety of proba- bilistic models. For each new application, however, it is currently necessary (cid:12)rst to derive the variational update equations, and then to implement them in application-speci(cid:12)c code. Each of these steps is both time consuming and error prone. In this paper we describe a general purpose inference engine called VIBES (‘Variational Infer- ence for Bayesian Networks’) which allows a wide variety of proba- bilistic models to be implemented and solved variationally without recourse to coding. New models are speci(cid:12)ed either through a simple script or via a graphical interface analogous to a drawing package. VIBES then automatically generates and solves the vari- ational equations. We illustrate the power and (cid:13)exibility of VIBES using examples from Bayesian mixture modelling.
A Convergent Form of Approximate Policy Iteration
Authors: Doina Precup Theodore J. Perkins
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We study a new, model-free form of approximate policy iteration which uses Sarsa updates with linear state-action value function approximation for policy evaluation, and a “policy improvement operator” to generate a new policy based on the learned state-action values. We prove that if the policy improvement operator produces  -soft policies and is Lipschitz continuous in the action values, with a constant that is not too large, then the approximate policy iteration algorithm converges to a unique solu- tion from any initial policy. To our knowledge, this is the ﬁrst conver- gence result for any form of approximate policy iteration under similar computational-resource assumptions.
Mean Field Approach to a Probabilistic Model in Information Retrieval
Authors: K. Wong, David Bodoff Bin Wu
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We study an explicit parametric model of documents, queries, and rel- evancy assessment for Information Retrieval (IR). Mean-ﬁeld methods are applied to analyze the model and derive efﬁcient practical algorithms to estimate the parameters in the problem. The hyperparameters are es- timated by a fast approximate leave-one-out cross-validation procedure based on the cavity method. The algorithm is further evaluated on several benchmark databases by comparing with standard algorithms in IR.
Exponential Family PCA for Belief Compression in POMDPs
Authors: Geoffrey J. Gordon Nicholas Roy
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Geoffrey Gordon
Department of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213 ggordon@cs.cmu.edu
Standard value function approaches to ﬁnding policies for Partially Observable Markov Decision Processes (POMDPs) are intractable for large models. The in- tractability of these algorithms is due to a great extent to their generating an optimal policy over the entire belief space. However, in real POMDP problems most belief states are unlikely, and there is a structured, low-dimensional manifold of plausible beliefs embedded in the high-dimensional belief space. We introduce a new method for solving large-scale POMDPs by taking advantage of belief space sparsity. We reduce the dimensionality of the belief space by exponential family Principal Components Analysis [1], which allows us to turn the sparse, high- dimensional belief space into a compact, low-dimensional representation in terms of learned features of the belief state. We then plan directly on the low-dimensional belief features. By planning in a low-dimensional space, we can ﬁnd policies for POMDPs that are orders of magnitude larger than can be handled by conventional techniques. We demonstrate the use of this algorithm on a synthetic problem and also on a mobile robot navigation task.
Dynamic Structure Super-Resolution
Authors: Amos J. Storkey
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: The problem of super-resolution involves generating feasible higher resolution images, which are pleasing to the eye and realistic, from a given low resolution image. This might be attempted by us- ing simple (cid:12)lters for smoothing out the high resolution blocks or through applications where substantial prior information is used to imply the textures and shapes which will occur in the images. In this paper we describe an approach which lies between the two extremes. It is a generic unsupervised method which is usable in all domains, but goes beyond simple smoothing methods in what it achieves. We use a dynamic tree-like architecture to model the high resolution data. Approximate conditioning on the low resolution image is achieved through a mean (cid:12)eld approach.
Independent Components Analysis through Product Density Estimation
Authors: Rob Tibshirani Trevor Hastie
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We  present a  simple direct approach for  solving the ICA problem,  using density estimation and maximum likelihood.  Given a  candi(cid:173) date  orthogonal frame,  we  model  each  of the  coordinates  using  a  semi-parametric density estimate based on cubic splines.  Since our  estimates have two continuous derivatives, we  can easily run a sec(cid:173) ond order search for  the frame  parameters.  Our method performs  very favorably  when compared to state-of-the-art techniques.
Discriminative Densities from Maximum Contrast Estimation
Authors: Thorsten Twellmann, Helge Ritter Peter Meinicke
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We propose a framework for classiﬁer design based on discriminative densities for representation of the differences of the class-conditional dis- tributions in a way that is optimal for classiﬁcation. The densities are selected from a parametrized set by constrained maximization of some objective function which measures the average (bounded) difference, i.e. the contrast between discriminative densities. We show that maximiza- tion of the contrast is equivalent to minimization of an approximation of the Bayes risk. Therefore using suitable classes of probability den- sity functions, the resulting maximum contrast classiﬁers (MCCs) can approximate the Bayes rule for the general multiclass case. In particular for a certain parametrization of the density functions we obtain MCCs which have the same functional form as the well-known Support Vec- tor Machines (SVMs). We show that MCC-training in general requires some nonlinear optimization but under certain conditions the problem is concave and can be tackled by a single linear program. We indicate the close relation between SVM- and MCC-training and in particular we show that Linear Programming Machines can be viewed as an approxi- mate realization of MCCs. In the experiments on benchmark data sets, the MCC shows a competitive classiﬁcation performance.
Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior
Authors: Aapo Hyvärinen Patrik O. Hoyer
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: The responses of cortical sensory neurons are notoriously variable, with the number of spikes evoked by identical stimuli varying signiﬁcantly from trial to trial. This variability is most often interpreted as ‘noise’, purely detrimental to the sensory system. In this paper, we propose an al- ternative view in which the variability is related to the uncertainty, about world parameters, which is inherent in the sensory stimulus. Speciﬁ- cally, the responses of a population of neurons are interpreted as stochas- tic samples from the posterior distribution in a latent variable model. In addition to giving theoretical arguments supporting such a representa- tional scheme, we provide simulations suggesting how some aspects of response variability might be understood in this framework.
Combining Features for BCI
Authors: Benjamin Blankertz, Gabriel Curio, Klaus-Robert Müller Guido Dornhege
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Recently, interest is growing to develop an effective communication in- terface connecting the human brain to a computer, the ’Brain-Computer Interface’ (BCI). One motivation of BCI research is to provide a new communication channel substituting normal motor output in patients with severe neuromuscular disabilities. In the last decade, various neuro- physiological cortical processes, such as slow potential shifts, movement related potentials (MRPs) or event-related desynchronization (ERD) of spontaneous EEG rhythms, were shown to be suitable for BCI, and, con- sequently, different independent approaches of extracting BCI-relevant EEG-features for single-trial analysis are under investigation. Here, we present and systematically compare several concepts for combining such EEG-features to improve the single-trial classiﬁcation. Feature combi- nations are evaluated on movement imagination experiments with 3 sub- jects where EEG-features are based on either MRPs or ERD, or both. Those combination methods that incorporate the assumption that the sin- gle EEG-features are physiologically mutually independent outperform the plain method of ’adding’ evidence where the single-feature vectors are simply concatenated. These results strengthen the hypothesis that MRP and ERD reﬂect at least partially independent aspects of cortical processes and open a new perspective to boost BCI effectiveness.
A Probabilistic Model for Learning Concatenative Morphology
Authors: Michael R. Brent Matthew G. Snover
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: This paper describes a system for the unsupervised learning of morpho- logical sufﬁxes and stems from word lists. The system is composed of a generative probability model and hill-climbing and directed search algo- rithms. By extracting and examining morphologically rich subsets of an input lexicon, the directed search identiﬁes highly productive paradigms. The hill-climbing algorithm then further maximizes the probability of the hypothesis. Quantitative results are shown by measuring the accuracy of the morphological relations identiﬁed. Experiments in English and Pol- ish, as well as comparisons with another recent unsupervised morphol- ogy learning algorithm demonstrate the effectiveness of this technique.
The Decision List Machine
Authors: Mario Marchand, Nathalie Japkowicz, John S. Shawe-taylor Marina Sokolova
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We introduce a new learning algorithm for decision lists to allow features that are constructed from the data and to allow a trade- oﬁ between accuracy and complexity. We bound its generalization error in terms of the number of errors and the size of the classiﬂer it ﬂnds on the training data. We also compare its performance on some natural data sets with the set covering machine and the support vector machine.
How to Combine Color and Shape Information for 3D Object Recognition: Kernels do the Trick
Authors: Gy. Dorkó B. Caputo
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: This paper presents a  kernel method that allows to combine  color  and shape information for  appearance-based object recognition.  It  doesn't require to define a new common representation, but use the  power of kernels to combine different representations together in an  effective manner.  These results are achieved using results of statis(cid:173) tical mechanics of spin glasses combined with Markov random fields  via kernel functions.  Experiments show  an increase in recognition  rate up  to 5.92%  with respect to conventional strategies.
Reconstructing Stimulus-Driven Neural Networks from Spike Times
Authors: Duane Q. Nykamp
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We present a method to distinguish direct connections between two neu- rons from common input originating from other, unmeasured neurons. The distinction is computed from the spike times of the two neurons in response to a white noise stimulus. Although the method is based on a highly idealized linear-nonlinear approximation of neural response, we demonstrate via simulation that the approach can work with a more re- alistic, integrate-and-ﬁre neuron model. We propose that the approach exempliﬁed by this analysis may yield viable tools for reconstructing stimulus-driven neural networks from data gathered in neurophysiology experiments.
Selectivity and Metaplasticity in a Unified Calcium-Dependent Model
Authors: Brian S. Blais, Leon N. Cooper, Harel Z. Shouval Luk Chong Yeung
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: A uni(cid:12)ed, biophysically motivated Calcium-Dependent Learning model has been shown to account for various rate-based and spike time-dependent paradigms for inducing synaptic plasticity. Here, we investigate the properties of this model for a multi-synapse neuron that receives inputs with di(cid:11)erent spike-train statistics. In addition, we present a physiological form of metaplasticity, an activity-driven regulation mechanism, that is essential for the ro- bustness of the model. A neuron thus implemented develops stable and selective receptive (cid:12)elds, given various input statistics
Learning to Detect Natural Image Boundaries Using Brightness and Texture
Authors: Charless C. Fowlkes, Jitendra Malik David R. Martin
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: The goal of this work is to accurately detect and localize boundaries in natural scenes using local image measurements. We formulate features that respond to characteristic changes in brightness and texture associated with natural boundaries. In order to combine the information from these features in an optimal way, a classiﬁer is trained using human labeled images as ground truth. We present precision-recall curves showing that the resulting detector outperforms existing approaches.
Concurrent Object Recognition and Segmentation by Graph Partitioning
Authors: Ralph Gross, Jianbo Shi Stella X. Yu
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Segmentation and recognition have long been treated as two separate pro(cid:173) cesses.  We  propose a  mechanism  based  on  spectral  graph  partitioning  that readily combine the  two processes into one.  A part-based recogni(cid:173) tion system detects object patches, supplies their partial segmentations as  well as knowledge about the spatial configurations of the object.  The goal  of patch grouping is to find a set of patches that conform best to the object  configuration,  while the goal  of pixel grouping is  to  find  a set of pixels  that have  the  best low-level  feature  similarity.  Through pixel-patch  in(cid:173) teractions and between-patch competition encoded in the solution space,  these two processes are realized in  one joint optimization problem.  The  globally optimal partition is obtained by solving a constrained eigenvalue  problem.  We  demonstrate that the  resulting  object segmentation elimi(cid:173) nates false  positives for  the part detection, while  overcoming occlusion  and weak contours for the low-level edge detection.
Approximate Linear Programming for Average-Cost Dynamic Programming
Authors: Daniela D. Farias Benjamin V. Roy
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: This paper extends our earlier analysis on approximate linear program- ming as an approach to approximating the cost-to-go function in a discounted-cost dynamic program [6]. In this paper, we consider the average-cost criterion and a version of approximate linear programming that generates approximations to the optimal average cost and differential cost function. We demonstrate that a naive version of approximate linear programming prioritizes approximation of the optimal average cost and that this may not be well-aligned with the objective of deriving a policy with low average cost. For that, the algorithm should aim at producing a good approximation of the differential cost function. We propose a two- phase variant of approximate linear programming that allows for external control of the relative accuracy of the approximation of the differential cost function over different portions of the state space via state-relevance weights. Performance bounds suggest that the new algorithm is compat- ible with the objective of optimizing performance and provide guidance on appropriate choices for state-relevance weights.
Learning Sparse Topographic Representations with Products of Student-t Distributions
Authors: Simon Osindero, Geoffrey E. Hinton Max Welling
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We propose a model for natural images in which the probability of an im- age is proportional to the product of the probabilities of some ﬁlter out- puts. We encourage the system to ﬁnd sparse features by using a Student- t distribution to model each ﬁlter output. If the t-distribution is used to model the combined outputs of sets of neurally adjacent ﬁlters, the sys- tem learns a topographic map in which the orientation, spatial frequency and location of the ﬁlters change smoothly across the map. Even though maximum likelihood learning is intractable in our model, the product form allows a relatively efﬁcient learning procedure that works well even for highly overcomplete sets of ﬁlters. Once the model has been learned it can be used as a prior to derive the “iterated Wiener ﬁlter” for the pur- pose of denoising images.
Margin Analysis of the LVQ Algorithm
Authors: Ran Gilad-bachrach, Amir Navot, Naftali Tishby Koby Crammer
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Prototypes based algorithms are commonly used to reduce the computa- tional complexity of Nearest-Neighbour (NN) classiﬁers. In this paper we discuss theoretical and algorithmical aspects of such algorithms. On the theory side, we present margin based generalization bounds that sug- gest that these kinds of classiﬁers can be more accurate then the 1-NN rule. Furthermore, we derived a training algorithm that selects a good set of prototypes using large margin principles. We also show that the 20 years old Learning Vector Quantization (LVQ) algorithm emerges natu- rally from our framework.
Feature Selection by Maximum Marginal Diversity
Authors: Nuno Vasconcelos
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We address the question of feature selection in the context of visual recognition. It is shown that, besides efﬁcient from a computational standpoint, the infomax principle is nearly optimal in the minimum Bayes error sense. The concept of marginal diversity is introduced, lead- ing to a generic principle for feature selection (the principle of maximum marginal diversity) of extreme computational simplicity. The relation- ships between infomax and the maximization of marginal diversity are identiﬁed, uncovering the existence of a family of classiﬁcation proce- dures for which near optimal (in the Bayes error sense) feature selection does not require combinatorial search. Examination of this family in light of recent studies on the statistics of natural images suggests that visual recognition problems are a subset of it.
A Neural Edge-Detection Model for Enhanced Auditory Sensitivity in Modulated Noise
Authors: Bradford J. May Alon Fishbach
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Psychophysical data suggest that temporal modulations of stimulus  amplitude envelopes play a prominent role in the perceptual  segregation of concurrent sounds. In particular, the detection of an  unmodulated signal can be significantly improved by adding  amplitude modulation to the spectral envelope of a competing  masking noise. This perceptual phenomenon  is known as  “Comodulation Masking Release” (CMR). Despite the obvious  influence of temporal structure on the perception of complex  auditory scenes, the physiological mechanisms that contribute to  CMR and auditory streaming are not well known. A recent  physiological study by Nelken and colleagues has demonstrated an  enhanced cortical representation of auditory signals in modulated  noise. Our study evaluates these CMR-like response patterns from  the perspective of a hypothetical auditory edge-detection neuron. It  is shown that this simple neural model for the detection of  amplitude transients can reproduce not only the physiological data  of Nelken et al., but also, in light of previous results, a variety of  physiological and psychoacoustical phenomena that are related to  the perceptual segregation of concurrent sounds.
Concentration Inequalities for the Missing Mass and for Histogram Rule Error
Authors: David A. McAllester Luis E. Ortiz
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: This paper gives distribution-free concentration inequalities for the miss- ing mass and the error rate of histogram rules. Negative association meth- ods can be used to reduce these concentration problems to concentration questions about independent sums. Although the sums are independent, they are highly heterogeneous. Such highly heterogeneous independent sums cannot be analyzed using standard concentration inequalities such as Hoeffding’s inequality, the Angluin-Valiant bound, Bernstein’s in- equality, Bennett’s inequality, or McDiarmid’s theorem.
Learning Sparse Multiscale Image Representations
Authors: Bruno A. Olshausen Phil Sallee
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We describe a method for learning sparse multiscale image repre- sentations using a sparse prior distribution over the basis function coe(cid:14)cients. The prior consists of a mixture of a Gaussian and a Dirac delta function, and thus encourages coe(cid:14)cients to have exact zero values. Coe(cid:14)cients for an image are computed by sampling from the resulting posterior distribution with a Gibbs sampler. The learned basis is similar to the Steerable Pyramid basis, and yields slightly higher SNR for the same number of active coe(cid:14)cients. De- noising using the learned image model is demonstrated for some standard test images, with results that compare favorably with other denoising methods.
The Effect of Singularities in a Learning Machine when the True Parameters Do Not Lie on such Singularities
Authors: Shun-ichi Amari Sumio Watanabe
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: A lot of learning machines with hidden variables used in infor- mation science have singularities in their parameter spaces. At singularities, the Fisher information matrix becomes degenerate, resulting that the learning theory of regular statistical models does not hold. Recently, it was proven that, if the true parameter is contained in singularities, then the coe(cid:14)cient of the Bayes gen- eralization error is equal to the pole of the zeta function of the Kullback information. In this paper, under the condition that the true parameter is almost but not contained in singularities, we show two results. (1) If the dimension of the parameter from in- puts to hidden units is not larger than three, then there exits a region of true parameters where the generalization error is larger than those of regular models, however, if otherwise, then for any true parameter, the generalization error is smaller than those of regular models. (2) The symmetry of the generalization error and the training error does not hold in singular models in general.
Learning About Multiple Objects in Images: Factorial Learning without Factorial Search
Authors: Michalis K. Titsias Christopher K. I. Williams
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We consider data which are images containing views of multiple objects. Our task is to learn about each of the objects present in the images. This task can be approached as a factorial learning problem, where each image must be explained by instantiating a model for each of the objects present with the correct instantiation parameters. A major problem with learning a factorial model is that as the number of objects increases, there is a combinatorial explosion of the number of conﬁgurations that need to be considered. We develop a method to extract object models sequentially from the data by making use of a robust statistical method, thus avoid- ing the combinatorial explosion, and present results showing successful extraction of objects from real images.
Distance Metric Learning with Application to Clustering with Side-Information
Authors: Michael I. Jordan, Stuart Russell, Andrew Y. Ng Eric P. Xing
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: @cs.berkeley.edu
Many algorithms rely critically on being given a good metric over their inputs. For instance, data can often be clustered in many “plausible” ways, and if a clustering algorithm such as K-means initially fails to ﬁnd one that is meaningful to a user, the only recourse may be for the user to manually tweak the metric until sufﬁciently good clusters are found. For these and other applications requiring good metrics, it is desirable that we provide a more systematic way for users to indicate what they con- sider “similar.” For instance, we may ask them to provide examples. In this paper, we present an algorithm that, given examples of similar (and, , learns a distance metric over if desired, dissimilar) pairs of points in that respects these relationships. Our method is based on posing met-  ric learning as a convex optimization problem, which allows us to give efﬁcient, local-optima-free algorithms. We also demonstrate empirically that the learned metrics can be used to signiﬁcantly improve clustering performance.

Location Estimation with a Differential Update Network
Authors: Trevor Darrell Ali Rahimi
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: Given a set of hidden variables with an a-priori Markov structure, we derive an online algorithm which approximately updates the posterior as pairwise measurements between the hidden variables become available. The update is performed using Assumed Density Filtering: to incorporate each pairwise measurement, we compute the optimal Markov structure which represents the true posterior and use it as a prior for incorporating the next measurement. We demonstrate the resulting algorithm by cal- culating globally consistent trajectories of a robot as it navigates along a 2D trajectory. To update a trajectory of length t, the update takes O(t). When all conditional distributions are linear-Gaussian, the algorithm can be thought of as a Kalman Filter which simpliﬁes the state covariance matrix after incorporating each measurement.
Hyperkernels
Authors: Robert C. Williamson, Alex J. Smola Cheng S. Ong
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We consider the problem of choosing a kernel suitable for estimation using a Gaussian Process estimator or a Support Vector Machine. A novel solution is presented which involves deﬁning a Reproducing Ker- nel Hilbert Space on the space of kernels itself. By utilizing an analog of the classical representer theorem, the problem of choosing a kernel from a parameterized family of kernels (e.g. of varying width) is reduced to a statistical estimation problem akin to the problem of minimizing a regularized risk functional. Various classical settings for model or kernel selection are special cases of our framework.
Prediction and Semantic Association
Authors: Mark Steyvers Thomas L. Griffiths
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We  explore  the  consequences  of  viewing  semantic  association  as  the result of attempting to predict the concepts likely to arise in a  particular context.  We  argue that the success of existing accounts  of semantic representation comes as a result of indirectly addressing  this problem, and show that a closer correspondence to human data  can be obtained by taking a  probabilistic approach that explicitly  models the generative structure of language.
A Maximum Entropy Approach to Collaborative Filtering in Dynamic, Sparse, High-Dimensional Domains
Authors: David M. Pennock Dmitry Y. Pavlov
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: We develop a maximum entropy (maxent) approach to generating recom- mendations in the context of a user’s current navigation stream, suitable for environments where data is sparse, high-dimensional, and dynamic— conditions typical of many recommendation applications. We address sparsity and dimensionality reduction by ﬁrst clustering items based on user access patterns so as to attempt to minimize the apriori probabil- ity that recommendations will cross cluster boundaries and then recom- mending only within clusters. We address the inherent dynamic nature of the problem by explicitly modeling the data as a time series; we show how this representational expressivity ﬁts naturally into a maxent frame- work. We conduct experiments on data from ResearchIndex, a popu- lar online repository of over 470,000 computer science documents. We show that our maxent formulation outperforms several competing algo- rithms in ofﬂine tests simulating the recommendation of documents to ResearchIndex users.
Annealing and the Rate Distortion Problem
Authors: Tomá\v S. Gedeon, Alexander G. Dimitrov Albert E. Parker
Published in: Advances in Neural Information Processing Systems 15  (NIPS 2002) (Main Conference Track), 2002

Abstract: In this paper we introduce methodology to determine the bifurcation structure of optima for a class of similar cost functions from Rate Distortion Theory, Determin- istic Annealing, Information Distortion and the Information Bottleneck Method. We also introduce a numerical algorithm which uses the explicit form of the bifur- cating branches to ﬁnd optima at a bifurcation point.