mtybilly/Instruct2DS huggingface dataset:
Support Vector Classification with Input Data Uncertainty
Authors: Jinbo Bi, Tong Zhang
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: This paper investigates a new learning model in which the input data is corrupted with noise. We present a general statistical framework to tackle this problem. Based on the statistical reasoning, we propose a novel formulation of support vector classiﬁcation, which allows uncer- tainty in input data. We derive an intuitive geometric interpretation of the proposed formulation, and develop algorithms to efﬁciently solve it. Empirical results are included to show that the newly formed method is superior to the standard SVM for problems with noisy input.
Expectation Consistent Free Energies for Approximate Inference
Authors: Manfred Opper, Ole Winther
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a novel a framework for deriving approximations for in-          tractable probabilistic models. This framework is based on a free energy          (negative log marginal likelihood) and can be seen as a generalization          of adaptive TAP [1, 2, 3] and expectation propagation (EP) [4, 5]. The          free energy is constructed from two approximating distributions which          encode different aspects of the intractable model such a single node con-          straints and couplings and are by construction consistent on a chosen set          of moments. We test the framework on a difficult benchmark problem          with binary variables on fully connected graphs and 2D grid graphs. We          find good performance using sets of moments which either specify fac-          torized nodes or a spanning tree on the nodes (structured approximation).          Surprisingly, the Bethe approximation gives very inferior results even on          grids.
Using Machine Learning to Break Visual Human Interaction Proofs (HIPs)
Authors: Kumar Chellapilla, Patrice Y. Simard
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Machine learning is often used to automatically solve human tasks. 
          In this paper, we look for tasks where machine learning algorithms 
          are not as good as humans with the hope of gaining insight into 
          their current limitations. We studied various Human Interactive 
          Proofs (HIPs) on the market, because they are systems designed to 
          tell computers and humans apart by posing challenges presumably 
          too hard for computers. We found that most HIPs are pure 
          recognition tasks which can easily be broken using machine 
          learning. The harder HIPs use a combination of segmentation and 
          recognition tasks. From this observation, we found that building 
          segmentation tasks is the most effective way to confuse machine 
          learning algorithms. This has enabled us to build effective HIPs 
          (which we deployed in MSN Passport), as well as design 
          challenging segmentation tasks for machine learning algorithms.
Making Latin Manuscripts Searchable using gHMM's
Authors: Jaety Edwards, Yee W. Teh, Roger Bock, Michael Maire, Grace Vesom, David A. Forsyth
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We describe a method that can make a scanned, handwritten mediaeval          latin manuscript accessible to full text search. A generalized HMM is          fitted, using transcribed latin to obtain a transition model and one exam-          ple each of 22 letters to obtain an emission model. We show results for          unigram, bigram and trigram models. Our method transcribes 25 pages          of a manuscript of Terence with fair accuracy (75% of letters correctly          transcribed). Search results are very strong; we use examples of vari-          ant spellings to demonstrate that the search respects the ink of the doc-          ument. Furthermore, our model produces fair searches on a document          from which we obtained no training data.
Nonparametric Transforms of Graph Kernels for Semi-Supervised Learning
Authors: Xiaojin Zhu, Jaz Kandola, Zoubin Ghahramani, John D. Lafferty
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present an algorithm based on convex optimization for constructing kernels for semi-supervised learning. The kernel matrices are derived from the spectral decomposition of graph Laplacians, and combine la- beled and unlabeled data in a systematic fashion. Unlike previous work using diffusion kernels and Gaussian random ﬁeld kernels, a nonpara- metric kernel approach is presented that incorporates order constraints during optimization. This results in ﬂexible kernels and avoids the need to choose among different parametric forms. Our approach relies on a quadratically constrained quadratic program (QCQP), and is compu- tationally feasible for large datasets. We evaluate the kernels on real datasets using support vector machines, with encouraging results.
Sparse Coding of Natural Images Using an Overcomplete Set of Limited Capacity Units
Authors: Eizaburo Doi, Michael S. Lewicki
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: It has been suggested that the primary goal of the sensory system is to            represent input in such a way as to reduce the high degree of redun-            dancy. Given a noisy neural representation, however, solely reducing            redundancy is not desirable, since redundancy is the only clue to reduce            the effects of noise. Here we propose a model that best balances redun-            dancy reduction and redundant representation. Like previous models, our            model accounts for the localized and oriented structure of simple cells,            but it also predicts a different organization for the population. With noisy,            limited-capacity units, the optimal representation becomes an overcom-            plete, multi-scale representation, which, compared to previous models,            is in closer agreement with physiological data. These results offer a new            perspective on the expansion of the number of neurons from retina to V1            and provide a theoretical model of incorporating useful redundancy into            efficient neural representations.
1     Introduction
Efficient coding theory posits that one of the primary goals of sensory coding is to eliminate redundancy from raw sensory signals, ideally representing the input by a set of statistically independent features [1]. Models for learning efficient codes, such as sparse coding [2] or ICA [3], predict the localized, oriented, and band-pass characteristics of simple cells. In this framework, units are assumed to be non-redundant and so the number of units should be identical to the dimensionality of the data.
Redundancy, however, can be beneficial if it is used to compensate for inherent noise in the system [4]. The models above assume that the system noise is low and negligible so that redundancy in the representation is not necessary. This is equivalent to assuming that the representational capacity of individual units is unlimited. Real neurons, however, have limited capacity [5], and this should place constraints on how a neural population can best encode a sensory signal. In fact, there are important characteristics of simple cells, such as the multi-scale representation, that cannot be explained by efficient coding theory.
The aim of this study is to evaluate how the optimal representation changes when the system
is constrained by limited capacity units. We propose a model that best balances redundancy reduction and redundant representation given the limited capacity units. In contrast to the efficient coding models, it is possible to have a larger number of units than the intrinsic dimensionality of the data. This further allows to introduce redundancy in the population, enabling precise reconstruction using the imprecise representation of a single unit.
Density Level Detection is Classification
Authors: Ingo Steinwart, Don Hush, Clint Scovel
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We show that anomaly detection can be interpreted as a binary classifi-          cation problem. Using this interpretation we propose a support vector          machine (SVM) for anomaly detection. We then present some theoret-          ical results which include consistency and learning rates. Finally, we          experimentally compare our SVM with the standard one-class SVM.
Learning Syntactic Patterns for Automatic Hypernym Discovery
Authors: Rion Snow, Daniel Jurafsky, Andrew Y. Ng
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Semantic taxonomies such as WordNet provide a rich source of knowl-
         edge for natural language processing applications, but are expensive to
         build, maintain, and extend. Motivated by the problem of automatically
         constructing and extending such taxonomies, in this paper we present a
         new algorithm for automatically learning hypernym (is-a) relations from
         text. Our method generalizes earlier work that had relied on using small
         numbers of hand-crafted regular expression patterns to identify hyper-
         nym pairs. Using "dependency path" features extracted from parse trees,
         we introduce a general-purpose formalization and generalization of these
         patterns. Given a training set of text containing known hypernym pairs,
         our algorithm automatically extracts useful dependency paths and applies
         them to new corpora to identify novel pairs. On our evaluation task (de-
         termining whether two nouns in a news article participate in a hypernym
         relationship), our automatically extracted database of hypernyms attains
         both higher precision and higher recall than WordNet.
Joint Tracking of Pose, Expression, and Texture using Conditionally Gaussian Filters
Authors: Tim K. Marks, J. C. Roddey, Javier R. Movellan, John R. Hershey
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present a generative model and stochastic filtering algorithm for si-           multaneous tracking of 3D position and orientation, non-rigid motion,           object texture, and background texture using a single camera. We show           that the solution to this problem is formally equivalent to stochastic fil-           tering of conditionally Gaussian processes, a problem for which well           known approaches exist [3, 8]. We propose an approach based on Monte           Carlo sampling of the nonlinear component of the process (object mo-           tion) and exact filtering of the object and background textures given the           sampled motion. The smoothness of image sequences in time and space           is exploited by using Laplace's method to generate proposal distributions           for importance sampling [7]. The resulting inference algorithm encom-           passes both optic flow and template-based tracking as special cases, and           elucidates the conditions under which these methods are optimal. We           demonstrate an application of the system to 3D non-rigid face tracking.
Hierarchical Clustering of a Mixture Model
Authors: Jacob Goldberger, Sam T. Roweis
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper we propose an efficient algorithm for reducing a large         mixture of Gaussians into a smaller mixture while still preserv-         ing the component structure of the original model; this is achieved         by clustering (grouping) the components. The method minimizes         a new, easily computed distance measure between two Gaussian         mixtures that can be motivated from a suitable stochastic model         and the iterations of the algorithm use only the model parameters,         avoiding the need for explicit resampling of datapoints. We demon-         strate the method by performing hierarchical clustering of scenery         images and handwritten digits.
Semi-parametric Exponential Family PCA
Authors: Sajama Sajama, Alon Orlitsky
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present a semi-parametric latent variable model based technique for            density modelling, dimensionality reduction and visualization. Unlike            previous methods, we estimate the latent distribution non-parametrically            which enables us to model data generated by an underlying low dimen-            sional, multimodal distribution. In addition, we allow the components            of latent variable models to be drawn from the exponential family which            makes the method suitable for special data types, for example binary or            count data. Simulations on real valued, binary and count data show fa-            vorable comparison to other related schemes both in terms of separating            different populations and generalization to unseen samples.
Self-Tuning Spectral Clustering
Authors: Lihi Zelnik-manor, Pietro Perona
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We study a number of open issues in spectral clustering: (i) Selecting the appropriate scale of analysis, (ii) Handling multi-scale data, (iii) Cluster- ing with irregular background clutter, and, (iv) Finding automatically the number of groups. We ﬁrst propose that a ‘local’ scale should be used to compute the afﬁnity between each pair of points. This local scaling leads to better clustering especially when the data includes multiple scales and when the clusters are placed within a cluttered background. We further suggest exploiting the structure of the eigenvectors to infer automatically the number of groups. This leads to a new algorithm in which the ﬁnal randomly initialized k-means stage is eliminated.
The Entire Regularization Path for the Support Vector Machine
Authors: Saharon Rosset, Robert Tibshirani, Ji Zhu, Trevor J. Hastie
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper we argue that the choice of the SVM cost parameter can be          critical. We then derive an algorithm that can fit the entire path of SVM          solutions for every value of the cost parameter, with essentially the same          computational cost as fitting one SVM model.
Neighbourhood Components Analysis
Authors: Jacob Goldberger, Geoffrey E. Hinton, Sam T. Roweis, Ruslan Salakhutdinov
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper we propose a novel method for learning a Mahalanobis              distance measure to be used in the KNN classification algorithm. The              algorithm directly maximizes a stochastic variant of the leave-one-out              KNN score on the training set. It can also learn a low-dimensional lin-              ear embedding of labeled data that can be used for data visualization              and fast classification. Unlike other methods, our classification model              is non-parametric, making no assumptions about the shape of the class              distributions or the boundaries between them. The performance of the              method is demonstrated on several data sets, both for metric learning and              linear dimensionality reduction.
Nonlinear Blind Source Separation by Integrating Independent Component Analysis and Slow Feature Analysis
Authors: Tobias Blaschke, Laurenz Wiskott
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In contrast to the equivalence of linear blind source separation and linear independent component analysis it is not possible to recover the origi- nal source signal from some unknown nonlinear transformations of the sources using only the independence assumption. Integrating the ob- jectives of statistical independence and temporal slowness removes this indeterminacy leading to a new method for nonlinear blind source sepa- ration. The principle of temporal slowness is adopted from slow feature analysis, an unsupervised method to extract slowly varying features from a given observed vectorial signal. The performance of the algorithm is demonstrated on nonlinearly mixed speech data.
Joint Probabilistic Curve Clustering and Alignment
Authors: Scott J. Gaffney, Padhraic Smyth
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Clustering and prediction of sets of curves is an important problem in          many areas of science and engineering. It is often the case that curves          tend to be misaligned from each other in a continuous manner, either in          space (across the measurements) or in time. We develop a probabilistic          framework that allows for joint clustering and continuous alignment of          sets of curves in curve space (as opposed to a fixed-dimensional feature-          vector space). The proposed methodology integrates new probabilistic          alignment models with model-based curve clustering algorithms. The          probabilistic approach allows for the derivation of consistent EM learn-          ing algorithms for the joint clustering-alignment problem. Experimental          results are shown for alignment of human growth data, and joint cluster-          ing and alignment of gene expression time-course data.
Economic Properties of Social Networks
Authors: Sham M. Kakade, Michael Kearns, Luis E. Ortiz, Robin Pemantle, Siddharth Suri
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We examine the marriage of recent probabilistic generative models           for social networks with classical frameworks from mathematical eco-           nomics. We are particularly interested in how the statistical structure of           such networks influences global economic quantities such as price vari-           ation. Our findings are a mixture of formal analysis, simulation, and           experiments on an international trade data set from the United Nations.
Beat Tracking the Graphical Model Way
Authors: Dustin Lang, Nando D. Freitas
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present a graphical model for beat tracking in recorded music. Using          a probabilistic graphical model allows us to incorporate local information          and global smoothness constraints in a principled manner. We evaluate          our model on a set of varied and difficult examples, and achieve impres-          sive results. By using a fast dual-tree algorithm for graphical model in-          ference, our system runs in less time than the duration of the music being          processed.
Large-Scale Prediction of Disulphide Bond Connectivity
Authors: Jianlin Cheng, Alessandro Vullo, Pierre F. Baldi
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The formation of disulphide bridges among cysteines is an important fea-          ture of protein structures. Here we develop new methods for the predic-          tion of disulphide bond connectivity. We first build a large curated data          set of proteins containing disulphide bridges and then use 2-Dimensional          Recursive Neural Networks to predict bonding probabilities between cys-          teine pairs. These probabilities in turn lead to a weighted graph matching          problem that can be addressed efficiently. We show how the method con-          sistently achieves better results than previous approaches on the same          validation data. In addition, the method can easily cope with chains with          arbitrary numbers of bonded cysteines. Therefore, it overcomes one of          the major limitations of previous approaches restricting predictions to          chains containing no more than 10 oxidized cysteines. The method can          be applied both to situations where the bonded state of each cysteine is          known or unknown, in which case bonded state can be predicted with          85% precision and 90% recall. The method also yields an estimate for          the total number of disulphide bridges in each chain.
Multi-agent Cooperation in Diverse Population Games
Authors: K. Wong, S. W. Lim, Z. Gao
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We consider multi-agent systems whose agents compete for resources by striving to be in the minority group. The agents adapt to the environment by reinforcement learning of the preferences of the policies they hold. Diversity of preferences of policies is introduced by adding random bi- ases to the initial cumulative payoffs of their policies. We explain and provide evidence that agent cooperation becomes increasingly important when diversity increases. Analyses of these mechanisms yield excellent agreement with simulations over nine decades of data.
Modeling Nonlinear Dependencies in Natural Images using Mixture of Laplacian Distribution
Authors: Hyun J. Park, Te W. Lee
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Capturing dependencies in images in an unsupervised manner is  important for many image processing applications. We propose a  new method for capturing nonlinear dependencies in images of  natural scenes. This method is an extension of the linear Independent  Component Analysis (ICA) method by building a hierarchical model  based on ICA and mixture of Laplacian distribution. The model  parameters are learned via an EM algorithm and it can accurately  capture variance correlation and other high order structures in a  simple manner. We visualize the learned variance structure and  demonstrate applications to image segmentation and denoising.
Dependent Gaussian Processes
Authors: Phillip Boyle, Marcus Frean
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Gaussian processes are usually parameterised in terms of their covari-          ance functions. However, this makes it difficult to deal with multiple          outputs, because ensuring that the covariance matrix is positive definite          is problematic. An alternative formulation is to treat Gaussian processes          as white noise sources convolved with smoothing kernels, and to param-          eterise the kernel instead. Using this, we extend Gaussian processes to          handle multiple, coupled outputs.
Synchronization of neural networks by mutual learning and its application to cryptography
Authors: Einat Klein, Rachel Mislovaty, Ido Kanter, Andreas Ruttor, Wolfgang Kinzel
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Two neural networks that are trained on their mutual output synchronize          to an identical time dependant weight vector. This novel phenomenon          can be used for creation of a secure cryptographic secret-key using a          public channel. Several models for this cryptographic system have been          suggested, and have been tested for their security under different sophis-          ticated attack strategies. The most promising models are networks that          involve chaos synchronization. The synchronization process of mutual          learning is described analytically using statistical physics methods.
Semigroup Kernels on Finite Sets
Authors: Marco Cuturi, Jean-philippe Vert
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Complex objects can often be conveniently represented by finite sets of          simpler components, such as images by sets of patches or texts by bags          of words. We study the class of positive definite (p.d.) kernels for two          such objects that can be expressed as a function of the merger of their          respective sets of components. We prove a general integral representa-          tion of such kernels and present two particular examples. One of them          leads to a kernel for sets of points living in a space endowed itself with a          positive definite kernel. We provide experimental results on a benchmark          experiment of handwritten digits image classification which illustrate the          validity of the approach.
A Hidden Markov Model for de Novo Peptide Sequencing
Authors: Bernd Fischer, Volker Roth, Jonas Grossmann, Sacha Baginsky, Wilhelm Gruissem, Franz Roos, Peter Widmayer, Joachim M. Buhmann
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: De novo Sequencing of peptides is a challenging task in proteome re-          search. While there exist reliable DNA-sequencing methods, the high-          throughput de novo sequencing of proteins by mass spectrometry is still          an open problem. Current approaches suffer from a lack in precision          to detect mass peaks in the spectrograms. In this paper we present a          novel method for de novo peptide sequencing based on a hidden Markov          model. Experiments effectively demonstrate that this new method signif-          icantly outperforms standard approaches in matching quality.
Instance-Based Relevance Feedback for Image Retrieval
Authors: Giorgio Gia\-cin\-to, Fabio Roli
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: High  retrieval  precision  in  content-based  image  retrieval  can  be           attained  by  adopting  relevance  feedback  mechanisms.  These           mechanisms require that the user judges the quality of the results of           the  query  by  marking  all  the  retrieved  images  as  being  either           relevant or not. Then, the search engine exploits this information to           adapt  the  search  to  better  meet  user's  needs.  At  present,  the  vast           majority  of  proposed  relevance  feedback  mechanisms  are           formulated in terms of search model that has to be optimized. Such           an  optimization  involves  the  modification  of  some  search           parameters so that the nearest neighbor of the query vector contains           the  largest  number  of  relevant  images.  In  this  paper,  a  different           approach  to  relevance  feedback  is  proposed.  After  the  user           provides the first feedback, following retrievals are not based on k-          nn  search,  but  on  the  computation  of  a  relevance  score  for  each           image of the database. This score is computed as a function of two           distances, namely the distance from the nearest non-relevant image           and  the  distance  from  the  nearest  relevant  one.  Images  are  then           ranked  according  to  this  score  and  the  top  k  images  are  displayed.           Reported  results  on  three  image  data  sets  show  that  the  proposed           mechanism  outperforms  other  state-of-the-art  relevance  feedback           mechanisms.
Learning Preferences for Multiclass Problems
Authors: Fabio Aiolli, Alessandro Sperduti
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Many interesting multiclass problems can be cast in the general frame-          work of label ranking defined on a given set of classes. The evaluation          for such a ranking is generally given in terms of the number of violated          order constraints between classes. In this paper, we propose the Prefer-          ence Learning Model as a unifying framework to model and solve a large          class of multiclass problems in a large margin perspective. In addition,          an original kernel-based method is proposed and evaluated on a ranking          dataset with state-of-the-art results.
Maximum Margin Clustering
Authors: Linli Xu, James Neufeld, Bryce Larson, Dale Schuurmans
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a new method for clustering based on finding maximum mar-          gin hyperplanes through data. By reformulating the problem in terms          of the implied equivalence relation matrix, we can pose the problem as          a convex integer program. Although this still yields a difficult com-          putational problem, the hard-clustering constraints can be relaxed to a          soft-clustering formulation which can be feasibly solved with a semidef-          inite program. Since our clustering technique only depends on the data          through the kernel matrix, we can easily achieve nonlinear clusterings in          the same manner as spectral clustering. Experimental results show that          our maximum margin clustering technique often obtains more accurate          results than conventional clustering methods. The real benefit of our ap-          proach, however, is that it leads naturally to a semi-supervised training          method for support vector machines. By maximizing the margin simul-          taneously on labeled and unlabeled training data, we achieve state of the          art performance by using a single, integrated learning principle.
On the Adaptive Properties of Decision Trees
Authors: Clayton Scott, Robert Nowak
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Decision trees are surprisingly adaptive in three important respects: They          automatically (1) adapt to favorable conditions near the Bayes decision          boundary; (2) focus on data distributed on lower dimensional manifolds;          (3) reject irrelevant features. In this paper we examine a decision tree          based on dyadic splits that adapts to each of these conditions to achieve          minimax optimal rates of convergence. The proposed classifier is the          first known to achieve these optimal rates while being practical and im-          plementable.
Similarity and Discrimination in Classical Conditioning: A Latent Variable Account
Authors: Aaron C. Courville, Nathaniel D. Daw, David S. Touretzky
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a probabilistic, generative account of configural learning          phenomena in classical conditioning. Configural learning experiments          probe how animals discriminate and generalize between patterns of si-          multaneously presented stimuli (such as tones and lights) that are dif-          ferentially predictive of reinforcement. Previous models of these issues          have been successful more on a phenomenological than an explanatory          level: they reproduce experimental findings but, lacking formal founda-          tions, provide scant basis for understanding why animals behave as they          do. We present a theory that clarifies seemingly arbitrary aspects of pre-          vious models while also capturing a broader set of data. Key patterns          of data, e.g. concerning animals' readiness to distinguish patterns with          varying degrees of overlap, are shown to follow from statistical inference.
Surface Reconstruction using Learned Shape Models
Authors: Jan E. Solem, Fredrik Kahl
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We consider the problem of geometrical surface reconstruction from one          or several images using learned shape models. While humans can effort-          lessly retrieve 3D shape information, this inverse problem has turned out          to be difficult to perform automatically. We introduce a framework based          on level set surface reconstruction and shape models for achieving this          goal. Through this merging, we obtain an efficient and robust method for          reconstructing surfaces of an object category of interest.          The shape model includes surface cues such as point, curve and silhou-          ette features. Based on ideas from Active Shape Models, we show how          both the geometry and the appearance of these features can be modelled          consistently in a multi-view context. The complete surface is obtained by          evolving a level set driven by a PDE, which tries to fit the surface to the          inferred 3D features. In addition, an a priori 3D surface model is used to          regularize the solution, in particular, where surface features are sparse.          Experiments are demonstrated on a database of real face images.
Maximum Likelihood Estimation of Intrinsic Dimension
Authors: Elizaveta Levina, Peter J. Bickel
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a new method for estimating intrinsic dimension of a dataset derived by applying the principle of maximum likelihood to the distances between close neighbors. We derive the estimator by a Poisson process approximation, assess its bias and variance theo- retically and by simulations, and apply it to a number of simulated and real datasets. We also show it has the best overall performance compared with two other intrinsic dimension estimators.
Distributed Information Regularization on Graphs
Authors: Adrian Corduneanu, Tommi S. Jaakkola
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We provide a principle for semi-supervised learning based on optimizing          the rate of communicating labels for unlabeled points with side informa-          tion. The side information is expressed in terms of identities of sets of          points or regions with the purpose of biasing the labels in each region          to be the same. The resulting regularization objective is convex, has a          unique solution, and the solution can be found with a pair of local prop-          agation operations on graphs induced by the regions. We analyze the          properties of the algorithm and demonstrate its performance on docu-          ment classification tasks.
Joint MRI Bias Removal Using Entropy Minimization Across Images
Authors: Erik G. Learned-miller, Parvez Ahammad
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The correction of bias in magnetic resonance images is an important          problem in medical image processing. Most previous approaches have          used a maximum likelihood method to increase the likelihood of the pix-          els in a single image by adaptively estimating a correction to the unknown          image bias field. The pixel likelihoods are defined either in terms of a          pre-existing tissue model, or non-parametrically in terms of the image's          own pixel values. In both cases, the specific location of a pixel in the im-          age is not used to calculate the likelihoods. We suggest a new approach          in which we simultaneously eliminate the bias from a set of images of          the same anatomy, but from different patients. We use the statistics from          the same location across different images, rather than within an image, to          eliminate bias fields from all of the images simultaneously. The method          builds a "multi-resolution" non-parametric tissue model conditioned on          image location while eliminating the bias fields associated with the orig-          inal image set. We present experiments on both synthetic and real MR          data sets, and present comparisons with other methods.
Chemosensory Processing in a Spiking Model of the Olfactory Bulb: Chemotopic Convergence and Center Surround Inhibition
Authors: Baranidharan Raman, Ricardo Gutierrez-osuna
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: This paper presents a neuromorphic model of two olfactory signal-          processing primitives: chemotopic convergence of olfactory           receptor neurons, and center on-off surround lateral inhibition in           the olfactory bulb. A self-organizing model of receptor           convergence onto glomeruli is used to generate a spatially           organized map, an olfactory image. This map serves as input to a           lattice of spiking neurons with lateral connections. The dynamics           of this recurrent network transforms the initial olfactory image into           a spatio-temporal pattern that evolves and stabilizes into odor- and           intensity-coding attractors. The model is validated using           experimental data from an array of temperature-modulated gas           sensors.  Our results are consistent with recent neurobiological           findings on the antennal lobe of the honeybee and the locust.
Using Random Forests in the Structured Language Model
Authors: Peng Xu, Frederick Jelinek
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper, we explore the use of Random Forests (RFs) in the struc- tured language model (SLM), which uses rich syntactic information in predicting the next word based on words already seen. The goal in this work is to construct RFs by randomly growing Decision Trees (DTs) us- ing syntactic information and investigate the performance of the SLM modeled by the RFs in automatic speech recognition. RFs, which were originally developed as classiﬁers, are a combination of decision tree classiﬁers. Each tree is grown based on random training data sampled independently and with the same distribution for all trees in the forest, and a random selection of possible questions at each node of the decision tree. Our approach extends the original idea of RFs to deal with the data sparseness problem encountered in language modeling. RFs have been studied in the context of n-gram language modeling and have been shown to generalize well to unseen data. We show in this paper that RFs using syntactic information can also achieve better performance in both perplexity (PPL) and word error rate (WER) in a large vocabulary speech recognition system, compared to a baseline that uses Kneser-Ney smoothing.
Exploration-Exploitation Tradeoffs for Experts Algorithms in Reactive Environments
Authors: Daniela D. Farias, Nimrod Megiddo
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: A reactive environment is one that responds to the actions of an agent rather than           evolving obliviously. In reactive environments, experts algorithms must balance           exploration and exploitation of experts more carefully than in oblivious ones. In           addition, a more subtle definition of a learnable value of an expert is required. A           general exploration-exploitation experts method is presented along with a proper           definition of value. The method is shown to asymptotically perform as well as           the best available expert. Several variants are analyzed from the viewpoint of the           exploration-exploitation tradeoff, including explore-then-exploit, polynomially           vanishing exploration, constant-frequency exploration, and constant-size explo-           ration phases. Complexity and performance bounds are proven.
Unsupervised Variational Bayesian Learning of Nonlinear Models
Authors: Antti Honkela, Harri Valpola
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper we present a framework for using multi-layer per- ceptron (MLP) networks in nonlinear generative models trained by variational Bayesian learning. The nonlinearity is handled by linearizing it using a Gauss–Hermite quadrature at the hidden neu- rons. This yields an accurate approximation for cases of large pos- terior variance. The method can be used to derive nonlinear coun- terparts for linear algorithms such as factor analysis, independent component/factor analysis and state-space models. This is demon- strated with a nonlinear factor analysis experiment in which even 20 sources can be estimated from a real world speech data set.
A Generalized Bradley-Terry Model: From Group Competition to Individual Skill
Authors: Tzu-kuo Huang, Chih-jen Lin, Ruby C. Weng
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The Bradley-Terry model for paired comparison has been popular in many areas. We propose a generalized version in which paired individual comparisons are extended to paired team comparisons. We introduce a simple algorithm with convergence proofs to solve the model and obtain individual skill. A useful application to multi-class probability estimates using error-correcting codes is demonstrated.
Rate- and Phase-coded Autoassociative Memory
Authors: Máté Lengyel, Peter Dayan
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Areas of the brain involved in various forms of memory exhibit patterns           of neural activity quite unlike those in canonical computational models.           We show how to use well-founded Bayesian probabilistic autoassociative           recall to derive biologically reasonable neuronal dynamics in recurrently           coupled models, together with appropriate values for parameters such as           the membrane time constant and inhibition. We explicitly treat two cases.           One arises from a standard Hebbian learning rule, and involves activity           patterns that are coded by graded firing rates. The other arises from a           spike timing dependent learning rule, and involves patterns coded by the           phase of spike times relative to a coherent local field potential oscillation.           Our model offers a new and more complete understanding of how neural           dynamics may support autoassociation.
Efficient Kernel Machines Using the Improved Fast Gauss Transform
Authors: Changjiang Yang, Ramani Duraiswami, Larry S. Davis
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The computation and memory required for kernel machines with N train-          ing samples is at least O(N 2). Such a complexity is significant even for          moderate size problems and is prohibitive for large datasets. We present          an approximation technique based on the improved fast Gauss transform          to reduce the computation to O(N ). We also give an error bound for the          approximation, and provide experimental results on the UCI datasets.
Responding to Modalities with Different Latencies
Authors: Fredrik Bissmarck, Hiroyuki Nakahara, Kenji Doya, Okihide Hikosaka
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Motor control depends on sensory feedback in multiple modalities with different latencies. In this paper we consider within the framework of re- inforcement learning how different sensory modalities can be combined and selected for real-time, optimal movement control. We propose an actor-critic architecture with multiple modules, whose output are com- bined using a softmax function. We tested our architecture in a simu- lation of a sequential reaching task. Reaching was initially guided by visual feedback with a long latency. Our learning scheme allowed the agent to utilize the somatosensory feedback with shorter latency when the hand is near the experienced trajectory. In simulations with different latencies for visual and somatosensory feedback, we found that the agent depended more on feedback with shorter latency.
Two-Dimensional Linear Discriminant Analysis
Authors: Jieping Ye, Ravi Janardan, Qi Li
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Linear Discriminant Analysis (LDA) is a well-known scheme for feature extraction and dimension reduction. It has been used widely in many ap- plications involving high-dimensional data, such as face recognition and image retrieval. An intrinsic limitation of classical LDA is the so-called singularity problem, that is, it fails when all scatter matrices are singu- lar. A well-known approach to deal with the singularity problem is to apply an intermediate dimension reduction stage using Principal Com- ponent Analysis (PCA) before LDA. The algorithm, called PCA+LDA, is used widely in face recognition. However, PCA+LDA has high costs in time and space, due to the need for an eigen-decomposition involving the scatter matrices. In this paper, we propose a novel LDA algorithm, namely 2DLDA, which stands for 2-Dimensional Linear Discriminant Analysis. 2DLDA over- comes the singularity problem implicitly, while achieving efﬁciency. The key difference between 2DLDA and classical LDA lies in the model for data representation. Classical LDA works with vectorized representa- tions of data, while the 2DLDA algorithm works with data in matrix representation. To further reduce the dimension by 2DLDA, the combi- nation of 2DLDA and classical LDA, namely 2DLDA+LDA, is studied, where LDA is preceded by 2DLDA. The proposed algorithms are ap- plied on face recognition and compared with PCA+LDA. Experiments show that 2DLDA and 2DLDA+LDA achieve competitive recognition accuracy, while being much more efﬁcient.
Learning Efficient Auditory Codes Using Spikes Predicts Cochlear Filters
Authors: Evan C. Smith, Michael S. Lewicki
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The representation of acoustic signals at the cochlear nerve must serve a           wide range of auditory tasks that require exquisite sensitivity in both time           and frequency. Lewicki (2002) demonstrated that many of the filtering           properties of the cochlea could be explained in terms of efficient coding           of natural sounds. This model, however, did not account for properties           such as phase-locking or how sound could be encoded in terms of action           potentials. Here, we extend this theoretical approach with algorithm for           learning efficient auditory codes using a spiking population code. Here,           we propose an algorithm for learning efficient auditory codes using a           theoretical model for coding sound in terms of spikes. In this model,           each spike encodes the precise time position and magnitude of a local-           ized, time varying kernel function. By adapting the kernel functions to           the statistics natural sounds, we show that, compared to conventional           signal representations, the spike code achieves far greater coding effi-           ciency. Furthermore, the inferred kernels show both striking similarities           to measured cochlear filters and a similar bandwidth versus frequency           dependence.