mtybilly/Instruct2DS huggingface dataset:
Neural mechanisms of contrast dependent receptive field size in V1
Authors: Jim Wielaard, Paul Sajda
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Based on a large scale spiking neuron model of the input layers 4C and  of macaque, we identify neural mechanisms for the observed contrast dependent receptive field size of V1 cells. We observe a rich variety of mechanisms for the phenomenon and analyze them based on the relative gain of excitatory and inhibitory synaptic inputs. We observe an average growth in the spatial extent of excitation and inhibition for low contrast, as predicted from phenomenological models. However, contrary to phenomenological models, our simulation results suggest this is neither sufficient nor necessary to explain the phenomenon.
Benchmarking Non-Parametric Statistical Tests
Authors: Mikaela Keller, Samy Bengio, Siew Y. Wong
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Although non-parametric tests have already been proposed for that purpose, statistical significance tests for non-standard measures (different from the classification error) are less often used in the literature. This paper is an attempt at empirically verifying how these tests compare with more classical tests, on various conditions. More precisely, using a very large dataset to estimate the whole "population", we analyzed the behavior of several statistical test, varying the class unbalance, the compared models, the performance measure, and the sample size. The main result is that providing big enough evaluation sets non-parametric tests are relatively reliable in all conditions.
Convergence and Consistency of Regularized Boosting Algorithms with Stationary B-Mixing Observations
Authors: Aurelie C. Lozano, Sanjeev R. Kulkarni, Robert E. Schapire
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We study the statistical convergence and consistency of regularized Boosting methods, where the samples are not independent and identi- cally distributed (i.i.d.) but come from empirical processes of stationary β-mixing sequences. Utilizing a technique that constructs a sequence of independent blocks close in distribution to the original samples, we prove the consistency of the composite classiﬁers resulting from a regulariza- tion achieved by restricting the 1-norm of the base classiﬁers’ weights. When compared to the i.i.d. case, the nature of sampling manifests in the consistency result only through generalization of the original condition on the growth of the regularization parameter.
Dynamic Social Network Analysis using Latent Space Models
Authors: Purnamrita Sarkar, Andrew W. Moore
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper explores two aspects of social network modeling. First, we generalize a successful static model of relationships into a dynamic model that accounts for friendships drifting over time. Second, we show how to make it tractable to learn such models from data, even as the number of entities n gets large. The generalized model associates each entity with a point in p-dimensional Euclidian latent space. The points can move as time progresses but large moves in latent space are improb- able. Observed links between entities are more likely if the entities are close in latent space. We show how to make such a model tractable (sub- quadratic in the number of entities) by the use of appropriate kernel func- tions for similarity in latent space; the use of low dimensional kd-trees; a new ef(cid:2)cient dynamic adaptation of multidimensional scaling for a (cid:2)rst pass of approximate projection of entities into latent space; and an ef(cid:2)- cient conjugate gradient update rule for non-linear local optimization in which amortized time per entity during an update is O(log n). We use both synthetic and real-world data on upto 11,000 entities which indicate linear scaling in computation time and improved performance over four alternative approaches. We also illustrate the system operating on twelve years of NIPS co-publication data. We present a detailed version of this work in [1].
Principles of real-time computing with feedback applied to cortical microcircuit models
Authors: Wolfgang Maass, Prashant Joshi, Eduardo D. Sontag
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: The network topology of neurons in the brain exhibits an abundance of feedback connections, but the computational function of these feedback connections is largely unknown. We present a computational theory that characterizes the gain in computational power achieved through feedback in dynamical systems with fading memory. It implies that many such systems acquire through feedback universal computational capabilities for analog computing with a non-fading memory. In particular, we show that feedback enables such systems to process time-varying input streams in diverse ways according to rules that are implemented through internal states of the dynamical system. In contrast to previous attractor-based computational models for neural networks, these ﬂexible internal states are high-dimensional attractors of the circuit dynamics, that still allow the circuit state to absorb new information from online input streams. In this way one arrives at novel models for working memory, integration of evidence, and reward expectation in cortical circuits. We show that they are applicable to circuits of conductance-based Hodgkin-Huxley (HH) neurons with high levels of noise that reﬂect experimental data on in- vivo conditions.
Location-based activity recognition
Authors: Lin Liao, Dieter Fox, Henry Kautz
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: Learning patterns of human behavior from sensor data is extremely important for high-level activity inference. We show how to extract and label a person's activities and significant places from traces of GPS data. In contrast to existing techniques, our approach simultaneously detects and classifies the significant locations of a person and takes the highlevel context into account. Our system uses relational Markov networks to represent the hierarchical activity model that encodes the complex relations among GPS readings, activities and significant places. We apply FFT-based message passing to perform efficient summation over large numbers of nodes in the networks. We present experiments that show significant improvements over existing techniques.
Modeling Memory Transfer and Saving in Cerebellar Motor Learning
Authors: Naoki Masuda, Shun-ichi Amari
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: There is a long-standing controversy on the site of the cerebellar motor learning. Different theories and experimental results suggest that either the cerebellar flocculus or the brainstem learns the task and stores the memory. With a dynamical system approach, we clarify the mechanism of transferring the memory generated in the flocculus to the brainstem and that of so-called savings phenomena. The brainstem learning must comply with a sort of Hebbian rule depending on Purkinje-cell activities. In contrast to earlier numerical models, our model is simple but it accommodates explanations and predictions of experimental situations as qualitative features of trajectories in the phase space of synaptic weights, without fine parameter tuning.
Gradient Flow Independent Component Analysis in Micropower VLSI
Authors: Abdullah Celik, Milutin Stanacevic, Gert Cauwenberghs
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present micropower mixed-signal VLSI hardware for real-time blind separation and localization of acoustic sources. Gradient flow representation of the traveling wave signals acquired over a miniature (1cm diameter) array of four microphones yields linearly mixed instantaneous observations of the time-differentiated sources, separated and localized by independent component analysis (ICA). The gradient flow and ICA processors each measure 3mm  3mm in 0.5 m CMOS, and consume 54 W and 180 W power, respectively, from a 3 V supply at 16 ks/s sampling rate. Experiments demonstrate perceptually clear (12dB) separation and precise localization of two speech sources presented through speakers positioned at 1.5m from the array on a conference room table. Analysis of the multipath residuals shows that they are spectrally diffuse, and void of the direct path.
An Alternative Infinite Mixture Of Gaussian Process Experts
Authors: Edward Meeds, Simon Osindero
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present an inﬁnite mixture model in which each component com- prises a multivariate Gaussian distribution over an input space, and a Gaussian Process model over an output space. Our model is neatly able to deal with non-stationary covariance functions, discontinuities, multi- modality and overlapping output signals. The work is similar to that by Rasmussen and Ghahramani [1]; however, we use a full generative model over input and output space rather than just a conditional model. This al- lows us to deal with incomplete data, to perform inference over inverse functional mappings as well as for regression, and also leads to a more powerful and consistent Bayesian speciﬁcation of the effective ‘gating network’ for the different experts.
Silicon growth cones map silicon retina
Authors: Brian Taba, Kwabena Boahen
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We demonstrate the ﬁrst fully hardware implementation of retinotopic self-organization, from photon transduction to neural map formation. A silicon retina transduces patterned illumination into correlated spike trains that drive a population of silicon growth cones to automatically wire a topographic mapping by migrating toward sources of a diffusible guidance cue that is released by postsynaptic spikes. We varied the pat- tern of illumination to steer growth cones projected by different retinal ganglion cell types to self-organize segregated or coordinated retinotopic maps.
Bayesian models of human action understanding
Authors: Chris Baker, Rebecca Saxe, Joshua B. Tenenbaum
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present a Bayesian framework for explaining how people reason about and predict the actions of an intentional agent, based on observ- ing its behavior. Action-understanding is cast as a problem of inverting a probabilistic generative model, which assumes that agents tend to act rationally in order to achieve their goals given the constraints of their en- vironment. Working in a simple sprite-world domain, we show how this model can be used to infer the goal of an agent and predict how the agent will act in novel situations or when environmental constraints change. The model provides a qualitative account of several kinds of inferences that preverbal infants have been shown to perform, and also ﬁts quantita- tive predictions that adult observers make in a new experiment.
Learning Influence among Interacting Markov Chains
Authors: Dong Zhang, Daniel Gatica-perez, Samy Bengio, Deb Roy
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We present a model that learns the influence of interacting Markov chains within a team. The proposed model is a dynamic Bayesian network (DBN) with a two-level structure: individual-level and group-level. Individual level models actions of each player, and the group-level models actions of the team as a whole. Experiments on synthetic multi-player games and a multi-party meeting corpus show the effectiveness of the proposed model.
A Probabilistic Interpretation of SVMs with an Application to Unbalanced Classification
Authors: Yves Grandvalet, Johnny Mariethoz, Samy Bengio
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: In this paper, we show that the hinge loss can be interpreted as the neg-log-likelihood of a semi-parametric model of posterior probabilities. From this point of view, SVMs represent the parametric component of a semi-parametric model ﬁtted by a maximum a posteriori estimation pro- cedure. This connection enables to derive a mapping from SVM scores to estimated posterior probabilities. Unlike previous proposals, the sug- gested mapping is interval-valued, providing a set of posterior probabil- ities compatible with each SVM score. This framework offers a new way to adapt the SVM optimization problem to unbalanced classiﬁca- tion, when decisions result in unequal (asymmetric) losses. Experiments show improvements over state-of-the-art procedures.
Oblivious Equilibrium: A Mean Field Approximation for Large-Scale Dynamic Games
Authors: Gabriel Y. Weintraub, Lanier Benkard, Benjamin Van Roy
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We propose a mean-ﬁeld approximation that dramatically reduces the computational complexity of solving stochastic dynamic games. We pro- vide conditions that guarantee our method approximates an equilibrium as the number of agents grow. We then derive a performance bound to assess how well the approximation performs for any given number of agents. We apply our method to an important class of problems in ap- plied microeconomics. We show with numerical experiments that we are able to greatly expand the set of economic problems that can be analyzed computationally.
Dynamical Synapses Give Rise to a Power-Law Distribution of Neuronal Avalanches
Authors: Anna Levina, Michael Herrmann
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: There is experimental evidence that cortical neurons show avalanche activity with the intensity of firing events being distributed as a power-law. We present a biologically plausible extension of a neural network which exhibits a power-law avalanche distribution for a wide range of connectivity parameters.
From Weighted Classification to Policy Search
Authors: Doron Blatt, Alfred O. Hero
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: This paper proposes an algorithm to convert a T -stage stochastic decision problem with a continuous state space to a sequence of supervised learning problems. The optimization problem associated with the trajectory tree and random trajectory methods of Kearns, Mansour, and Ng, 2000, is solved using the Gauss-Seidel method. The algorithm breaks a multistage reinforcement learning problem into a sequence of single-stage reinforcement learning subproblems, each of which is solved via an exact reduction to a weighted-classification problem that can be solved using off-the-self methods. Thus the algorithm converts a reinforcement learning problem into simpler supervised learning subproblems. It is shown that the method converges in a finite number of steps to a solution that cannot be further improved by componentwise optimization. The implication of the proposed algorithm is that a plethora of classification methods can be applied to find policies in the reinforcement learning problem.
Off-Road Obstacle Avoidance through End-to-End Learning
Authors: Urs Muller, Jan Ben, Eric Cosatto, Beat Flepp, Yann L. Cun
Published in: Advances in Neural Information Processing Systems 18  (NIPS 2005) (Main Conference Track), 2005

Abstract: We describe a vision-based obstacle avoidance system for off-road mobile robots. The system is trained from end to end to map raw in put images to steering angles. It is trained in supervised mode to predict the steering angles provided by a human driver during training r uns collected in a wide variety of terrains, weather conditions, lighting conditions, and obstacle types. The robot is a 50cm off-road truck, with two f orwardpointing wireless color cameras. A remote computer process es the video and controls the robot via radio. The learning system is a lar ge 6-layer convolutional network whose input is a single left/right pair of unprocessed low-resolution images. The robot exhibits an excellent ability to detect obstacles and navigate around them in real time at speeds of 2 m/s.
PAC-Bayes Learning of Conjunctions and Classification of Gene-Expression Data
Authors: Mario Marchand, Mohak Shah
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a “soft greedy” learning algorithm for building small conjunctions of simple threshold functions, called rays, deﬁned on single real-valued attributes. We also propose a PAC-Bayes risk bound which is minimized for classiﬁers achieving a non-trivial tradeoﬀ between sparsity (the number of rays used) and the mag- nitude of the separating margin of each ray. Finally, we test the soft greedy algorithm on four DNA micro-array data sets.
A Second Order Cone programming Formulation for Classifying Missing Data
Authors: Chiranjib Bhattacharyya, Pannagadatta K. Shivaswamy, Alex J. Smola
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a convex optimization based strategy to deal with uncertainty          in the observations of a classification problem. We assume that instead          of a sample (xi, yi) a distribution over (xi, yi) is specified. In particu-          lar, we derive a robust formulation when the distribution is given by a          normal distribution. It leads to Second Order Cone Programming formu-          lation. Our method is applied to the problem of missing data, where it          outperforms direct imputation.
Learning first-order Markov models for control
Authors: Pieter Abbeel, Andrew Y. Ng
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: First-order Markov models have been successfully applied to many prob- lems, for example in modeling sequential data using Markov chains, and modeling control problems using the Markov decision processes (MDP) formalism. If a ﬁrst-order Markov model’s parameters are estimated from data, the standard maximum likelihood estimator considers only the ﬁrst-order (single-step) transitions. But for many problems, the ﬁrst- order conditional independence assumptions are not satisﬁed, and as a re- sult the higher order transition probabilities may be poorly approximated. Motivated by the problem of learning an MDP’s parameters for control, we propose an algorithm for learning a ﬁrst-order Markov model that ex- plicitly takes into account higher order interactions during training. Our algorithm uses an optimization criterion different from maximum likeli- hood, and allows us to learn models that capture longer range effects, but without giving up the beneﬁts of using ﬁrst-order Markov models. Our experimental results also show the new algorithm outperforming conven- tional maximum likelihood estimation in a number of control problems where the MDP’s parameters are estimated from data.
Who's In the Picture
Authors: Tamara L. Berg, Alexander C. Berg, Jaety Edwards, David A. Forsyth
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The context in which a name appears in a caption provides powerful cues           as to who is depicted in the associated image. We obtain 44,773 face im-           ages, using a face detector, from approximately half a million captioned           news images and automatically link names, obtained using a named en-           tity recognizer, with these faces. A simple clustering method can pro-           duce fair results. We improve these results significantly by combining           the clustering process with a model of the probability that an individual           is depicted given its context. Once the labeling procedure is over, we           have an accurately labeled set of faces, an appearance model for each           individual depicted, and a natural language model that can produce ac-           curate results on captions in isolation.
Optimal Aggregation of Classifiers and Boosting Maps in Functional Magnetic Resonance Imaging
Authors: Vladimir Koltchinskii, Manel Martínez-ramón, Stefan Posse
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We study a method of optimal data-driven aggregation of classifiers in a           convex combination and establish tight upper bounds on its excess risk           with respect to a convex loss function under the assumption that the so-           lution of optimal aggregation problem is sparse. We use a boosting type           algorithm of optimal aggregation to develop aggregate classifiers of ac-           tivation patterns in fMRI based on locally trained SVM classifiers. The           aggregation coefficients are then used to design a "boosting map" of the           brain needed to identify the regions with most significant impact on clas-           sification.
Binet-Cauchy Kernels
Authors: Alex J. Smola, S.v.n. Vishwanathan
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a family of kernels based on the Binet-Cauchy theorem and its ex- tension to Fredholm operators. This includes as special cases all currently known kernels derived from the behavioral framework, diffusion processes, marginalized kernels, kernels on graphs, and the kernels on sets arising from the subspace angle approach. Many of these kernels can be seen as the extrema of a new continuum of kernel functions, which leads to numerous new special cases. As an application, we apply the new class of kernels to the problem of clustering of video sequences with encouraging results.
The power of feature clustering: An application to object detection
Authors: Shai Avidan, Moshe Butman
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We give a fast rejection scheme that is based on image segments and          demonstrate it on the canonical example of face detection. However, in-          stead of focusing on the detection step we focus on the rejection step and          show that our method is simple and fast to be learned, thus making it          an excellent pre-processing step to accelerate standard machine learning          classifiers, such as neural-networks, Bayes classifiers or SVM. We de-          compose a collection of face images into regions of pixels with similar          behavior over the image set. The relationships between the mean and          variance of image segments are used to form a cascade of rejectors that          can reject over 99.8% of image patches, thus only a small fraction of the          image patches must be passed to a full-scale classifier. Moreover, the          training time for our method is much less than an hour, on a standard PC.          The shape of the features (i.e. image segments) we use is data-driven,          they are very cheap to compute and they form a very low dimensional          feature space in which exhaustive search for the best features is tractable.
A Probabilistic Model for Online Document Clustering with Application to Novelty Detection
Authors: Jian Zhang, Zoubin Ghahramani, Yiming Yang
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper we propose a probabilistic model for online document clus-          tering. We use non-parametric Dirichlet process prior to model the grow-          ing number of clusters, and use a prior of general English language          model as the base distribution to handle the generation of novel clusters.          Furthermore, cluster uncertainty is modeled with a Bayesian Dirichlet-          multinomial distribution. We use empirical Bayes method to estimate          hyperparameters based on a historical dataset. Our probabilistic model          is applied to the novelty detection task in Topic Detection and Tracking          (TDT) and compared with existing approaches in the literature.
Non-Local Manifold Tangent Learning
Authors: Yoshua Bengio, Martin Monperrus
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We claim and present arguments to the effect that a large class of man-          ifold learning algorithms that are essentially local and can be framed as          kernel learning algorithms will suffer from the curse of dimensionality,          at the dimension of the true underlying manifold. This observation sug-          gests to explore non-local manifold learning algorithms which attempt to          discover shared structure in the tangent planes at different positions. A          criterion for such an algorithm is proposed and experiments estimating a          tangent plane prediction function are presented, showing its advantages          with respect to local manifold learning algorithms: it is able to general-          ize very far from training data (on learning handwritten character image          rotations), where a local non-parametric method fails.
Maximal Margin Labeling for Multi-Topic Text Categorization
Authors: Hideto Kazawa, Tomonori Izumitani, Hirotoshi Taira, Eisaku Maeda
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper, we address the problem of statistical learning for multi- topic text categorization (MTC), whose goal is to choose all relevant top- ics (a label) from a given set of topics. The proposed algorithm, Max- imal Margin Labeling (MML), treats all possible labels as independent classes and learns a multi-class classiﬁer on the induced multi-class cate- gorization problem. To cope with the data sparseness caused by the huge number of possible labels, MML combines some prior knowledge about label prototypes and a maximal margin criterion in a novel way. Experi- ments with multi-topic Web pages show that MML outperforms existing learning algorithms including Support Vector Machines.
1 Multi-topic Text Categorization (MTC)
This paper addresses the problem of learning for multi-topic text categorization (MTC), whose goal is to select all topics relevant to a text from a given set of topics. In MTC, multiple topics may be relevant to a single text. We thus call a set of topics label, and say that a text is assigned a label, not a topic.
In almost all previous text categorization studies (e.g. [1, 2]), the label is predicted by judging each topic’s relevance to the text. In this decomposition approach, the features speciﬁc to a topic, not a label, are regarded as important features. However, the approach may result in inefﬁcient learning as we will explain in the following example.
Imagine an MTC problem of scientiﬁc papers where quantum computing papers are as- signed multi-topic label “quantum physics (QP) & computer science (CS)”. (QP and CS are topics in this example.) Since there are some words speciﬁc to quantum computing such as “qbit1”, one can say that efﬁcient MTC learners should use such words to assign label QP & CS. However, the decomposition approach is likely to ignore these words since they are only speciﬁc to a small portion of the whole QP or CS papers (there are many more QP and CS papers than quantum computing papers), and therefore are not discriminative features for either topic QP or CS.
1Qbit is a unit of quantum information, and frequently appears in real quantum computing litera-
tures, but rarely seen in other literatures.
Symbol x(∈ Rd) t1, t2, . . . , tl T L, λ(⊂ T ) L[j] Λ(= 2T ) {(xi, Li)}m
i=1
Meaning A document vector Topics The set of all topics A label The binary representation of L. 1 if tj ∈ L and 0 otherwise. The set of all possible labels Training samples
Conditional Random Fields for Object Recognition
Authors: Ariadna Quattoni, Michael Collins, Trevor Darrell
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present a discriminative part-based approach for the recognition of          object classes from unsegmented cluttered scenes. Objects are modeled          as flexible constellations of parts conditioned on local observations found          by an interest operator. For each object class the probability of a given          assignment of parts to local features is modeled by a Conditional Ran-          dom Field (CRF). We propose an extension of the CRF framework that          incorporates hidden variables and combines class conditional CRFs into          a unified framework for part-based object recognition. The parameters of          the CRF are estimated in a maximum likelihood framework and recogni-          tion proceeds by finding the most likely class under our model. The main          advantage of the proposed CRF framework is that it allows us to relax the          assumption of conditional independence of the observed data (i.e. local          features) often used in generative approaches, an assumption that might          be too restrictive for a considerable number of object classes.
Inference, Attention, and Decision in a Bayesian Neural Architecture
Authors: Angela J. Yu, Peter Dayan
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We study the synthesis of neural coding, selective attention and percep-          tual decision making. A hierarchical neural architecture is proposed,          which implements Bayesian integration of noisy sensory input and top-          down attentional priors, leading to sound perceptual discrimination. The          model offers an explicit explanation for the experimentally observed          modulation that prior information in one stimulus feature (location) can          have on an independent feature (orientation). The network's intermediate          levels of representation instantiate known physiological properties of vi-          sual cortical neurons. The model also illustrates a possible reconciliation          of cortical and neuromodulatory representations of uncertainty.
Exponential Family Harmoniums with an Application to Information Retrieval
Authors: Max Welling, Michal Rosen-zvi, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Directed graphical models with one layer of observed random variables and one or more layers of hidden random variables have been the dom- inant modelling paradigm in many research ﬁelds. Although this ap- proach has met with considerable success, the causal semantics of these models can make it difﬁcult to infer the posterior distribution over the hidden variables. In this paper we propose an alternative two-layer model based on exponential family distributions and the semantics of undi- rected models. Inference in these “exponential family harmoniums” is fast while learning is performed by minimizing contrastive divergence. A member of this family is then studied as an alternative probabilistic model for latent semantic indexing. In experiments it is shown that they perform well on document retrieval tasks and provide an elegant solution to searching with keywords.
An Investigation of Practical Approximate Nearest Neighbor Algorithms
Authors: Ting Liu, Andrew W. Moore, Ke Yang, Alexander G. Gray
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: This paper concerns approximate nearest neighbor searching algorithms,          which have become increasingly important, especially in high dimen-          sional perception areas such as computer vision, with dozens of publica-          tions in recent years. Much of this enthusiasm is due to a successful new          approximate nearest neighbor approach called Locality Sensitive Hash-          ing (LSH). In this paper we ask the question: can earlier spatial data          structure approaches to exact nearest neighbor, such as metric trees, be          altered to provide approximate answers to proximity queries and if so,          how? We introduce a new kind of metric tree that allows overlap: certain          datapoints may appear in both the children of a parent. We also intro-          duce new approximate k-NN search algorithms on this structure. We          show why these structures should be able to exploit the same random-          projection-based approximations that LSH enjoys, but with a simpler al-          gorithm and perhaps with greater efficiency. We then provide a detailed          empirical evaluation on five large, high dimensional datasets which show          up to 31-fold accelerations over LSH. This result holds true throughout          the spectrum of approximation levels.
Multiple Relational Embedding
Authors: Roland Memisevic, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We describe a way of using multiple different types of similarity rela-          tionship to learn a low-dimensional embedding of a dataset. Our method          chooses different, possibly overlapping representations of similarity by          individually reweighting the dimensions of a common underlying latent          space. When applied to a single similarity relation that is based on Eu-          clidean distances between the input data points, the method reduces to          simple dimensionality reduction. If additional information is available          about the dataset or about subsets of it, we can use this information to          clean up or otherwise improve the embedding. We demonstrate the po-          tential usefulness of this form of semi-supervised dimensionality reduc-          tion on some simple examples.
Conditional Models of Identity Uncertainty with Application to Noun Coreference
Authors: Andrew McCallum, Ben Wellner
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Coreference analysis, also known as record linkage or identity uncer-          tainty, is a difficult and important problem in natural language process-          ing, databases, citation matching and many other tasks. This paper intro-          duces several discriminative, conditional-probability models for coref-          erence analysis, all examples of undirected graphical models. Unlike          many historical approaches to coreference, the models presented here          are relational--they do not assume that pairwise coreference decisions          should be made independently from each other. Unlike other relational          models of coreference that are generative, the conditional model here can          incorporate a great variety of features of the input without having to be          concerned about their dependencies--paralleling the advantages of con-          ditional random fields over hidden Markov models. We present positive          results on noun phrase coreference in two standard text data sets.
Generative Affine Localisation and Tracking
Authors: John Winn, Andrew Blake
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We present an extension to the Jojic and Frey (2001) layered sprite model          which allows for layers to undergo affine transformations. This extension          allows for affine object pose to be inferred whilst simultaneously learn-          ing the object shape and appearance. Learning is carried out by applying          an augmented variational inference algorithm which includes a global          search over a discretised transform space followed by a local optimisa-          tion. To aid correct convergence, we use bottom-up cues to restrict the          space of possible affine transformations. We present results on a number          of video sequences and show how the model can be extended to track an          object whose appearance changes throughout the sequence.
Schema Learning: Experience-Based Construction of Predictive Action Models
Authors: Michael P. Holmes, Charles Jr.
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: Schema learning is a way to discover probabilistic, constructivist, pre- dictive action models (schemas) from experience. It includes meth- ods for ﬁnding and using hidden state to make predictions more accu- rate. We extend the original schema mechanism [1] to handle arbitrary discrete-valued sensors, improve the original learning criteria to handle POMDP domains, and better maintain hidden state by using schema pre- dictions. These extensions show large improvement over the original schema mechanism in several rewardless POMDPs, and achieve very low prediction error in a difﬁcult speech modeling task. Further, we compare extended schema learning to the recently introduced predictive state rep- resentations [2], and ﬁnd their predictions of next-step action effects to be approximately equal in accuracy. This work lays the foundation for a schema-based system of integrated learning and planning.
Machine Learning Applied to Perception: Decision Images for Gender Classification
Authors: Felix A. Wichmann, Arnulf B. Graf, Heinrich H. Bülthoff, Eero P. Simoncelli, Bernhard Schölkopf
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We study gender discrimination of human faces using a combination of psychophysical classiﬁcation and discrimination experiments together with methods from machine learning. We reduce the dimensionality of a set of face images using principal component analysis, and then train a set of linear classiﬁers on this reduced representation (linear support vec- tor machines (SVMs), relevance vector machines (RVMs), Fisher linear discriminant (FLD), and prototype (prot) classiﬁers) using human clas- siﬁcation data. Because we combine a linear preprocessor with linear classiﬁers, the entire system acts as a linear classiﬁer, allowing us to visu- alise the decision-image corresponding to the normal vector of the separ- ating hyperplanes (SH) of each classiﬁer. We predict that the female-to- maleness transition along the normal vector for classiﬁers closely mim- icking human classiﬁcation (SVM and RVM [1]) should be faster than the transition along any other direction. A psychophysical discrimina- tion experiment using the decision images as stimuli is consistent with this prediction.
An Information Maximization Model of Eye Movements
Authors: Laura W. Renninger, James M. Coughlan, Preeti Verghese, Jitendra Malik
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a sequential information maximization model as a  general strategy for programming eye movements. The model  reconstructs high-resolution visual information from a sequence of  fixations, taking into account the fall-off in resolution from the  fovea to the periphery. From this framework we get a simple rule  for predicting fixation sequences: after each fixation, fixate next at  the location that minimizes uncertainty (maximizes information)  about the stimulus. By comparing our model performance to human  eye movement data and to predictions from a saliency and random  model, we demonstrate that our model is best at predicting fixation  locations. Modeling additional biological constraints will improve  the prediction of fixation sequences. Our results suggest that  information maximization is a useful principle for programming  eye movements.
Learning Hyper-Features for Visual Identification
Authors: Andras D. Ferencz, Erik G. Learned-miller, Jitendra Malik
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We address the problem of identifying specific instances of a class (cars)           from a set of images all belonging to that class. Although we cannot build           a model for any particular instance (as we may be provided with only one           "training" example of it), we can use information extracted from observ-           ing other members of the class. We pose this task as a learning problem,           in which the learner is given image pairs, labeled as matching or not, and           must discover which image features are most consistent for matching in-           stances and discriminative for mismatches. We explore a patch based           representation, where we model the distributions of similarity measure-           ments defined on the patches. Finally, we describe an algorithm that           selects the most salient patches based on a mutual information criterion.           This algorithm performs identification well for our challenging dataset           of car images, after matching only a few, well chosen patches.
Parametric Embedding for Class Visualization
Authors: Tomoharu Iwata, Kazumi Saito, Naonori Ueda, Sean Stromsten, Thomas L. Griffiths, Joshua B. Tenenbaum
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: In this paper, we propose a new method, Parametric Embedding (PE), for visualizing the posteriors estimated over a mixture model. PE simultane- ously embeds both objects and their classes in a low-dimensional space. PE takes as input a set of class posterior vectors for given data points, and tries to preserve the posterior structure in an embedding space by minimizing a sum of Kullback-Leibler divergences, under the assump- tion that samples are generated by a Gaussian mixture with equal covari- ances in the embedding space. PE has many potential uses depending on the source of the input data, providing insight into the classiﬁer’s be- havior in supervised, semi-supervised and unsupervised settings. The PE algorithm has a computational advantage over conventional embedding methods based on pairwise object relations since its complexity scales with the product of the number of objects and the number of classes. We demonstrate PE by visualizing supervised categorization of web pages, semi-supervised categorization of digits, and the relations of words and latent topics found by an unsupervised algorithm, Latent Dirichlet Allo- cation.
A Topographic Support Vector Machine: Classification Using Local Label Configurations
Authors: Johannes Mohr, Klaus Obermayer
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: The standard approach to the classification of objects is to consider the          examples as independent and identically distributed (iid). In many real          world settings, however, this assumption is not valid, because a topo-          graphical relationship exists between the objects. In this contribution we          consider the special case of image segmentation, where the objects are          pixels and where the underlying topography is a 2D regular rectangular          grid. We introduce a classification method which not only uses measured          vectorial feature information but also the label configuration within a to-          pographic neighborhood. Due to the resulting dependence between the          labels of neighboring pixels, a collective classification of a set of pixels          becomes necessary. We propose a new method called 'Topographic Sup-          port Vector Machine' (TSVM), which is based on a topographic kernel          and a self-consistent solution to the label assignment shown to be equiv-          alent to a recurrent neural network. The performance of the algorithm is          compared to a conventional SVM on a cell image segmentation task.
New Criteria and a New Algorithm for Learning in Multi-Agent Systems
Authors: Rob Powers, Yoav Shoham
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We propose a new set of criteria for learning algorithms in multi-agent          systems, one that is more stringent and (we argue) better justified than          previous proposed criteria. Our criteria, which apply most straightfor-          wardly in repeated games with average rewards, consist of three require-          ments: (a) against a specified class of opponents (this class is a parameter          of the criterion) the algorithm yield a payoff that approaches the payoff          of the best response, (b) against other opponents the algorithm's payoff          at least approach (and possibly exceed) the security level payoff (or max-          imin value), and (c) subject to these requirements, the algorithm achieve          a close to optimal payoff in self-play. We furthermore require that these          average payoffs be achieved quickly. We then present a novel algorithm,          and show that it meets these new criteria for a particular parameter class,          the class of stationary opponents. Finally, we show that the algorithm          is effective not only in theory, but also empirically. Using a recently          introduced comprehensive game theoretic test suite, we show that the          algorithm almost universally outperforms previous learning algorithms.
The Cerebellum Chip: an Analog VLSI Implementation of a Cerebellar Model of Classical Conditioning
Authors: Constanze Hofstoetter, Manuel Gil, Kynan Eng, Giacomo Indiveri, Matti Mintz, Jörg Kramer, Paul F. Verschure
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: We  present  a  biophysically  constrained  cerebellar  model  of                classical  conditioning,  implemented  using  a  neuromorphic  analog                VLSI (aVLSI) chip.  Like its biological counterpart, our cerebellar                model  is  able  to  control  adaptive  behavior  by  predicting  the                precise timing of events.  Here we describe the functionality of the                chip  and  present  its  learning  performance,  as  evaluated  in                simulated  conditioning  experiments  at  the  circuit  level  and  in                behavioral  experiments  using  a  mobile  robot.    We  show  that  this                aVLSI model supports the acquisition and extinction of adaptively                timed  conditioned  responses  under  real-world  conditions  with                ultra-low power consumption.
Pictorial Structures for Molecular Modeling: Interpreting Density Maps
Authors: Frank Dimaio, George Phillips, Jude W. Shavlik
Published in: Advances in Neural Information Processing Systems 17  (NIPS 2004) (Main Conference Track), 2004

Abstract: X-ray crystallography is currently the most common way protein  structures are elucidated. One of the most time-consuming steps in  the crystallographic process is interpretation of the electron density  map, a task that involves finding patterns in a three-dimensional  picture of a protein. This paper describes DEFT (DEFormable  Template), an algorithm using pictorial structures to build a  flexible protein model from the protein's amino-acid sequence.  Matching this pictorial structure into the density map is a way of  automating density-map interpretation. Also described are several  extensions to the pictorial structure matching algorithm necessary  for this automated interpretation. DEFT is tested on a set of  density maps ranging from 2 to 4Å resolution, producing root- mean-squared errors ranging from 1.38 to 1.84Å.