mtybilly/Instruct2DS huggingface dataset:
An Efficient Implementation of the Back-propagation Algorithm on the Connection Machine CM-2
Authors: Michael McKenna, Jill P. Mesirov, David L. Waltz Xiru Zhang
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: In this paper, we  present a novel implementation of the widely used  Back-propagation neural net learning algorithm on the Connection  Machine  CM-2  - a  general  purpose,  massively  parallel  computer  with a hypercube topology.  This implementation runs at about 180  million  interconnections per second  (IPS)  on a  64K  processor  CM- 2.  The  main  interprocessor  communication operation  used  is  2D  nearest  neighbor  communication.  The  techniques  developed  here  can  be  easily  extended  to implement  other algorithms for  layered  neural nets on  the CM-2,  or on other massively parallel computers  which have 2D or higher degree connections among their processors.
Algorithms for Better Representation and Faster Learning in Radial Basis Function Networks
Authors: James D. Keeler Avijit Saha
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: in 
learning,  using  radial  basis  functions 
In  this  paper  we  present  upper  bounds  for  the  learning  rates  for  hybrid  models  that  employ  a  combination  of  both  self-organized  and  supervised  to  build  receptive  field  representations  the  hidden  units.  The  learning  performance  in  such  networks  with  nearest  neighbor  heuristic  can  be  improved  upon  by  multiplying  the  individual  receptive  field  widths  by  a  suitable  overlap  factor.  We  present  results  indicat!ng  optimal  values  for  such  overlap  factors.  We  also  present  a  new  algorithm  for  determining  receptive  field  centers.  This  method  negotiates  more  hidden  units  in  the  regions  of the  input  space  as  a  function  of the  output  and  is  conducive  to  better  learning  when  the  number of patterns (hidden units) is small.
The Cocktail Party Problem: Speech/Data Signal Separation Comparison between Backpropagation and SONN
Authors: Manoel Fernando Tenorio, Christoph Schaefers John Kassebaum
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: This  work  introduces  a  new  method  called  Self  Organizing  Neural  Network  (SONN)  algorithm  and  compares  its  performance  with Back  Propagation  in  a  signal  separation  application.  The  problem  is  to  separate  two  signals;  a  modem  data signal  and  a  male  speech  signal,  added and transmitted  through  a  4 khz  channel.  The signals  are sam(cid:173) pled  at  8  khz,  and  using  supervised  learning,  an  attempt  is  made  to  reconstruct  them.  The  SONN  is  an  algorithm  that  constructs  its  own  network  topology  during  training,  which  is  shown  to  be  much  smaller  than  the  BP  network,  faster  to  trained,  and  free  from  the  trial-and(cid:173) error network design that characterize BP.
Real-Time Computer Vision and Robotics Using Analog VLSI Circuits
Authors: Wyeth Bair, John G. Harris, Timothy K. Horiuchi, Andrew Hsu, Jin Luo Christof Koch
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The long-term goal of our laboratory is  the development of analog  resistive  network-based  VLSI implementations of early  and  inter(cid:173) mediate  vision  algorithms.  We  demonstrate  an  experimental cir(cid:173) cuit  for  smoothing  and  segmenting  noisy  and  sparse  depth  data  using  the  resistive  fuse  and  a  1-D  edge-detection  circuit  for  com(cid:173) puting zero-crossings using two  resistive grids with different space(cid:173) constants.  To  demonstrate  the  robustness  of our  algorithms  and  of the fabricated analog CMOS VLSI chips, we  are mounting these  circuits  onto  small  mobile  vehicles  operating in  a  real-time,  labo(cid:173) ratory environment.
The CHIR Algorithm for Feed Forward Networks with Binary Weights
Authors: Tal Grossman
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: A new learning algorithm, Learning by Choice of Internal Rep(cid:173) resetations  (CHIR), was  recently introduced.  Whereas many algo(cid:173) rithms  reduce  the  learning  process  to  minimizing a  cost  function  over the  weights, our method treats the  internal representations as  the fundamental entities to  be determined.  The algorithm applies  a  search  procedure  in the  space  of internal representations,  and  a  cooperative adaptation of the weights (e.g.  by using the perceptron  learning rule).  Since the introduction of its basic, single output ver(cid:173) sion, the CHIR algorithm was generalized to train any feed  forward  network of binary neurons.  Here we present the generalised version  of the  CHIR algorithm,  and further  demonstrate  its  versatility by  describing  how it can  be  modified  in order  to  train networks with  binary  (±1)  weights.  Preliminary  tests  of this  binary  version  on  the random teacher  problem are  also  reported.
A Large-Scale Neural Network Which Recognizes Handwritten Kanji Characters
Authors: Kazuki Joe Yoshihiro Mori
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We propose a  new  way  to construct a  large-scale neural  network  for  3.000 handwritten  Kanji  characters  recognition.  This  neural  network  consists  of 3  parts:  a  collection  of small-scale  networks  which  are  trained individually on a small number of Kanji characters; a network  which  integrates  the  output  from  the  small-scale  networks,  and  a  process to facilitate  the integration of these neworks. The recognition  rate of the  total  system  is  comparable  with  those  of the  small-scale  networks. Our results indicate that the proposed method is effective for  constructing  a  large-scale  network  without  loss  of  recognition  performance.
Mechanisms for Neuromodulation of Biological Neural Networks
Authors: Ronald M. Harris-Warrick
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The pyloric Central Pattern Generator of the crustacean stomatogastric  ganglion is a  well-defined  biological neural  network.  This  14-neuron  network  is  modulated  by  many  inputs.  These inputs reconfigure  the  network  to  produce  multiple  output  patterns  by  three  simple  mechanisms:  1) detennining which  cells are active; 2) modulating the  synaptic  efficacy;  3)  changing  the  intrinsic  response  properties  of  individual  neurons.  The importance of modifiable  intrinsic  response  properties of neurons for network function and modulation is discussed.
TRAFFIC: Recognizing Objects Using Hierarchical Reference Frame Transformations
Authors: Michael Mozer, Geoffrey E. Hinton Richard S. Zemel
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  describe a model that can recognize  two-dimensional shapes in  an  unsegmented  image,  independent  of their orientation,  position,  and scale.  The model,  called TRAFFIC, efficiently  represents  the  structural  relation  between  an  object  and  each  of its  component  features  by  encoding  the fixed  viewpoint-invariant transformation  from the feature's reference frame to the object's in the weights of a  connectionist  network.  Using  a  hierarchy  of such  transformations,  with increasing complexity of features  at each successive  layer,  the  network  can  recognize  multiple objects  in parallel.  An implemen(cid:173) tation  of TRAFFIC  is  described,  along with  experimental  results  demonstrating  the  network's  ability to  recognize  constellations  of  stars in  a viewpoint-invariant manner.
Performance of Connectionist Learning Algorithms on 2-D SIMD Processor Arrays
Authors: José A. B. Fortes Fernando J. Nuñez
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The  mapping  of  the  back-propagation  and  mean  field  theory  learning  algorithms  onto  a  generic  2-D  SIMD  computer  is  described.  This  architecture proves to  be very  adequate for  these  applications  since  efficiencies  close  to  the  optimum  can  be  attained.  Expressions  to  find  the  learning  rates  are  given  and  then particularized to the DAP  array procesor.
Neurally Inspired Plasticity in Oculomotor Processes
Authors: Paul A. Viola
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  have  constructed  a  two axis  camera positioning system which  is roughly analogous to a single human eye.  This Artificial-Eye (A(cid:173) eye)  combines  the  signals  generated  by  two  rate gyroscopes  with  motion  information  extracted from  visual  analysis  to stabilize  its  camera.  This stabilization process is similar to the vestibulo-ocular  response  (VOR);  like the  VOR, A-eye  learns a  system model that  can  be incrementally modified to adapt to changes in its structure,  performance and environment.  A-eye is an example of a robust sen(cid:173) sory system that performs computations that can be of significant  use  to the designers of mobile robots.
Combining Visual and Acoustic Speech Signals with a Neural Network Improves Intelligibility
Authors: Ben P. Yuhas, Moise H. Goldstein Jr., Robert E. Jenkins Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: R.E. Jenkins  The Applied  Physics  Laboratory  The Johns Hopkins  University  Laurel,  MD  20707 
Acoustic speech recognition degrades in the presence of noise.  Com(cid:173) pensatory  information  is  available  from  the  visual  speech  signals  around  the  speaker's  mouth.  Previous  attempts  at  using  these  visual speech signals to improve automatic speech  recognition sys(cid:173) tems have combined the acoustic and visual speech information at a  symbolic level  using heuristic rules.  In this paper, we  demonstrate  an  alternative  approach  to fusing  the  visual  and  acoustic  speech  information  by  training  feedforward  neural  networks  to  map  the  visual signal onto the corresponding short-term spectral amplitude  envelope  (STSAE)  of the  acoustic  signal.  This  information  can  be  directly  combined  with  the  degraded  acoustic  STSAE.  Signif(cid:173) icant  improvements  are  demonstrated  in  vowel  recognition  from  noise-degraded  acoustic signals.  These results are compared to the  performance of humans, as well  as other pattern matching and es(cid:173) timation algorithms.
The Computation of Sound Source Elevation in the Barn Owl
Authors: John C. Pearson Clay D. Spence
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The midbrain of the barn owl contains a map-like representation of  sound source direction which is used to precisely orient the head to(cid:173) ward targets of interest. Elevation is computed from the interaural  difference in sound level. We present models and computer simula(cid:173) tions of two stages of level difference processing which qualitatively  agree with known anatomy and physiology, and make several strik(cid:173) ing predictions.
Ordinal Regression by Extended Binary Classification
Authors: Hsuan-tien Lin Ling Li
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We present a reduction framework from ordinal regression to binary classification based on extended examples. The framework consists of three steps: extracting extended examples from the original examples, learning a binary classifier on the extended examples with any binary classification algorithm, and constructing a ranking rule from the binary classifier. A weighted 0/1 loss of the binary classifier would then bound the mislabeling cost of the ranking rule. Our framework allows not only to design good ordinal regression algorithms based on well-tuned binary classification approaches, but also to derive new generalization bounds for ordinal regression from known bounds for binary classification. In addition, our framework unifies many existing ordinal regression algorithms, such as perceptron ranking and support vector ordinal regression. When compared empirically on benchmark data sets, some of our newly designed algorithms enjoy advantages in terms of both training speed and generalization performance over existing algorithms, which demonstrates the usefulness of our framework.
Learning to Traverse Image Manifolds
Authors: Vincent Rabaud, Serge J. Belongie Piotr Dollár
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We present a new algorithm, Locally Smooth Manifold Learning (LSML), that learns a warping function from a point on an manifold to its neighbors. Important characteristics of LSML include the ability to recover the structure of the manifold in sparsely populated regions and beyond the support of the provided data. Appli- cations of our proposed technique include embedding with a natural out-of-sample extension and tasks such as tangent distance estimation, frame rate up-conversion, video compression and motion transfer.
Game Theoretic Algorithms for Protein-DNA binding
Authors: Luis E. Ortiz, Chen-hsiang Yeang, Tommi S. Jaakkola Luis Pérez-breva
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We develop and analyze game-theoretic algorithms for predicting coordinate binding of multiple DNA binding regulators. The allocation of proteins to local neighborhoods and to sites is carried out with resource constraints while explicating competing and coordinate binding relations among proteins with affinity to the site or region. The focus of this paper is on mathematical foundations of the approach. We also briefly demonstrate the approach in the context of the -phage switch.
Kernel Maximum Entropy Data Transformation and an Enhanced Spectral Clustering Algorithm
Authors: Torbjørn Eltoft, Mark Girolami, Deniz Erdogmus Robert Jenssen
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We propose a new kernel-based data transformation technique. It is founded on the principle of maximum entropy (MaxEnt) preservation, hence named kernel MaxEnt. The key measure is Renyi's entropy estimated via Parzen windowing. We show that kernel MaxEnt is based on eigenvectors, and is in that sense similar to kernel PCA, but may produce strikingly different transformed data sets. An enhanced spectral clustering algorithm is proposed, by replacing kernel PCA by kernel MaxEnt as an intermediate step. This has a major impact on performance.
Graph Laplacian Regularization for Large-Scale Semidefinite Programming
Authors: Fei Sha, Qihui Zhu, Lawrence K. Saul Kilian Q. Weinberger
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: In many areas of science and engineering, the problem arises how to discover low dimensional representations of high dimensional data. Recently, a number of researchers have converged on common solutions to this problem using methods from convex optimization. In particular, many results have been obtained by constructing semidefinite programs (SDPs) with low rank solutions. While the rank of matrix variables in SDPs cannot be directly constrained, it has been observed that low rank solutions emerge naturally by computing high variance or maximal trace solutions that respect local distance constraints. In this paper, we show how to solve very large problems of this type by a matrix factorization that leads to much smaller SDPs than those previously studied. The matrix factorization is derived by expanding the solution of the original problem in terms of the bottom eigenvectors of a graph Laplacian. The smaller SDPs obtained from this matrix factorization yield very good approximations to solutions of the original problem. Moreover, these approximations can be further refined by conjugate gradient descent. We illustrate the approach on localization in large scale sensor networks, where optimizations involving tens of thousands of nodes can be solved in just a few minutes.
Multi-Task Feature Learning
Authors: Theodoros Evgeniou, Massimiliano Pontil Andreas Argyriou
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We present a method for learning a low-dimensional representation which is shared across a set of multiple related tasks. The method builds upon the well- known 1-norm regularization problem using a new regularizer which controls the number of learned features common for all the tasks. We show that this problem is equivalent to a convex optimization problem and develop an iterative algorithm for solving it. The algorithm has a simple interpretation: it alternately performs a supervised and an unsupervised step, where in the latter step we learn common- across-tasks representations and in the former step we learn task-speciﬁc functions using these representations. We report experiments on a simulated and a real data set which demonstrate that the proposed method dramatically improves the per- formance relative to learning each task independently. Our algorithm can also be used, as a special case, to simply select – not learn – a few common features across the tasks.
Predicting spike times from subthreshold dynamics of a neuron
Authors: Shigeru Shinomoto Ryota Kobayashi
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: It has been established that a neuron reproduces highly precise spike response to identical fluctuating input currents. We wish to accurately predict the firing times of a given neuron for any input current. For this purpose we adopt a model that mimics the dynamics of the membrane potential, and then take a cue from its dynamics for predicting the spike occurrence for a novel input current. It is found that the prediction is significantly improved by observing the state space of the membrane potential and its time derivative(s) in advance of a possible spike, in comparison to simply thresholding an instantaneous value of the estimated potential.
Blind source separation for over-determined delayed mixtures
Authors: Martin Giese Lars Omlor
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Blind source separation, i.e. the extraction of unknown sources from a set of given signals, is relevant for many applications. A special case of this problem is dimension reduction, where the goal is to approximate a given set of signals by superpositions of a minimal number of sources. Since in this case the signals outnumber the sources the problem is over-determined. Most popular approaches for addressing this problem are based on purely linear mixing models. However, many applications like the modeling of acoustic signals, EMG signals, or movement trajectories, require temporal shift-invariance of the extracted components. This case has only rarely been treated in the computational literature, and specifically for the case of dimension reduction almost no algorithms have been proposed. We present a new algorithm for the solution of this problem, which is based on a timefrequency transformation (Wigner-Ville distribution) of the generative model. We show that this algorithm outperforms classical source separation algorithms for linear mixtures, and also a related method for mixtures with delays. In addition, applying the new algorithm to trajectories of human gaits, we demonstrate that it is suitable for the extraction of spatio-temporal components that are easier to interpret than components extracted with other classical algorithms.
Multiple timescales and uncertainty in motor adaptation
Authors: Joshua B. Tenenbaum, Reza Shadmehr Konrad P. Körding
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Our motor system changes due to causes that span multiple timescales. For example, muscle response can change because of fatigue, a condition where the disturbance has a fast timescale or because of disease where the disturbance is much slower. Here we hypothesize that the nervous system adapts in a way that reflects the temporal properties of such potential disturbances. According to a Bayesian formulation of this idea, movement error results in a credit assignment problem: what timescale is responsible for this disturbance? The adaptation schedule influences the behavior of the optimal learner, changing estimates at different timescales as well as the uncertainty. A system that adapts in this way predicts many properties observed in saccadic gain adaptation. It well predicts the timecourses of motor adaptation in cases of partial sensory deprivation and reversals of the adaptation direction.
Learning from Multiple Sources
Authors: Michael Kearns, Jennifer Wortman Koby Crammer
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We consider the problem of learning accurate models from multiple sources of "nearby" data. Given distinct samples from multiple data sources and estimates of the dissimilarities between these sources, we provide a general theory of which samples should be used to learn models for each source. This theory is applicable in a broad decision-theoretic learning framework, and yields results for classification and regression generally, and for density estimation within the exponential family. A key component of our approach is the development of approximate triangle inequalities for expected loss, which may be of independent interest.
Modeling Human Motion Using Binary Latent Variables
Authors: Geoffrey E. Hinton, Sam T. Roweis Graham W. Taylor
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We propose a non-linear generative model for human motion data that uses an undirected model with binary latent variables and real-valued "visible" variables that represent joint angles. The latent and visible variables at each time step receive directed connections from the visible variables at the last few time-steps. Such an architecture makes on-line inference efficient and allows us to use a simple approximate learning procedure. After training, the model finds a single set of parameters that simultaneously capture several different kinds of motion. We demonstrate the power of our approach by synthesizing various motion sequences and by performing on-line filling in of data lost during motion capture. Website: http://www.cs.toronto.edu/gwtaylor/publications/nips2006mhmublv/
Bayesian Image Super-resolution, Continued
Authors: David P. Capel, Stephen J. Roberts, Andrew Zisserman Lyndsey C. Pickup
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: This paper develops a multi-frame image super-resolution approach from a Bayesian view-point by marginalizing over the unknown registration parameters relating the set of input low-resolution views. In Tipping and Bishop’s Bayesian image super-resolution approach [16], the marginalization was over the super- resolution image, necessitating the use of an unfavorable image prior. By inte- grating over the registration parameters rather than the high-resolution image, our method allows for more realistic prior distributions, and also reduces the dimen- sion of the integral considerably, removing the main computational bottleneck of the other algorithm. In addition to the motion model used by Tipping and Bishop, illumination components are introduced into the generative model, allowing us to handle changes in lighting as well as motion. We show results on real and synthetic datasets to illustrate the efﬁcacy of this approach.
Geometric entropy minimization (GEM) for anomaly detection and localization
Authors: Alfred O. Hero
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We introduce a novel adaptive non-parametric anomaly detection approach, called GEM, that is based on the minimal covering properties of K-point entropic graphs when constructed on N training samples from a nominal probability distribution. Such graphs have the property that as N   their span recovers the entropy minimizing set that supports at least  = K/N (100)% of the mass of the Lebesgue part of the distribution. When a test sample falls outside of the entropy minimizing set an anomaly can be declared at a statistical level of significance  = 1 - . A method for implementing this non-parametric anomaly detector is proposed that approximates this minimum entropy set by the influence region of a K-point entropic graph built on the training data. By implementing an incremental leave-one-out k-nearest neighbor graph on resampled subsets of the training data GEM can efficiently detect outliers at a given level of significance and compute their empirical p-values. We illustrate GEM for several simulated and real data sets in high dimensional feature spaces.
Bayesian Ensemble Learning
Authors: Edward I. George, Robert E. Mcculloch Hugh A. Chipman
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We develop a Bayesian "sum-of-trees" model, named BART, where each tree is constrained by a prior to be a weak learner. Fitting and inference are accomplished via an iterative backfitting MCMC algorithm. This model is motivated by ensemble methods in general, and boosting algorithms in particular. Like boosting, each weak learner (i.e., each weak tree) contributes a small amount to the overall model. However, our procedure is defined by a statistical model: a prior and a likelihood, while boosting is defined by an algorithm. This model-based approach enables a full and accurate assessment of uncertainty in model predictions, while remaining highly competitive in terms of predictive accuracy.
The Robustness-Performance Tradeoff in Markov Decision Processes
Authors: Shie Mannor Huan Xu
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Computation of a satisfactory control policy for a Markov decision process when the parameters of the model are not exactly known is a problem encountered in many practical applications. The traditional robust approach is based on a worstcase analysis and may lead to an overly conservative policy. In this paper we consider the tradeoff between nominal performance and the worst case performance over all possible models. Based on parametric linear programming, we propose a method that computes the whole set of Pareto efficient policies in the performancerobustness plane when only the reward parameters are subject to uncertainty. In the more general case when the transition probabilities are also subject to error, we show that the strategy with the "optimal" tradeoff might be non-Markovian and hence is in general not tractable.
Combining causal and similarity-based reasoning
Authors: Patrick Shafto, Allison Berke, Joshua B. Tenenbaum Charles Kemp
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Everyday inductive reasoning draws on many kinds of knowledge, including knowledge about relationships between properties and knowledge about relationships between objects. Previous accounts of inductive reasoning generally focus on just one kind of knowledge: models of causal reasoning often focus on relationships between properties, and models of similarity-based reasoning often focus on similarity relationships between objects. We present a Bayesian model of inductive reasoning that incorporates both kinds of knowledge, and show that it accounts well for human inferences about the properties of biological species.
Detecting Humans via Their Pose
Authors: Ming-Hsuan Yang, Stefano Soatto Alessandro Bissacco
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We consider the problem of detecting humans and classifying their pose from a single image. Specifically, our goal is to devise a statistical model that simultaneously answers two questions: 1) is there a human in the image? and, if so, 2) what is a low-dimensional representation of her pose? We investigate models that can be learned in an unsupervised manner on unlabeled images of human poses, and provide information that can be used to match the pose of a new image to the ones present in the training set. Starting from a set of descriptors recently proposed for human detection, we apply the Latent Dirichlet Allocation framework to model the statistics of these features, and use the resulting model to answer the above questions. We show how our model can efficiently describe the space of images of humans with their pose, by providing an effective representation of poses for tasks such as classification and matching, while performing remarkably well in human/non human decision problems, thus enabling its use for human detection. We validate the model with extensive quantitative experiments and comparisons with other approaches on human detection and pose matching.
Hidden Markov Dirichlet Process: Modeling Genetic Recombination in Open Ancestral Space
Authors: Eric P. Xing Kyung-ah Sohn
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We present a new statistical framework called hidden Markov Dirichlet process (HMDP) to jointly model the genetic recombinations among possibly infinite number of founders and the coalescence-with-mutation events in the resulting genealogies. The HMDP posits that a haplotype of genetic markers is generated by a sequence of recombination events that select an ancestor for each locus from an unbounded set of founders according to a 1st-order Markov transition process. Conjoining this process with a mutation model, our method accommodates both between-lineage recombination and within-lineage sequence variations, and leads to a compact and natural interpretation of the population structure and inheritance process underlying haplotype data. We have developed an efficient sampling algo rithm for HMDP based on a two-level nested Polya urn scheme. On both simulated and real SNP haplotype data, our method performs competitively or significantly better than extant methods in uncovering the recombination hotspots along chromosomal loci; and in addition it also infers the ancestral genetic patterns and offers a highly accurate map of ancestral compositions of modern populations.
Online Classification for Complex Problems Using Simultaneous Projections
Authors: Shai Shalev-shwartz, Yoram Singer Yonatan Amit
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We describe and analyze an algorithmic framework for online classiﬁcation where each online trial consists of multiple prediction tasks that are tied together. We tackle the problem of updating the online hypothesis by deﬁning a projection problem in which each prediction task corresponds to a single linear constraint. These constraints are tied together through a single slack parameter. We then in- troduce a general method for approximately solving the problem by projecting simultaneously and independently on each constraint which corresponds to a pre- diction sub-problem, and then averaging the individual solutions. We show that this approach constitutes a feasible, albeit not necessarily optimal, solution for the original projection problem. We derive concrete simultaneous projection schemes and analyze them in the mistake bound model. We demonstrate the power of the proposed algorithm in experiments with online multiclass text categorization. Our experiments indicate that a combination of class-dependent features with the simultaneous projection method outperforms previously studied algorithms.
Towards a general independent subspace analysis
Authors: Fabian J. Theis
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: The increasingly popular independent component analysis (ICA) may only be applied to data following the generative ICA model in order to guarantee algorithmindependent and theoretically valid results. Subspace ICA models generalize the assumption of component independence to independence between groups of components. They are attractive candidates for dimensionality reduction methods, however are currently limited by the assumption of equal group sizes or less general semi-parametric models. By introducing the concept of irreducible independent subspaces or components, we present a generalization to a parameter-free mixture model. Moreover, we relieve the condition of at-most-one-Gaussian by including previous results on non-Gaussian component analysis. After introducing this general model, we discuss joint block diagonalization with unknown block sizes, on which we base a simple extension of JADE to algorithmically perform the subspace analysis. Simulations confirm the feasibility of the algorithm.
1 Independent subspace analysis A random vector Y is called an independent component of the random vector X, if there exists an invertible matrix A and a decomposition X = A(Y, Z) such that Y and Z are stochastically independent. The goal of a general independent subspace analysis (ISA) or multidimensional independent component analysis is the decomposition of an arbitrary random vector X into independent components. If X is to be decomposed into one-dimensional components, this coincides with ordinary independent component analysis (ICA). Similarly, if the independent components are required to be of the same dimension k , then this is denoted by multidimensional ICA of fixed group size k or simply k -ISA. So 1-ISA is equivalent to ICA. 1.1 Why extend ICA? An important structural aspect in the search for decompositions is the knowledge of the number of solutions i.e. the indeterminacies of the problem. Without it, the result of any ICA or ISA algorithm cannot be compared with other solutions, so for instance blind source separation (BSS) would be impossible. Clearly, given an ISA solution, invertible transforms in each component (scaling matrices L) as well as permutations of components of the same dimension (permutation matrices P) give again an ISA of X. And indeed, in the special case of ICA, scaling and permutation are already all indeterminacies given that at most one Gaussian is contained in X [6]. This is one of the key theoretical results in ICA, allowing the usage of ICA for solving BSS problems and hence stimulating many applications. It has been shown that also for k -ISA, scalings and permutations as above are the only indeterminacies [11], given some additional rather weak restrictions to the model. However, a serious drawback of k -ISA (and hence of ICA) lies in the fact that the requirement fixed group-size k does not allow us to apply this analysis to an arbitrary random vector. Indeed,
4
crosstalking error
In-Network PCA and Anomaly Detection
Authors: Xuanlong Nguyen, Minos Garofalakis, Michael I. Jordan, Anthony Joseph, Nina Taft Ling Huang
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We consider the problem of network anomaly detection in large distributed systems. In this setting, Principal Component Analysis (PCA) has been proposed as a method for discover- ing anomalies by continuously tracking the projection of the data onto a residual subspace. This method was shown to work well empirically in highly aggregated networks, that is, those with a limited number of large nodes and at coarse time scales. This approach, how- ever, has scalability limitations. To overcome these limitations, we develop a PCA-based anomaly detector in which adaptive local data (cid:2)lters send to a coordinator just enough data to enable accurate global detection. Our method is based on a stochastic matrix perturba- tion analysis that characterizes the tradeoff between the accuracy of anomaly detection and the amount of data communicated over the network.
Robotic Grasping of Novel Objects
Authors: Justin Driemeyer, Justin Kearns, Andrew Y. Ng Ashutosh Saxena
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We consider the problem of grasping novel objects, specifically ones that are being seen for the first time through vision. We present a learning algorithm that neither requires, nor tries to build, a 3-d model of the object. Instead it predicts, directly as a function of the images, a point at which to grasp the object. Our algorithm is trained via supervised learning, using synthetic images for the training set. We demonstrate on a robotic manipulation platform that this approach successfully grasps a wide variety of objects, such as wine glasses, duct tape, markers, a translucent box, jugs, knife-cutters, cellphones, keys, screwdrivers, staplers, toothbrushes, a thick coil of wire, a strangely shaped power horn, and others, none of which were seen in the training set.
Training Conditional Random Fields for Maximum Labelwise Accuracy
Authors: Olga Russakovsky, Chuong B. Do, Serafim Batzoglou Samuel S. Gross
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We consider the problem of training a conditional random field (CRF) to maximize per-label predictive accuracy on a training set, an approach motivated by the principle of empirical risk minimization. We give a gradient-based procedure for minimizing an arbitrarily accurate approximation of the empirical risk under a Hamming loss function. In experiments with both simulated and real data, our optimization procedure gives significantly better testing performance than several current approaches for CRF training, especially in situations of high label noise.
Reducing Calibration Time For Brain-Computer Interfaces: A Clustering Approach
Authors: Michael Schröder, Benjamin Blankertz, Klaus-Robert Müller Matthias Krauledat
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Up to now even subjects that are experts in the use of machine learning based BCI systems still have to undergo a calibration session of about 20-30 min. From this data their (movement) intentions are so far infered. We now propose a new paradigm that allows to completely omit such calibration and instead transfer knowledge from prior sessions. To achieve this goal we first define normalized CSP features and distances in-between. Second, we derive prototypical features across sessions: (a) by clustering or (b) by feature concatenation methods. Finally, we construct a classifier based on these individualized prototypes and show that, indeed, classifiers can be successfully transferred to a new session for a number of subjects.
Theory and Dynamics of Perceptual Bistability
Authors: Rashmi Sundareswara Paul R. Schrater
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Perceptual Bistability refers to the phenomenon of spontaneously switching between two or more interpretations of an image under continuous viewing. Although switching behavior is increasingly well characterized, the origins remain elusive. We propose that perceptual switching naturally arises from the brain's search for best interpretations while performing Bayesian inference. In particular, we propose that the brain explores a posterior distribution over image interpretations at a rapid time scale via a sampling-like process and updates its interpretation when a sampled interpretation is better than the discounted value of its current interpretation. We formalize the theory, explicitly derive switching rate distributions and discuss qualitative properties of the theory including the effect of changes in the posterior distribution on switching rates. Finally, predictions of the theory are shown to be consistent with measured changes in human switching dynamics to Necker cube stimuli induced by context.
Aggregating Classification Accuracy across Time: Application to Single Trial EEG
Authors: Christin Schäfer, Gabriel Curio Steven Lemm
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We present a method for binary on-line classification of triggered but temporally blurred events that are embedded in noisy time series in the context of on-line discrimination between left and right imaginary hand-movement. In particular the goal of the binary classification problem is to obtain the decision, as fast and as reliably as possible from the recorded EEG single trials. To provide a probabilistic decision at every time-point t the presented method gathers information from two distinct sequences of features across time. In order to incorporate decisions from prior time-points we suggest an appropriate weighting scheme, that emphasizes time instances, providing a higher discriminatory power between the instantaneous class distributions of each feature, where the discriminatory power is quantified in terms of the Bayes error of misclassification. The effectiveness of this procedure is verified by its successful application in the 3rd BCI competition. Disclosure of the data after the competition revealed this approach to be superior with single trial error rates as low as 10.7, 11.5 and 16.7% for the three different sub jects under study.
Blind Motion Deblurring Using Image Statistics
Authors: Anat Levin
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We address the problem of blind motion deblurring from a single image, caused by a few moving objects. In such situations only part of the image may be blurred, and the scene consists of layers blurred in different degrees. Most of of existing blind deconvolution research concentrates at recovering a single blurring kernel for the entire image. However, in the case of different motions, the blur cannot be modeled with a single kernel, and trying to deconvolve the entire image with the same kernel will cause serious artifacts. Thus, the task of deblurring needs to involve segmentation of the image into regions with different blurs. Our approach relies on the observation that the statistics of derivative filters in images are significantly changed by blur. Assuming the blur results from a constant velocity motion, we can limit the search to one dimensional box filter blurs. This enables us to model the expected derivatives distributions as a function of the width of the blur kernel. Those distributions are surprisingly powerful in discriminating regions with different blurs. The approach produces convincing deconvolution results on real world images with rich texture.
Accelerated Variational Dirichlet Process Mixtures
Authors: Max Welling, Nikos Vlassis Kenichi Kurihara
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Dirichlet Process (DP) mixture models are promising candidates for clustering applications where the number of clusters is unknown a priori. Due to compu- tational considerations these models are unfortunately unsuitable for large scale data-mining applications. We propose a class of deterministic accelerated DP mixture models that can routinely handle millions of data-cases. The speedup is achieved by incorporating kd-trees into a variational Bayesian algorithm for DP mixtures in the stick-breaking representation, similar to that of Blei and Jordan (2005). Our algorithm differs in the use of kd-trees and in the way we handle truncation: we only assume that the variational distributions are ﬁxed at their pri- ors after a certain level. Experiments show that speedups relative to the standard variational algorithm can be signiﬁcant.
Online Clustering of Moving Hyperplanes
Authors: René Vidal
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: We propose a recursive algorithm for clustering trajectories lying in multiple moving hyperplanes. Starting from a given or random initial condition, we use normalized gradient descent to update the coefficients of a time varying polynomial whose degree is the number of hyperplanes and whose derivatives at a trajectory give an estimate of the vector normal to the hyperplane containing that trajectory. As time proceeds, the estimates of the hyperplane normals are shown to track their true values in a stable fashion. The segmentation of the trajectories is then obtained by clustering their associated normal vectors. The final result is a simple recursive algorithm for segmenting a variable number of moving hyperplanes. We test our algorithm on the segmentation of dynamic scenes containing rigid motions and dynamic textures, e.g., a bird floating on water. Our method not only segments the bird motion from the surrounding water motion, but also determines patterns of motion in the scene (e.g., periodic motion) directly from the temporal evolution of the estimated polynomial coefficients. Our experiments also show that our method can deal with appearing and disappearing motions in the scene.
Efficient sparse coding algorithms
Authors: Alexis Battle, Rajat Raina, Andrew Y. Ng Honglak Lee
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Sparse coding provides a class of algorithms for finding succinct representations of stimuli; given only unlabeled input data, it discovers basis functions that capture higher-level features in the data. However, finding sparse codes remains a very difficult computational problem. In this paper, we present efficient sparse coding algorithms that are based on iteratively solving two convex optimization problems: an L1 -regularized least squares problem and an L2 -constrained least squares problem. We propose novel algorithms to solve both of these optimization problems. Our algorithms result in a significant speedup for sparse coding, allowing us to learn larger sparse codes than possible with previously described algorithms. We apply these algorithms to natural images and demonstrate that the inferred sparse codes exhibit end-stopping and non-classical receptive field surround suppression and, therefore, may provide a partial explanation for these two phenomena in V1 neurons.
Approximate Correspondences in High Dimensions
Authors: Trevor Darrell Kristen Grauman
Published in: Advances in Neural Information Processing Systems 19  (NIPS 2006) (Main Conference Track), 2006

Abstract: Pyramid intersection is an efﬁcient method for computing an approximate partial matching between two sets of feature vectors. We introduce a novel pyramid em- bedding based on a hierarchy of non-uniformly shaped bins that takes advantage of the underlying structure of the feature space and remains accurate even for sets with high-dimensional feature vectors. The matching similarity is computed in linear time and forms a Mercer kernel. Whereas previous matching approxima- tion algorithms suffer from distortion factors that increase linearly with the fea- ture dimension, we demonstrate that our approach can maintain constant accuracy even as the feature dimension increases. When used as a kernel in a discrimina- tive classiﬁer, our approach achieves improved object recognition results over a state-of-the-art set kernel.