bluuebunny/crossref_metadata_2025 huggingface dataset:
Beyond Babble: A Text-Generation Method and Computer Program for Composing Text, Music and Poetry
Authors: Polashek Timothy D.
Published: May 2006

The author presents a method for creating sound poetry and text/sound music. According to his theory on the musical nature of speech and through the quantification of syllabic stress, the author presents his aesthetics of text/sound music and a detailed description of his original algorithms for manipulating characters representing phonemes. Redefining compositional techniques through his Beyond Babble computer program, The Babble Poet, he synthesizes vocalisms that sound like speech, yet are not, allowing nonsensical words to exist without semantic content and its inherent perceptual baggage. He also explores the applications of his program in composition by composing text/sound études synthesizing well-known texts with his method.
Sounds of a Community: Cultural Identity and Interactive Art
Authors: Gluck Robert J.
Published: May 2006

Sounds of a Community (2001–2003), a sound installation, is a series of sculptures modeled upon traditional Jewish ritual objects. The sculptures also serve as musical instruments. The installation is designed to offer an engaging musical experience but also a means by which people can explore complex issues relating to religious and cultural identity. Basic issues involved in the design, construction and programming of this work are presented, along with an exploration of how this work seeks to engage visitors in questions about their relationship to religious settings that involve the place of an individual within a group.
Experiments with Whistling Machines
Authors: Böhlen Marc , Rinker J.T.
Published: May 2006

This text describes a series of experiments performed with a machine capable of synthesizing human whistles and canary song. We call this device The Universal Whistling Machine (U.W.M.). It senses the presence of living creatures in its vicinity and attracts them with a signa-ture whistle. Given a response whistle, U.W.M. counters with its own composition, based on a time-frequency analysis of the response. Here the authors present a broad overview of all issues U.W.M. touches upon, including animal-machine interaction.
Composer's Notebook: Interpreting
Authors: Goode Daniel
Published: May 2006

The text is a 10-minute rant delivered by narrator William Hellermann, composer/performer and co-director/founder of the DownTown Ensemble, which seems to be about him (Hellermann), although approximately every other biographical fact seems questionable and is in fact false. That still leaves much that is true. So while Bill seems in some sense to be “Bill,” as things go on, the rant turns sour, then melancholy, totally political and outrageous. Finally, one should, based on observation, begin to suspect that the text is by the composer, who is sitting in the ensemble playing clarinet. The performance begins when a faux-emcee Bill, under appropriately harsh emcee lighting, comes onstage from the wings to (pretend to) introduce the Ensemble. He takes the mike while the ensemble is still tuning up (as specified in the score). The ensemble plays on while the text referentially points out what is happening: the Ensemble is “at this moment” strategizing political action to itself while playing. Further observation leads to another conclusion: The Ensemble and the Speaker seem to have nothing to do with each other—that is, until one focuses on the drummer (Jim Pugliese) who, entrained by the spoken voice, is responding to and improvising off the vocal rhythms of Hellermann with a softish and insistent wash of brushes and kick-drum. So although seated with the Ensemble, he is really part of Bill's “ensemble.”

The text is part of the score of Interpreting, composed for the Down Town Ensemble: clarinet (Daniel Goode), trombone (Peter Zummo), cello (Matt Goeke), piano (Joseph Kubera), percussion (Jim Pugliese) and soprano voice (Mary Jane Leach).The piece premiered at the Sounds Like Now festival at La Mama ETC, 16 October 2004.
Sounding the Outer Limits
Authors: Blonk Jaap , Peer René van
Published: May 2006

In an exchange with music journalist René van Peer, composer and sound poet Jaap Blonk discusses aspects of his work that hitherto have not been given much attention: his approach to composition, his use of electronic equipment and software, and his thoughts about recordings of his work.
Partial BFGS Update and Efficient Step-Length Calculation for Three-Layer Neural Networks
Authors: Saito Kazumi , Nakano Ryohei
Published: May 2006

Second-order learning algorithms based on quasi-Newton methods have two problems. First, standard quasi-Newton methods are impractical for large-scale problems because they require N2 storage space to maintain an approximation to an inverse Hessian matrix (N is the number of weights). Second, a line search to calculate areasonably accurate step length is indispensable for these algorithms. In order to provide desirable performance, an efficient and reasonably accurate line search is needed.

To overcome these problems, we propose a new second-order learning algorithm. Descent direction is calculated on the basis of a partial Broydon-Fletcher-Goldfarb-Shanno (BFGS) update with 2Ns memory space (s « N), and a reasonably accurate step length is efficiently calculated as the minimal point of a second-order approximation to the objective function with respect to the step length. Our experiments, which use a parity problem and a speech synthesis problem, have shown that the proposed algorithm outperformed major learning algorithms. Moreover, it turned out that an efficient and accurate step-length calculation plays an important role for the convergence of quasi-Newton algorithms, and a partial BFGS update greatly saves storage space without losing the convergence performance.
Selecting Optimal Experiments for Multiple Output Multilayer Perceptrons
Authors: Belue Lisa M. , Bauer Kenneth W. , Ruck Dennis W.
Published: May 2006

Where should a researcher conduct experiments to provide training data for a multilayer perceptron? This question is investigated, and a statistical method for selecting optimal experimental design points for multiple output multilayer perceptrons is introduced. Multiple class discrimination problems are examined using a framework in which the multilayer perceptron is viewed as a multivariate nonlinear regression model. Following a Bayesian formulation for the case where the variance-covariance matrix of the responses is unknown, a selection criterion is developed. This criterion is based on the volume of the joint confidence ellipsoid for the weights in a multilayer perceptron. An example is used to demonstrate the superiority of optimally selected design points over randomly chosen points, as well as points chosen in a grid pattern. Simplification of the basic criterion is offered through the use of Hadamard matrices to produce uncorrelated outputs.
Lyapunov Functions for Neural Nets with Nondifferentiable Input-Output Characteristics
Authors: Feng Jianfeng
Published: May 2006

I construct Lyapunov functions for asynchronous dynamics and synch ronous dynamics of neural networks with nondifferentiable input-output characteristics.
Using Expectation-Maximization for Reinforcement Learning
Authors: Dayan Peter , Hinton Geoffrey E.
Published: May 2006

We discuss Hinton's (1989) relative payoff procedure (RPP), a static reinforcement learning algorithm whose foundation is not stochastic gradient ascent. We show circumstances under which applying the RPP is guaranteed to increase the mean return, even though it can make large changes in the values of the parameters. The proof is based on a mapping between the RPP and a form of the expectation-maximization procedure of Dempster, Laird, and Rubin (1977).
Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions
Authors: Balasubramanian Vijay
Published: May 2006

The task of parametric model selection is cast in terms of a statistical mechanics on the space of probability distributions. Using the techniques of low-temperature expansions, I arrive at a systematic series for the Bayesian posterior probability of a model family that significantly extends known results in the literature. In particular, I arrive at a precise understanding of how Occam's razor, the principle that simpler models should be preferred until the data justify more complex models, is automatically embodied by probability theory. These results require a measure on the space of model parameters and I derive and discuss an interpretation of Jeffreys' prior distribution as a uniform prior over the distributions indexed by a family. Finally, I derive a theoretical index of the complexity of a parametric family relative to some true distribution that I call the razor of the model. The form of the razor immediately suggests several interesting questions in the theory of learning that can be studied using the techniques of statistical mechanics.
The Behavior of Forgetting Learning in Bidirectional Associative Memory
Authors: Leung Chi Sing , Chan Lai Wan
Published: May 2006

Forgetting learning is an incremental learning rule in associative memories. With it, the recent learning items can be encoded, and the old learning items will be forgotten. In this article, we analyze the storage behavior of bidirectional associative memory (BAM) under the forgetting learning. That is, “Can the most recent k learning item be stored as a fixed point?” Also, we discuss how to choose the forgetting constant in the forgetting learning such that the BAM can correctly store as many as possible of the most recent learning items. Simulation is provided to verify the theoretical analysis.
A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks
Authors: Yingwei Lu , Sundararajan N. , Saratchandran P.
Published: May 2006

This article presents a sequential learning algorithm for function approximation and time-series prediction using a minimal radial basis function neural network (RBFNN). The algorithm combines the growth criterion of the resource-allocating network (RAN) of Platt (1991) with a pruning strategy based on the relative contribution of each hidden unit to the overall network output. The resulting network leads toward a minimal topology for the RBFNN.

The performance of the algorithm is compared with RAN and the enhanced RAN algorithm of Kadirkamanathan and Niranjan (1993) for the following benchmark problems: (1) hearta from the benchmark problems database PROBEN1, (2) Hermite polynomial, and (3) Mackey-Glass chaotic time series. For these problems, the proposed algorithm is shown to realize RBFNNs with far fewer hidden neurons with better or same accuracy.
A Simple Model of Transmitter Release and Facilitation
Authors: Bertram Richard
Published: May 2006

We describe a model of synaptic transmitter release and presynaptic facilitation that is based on activation of release sites by single Ca2+ C microdomains. Facilitation is due to Ca2+ that remains bound to release sites between impulses. This model is inherently stochastic, but deterministic equations can be derived for the mean release. The number of equations required to describe the mean release is small, so it is practical to use the model with models of neuronal electrical activity to investigate the effects of different input spike patterns on presynaptic facilitation. We use it in conjunction with a model of dopamine-secreting neurons of the basal ganglia to demonstrate that transmitter release is greater when the neuron bursts than when it spikes continuously, due to the greater facilitation generated by the bursting impulse pattern. Finally, a minimal form of the model is described that is coupled to simple models of postsynaptic receptors and passive membrane to compute the postsynaptic voltage response to a train of presynaptic stimuli. This form of the model is appropriate for neural network simulations.
Possible Roles of Spontaneous Waves and Dendritic Growth for Retinal Receptive Field Development
Authors: Burgi Pierre-Yves , Grzywacz Norberto M.
Published: May 2006

Several models of cortical development postulate that a Hebbian process fed by spontaneous activity amplifies orientation biases occurring randomly in early wiring, to form orientation selectivity. These models are not applicable to the development of retinal orientation selectivity, since they neglect the polarization of the retina's poorly branched early dendritic trees and the wavelike organization of the retina's early noise. There is now evidence that dendritic polarization and spontaneous waves are key in the development of retinal receptive fields. When models of cortical development are modified to take these factors into account, one obtains a model of retinal development in which early dendritic polarization is the seed of orientation selectivity, while the spatial extent of spontaneous waves controls the spatial profile of receptive fields and their tendency to be isotropic.
Self-Organization of Firing Activities in Monkey's Motor Cortex: Trajectory Computation from Spike Signals
Authors: Lin Siming , Si Jennie , Schwartz A. B.
Published: May 2006

The population vector method has been developed to combine the simultaneous direction-related activities of a population of motor cortical neurons to predict the trajectory of the arm movement. In this article, we consider a self-organizing model of a neural representation of the arm trajectory based on neuronal discharge rates. A self-organizing feature map (SOFM) is used to select the optimal set of weights in the model to determine the contribution of an individual neuron to an overall movement representation. The correspondence between movement directions and discharge patterns of the motor cortical neurons is established in the output map. The topology-preserving property of the SOFM is used to analyze the recorded data of a behaving monkey. The data used in this analysis were taken while the monkey was tracing spirals and doing center→out movements. The arm trajectory could be well predicted using such a statistical model based on the motor cortex neuronal firing information. The SOFM method is compared with the population vector method, which extracts information related to trajectory by assuming that each cell has a fixed preferred direction during the task. This implies that these cells are acting along lines labeled only for direction. However, extradirectional information is carried in these cell responses. The SOFM has the capability of extracting not only direction-related information but also other parameters that are consistently represented in the activity of the recorded population of cells.
Supervised Networks That Self-Organize Class Outputs
Authors: Sarukkai Ramesh R.
Published: May 2006

Supervised, neural network, learning algorithms have proved very successful at solving a variety of learning problems; however, they suffer from a common problem of requiring explicit output labels. In this article, it is shown that pattern classification can be achieved, in a multilayered, feedforward, neural network, without requiring explicit output labels, by a process of supervised self-organization. The class projection is achieved by optimizing appropriate within-class uniformity and between-class discernibility criteria. The mapping function and the class labels are developed together iteratively using the derived self organizing backpropagation algorithm. The ability of the self-organizing network to generalize on unseen data is also experimentally evaluated on real data sets and compares favorably with the traditional labeled supervision with neural networks. In addition, interesting features emerge out of the proposed self-organizing supervision, which are absent in conventional approaches.
How Well Can We Estimate the Information Carried in Neuronal Responses from Limited Samples?
Authors: Golomb David , Hertz John , Panzeri Stefano , Treves Alessandro , Richmond Barry
Published: May 2006

It is difficult to extract the information carried by neuronal responses about a set of stimuli because limited data samples result in biased es timates. Recently two improved procedures have been developed to calculate information from experimental results: a binning-and-correcting procedure and a neural network procedure. We have used data produced from a model of the spatiotemporal receptive fields of parvocellular and magnocellular lateral geniculate neurons to study the performance of these methods as a function of the number of trials used. Both procedures yield accurate results for one-dimensional neuronal codes. They can also be used to produce a reasonable estimate of the extra information in a three-dimensional code, in this instance, within 0.05-0.1 bit of the asymptotically calculated value—about 10% of the total transmitted information. We believe that this performance is much more accurate than previous procedures.
A Mobile Robot That Learns Its Place
Authors: Oore Sageev , Hinton Geoffrey E. , Dudek Gregory
Published: May 2006

We show how a neural network can be used to allow a mobile robot to derive an accurate estimate of its location from noisy sonar sensors and noisy motion information. The robot's model of its location is in the form of a probability distribution across a grid of possible locations. This distribution is updated using both the motion information and the predictions of a neural network that maps locations into likelihood distributions across possible sonar readings. By predicting sonar readings from locations, rather than vice versa, the robot can handle the very nongaussian noise in the sonar sensors. By using the constraint provided by the noisy motion information, the robot can use previous readings to improve its estimate of its current location. By treating the resulting estimates as if they were correct, the robot can learn the relationship between location and sonar readings without requiring an external supervision signal that specifies the actual location of the robot. It can learn to locate itself in a new environment with almost no supervision, and it can maintain its location ability even when the environment is nonstationary.
Lower Bound on VC-Dimension by Local Shattering
Authors: Erlich Yossi , Chazan Dan , Petrack Scott , Levy Avraham
Published: May 2006

We show that the VC-dimension of a smoothly parameterized function class is not less than the dimension of any manifold in the parameter space, as long as distinct parameter values induce distinct decision boundaries. A similar theorem was published recently and used to introduce lower bounds on VC-dimension for several cases (Lee, Bartlett, & Williamson, 1995). This theorem is not correct, but our theorem could replace it for those cases and many other practical ones.
Image Segmentation Based on Oscillatory Correlation
Authors: Wang DeLiang , Terman David
Published: May 2006

We study image segmentation on the basis of locally excitatory, globally inhibitory oscillator networks (LEGION), whereby the phases of oscillators encode the binding of pixels. We introduce a lateral potential for each oscillator so that only oscillators with strong connections from their neighborhood can develop high potentials. Based on the concept of the lateral potential, a solution to remove noisy regions in an image is proposed for LEGION, so that it suppresses the oscillators corresponding to noisy regions but without affecting those corresponding to major regions. We show that the resulting oscillator network separates an image into several major regions, plus a background consisting of all noisy regions, and we illustrate network properties by computer simulation. The network exhibits a natural capacity in segmenting images. The oscillatory dynamics leads to a computer algorithm, which is applied successfully to segmenting real gray-level images. A number of issues regarding biological plausibility and perceptual organization are discussed. We argue that LEGION provides a novel and effective framework for image segmentation and figure-ground segregation.
Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience
Authors: Williams Lance R. , Jacobs David W.
Published: May 2006

We describe an algorithm and representation-level theory of illusory contour shape and salience. Unlike previous theories, our model is derived from a single assumption: that the prior probability distribution of boundary completion shape can be modeled by a random walk in a lattice whose points are positions and orientations in the image plane (i.e., the space that one can reasonably assume is represented by neurons of the mammalian visual cortex). Our model does not employ numerical relaxation or other explicit minimization, but instead relies on the fact that the probability that a particle following a random walk will pass through a given position and orientation on a path joining two boundary fragments can be computed directly as the product of two vector-field convolutions. We show that for the random walk we define, the maximum likelihood paths are curves of least energy, that is, on average, random walks follow paths commonly assumed to model the shape of illusory contours. A computer model is demonstrated on numerous illusory contour stimuli from the literature.
Local Parallel Computation of Stochastic Completion Fields
Authors: Williams Lance R. , Jacobs David W.
Published: May 2006

We describe a local parallel method for computing the stochastic completion field introduced in the previous article (Williams and Jacobs, 1997). The stochastic completion field represents the likelihood that a completion joining two contour fragments passes through any given position and orientation in the image plane. It is based on the assumption that the prior probability distribution of completion shape can be modeled as a random walk in a lattice of discrete positions and orientations. The local parallel method can be interpreted as a stable finite difference scheme for solving the underlying Fokker-Planck equation identified by Mumford (1994). The resulting algorithm is significantly faster than the previously employed method, which relied on convolution with large-kernel filters computed by Monte Carlo simulation. The complexity of the new method is O (n3m), while that of the previous algorithm was O(n4m2 (for an n × n image with m discrete orientations). Perhaps most significant, the use of a local method allows us to model the probability distribution of completion shape using stochastic processes that are neither homogeneous nor isotropic. For example, it is possible to modulate particle decay rate by a directional function of local image brightnesses (i.e., anisotropic decay). The effect is that illusory contours can be made to respect the local image brightness structure. Finally, we note that the new method is more plausible as a neural model since (1) unlike the previous method, it can be computed in a sparse, locally connected network, and (2) the network dynamics are consistent with psychophysical measurements of the time course of illusory contour formation.
Activation Functions, Computational Goals, and Learning Rules for Local Processors with Contextual Guidance
Authors: Kay Jim , Phillips W. A.
Published: May 2006

Information about context can enable local processors to discover latent variables that are relevant to the context within which they occur, and it can also guide short-term processing. For example, Becker and Hinton (1992) have shown how context can guide learning, and Hummel and Biederman (1992) have shown how it can guide processing in a large neural net for object recognition. This article studies the basic capabilities of a local processor with two distinct classes of inputs: receptive field inputs that provide the primary drive and contextual inputs that modulate their effects. The contextual predictions are used to guide processing without confusing them with receptive field inputs. The processor's transfer function must therefore distinguish these two roles. Given these two classes of input, the information in the output can be decomposed into four disjoint components to provide a space of possible goals in which the unsupervised learning of Linsker (1988) and the internally supervised learning of Becker and Hinton (1992) are special cases. Learning rules are derived from an information-theoretic objective function, and simulations show that a local processor trained with these rules and using an appropriate activation function has the elementary properties required.
Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates
Authors: Holt Gary R. , Koch Christof
Published: May 2006

Shunting inhibition, a conductance increase with a reversal potential close to the resting potential of the cell, has been shown to have a divisive effect on subthreshold excitatory postsynaptic potential amplitudes. It has therefore been assumed to have the same divisive effect on firing rates. We show that shunting inhibition actually has a subtractive effecton the firing rate in most circumstances. Averaged over several interspike intervals, the spiking mechanism effectively clamps the somatic membrane potential to a value significantly above the resting potential, so that the current through the shunting conductance is approximately independent of the firing rate. This leads to a subtractive rather than a divisive effect. In addition, at distal synapses, shunting inhibition will also have an approximately subtractive effect if the excitatory conductance is not small compared to the inhibitory conductance. Therefore regulating a cell's passive membrane conductance—for instance, via massive feedback—is not an adequate mechanism for normalizing or scaling its output.
The Faulty Behavior of Feedforward Neural Networks with Hard-Limiting Activation Function
Authors: Tian Zhiyu , Lin Ting-Ting Y. , Yang Shiyuan , Tong Shibai
Published: May 2006

With the progress in hardware implementation of artificial neural networks, the ability to analyze their faulty behavior has become increasingly important to their diagnosis, repair, reconfiguration, and reliable application. The behavior of feedforward neural networks with hard limiting activation function under stuck-at faults is studied in this article. It is shown that the stuck-at-M faults have a larger effect on the network's performance than the mixed stuck-at faults, which in turn have a larger effect than that of stuck-at-0 faults. Furthermore, the fault-tolerant ability of the network decreases with the increase of its size for the same percentage of faulty interconnections. The results of our analysis are validated by Monte-Carlo simulations.
Physiological Gain Leads to High ISI Variability in a Simple Model of a Cortical Regular Spiking Cell
Authors: Troyer Todd W. , Miller Kenneth D.
Published: May 2006

To understand the interspike interval (ISI) variability displayed by visual cortical neurons (Softky & Koch, 1993), it is critical to examine the dynamics of their neuronal integration, as well as the variability in their synaptic input current. Most previous models have focused on the latter factor. We match a simple integrate-and-fire model to the experimentally measured integrative properties of cortical regular spiking cells (McCormick, Connors, Lighthall, & Prince, 1985). After setting RC parameters, the postspike voltage reset is set to match experimental measurements of neuronal gain (obtained from in vitro plots of firing frequency versus injected current). Examination of the resulting model leads to an intuitive picture of neuronal integration that unifies the seemingly contradictory [Formula: see text] and random walk pictures that have previously been proposed. When ISIs are dominated by postspike recovery,[Formula: see text] arguments hold and spiking is regular; after the “memory” of the last spike becomes negligible, spike threshold crossing is caused by input variance around a steady state and spiking is Poisson. In integrate-and-fire neurons matched to cortical cell physiology, steady-state behavior is predominant, and ISIs are highly variable at all physiological firing rates and for a wide range of inhibitory and excitatory inputs.
Role of Temporal Integration and Fluctuation Detection in the Highly Irregular Firing of a Leaky Integrator Neuron Model with Partial Reset
Authors: Bugmann Guido , Christodoulou Chris , Taylor John G.
Published: May 2006

Partial reset is a simple and powerful tool for controlling the irregularity of spike trains fired by a leaky integrator neuron model with random inputs. In particular, a single neuron model with a realistic membrane time constant of 10 ms can reproduce the highly irregular firing of cortical neurons reported by Softky and Koch (1993). In this article, the mechanisms by which partial reset affects the firing pattern are investigated. Itisshown theoretically that partial reset is equivalent to the use of a time-dependent threshold, similar to a technique proposed by Wilbur and Rinzel (1983) to produce high irregularity. This equivalent model allows establishing that temporal integration and fluctuation detection can coexist and cooperate to cause highly irregular firing. This study also reveals that reverse correlation curves cannot be used reliably to assess the causes of firing. For instance, they do not reveal temporal integration when it takes place. Further, the peak near time zero does not always indicate coincidence detection. An alternative qualitative method is proposed here for that later purpose. Finally, it is noted that as the reset becomes weaker, the firing pattern shows a progressive transition from regular firing, to random, to temporally clustered, and eventually to bursting firing. Concurrently the slope of the transfer function increases. Thus, simulations suggest a correlation between high gain and highly irregular firing.
The NEURON Simulation Environment
Authors: Hines M. L. , Carnevale N. T.
Published: May 2006

The moment-to-moment processing of information by the nervous system involves the propagation and interaction of electrical and chemical signals that are distributed in space and time. Biologically realistic modeling is needed to test hypotheses about the mechanisms that govern these signals and how nervous system function emerges from the operation of these mechanisms. The NEURON simulation program provides a powerful and flexible environment for implementing such models of individual neurons and small networks of neurons. It is particularly useful when membrane potential is nonuniform and membrane currents are complex. We present the basic ideas that would help informed users make the most efficient use of NEURON.
Gamma Oscillation Model Predicts Intensity Coding by Phase Rather than Frequency
Authors: Traub Roger D. , Whittington Miles A. , Jefferys John G. R.
Published: May 2006

Gamma-frequency electroencephalogram oscillations may be important for cognitive processes such as feature binding. Gamma oscillations occur in hippocampus in vivo during the theta state, following physiological sharp waves, and after seizures, and they can be evoked in vitro by tetanic stimulation. In neocortex, gamma oscillations occur under conditions of sensory stimulation as well as during sleep. After tetanic or sensory stimulation, oscillations in regions separated by several millimeters or more occur at the same frequency, but with phase lags ranging from less than 1 ms to 10 ms, depending on the conditions of stimulation. We have constructed a distributed network model of pyramidal cells and interneurons, based on a variety of experiments, that accounts for near-zero phase lag synchrony of oscillations over long distances (with axon conduction delays totaling 16 ms or more). Here we show that this same model can also account for fixed positive phase lags between nearby cell groups coexisting with near-zero phase lags between separated cell groups, a phenomenon known to occur in visual cortex. The model achieves this because interneurons fire spike doublets and triplets that have average zero phase difference throughout the network; this provides a temporal framework on which pyramidal cell phase lags can be superimposed, the lag depending on how strongly the pyramidal cells are excited.
Self-Organized Formation of Various Invariant-Feature Filters in the Adaptive-Subspace SOM
Authors: Kohonen Teuvo , Kaski Samuel , Lappalainen Harri
Published: May 2006

The adaptive-subspace self-organizing map (ASSOM) is a modular neural network architecture, the modules of which learn to identify input patterns subject to some simple transformations. The learning process is unsupervised, competitive, and related to that of the traditional SOM (self-organizing map). Each neural module becomes adaptively specific to some restricted class of transformations, and modules close to each other in the network become tuned to similar features in an orderly fashion. If different transformations exist in the input signals, different subsets of ASSOM units become tuned to these transformation classes.
Information Controller to Maximize and Minimize Information
Authors: Kamimura Ryotaro
Published: May 2006

We propose an information controller to maximize and simultaneously minimize information in neural networks. The controller can be used to improve generalization performance and interpret explicitly internal representations. An apparently contradictory operation of the simultaneous maximization and minimization is possible by assuming two kinds of information: collective information and individual information, representing collective activities of hidden units and activities of individual input-hidden connections, respectively. By maximizing the collective information and minimizing the individual information, simple networks can be generated in terms of the number of connections and number of hidden units. This method was applied to the inference of the maximum onset principle and the sonority principle of artificial languages. In these problems, it was shown that the individual information could be sufficiently minimized and simultaneously the collective information was maximized. Experimental results confirmed improved generalization performance by the suppression of overtraining. In addition, internal representations created by information maximization and minimization could easily be interpreted.
Redundancy Reduction and Independent Component Analysis: Conditions on Cumulants and Adaptive Approaches
Authors: Nadal Jean-Pierre , Parga Nestor
Published: May 2006

In the context of both sensory coding and signal processing, building factorized codes has been shown to be an efficient strategy. In a wide variety of situations, the signal to be processed is a linear mixture of statistically independent sources. Building a factorized code is then equivalent to performing blind source separation. Thanks to the linear structure of the data, this can be done, in the language of signal processing, by finding an appropriate linear filter, or equivalently, in the language of neural modeling, by using a simple feedforward neural network.

In this article, we discuss several aspects of the source separation problem. We give simple conditions on the network output that, if satisfied, guarantee that source separation has been obtained. Then we study adaptive approaches, in particular those based on redundancy reduction and maximization of mutual information. We show how the resulting updating rules are related to the BCM theory of synaptic plasticity. Eventually we briefly discuss extensions to the case of nonlinear mixtures. Through out this article, we take care to put into perspective our work with other studies on source separation and redundancy reduction. In particular we review algebraic solutions, pointing out their simplicity but also their drawbacks.
A Fast Fixed-Point Algorithm for Independent Component Analysis
Authors: Hyvärinen Aapo , Oja Erkki
Published: May 2006

We introduce a novel fast algorithm for independent component analysis, which can be used for blind source separation and feature extraction. We show how a neural network learning rule can be transformed into a fixedpoint iteration, which provides an algorithm that is very simple, does not depend on any user-defined parameters, and is fast to converge to the most accurate solution allowed by the data. The algorithm finds, one at a time, all nongaussian independent components, regardless of their probability distributions. The computations can be performed in either batch mode or a semiadaptive manner. The convergence of the algorithm is rigorously proved, and the convergence speed is shown to be cubic. Some comparisons to gradient-based algorithms are made, showing that the new algorithm is usually 10 to 100 times faster, sometimes giving the solution in just a few iterations.
Dimension Reduction by Local Principal Component Analysis
Authors: Kambhatla Nandakishore , Leen Todd K.
Published: May 2006

Reducing or eliminating statistical redundancy between the components of high-dimensional vector data enables a lower-dimensional representation without significant loss of information. Recognizing the limitations of principal component analysis (PCA), researchers in the statistics and neural network communities have developed nonlinear extensions of PCA. This article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute. We exercise the algorithms on speech and image data, and compare performance with PCA and with neural network implementations of nonlinear PCA. We find that both nonlinear techniques can provide more accurate representations than PCA and show that the local linear techniques outperform neural network implementations.
Shape Quantization and Recognition with Randomized Trees
Authors: Amit Yali , Geman Donald
Published: May 2006

We explore a new approach to shape recognition based on a virtually infinite family of binary features (queries) of the image data, designed to accommodate prior information about shape invariance and regularity. Each query corresponds to a spatial arrangement of several local topographic codes (or tags), which are in themselves too primitive and common to be informative about shape. All the discriminating power derives from relative angles and distances among the tags. The important attributes of the queries are a natural partial ordering corresponding to increasing structure and complexity; semi-invariance, meaning that most shapes of a given class will answer the same way to two queries that are successive in the ordering; and stability, since the queries are not based on distinguished points and substructures.

No classifier based on the full feature set can be evaluated, and it is impossible to determine a priori which arrangements are informative. Our approach is to select informative features and build tree classifiers at the same time by inductive learning. In effect, each tree provides an approximation to the full posterior where the features chosen depend on the branch that is traversed. Due to the number and nature of the queries, standard decision tree construction based on a fixed-length feature vector is not feasible. Instead we entertain only a small random sample of queries at each node, constrain their complexity to increase with tree depth, and grow multiple trees. The terminal nodes are labeled by estimates of the corresponding posterior distribution over shape classes. An image is classified by sending it down every tree and aggregating the resulting distributions.

The method is applied to classifying handwritten digits and synthetic linear and nonlinear deformations of three hundred [Formula: see text] symbols. State-of-the-art error rates are achieved on the National Institute of Standards and Technology database of digits. The principal goal of the experiments on [Formula: see text] symbols is to analyze invariance, generalization error and related issues, and a comparison with artificial neural networks methods is presented in this context.

[Figure: see text]
Airline Crew Scheduling with Potts Neurons
Authors: Lagerholm Martin , Peterson Carsten , Söderberg Bo
Published: May 2006

A Potts feedback neural network approach for finding good solutions to resource allocation problems with a nonfixed topology is presented. As a target application, the airline crew scheduling problem is chosen. The topological complication is handled by means of a propagator defined in terms of Potts neurons. The approach is tested on artificial random problems tuned to resemble real-world conditions. Very good results are obtained for a variety of problem sizes. The computer time demand for the approach only grows like (number of flights)3. A realistic problem typically is solved within minutes, partly due to a prior reduction of the problem size, based on an analysis of the local arrival and departure structure at the single airports.
Online Learning in Radial Basis Function Networks
Authors: Freeman Jason A. S. , Saad David
Published: May 2006

An analytic investigation of the average case learning and generalization properties of radial basis function (RBFs) networks is presented, utilizing online gradient descent as the learning rule. The analytic method employed allows both the calculation of generalization error and the examination of the internal dynamics of the network. The generalization error and internal dynamics are then used to examine the role of the learning rate and the specialization of the hidden units, which gives insight into decreasing the time required for training. The realizable and some over realizable cases are studied in detail: the phase of learning in which the hidden units are unspecialized (symmetric phase) and the phase in which asymptotic convergence occurs are analyzed, and their typical properties found. Finally, simulations are performed that strongly confirm the analytic results.
A Local Learning Rule That Enables Information Maximization for Arbitrary Input Distributions
Authors: Linsker Ralph
Published: May 2006

This note presents a local learning rule that enables a network to maximize the mutual information between input and output vectors. The network's output units may be nonlinear, and the distribution of input vectors is arbitrary. The local algorithm also serves to compute the inverse C−1 of an arbitrary square connection weight matrix.
Convergence and Ordering of Kohonen's Batch Map
Authors: Cheng Yizong
Published: May 2006

The convergence and ordering of Kohonen's batch-mode self-organizing map with Heskes and Kappen's (1993) winner selection are proved. Selim and Ismail's (1984) objective function for k-means clustering is generalized in the convergence proof of the self-organizing map. It is shown that when the neighborhood relation is doubly decreasing, order in the map is preserved. An unordered map becomes ordered when a degenerate state of ordering is entered, where the number of distinct winners is one or two. One strategy to enter this state is to run the algorithm with a broad neighborhood relation.
Time-Series Segmentation Using Predictive Modular Neural Networks
Authors: Kehagias Athanasios , Petridis Vassilios
Published: May 2006

A predictive modular neural network method is applied to the problem of unsupervised time-series segmentation. The method consists of the concurrent application of two algorithms: one for source identification, the other for time-series classification. The source identification algorithm discovers the sources generating the time series, assigns data to each source, and trains one predictor for each source. The classification algorithm recursively computes a credit function for each source, based on the competition of the respective predictors, according to their predictive accuracy; the credit function is used for classification of the time-series observation at each time step. The method is tested by numerical experiments.
Factor Analysis Using Delta-Rule Wake-Sleep Learning
Authors: Neal Radford M. , Dayan Peter
Published: May 2006

We describe a linear network that models correlations between real-valued visible variables using one or more real-valued hidden variables—a factor analysis model. This model can be seen as a linear version of the Helmholtz machine, and its parameters can be learned using the wake sleep method, in which learning of the primary generative model is as sisted by a recognition model, whose role is to fill in the values of hidden variables based on the values of visible variables. The generative and recognition models are jointly learned in wake and sleep phases, using just the delta rule. This learning procedure is comparable in simplicity to Hebbian learning, which produces a somewhat different representation of correlations in terms of principal components. We argue that the simplicity of wake-sleep learning makes factor analysis a plausible alternative to Hebbian learning as a model of activity-dependent cortical plasticity.
Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm
Authors: Attias H. , Schreiner C. E.
Published: May 2006

We derive a novel family of unsupervised learning algorithms for blind separation of mixed and convolved sources. Our approach is based on formulating the separation problem as a learning task of a spatiotemporal generative model, whose parameters are adapted iteratively to minimize suitable error functions, thus ensuring stability of the algorithms. The resulting learning rules achieve separation by exploiting high-order spatiotemporal statistics of the mixture data. Different rules are obtained by learning generative models in the frequency and time domains, whereas a hybrid frequency-time model leads to the best performance. These algorithms generalize independent component analysis to the case of convolutive mixtures and exhibit superior performance on instantaneous mixtures. An extension of the relative-gradient concept to the spatiotemporal case leads to fast and efficient learning rules with equivariant properties. Our approach can incorporate information about the mixing situation when available, resulting in a “semiblind” separation method. The spatiotemporal redundancy reduction performed by our algorithms is shown to be equivalent to information-rate maximization through a simple network. We illustrate the performance of these algorithms by successfully separating instantaneous and convolutive mixtures of speech and noise signals.
Receptive Field Structures for Recognition
Authors: Balas Benjamin J. , Sinha Pawan
Published: May 2006

Localized operators, like Gabor wavelets and difference-of-gaussian filters, are considered useful tools for image representation. This is due to their ability to form a sparse code that can serve as a basis set for high-fidelity reconstruction of natural images. However, for many visual tasks, the more appropriate criterion of representational efficacy is recognition rather than reconstruction. It is unclear whether simple local features provide the stability necessary to subserve robust recognition of complex objects. In this article, we search the space of two-lobed differential operators for those that constitute a good representational code under recognition and discrimination criteria. We find that a novel operator, which we call the dissociated dipole, displays useful properties in this regard. We describe simple computational experiments to assess the merits of such dipoles relative to the more traditional local operators. The results suggest that nonlocal operators constitute a vocabulary that is stable across a range of image transformations.