nils-herrmann/clean_scirepeval huggingface dataset:
Learning viewpoint-invariant face representations from visual experience in an attractor network.

In natural visual experience, different views of an object or face tend to appear in close temporal proximity as an animal manipulates the object or navigates around it, or as a face changes expression or pose. A set of simulations is presented which demonstrate how viewpoint-invariant representations of faces can be developed from visual experience by capturing the temporal relationships among the input patterns. The simulations explored the interaction of temporal smoothing of activity signals with Hebbian learning in both a feedforward layer and a second, recurrent layer of a network. The feedforward connections were trained by competitive Hebbian learning with temporal smoothing of the post-synaptic unit activities. The recurrent layer was a generalization of a Hopfield network with a low-pass temporal filter on all unit activities. The combination of basic Hebbian learning with temporal smoothing of unit activities produced an attractor network learning rule that associated temporally proximal input patterns into basins of attraction. These two mechanisms were demonstrated in a model that took grey-level images of faces as input. Following training on image sequences of faces as they changed pose, multiple views of a given face fell into the same basin of attraction, and the system acquired representations of faces that were approximately viewpoint-invariant.
Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells.

Physical variables such as the orientation of a line in the visual field or the location of the body in space are coded as activity levels in populations of neurons. Reconstruction or decoding is an inverse problem in which the physical variables are estimated from observed neural activity. Reconstruction is useful first in quantifying how much information about the physical variables is present in the population and, second, in providing insight into how the brain might use distributed representations in solving related computational problems such as visual object recognition and spatial navigation. Two classes of reconstruction methods, namely, probabilistic or Bayesian methods and basis function methods, are discussed. They include important existing methods as special cases, such as population vector coding, optimal linear estimation, and template matching. As a representative example for the reconstruction problem, different methods were applied to multi-electrode spike train data from hippocampal place cells in freely moving rats. The reconstruction accuracy of the trajectories of the rats was compared for the different methods. Bayesian methods were especially accurate when a continuity constraint was enforced, and the best errors were within a factor of two of the information-theoretic limit on how accurate any reconstruction can be and were comparable with the intrinsic experimental errors in position tracking. In addition, the reconstruction analysis uncovered some interesting aspects of place cell activity, such as the tendency for erratic jumps of the reconstructed trajectory when the animal stopped running. In general, the theoretical values of the minimal achievable reconstruction errors quantify how accurately a physical variable is encoded in the neuronal population in the sense of mean square error, regardless of the method used for reading out the information. One related result is that the theoretical accuracy is independent of the width of the Gaussian tuning function only in two dimensions. Finally, all the reconstruction methods considered in this paper can be implemented by a unified neural network architecture, which the brain feasibly could use to solve related problems.
Coding Time-Varying Signals Using Sparse, Shift-Invariant Representations

A common way to represent a time series is to divide it into short-duration blocks, each of which is then represented by a set of basis functions. A limitation of this approach, however, is that the temporal alignment of the basis functions with the underlying structure in the time series is arbitrary. We present an algorithm for encoding a time series that does not require blocking the data. The algorithm finds an efficient representation by inferring the best temporal positions for functions in a kernel basis. These can have arbitrary temporal extent and are not constrained to be orthogonal. This allows the model to capture structure in the signal that may occur at arbitrary temporal positions and preserves the relative temporal structure of underlying events. The model is shown to be equivalent to a very sparse and highly overcomplete basis. Under this model, the mapping from the data to the representation is nonlinear, but can be computed efficiently. This form also allows the use of existing methods for adapting the basis itself to data. This approach is applied to speech data and results in a shift invariant, spikelike representation that resembles coding in the cochlear nerve.
Unsupervised Classification with Non-Gaussian Mixture Models Using ICA

We present an unsupervised classification algorithm based on an ICA mixture model. The ICA mixture model assumes that the observed data can be categorized into several mutually exclusive data classes in which the components in each class are generated by a linear mixture of independent sources. The algorithm finds the independent sources, the mixing matrix for each class and also computes the class membership probability for each data point. This approach extends the Gaussian mixture model so that the classes can have non-Gaussian structure. We demonstrate that this method can learn efficient codes to represent images of natural scenes and text. The learned classes of basis functions yield a better approximation of the underlying distributions of the data, and thus can provide greater coding efficiency. We believe that this method is well suited to modeling structure in high-dimensional data and has many potential applications.
Unsupervised statistical methods for processing of image sequences

The performance of pattern recognition systems depends on preprocessing. Unsupervised learning techniques can be used to construct statistically efficient preprocessing filters from an ensemble of images. The efficiency of local and global principal component and independent component image representations was examined on the task of visually recognizing the first four digits spoken in English using a hidden Markov model (HMM) for the recognition system. Local representations consistently outperformed global representations in generalizing to new speakers. In addition, the use of a novel regression-based variable selection technique substantially boosted performance. Global representations, on the other hand, performed better than local ones for speaker identification tasks. These results imply that local structure in an image contains more information than global structure for lipreading, a task that requires the extraction of speaker invariant information from sequences of images.
Analysis of fMRI data by blind separation into independent spatial components

Current analytical techniques applied to functional magnetic resonance imaging (fMRI) data require a priori knowledge or specific assumptions about the time courses of processes contributing to the measured signals. Here we describe a new method for analyzing fMRI data based on the independent component analysis (ICA) algorithm of Bell and Sejnowski ([1995]: Neural Comput 7:1129–1159). We decomposed eight fMRI data sets from 4 normal subjects performing Stroop color‐naming, the Brown and Peterson word/number task, and control tasks into spatially independent components. Each component consisted of voxel values at fixed three‐dimensional locations (a component “map”), and a unique associated time course of activation. Given data from 144 time points collected during a 6‐min trial, ICA extracted an equal number of spatially independent components. In all eight trials, ICA derived one and only one component with a time course closely matching the time course of 40‐sec alternations between experimental and control tasks. The regions of maximum activity in these consistently task‐related components generally overlapped active regions detected by standard correlational analysis, but included frontal regions not detected by correlation. Time courses of other ICA components were transiently task‐related, quasiperiodic, or slowly varying. By utilizing higher‐order statistics to enforce successively stricter criteria for spatial independence between component maps, both the ICA algorithm and a related fourth‐order decomposition technique (Comon [1994]: Signal Processing 36:11–20) were superior to principal component analysis (PCA) in determining the spatial and temporal extent of task‐related activation. For each subject, the time courses and active regions of the task‐related ICA components were consistent across trials and were robust to the addition of simulated noise. Simulated movement artifact and simulated task‐related activations added to actual fMRI data were clearly separated by the algorithm. ICA can be used to distinguish between nontask‐related signal components, movements, and other artifacts, as well as consistently or transiently task‐related fMRI activations, based on only weak assumptions about their spatial distributions and without a priori assumptions about their time courses. ICA appears to be a highly promising method for the analysis of fMRI data from normal and clinical populations, especially for uncovering unpredictable transient patterns of brain activity associated with performance of psychomotor tasks.  Brain Mapping 6:160–188, 1998.
Local Filter Selection Boosts Performance of Automatic Speechreading

We examine general purpose unsupervised techniques for visual preprocesing in machine vision tasks. In particular we analyze a wide variety of principal component and independent component techniques in combination with stepwise regression methods for variable selection. The task at hand is recognition of the first four digits spoken in English using hidden Markov models (HMM) for the recognition system. Local representations consistently outperformed global representations in generalizing to new speakers while global representations performed better than local ones for speaker identification tasks. In addition, the use of a novel regression-based variable selection technique substantially boosted performance.
COMPUTATIONAL MODELS CONSTRAINED BY VOLTAGE-CLAMP DATA FOR INVESTIGATING DENDRITIC CURRENTS

We have investigated dendritic currents using a combination of experimental and modeling approaches. Voltage-clamp recordings were obtained in neurons with and without dendrites (intact vs. dissociated cells), and the recordings were compared to simulations that incorporated the corresponding dendritic morphology. This approach was applied to thalamic reticular cells to examine dendritic location of the low-threshold calcium current (ITs). The simulations matched the recordings only if ITs was located in distal dendrites with high density. Moreover, dendritic ITs was also necessary to reproduce the typical responses of these cells seen in intracellular recordings in vivo. This approach can be used to investigate other types of neurons with active dendritic currents.  Thalamic reticular (RE) cells typically re burst discharges, which play a central role in the genesis of various types of synchronized sleep oscillations and epileptic behavior (Steriade et al., 1993). A low-threshold Ca current, ITs, underlies the production of these bursts (Bal and McCormick,1993; Huguenard and Prince 1992).
Cellular and network models for intrathalamic augmenting responses during 10-Hz stimulation.

Repetitive stimulation of the thalamus at 7-14 Hz evokes responses of increasing amplitude in the thalamus and the areas of the neocortex to which the stimulated foci project. Possible mechanisms underlying the thalamic augmenting responses during repetitive stimulation were investigated with computer models of interacting thalamocortical (TC) and thalamic reticular (RE) cells. The ionic currents in these cells were modeled with Hodgkin-Huxley type of kinetics, and the results of the model were compared with in vivo thalamic recordings from decorticated cats. The simplest network model demonstrating an augmenting response was a single pair of coupled RE and TC cells, in which RE-induced inhibitory postsynaptic potentials (IPSPs) in the TC cell led to progressive deinactivation of a low-threshold Ca2+ current. The augmenting responses in two reciprocally interacting chains of RE and TC cells depended also on gamma-aminobutyric acid-B (GABAB) IPSPs. Lateral GABAA inhibition between identical RE cells, which weakened bursts in these cells, diminished GABAB IPSPs and delayed the augmenting response in TC cells. The results of these simulations show that the interplay between existing mechanisms in the thalamus explains the basic properties of the intrathalamic augmenting responses.
Submicrosecond pacemaker precision is behaviorally modulated: the gymnotiform electromotor pathway.

What are the limits and modulators of neural precision? We address this question in the most regular biological oscillator known, the electric organ command nucleus in the brainstem of wave-type gymnotiform fish. These fish produce an oscillating electric field, the electric organ discharge (EOD), used in electrolocation and communication. We show here that the EOD precision, measured by the coefficient of variation (CV = SD/mean period) is as low as 2 x 10(-4) in five species representing three families that range widely in species and individual mean EOD frequencies (70-1,250 Hz). Intracellular recording in the pacemaker nucleus (Pn), which commands the EOD cycle by cycle, revealed that individual Pn neurons of the same species also display an extremely low CV (CV = 6 x 10(-4), 0.8 micro sec SD). Although the EOD CV can remain at its minimum for hours, it varies with novel environmental conditions, during communication, and spontaneously. Spontaneous changes occur as abrupt steps (250 ms), oscillations (3-5 Hz), or slow ramps (10-30 s). Several findings suggest that these changes are under active control and depend on behavioral state: mean EOD frequency and CV can change independently; CV often decreases in response to behavioral stimuli; and lesions of one of the two inputs to the Pn had more influence on CV than lesions of the other input.
Combining time-delayed decorrelation and ICA: towards solving the cocktail party problem

We present methods to separate blindly mixed signals recorded in a room. The learning algorithm is based on the information maximization in a single layer neural network. We focus on the implementation of the learning algorithm and on issues that arise when separating speakers in room recordings. We used an infomax approach in a feedforward neural network implemented in the frequency domain using the polynomial filter matrix algebra technique. A fast convergence speed was achieved by using a time-delayed decorrelation method as a preprocessing step. Under minimum-phase mixing conditions this preprocessing step was sufficient for the separation of signals. These methods successfully separated a recorded voice with music in the background (cocktail party problem). Finally, we discuss problems that arise in real world recordings and their potential solutions.
Objective Functions for Neural Map Formation

A unifying framework for analyzing models of neural map formation is presented based on growth rules derived from objective functions and normalization rules derived from constraint functions. Coordinate transformations play an important role in deriving various rules from the same function. Ten different models from the literature are classified within the objective function framework presented here. Though models may look different, they may actually be equivalent in terms of their stable solutions. The techniques used in this analysis may also be useful in investigating other types of neural dynamics.
Spatiotemporal Patterns of Spindle Oscillations in Cortex and Thalamus

Spindle oscillations (7–14 Hz) appear in the thalamus and cortex during early stages of sleep. They are generated by the combination of intrinsic properties and connectivity patterns of thalamic neurons and distributed to cortical territories by thalamocortical axons. The corticothalamic feedback is a major factor in producing coherent spatiotemporal maps of spindle oscillations in widespread thalamic territories. Here we have investigated the spatiotemporal patterns of spontaneously occurring and evoked spindles by means of multisite field potential and unit recordings in intact cortex and decorticated animals. We show that (1) spontaneous spindle oscillations are synchronized over large cortical areas during natural sleep and barbiturate anesthesia; (2) under barbiturate anesthesia, the cortical coherence is not disrupted by transection of intracortical synaptic linkages; (3) in intact cortex animals, spontaneously occurring barbiturate spindle sequences occur nearly simultaneously over widespread thalamic territories; (4) in the absence of cortex, the spontaneous spindle oscillations throughout the thalamus are less organized, but the local coherence (within 2–4 mm) is still maintained; and (5) spindling propagation is observed in intact cortex animals only when elicited by low intensity cortical stimulation, applied shortly before the initiation of a spontaneous spindle sequence; propagation velocities are between 1 and 3 mm/sec, measured in the anteroposterior axis of the thalamus; increasing the intensity of cortical stimulation triggers spindle oscillations, which start simultaneously in all leads. We propose that, in vivo, the coherence of spontaneous spindle oscillations in corticothalamic networks is attributable to the combined action of continuous background corticothalamic input initiating spindle sequences in several thalamic sites at the same time and divergent corticothalamic and intrathalamic connectivity.
Objective Functions for Topography: A Comparison of Optimal Maps

Topographic mappings are important in several contexts, including data visualization, connectionist representation, and cortical structure. Many different ways of quantifying the degree of topography of a mapping have been proposed. In order to investigate the consequences of the varying assumptions that these different approaches embody, we have optimized the mapping with respect to a number of different measures for a very simple problem — the mapping from a square to a line. The principal results are that (1) different objective functions can produce very different maps, (2) only a small number of these functions produce mappings which match common intuitions as to what a topographic mapping “should” actually look like for this problem, (3) the objective functions can be put into certain broad categories based on the overall form of the maps, and (4) certain categories of objective functions may be more appropriate for particular types of problem than other categories.
Estimating alertness from the EEG power spectrum

In tasks requiring sustained attention, human alertness varies on a minute time scale. This can have serious consequences in occupations ranging from air traffic control to monitoring of nuclear power plants. Changes in the electroencephalographic (EEG) power spectrum accompany these fluctuations in the level of alertness, as assessed by measuring simultaneous changes in EEG and performance on an auditory monitoring task. By combining power spectrum estimation, principal component analysis and artificial neural networks, the authors show that continuous, accurate, noninvasive, and near real-time estimation of an operator's global level of alertness is feasible using EEC; measures recorded from as few as two central scalp sites. This demonstration could lead to a practical system for noninvasive monitoring of the cognitive state of human operators in attention-critical settings.
Woods Hole Workshop on Computational Neuroscience

The Woods Hole Workshop on Computational Neuroscience was held at the Marine Biological Laboratory on August 26 to August 31, 1996. Twenty-two investigators attended the workshop on the computational functions of nervous systems. The topics in computational neuroscience that were discussed included neural systems for learning and long term memory, non-classical responses of cortical neurons, and active perception. In addition, some members of the workshop lectured in the concurrent Computational Neuroscience Course at the Marine Biological Laboratory, and students taking the course were invited to attend the workshop. The Workshop was highly successful at bringing together experimentalists and computational neuroscientists for intense discussions at the forefront of computational neuroscience.
The "Independent Components" of Scenes are Edge Filters

It has previously been suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and it has been reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that a new unsupervised learning algorithm based on information maximization, a nonlinear "infomax" network, when applied to an ensemble of natural scenes produces sets of visual filters that are localized and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maxim ization network. In addition, the outputs of these filters are as independent as possible, since this infomax network performs Independent Components Analysis or ICA, for sparse (super-gaussian) component distributions. We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form a natural, information-theoretic coordinate system for natural images.  Information theory Independent components Neural network learning
The Year of the Dendrite

Octopus-like neurons, the cells of the brain, typically have many input “arms”—the dendrites, which receive signals from other neurons—and one output arm, the axon. Two new reports in this week's Science show that the dendrites process information in a manner even more complex than expected. In his Perspective, Sejnowski explains how the observations of Magee and Johnston (p. 209) and Markram et al. (p. 213) on backpropagating action potentials illuminate the mechanism of Hebbian learning, the synaptic strengthening that occurs when a receiving neuron is active at the same time that a signal comes in.
A unifying measure for neighbourhood preservation in topographic mappings

In this paper, the abstract computational principles underlying topographic maps are discussed. We give a definition of a “perfectly neighbourhood preserving” map, which we call a topographic homeomorphism, and we prove that this has certain desirable properties. It is argued that when a topographic homeomorphism does not exist (the usual case), many equally valid choices are available for quantifying the quality of a map. We introduce a particular measure that encompasses several previous proposals, and discuss its relation to other work. This formulation of the problem sets it within the well-known class of quadratic assignment problems.