nils-herrmann/clean_scirepeval huggingface dataset:
Independent Components of Magnetoencephalography: Localization

We applied second-order blind identification (SOBI), an independent component analysis method, to MEG data collected during cognitive tasks. We explored SOBI's ability to help isolate underlying neuronal sources with relatively poor signal-to-noise ratios, allowing their identification and localization. We compare localization of the SOBI-separated components to localization from unprocessed sensor signals, using an equivalent current dipole modeling method. For visual and somatosensory modalities, SOBI preprocessing resulted in components that can be localized to physiologically and anatomically meaningful locations. Furthermore, this preprocessing allowed the detection of neuronal source activations that were otherwise undetectable. This increased probability of neuronal source detection and localization can be particularly beneficial for MEG studies of higher-level cognitive functions, which often
Algorithmic Differentiation, Functional Programming, and Iterate-to-Fixedpoint

Algorithmic differentiation (AD) transforms straight lin e numeric code so that it calculates the derivative of the function originally calculated (Corliss et al., 2001). The re are two varieties of AD: forward mode (Wengert, 1964), in which the transformed program calculates the product of the Jacobian of the original program with an input vector, and reverse mode (Speelpenning, 1980; Rall, 1981; Rumelhart et al., 1986), in which it calculates the product of the transpose of the Jacobian with an input vector. The functions produced by the forward and reverse mode AD of a functionf can be characterized in terms of the standard derivativeDf x = (dfi(x)/dxj)ij
Fast robust MEG source localization using MLPs

Source localization from MEG data in real time requires algorithms which are robust, fully automatic, and very fast. We present two neural network systems which are able to localize a single dipole to reasonable accuracy within a fraction of a millisecond, even when the signals are contaminated by considerable noise. The first network is a multilayer perceptron (MLP) which takes the sensor measurements as inputs, uses two hidden layers, and outputs source location in Cartesian coordinates. After training with random dipolar sources contaminated by real noise, localization of a single dipole could be performed within 300 microseconds on an 800 Mhz Athlon workstation, with an average localization error of 1.15 cm. To improve the accuracy to 0.28 cm, one can apply a few iterations of conventional Levenberg-Marquardt (LM) minimization using the MLP output as the initial guess. The combined method is about twenty times faster than ultistart LM localization with comparable accuracy. In a second network with only one hidden layer, the outputs were the amplitudes of 193 evenly distributed Gaussian functions holding a soft distributed representation of the dipole location. We trained this network on dipolar sources with real noise, and externally converted the network’s output into an explicit Cartesian coordinate representation of the dipole location. This new network had an improved localization accuracy of 0.87 cm, while localization time was lengthened to about 800 microseconds.
Fast accurate MEG source localization using a multilayer perceptron trained with real brain noise.

Iterative gradient methods such as Levenberg-Marquardt (LM) are in widespread use for source localization from electroencephalographic (EEG) and magnetoencephalographic (MEG) signals. Unfortunately, LM depends sensitively on the initial guess, necessitating repeated runs. This, combined with LM's high per-step cost, makes its computational burden quite high. To reduce this burden, we trained a multilayer perceptron (MLP) as a real-time localizer. We used an analytical model of quasistatic electromagnetic propagation through a spherical head to map randomly chosen dipoles to sensor activities according to the sensor geometry of a 4D Neuroimaging Neuromag-122 MEG system, and trained a MLP to invert this mapping in the absence of noise or in the presence of various sorts of noise such as white Gaussian noise, correlated noise, or real brain noise. A MLP structure was chosen to trade off computation and accuracy. This MLP was trained four times, with each type of noise. We measured the effects of initial guesses on LM performance, which motivated a hybrid MLP-start-LM method, in which the trained MLP initializes LM. We also compared the localization performance of LM, MLPs, and hybrid MLP-start-LMs for realistic brain signals. Trained MLPs are much faster than other methods, while the hybrid MLP-start-LMs are faster and more accurate than fixed-4-start-LM. In particular, the hybrid MLP-start-LM initialized by a MLP trained with the real brain noise dataset is 60 times faster and is comparable in accuracy to random-20-start-LM, and this hybrid system (localization error: 0.28 cm, computation time: 36 ms) shows almost as good performance as optimal-1-start-LM (localization error: 0.23 cm, computation time: 22 ms), which initializes LM with the correct dipole location. MLPs trained with noise perform better than the MLP trained without noise, and the MLP trained with real brain noise is almost as good an initial guesser for LM as the correct dipole location.
Independent Components of Magnetoencephalography: Single-Trial Response Onset Times

Werecently demonstrated that second-order blind identification (SOBI), an independent component analysis (ICA) method, can separate the mixture of neuronal and noise signals in magnetoencephalographic (MEG) data into neuroanatomically and neurophysiologically meaningful components. When the neuronal signals had relatively higher trial-to-trial variability, SOBI offered a particular advantage in identifying and localizing neuronal source activations with increased source detectability (A. C. Tang et al., 2002, Neural Comput. 14, 1827-1858). Here, we explore the utility of SOBI in the analysis of temporal aspects of neuromagnetic signals from MEG data. From SOBI components, we were able to measure single-trial response onset times of neuronal populations in visual, auditory, and somatosensory modalities during cognitive and sensory activation tasks, with a detection rate as high as 96% under optimal conditions. Comparing the SOBI-aided detection results with those obtained directly from the sensors, we found that with SOBI preprocessing, we were able to measure, among a greater proportion of trials, single-trial response onset times that are above background neuronal activity. We suggest that SOBI ICA can improve our current capability in measuring single-trial responses from human subjects using the noninvasive brain imaging method MEG.
Linear Spatial Integration for Single-Trial Detection in Encephalography

Conventional analysis of electroencephalography (EEG) and magnetoencephalography (MEG) often relies on averaging over multiple trials to extract statistically relevant differences between two or more experimental conditions. In this article we demonstrate single-trial detection by linearly integrating information over multiple spatially distributed sensors within a predefined time window. We report an average, single-trial discrimination performance of Az approximately 0.80 and faction correct between 0.70 and 0.80, across three distinct encephalographic data sets. We restrict our approach to linear integration, as it allows the computation of a spatial distribution of the discriminating component activity. In the present set of experiments the resulting component activity distributions are shown to correspond to the functional neuroanatomy consistent with the task (e.g., contralateral sensorymotor cortex and anterior cingulate). Our work demonstrates how a purely data-driven method for learning an optimal spatial weighting of encephalographic activity can be validated against the functional neuroanatomy.
Realtime MEG source localization

Iterative gradient methods like Levenberg-Marquardt (LM) are in widespread use for source localization from electroencephalographic (EEG) and magnetoencephalographic (MEG) signals. Unfortunately LM depends sensitively on the initial guess, particularly (and counterintuitively) at higher signal-to-noise ratios, necessitating repeated runs. This, combined with LM's high per-step cost, makes its computational burden quite high. To reduce this burden, we trained a multilayer perceptron (MLP) as a real-time localizer. We used an analytical model of quasistatic electromagnetic propagation through the head to map randomly chosen dipoles to sensor activities, and trained an MLP to invert this mapping in the presence of various sorts of noise. With realistic noise, our MLP is about five hundred times faster than n-start-LM with n = 4 to match accuracies, while our hybrid MLP-start-LM is about four times more accurate and thirteen times faster than 4-start-LM.
Realtime MEG source localization with realistic noise

Iterative gradient methods like Levenberg-Marquardt (LM) are in widespread use for source localization from electroenceph alographic (EEG) and magnetoencephalographic (MEG) signals. Unfortu nately LM depends sensitively on the initial guess, particularly ( and counterintuitively) at higher signal-to-noise ratios, necessita ting repeated runs. This, combined with LM’s high per-step cost, makes its compu tational burden quite high. To reduce this burden, we trained a multil ayer perceptron (MLP) as a real-time localizer. We used an analytica l model of quasistatic electromagnetic propagation through the he ad to map randomly chosen dipoles to sensor activities, and trained an ML P to invert this mapping in the presence of various sorts of noise. With realistic noise, our MLP is about five hundred times faster than n-start-LM with n = 4 to match accuracies, while our hybrid MLP-start-LM is about four times more accurate and thirteen times faster than 4-startLM.
Multiscale blind source separation

The concern of the blind source separation problem is to extract the underlying source signals from a set of their linear mixtures, where the mixing matrix is unknown. It was discovered recently, that use of sparsity of source representation in some signal dictionary dramatically improves the quality of separation. In this work we use the property of multiscale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. Experiments with simulated signals, musical sounds and images demonstrate significant improvement of separation quality.
Blind Source Separation via Multinode Sparse Representation

We consider a problem of blind source separation from a set of instantaneous linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in an appropriate representation according to some signal dictionary, dramatically improves the quality of separation. In this work we use the property of multiscale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. The performance of the algorithm is verified on noise-free and noisy data. Experiments with simulated signals, musical sounds and images demonstrate significant improvement of separation quality over previously reported results.
Blind Source Separation by Sparse Decomposition in a Signal Dictionary

The blind source separation problem is to extract the underlying source signals from a set of linear mixtures, where the mixing matrix is unknown. This situation is common in acoustics, radio, medical signal and image processing, hyperspectral imaging, and other areas. We suggest a two-stage separation process: a priori selection of a possibly overcomplete signal dictionary (for instance, a wavelet frame or a learned dictionary) in which the sources are assumed to be sparsely representable, followed by unmixing the sources by exploiting the their sparse representability. We consider the general case of more sources than mixtures, but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures. Experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques.
Localization of Independent Components from Magnetoencephalography

Blind source separation (BSS) decomposes a multidimensional time series into a set of sources, each with a one-dimensional time course and a xed spatial distribution. For EEG and MEG, the former corresponds to the simultaneously separated and temporally overlapping signals for continuous non-averaged data; the latter corresponds to the set of attenuations from the sources to the sensors. These sensor projection vectors give information on the spatial locations of the sources. Here we use standard Neuromag dipole-tting software to localize BSS-separated components of MEG data collected in several tasks in which visual, auditory, and somatosensory stimuli all play a role. We found that BSS-separated components with stimulusor motor-locked responses can be localized to physiological and anatomically meaningful locations within the brain.
Differentiating Functions of the Jacobianwith Respect to the Weights

For many problems, the correct behavior of a model depends not only on 
its input-output mapping but also on properties of its Jacobian matrix, the 
matrix of partial derivatives of the model's outputs with respect to its inputs. 
We introduce the J-prop algorithm, an efficient general method for 
computing the exact partial derivatives of a variety of simple functions of 
the Jacobian of a model with respect to its free parameters. The algorithm 
applies to any parametrized feedforward model, including nonlinear regression, 
multilayer perceptrons, and radial basis function networks.
Blind separation of sources with sparse representations in a given signal dictionary

The blind source separation problem is to extract the underlying source signals from a set of linear mixtures, where the mixing matrix is unknown. We consider a two-stage separation process. First, a priori selection of a possibly overcomplete signal dictionary (e.g. wavelet frame, learned dictionary, etc.) in which the sources are assumed to be sparsely representable. Second, unmixing the sources by exploiting the their sparse representability. We consider the general case of more sources than mixtures. But also derive a more efficient algorithm in the case of a non-overcomplete dictionary and equal numbers of sources and mixtures. Experiments with artificial signals and with musical sounds demonstrate significantly better separation than other known techniques.
Blind source separation by sparse decomposition

The blind source separation problem is to extract the underlying source signals from a set of their linear mixtures, where the mixing matrix is unknown. This situation is common, eg in acoustics, radio, and medical signal processing. We exploit the property of the sources to have a sparse representation in a corresponding signal dictionary. Such a dictionary may consist of wavelets, wavelet packets, etc., or be obtained by learning from a given family of signals. Starting from the maximum a posteriori framework, which is applicable to the case of more sources than mixtures, we derive a few other categories of objective functions, which provide faster and more robust computations, when there are an equal number of sources and mixtures. Our experiments with artificial signals and with musical sounds demonstrate significantly better separation than other known techniques.
Visualizing communication between neurons in the lamina ganglionaris of Musca domestica

A simplified model of the fly's early visual processing system was simulated. The model contains coupled nonlinear differential equations, increasing the complexity of software and its computational burden. An extension of the model to large arrays of retinal cartridges required high performance computing which further increased the complexity. Conventional visualization methods limited software development and model analysis. As a solution, more sophisticated visualization tools, such as IBM's open visualization data explorer and UNM's Flatland, were employed. These tools expedited software debugging and development. The accuracy and added insight afforded by these visualization tools provided a more natural environment for simulation analysis.
An MEG Study of Response Latency and Variability in the Human Visual System During a Visual-Motor Integration Task

Human reaction times during sensory-motor tasks vary considerably. To begin to understand how this variability arises, we examined neuronal populational response time variability at early versus late visual processing stages. The conventional view is that precise temporal information is gradually lost as information is passed through a layered network of mean-rate "units." We tested in humans whether neuronal populations at different processing stages behave like mean-rate "units". A blind source separation algorithm was applied to MEG signals from sensory-motor integration tasks. Response time latency and variability for multiple visual sources were estimated by detecting single-trial stimulus-locked events for each source. In two subjects tested on four visual reaction time tasks, we reliably identified sources belonging to early and late visual processing stages. The standard deviation of response latency was smaller for early rather than late processing stages. This supports the hypothesis that human populational response time variability increases from early to late visual processing stages.
Differentiating Functions of the Jacobian with Respect to the Weights

For many problems, the correct behavior of a model depends not only on its input-output mapping but also on properties of its Jacobian matrix, the matrix of partial derivatives of the model's outputs with respect to its inputs. We introduce the J-prop algorithm, an efficient general method for computing the exact partial derivatives of a variety of simple functions of the Jacobian of a model with respect to its free parameters. The algorithm applies to any parametrized feedforward model, including nonlinear regression, multilayer perceptrons, and radial basis function networks.
Detecting intrusions using system calls: alternative data models

Intrusion detection systems rely on a wide variety of observable data to distinguish between legitimate and illegitimate activities. We study one such observable-sequences of system calls into the kernel of an operating system. Using system-call data sets generated by several different programs, we compare the ability of different data modeling methods to represent normal behavior accurately and to recognize intrusions. We compare the following methods: simple enumeration of observed sequences; comparison of relative frequencies of different sequences; a rule induction technique; and hidden Markov models (HMMs). We discuss the factors affecting the performance of each method and conclude that for this particular problem, weaker methods than HMMs are likely sufficient.
Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm

This paper first describes the structure and results of the Abbadingo One DFA Learning Competition. The competition was designed to encourage work on algorithms that scale well—both to larger DFAs and to sparser training data. We then describe and discuss the winning algorithm of Rodney Price, which orders state merges according to the amount of evidence in their favor. A second winning algorithm, of Hugues Juille, will be described in a separate paper.
Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA

In the square linear blind source separation problem, one must find a linear unmixing operator which can detangle the result xi(t) of mixing n unknown independent sources si(t) through an unknown n × n mixing matrix A(t) of causal linear filters: xi = Σj aij * sj. We cast the problem as one of maximum likelihood density estimation, and in that framework introduce an algorithm that searches for independent components using both temporal and spatial cues. We call the resulting algorithm "Contextual ICA," after the (Bell and Sejnowski 1995) Infomax algorithm, which we show to be a special case of cICA. Because cICA can make use of the temporal structure of its input, it is able separate in a number of situations where standard methods cannot, including sources with low kurtosis, colored Gaussian sources, and sources which have Gaussian histograms.
VC Dimension of an Integrate-and-Fire Neuron Model

We compute the VC dimension of a leaky integrate-and-fire neuron model. The VC dimension quantifies the ability of a function class to partition an input pattern space, and can be considered a measure of computational capacity. In this case, the function class is the class of integrate-and-fire models generated by varying the integration time constant T and the threshold θ, the input space they partition is the space of continuous-time signals, and the binary partition is specified by whether or not the model reaches threshold at some specified time. We show that the VC dimension diverges only logarithmically with the input signal bandwidth N. We also extend this approach to arbitrary passive dendritic trees. The main contributions of this work are (1) it offers a novel treatment of computational capacity of this class of dynamic system; and (2) it provides a framework for analyzing the computational capabilities of the dynamic systems defined by networks of spiking neurons.
An investigation of the gradient descent process in neural networks

Usually gradient descent is merely a way to find a minimum, abandoned if a more efficient technique is available. Here we investigate the detailed properties of the gradient descent process, and the related topics of how gradients can be computed, what the limitations on gradient descent are, and how the second-order information that governs the dynamics of gradient descent can be probed. To develop our intuitions, gradient descent is applied to a simple robot arm dynamics compensation problem, using backpropagation on a temporal windows architecture. The results suggest that smooth filters can be easily learned, but that the deterministic gradient descent process can be slow and can exhibit oscillations. Algorithms to compute the gradient of recurrent networks are then surveyed in a general framework, leading to some unifications, a deeper understanding of recurrent networks, and some algorithmic extensions. By regarding deterministic gradient descent as a dynamic system we obtain results concerning its convergence, and a quantitative theory of its behavior when a saturating or "robust" error measure is used. Since second-order structure enters into the dynamics, an efficient exact technique for multiplying a vector by the Hessian of a gradient system is derived and applied.