nils-herrmann/clean_scirepeval huggingface dataset:
An Augmented Echo State Network for Nonlinear Adaptive Filtering of Complex Noncircular Signals

A novel complex echo state network (ESN), utilizing full second-order statistical information in the complex domain, is introduced. This is achieved through the use of the so-called augmented complex statistics, thus making complex ESNs suitable for processing the generality of complex-valued signals, both second-order circular (proper) and noncircular (improper). Next, in order to deal with nonstationary processes with large nonlinear dynamics, a nonlinear readout layer is introduced and is further equipped with an adaptive amplitude of the nonlinearity. This combination of augmented complex statistics and enhanced adaptivity within ESNs also facilitates the processing of bivariate signals with strong component correlations. Simulations in the prediction setting on both circular and noncircular synthetic benchmark processes and real-world noncircular and nonstationary wind signals support the analysis.
Adaptive filtering based on extended kernel recursive maximum correntropy

In this paper, an adaptive filtering algorithm, termed the extended kernel recursive maximum correntropy (EX-KRMC) algorithm is proposed as a novel approach of traditional recursion based adaptive filtering algorithms. Maximum correntropy criterion is employed to better the robustness to non-Gaussian noise and kernel methods are used to enable the capacity for nonlinear systems. It is verified by simulation experiments that EX-KRMC outperforms existing adaptive filtering algorithms when dealing with non-Gaussian noise for nonlinear time-variant systems.
Asynchronous biphasic pulse signal coding and its CMOS realization

An asynchronous biphasic pulse train signal representation mechanism is proposed for low bandwidth, low power sensor applications. We prove that this novel pulse coding method provides lossless encoding theoretically. We also present a CMOS circuit realization using the AMI 0.6 mum process. Cadence simulation results show that the reconstructed signal has 8 effective bits of resolution with less than 100 muW of total power consumption
Guided Policy Exploration for Markov Decision Processes Using an Uncertainty-Based Value-of-Information Criterion

Reinforcement learning in environments with many action–state pairs is challenging. The issue is the number of episodes needed to thoroughly search the policy space. Most conventional heuristics address this search problem in a stochastic manner. This can leave large portions of the policy space unvisited during the early training stages. In this paper, we propose an uncertainty-based, information-theoretic approach for performing guided stochastic searches that more effectively cover the policy space. Our approach is based on the value of information, a criterion that provides the optimal tradeoff between expected costs and the granularity of the search process. The value of information yields a stochastic routine for choosing actions during learning that can explore the policy space in a coarse to fine manner. We augment this criterion with a state-transition uncertainty factor, which guides the search process into previously unexplored regions of the policy space. We evaluate the uncertainty-based value-of-information policies on the games Centipede and Crossy Road. Our results indicate that our approach yields better performing policies in fewer episodes than stochastic-based exploration strategies. We show that the training rate for our approach can be further improved by using the policy cross entropy to guide our criterion’s hyperparameter selection.
Information transfer through classifiers and its relation to probability of error

Fano's (1961) bound identifies a lower bound for the classification error probability and indicates how the information transfer through classifier affects its performance. It was an important step towards linking the information theory and pattern recognition. In this paper, a family of lower bounds is derived using Renyi's entropy, which yields Fano's lower bound as a special case. Using a different set of entropy orders, Renyi's definition also allows the construction a family of upper bounds for the probability of error. This is impossible using Shannon's definition of entropy. Further analysis to obtain the tightest lower and upper bounds revealed the fact that Fano's bound is indeed the tightest lower bound, and the upper bounds become tighter as the entropy order approaches to one from below. Numerical evaluations of the bounds are presented for three digital modulation schemes under AWGN channel.
Exploring the time-frequency microstructure of speech for blind source separation

This paper explores the different frequency contents in short time segments (temporal microstructure) of speech to identify the mixing matrix in blind source separation. We propose a new method based on the eigenspread in different frequency bands to identify the segments which contain only one of the mixtures. It is much simpler to accurately estimate the mixing matrices from these segments. This short-time subband analysis trains very fast and estimates reliably the column vectors of the linear mixture. Simulation results show that our proposed method outperforms the existing model-based and competitive learning approaches in the identification of the mixing matrix for both sensor-sufficient (as many sensors as sources) and sensor-deficient (less sensors than sources) cases.
Learning multiscale neural metrics via entropy minimization

In order to judiciously compare neural responses between repeated trials or stimuli, a well-suited distance metric is necessary. With multi-electrode recordings, a neural response is a spatiotemporal pattern, but not all of the dimensions of space and time should be treated equally. In order to understand which dimensions of the input are more discriminative and to improve the classification performance, we propose a metric-learning approach that can be used across scales. This extends previous work that used a linear projection into lower dimensional space; here, multiscale metrics or kernels are learned as the weighted combinations of different metrics or kernels on each of the neural response's dimensions. Preliminary results are explored on a cortical recording of a rat during a tactile stimulation experiment. Metrics on both local field potential and spiking data are explored. The learned weights reveal important dimensions of the response, and the learned metrics improve nearest-neighbor classification performance.
Wireless transmission of neuronal recordings using a portable real-time discrimination/compression algorithm

A design challenge of portable wireless neural recording systems is the tradeoff between bandwidth and power consumption. This paper investigates the compression of neuronal recordings in real-time using a novel discriminating Linde-Buzo-Gray algorithm (DLBG) that preserves spike shapes while filtering background noise. The technique is implemented in a low power digital signal processor (DSP) which is capable of wirelessly transmitting raw neuronal recordings. Depending on the signal to noise ratio of the recording, the compression ratio can be tailored to the data to maximally preserve power and bandwidth. The approach was tested in real and synthetic data and achieved compression ratios between 184:1 and 10:1.
Learning mappings in brain machine interfaces with echo state networks

Brain machine interfaces (BMI) utilize linear or non-linear models to map the neural activity to the associated behavior which is typically the 2D or 3D hand position of a primate. Linear models are plagued by the massive disparity of the input and output dimensions thereby leading to poor generalization. A solution would be to use non-linear models like the recurrent multi-layer perceptron (RMLP) that provide parsimonious mapping functions with better generalization. However, this results in a drastic increase in the training complexity, which can be critical for practical use of a BMI. This paper bridges the gap between superior performance per trained weight and model learning complexity. Towards this end, we propose to use echo state networks (ESN) to transform the neuronal firing activity into a higher dimensional space and then derive an optimal sparse linear mapping in the transformed space to match the hand position. The sparse mapping is obtained using a weight constrained cost function whose optimal solution is determined using a stochastic gradient algorithm.
An information-theoretic approach to motor action decoding with a reconfigurable parallel architecture

Methods for decoding movements from neural spike counts using adaptive filters often rely on minimizing the mean-squared error. However, for non-Gaussian distribution of errors, this approach is not optimal for performance. Therefore, rather than using probabilistic modeling, we propose an alternate non-parametric approach. In order to extract more structure from the input signal (neuronal spike counts) we propose using minimum error entropy (MEE), an information-theoretic approach that minimizes the error entropy as part of an iterative cost function. However, the disadvantage of using MEE as the cost function for adaptive filters is the increase in computational complexity. In this paper we present a comparison between the decoding performance of the analytic Wiener filter and a linear filter trained with MEE, which is then mapped to a parallel architecture in reconfigurable hardware tailored to the computational needs of the MEE filter. We observe considerable speedup from the hardware design. The adaptation of filter weights for the multiple-input, multiple-output linear filters, necessary in motor decoding, is a highly parallelizable algorithm. It can be decomposed into many independent computational blocks with a parallel architecture readily mapped to a field-programmable gate array (FPGA) and scales to large numbers of neurons. By pipelining and parallelizing independent computations in the algorithm, the proposed parallel architecture has sublinear increases in execution time with respect to both window size and filter order.
Spike detection using a syntactic pattern recognition approach

Syntactic methods were used to detect spikes utilizing time-domain features (slope and duration). The difference between this and earlier work is that here rules to discriminate each subject's spikes are automatically inferred from a training set consisting of sample spike waveforms obtained using expert knowledge, whereas the previous work used expert human knowledge to determine a set of universal rules. Therefore, the intersubject spike variability is accommodated in the design, without jeopardizing objectivity. The structural combination of the features is used by the syntactic method, providing a strict, explicit set of rules by which the spikes are identified. As a result, a rule-based definition for each subject's spike is obtained.<<ETX>>
Content addressable memories in reproducing Kernel Hilbert spaces

Content addressable memories (CAM) are one of the few technologies that provide the capability to store and retrieve information based on content. Even more useful is their ability to recall data from noisy or incomplete inputs. However, the input data dimension limits the amount of data that CAMs can store and successfully retrieve. We propose to increase the amount of information that can be stored by implementing CAMs in a reproducing kernel Hilbert space where the input dimension is practically infinite, effectively lifting this CAM limitation. We show the advantages of kernel CAMs over CAMs by comparing their performance in information retrieval, generalization, storage, and online learning.
Spatio-Temporal Clustering of Epileptic ECOG

The spatio-temporal mechanisms underlying the generation of epileptic seizures is not yet clearly understood. In this study, we attempt to quantify the spatio-temporal interactions of an epileptic brain by using a previously proposed SOM-based similarity index (SI) measure. We further show that spectral clustering approach can be appropriately used to determine the average spatial mappings in the brain at different stages of a seizure, by interpreting the SOM-SI values as affinity matrices. Results involving two pairs of seizures of an epileptic patient suggest that there may not be a regular pattern associated with channels's spatio-temporal dynamics during the inter-ictal to pre-post ictal transition
Kernel Based Synthetic Discriminant Function for Object Recognition

In this paper a non-linear extension to the synthetic discriminant function (SDF) is proposed. The SDF is a well known 2-D correlation filter for object recognition. The proposed nonlinear version of the SDF is derived from kernel-based learning. The kernel SDF is implemented in a nonlinear high dimensional space by using the kernel trick and it can improve the performance of the linear SDF by incorporating the image's class higher order moments. We show that this kernelized composite correlation filter has an intrinsic connection with the recently proposed correntropy function. We apply this kernel SDF to face recognition and simulations show that the kernel SDF significantly outperforms the traditional SDF as well as is robust in noisy data environments
Integrate and fire circuit as an ADC replacement

We evaluate the performance of an asynchronous analog-to-digital converter known as the Integrate and Fire (I&F) circuit. The reconstruction algorithm for the sampler is presented along with the analytical results for the signal-to-noise ratio (SNR) of the reconstructed signal. We present the power vs. SNR tradeoff, based on the measurements from the chip fabricated in AMI 0.6 µm technology. The figure-of-merit (FOM) of the I&F sampler is 0.6 pJ/conversion, which is comparable to the reported FOM for other synchronous and asynchronous ADCs, even though the I&F is implemented in a much older technology as compared to the other ADCs.
Generalized anti-Hebbian learning for source separation

The information-theoretic framework for source separation is highly suitable. However the choice of the nonlinearity or the estimation of the multidimensional joint probability density function are nontrivial. We propose here a generalized Gaussian model to construct a generalized blind source separation network based on the minimum entropy principle. This new separation network can suppress the interference to a significant amount compared to the traditional LMS-echo-canceler. The simulation is given to show the disparity of the performance as a varies. Finally how to choose the appropriate a in our generalized anti-Hebbian rule is discussed.
A principal component network for generalized eigen-decomposition

This paper presents a novel principal component network with simple online local learning rules to obtain generalized eigenvalues and their corresponding eigenvectors of input data. The network is composed of a set of forward linear projections and a set of linear lateral inhibition connections between them. The rules for both forward and lateral connections are not only mathematically rooted but also local which confers them biological plausibility. Stationary points of the rules and their stability are analyzed. Simulations are given to verify, the validity and effectiveness of the proposed method.
A reconfigurable parallel acceleration platform for evaluation of permutation entropy

In recent years, permutation entropy is widely used to characterize the complexity of EEG time series and can be applied to predict the onset of serious brain diseases, such as the epileptic seizure. In many practical situations, the number of EEG time series that need to be analyzed simultaneously is very large, so the computation of the permutation entropy is time-consuming and should be accelerated so that the real-time analysis is possible. Noting that mathematical operations can be sped up effectively with hardware implementation, we design a parallel FPGA platform consisting of 128 reconfigurable pipelines, which are used to calculate the permutation entropy for a single EEG time series. When the platform works at 150MHz and the embedding dimension is 5, an average speedup of 5553 for different window sizes is achieved compared with C codes running on a 3GHz Intel(R) Core(TM) i5-2320 CPU. Meanwhile, the hardware cost is very low.
An analysis of the gamma memory in dynamic neural networks

Presents a vector space framework to study short-term memory filters in dynamic neural networks. The authors define parameters to quantify the function of feedforward and recursive linear memory filters. They show, using vector spaces, what is the optimization problem solved by the PEs of the first hidden layer of the single input focused network architecture. Due to the special properties of the gamma bases, recursion brings an extra parameter lambda (the time constant of the leaky integrator) that displaces the memory manifold towards the desired signal when the mean square error is minimized. In contrast, for the feedforward memory filter the angle between the desired signal and the memory manifold is fixed for a given memory order. The adaptation of the feedback parameter can be done using gradient descent, but the optimization is nonconvex.
Quantized Kernel Least Mean Square Algorithm

In this paper, we propose a quantization approach, as an alternative of sparsification, to curb the growth of the radial basis function structure in kernel adaptive filtering. The basic idea behind this method is to quantize and hence compress the input (or feature) space. Different from sparsification, the new approach uses the “redundant” data to update the coefficient of the closest center. In particular, a quantized kernel least mean square (QKLMS) algorithm is developed, which is based on a simple online vector quantization method. The analytical study of the mean square convergence has been carried out. The energy conservation relation for QKLMS is established, and on this basis we arrive at a sufficient condition for mean square convergence, and a lower and upper bound on the theoretical value of the steady-state excess mean square error. Static function estimation and short-term chaotic time-series prediction examples are presented to demonstrate the excellent performance.
TDAT-time domain analysis tool for EEG analysis

An interactive design and analysis tool for displaying and quantifying multiple channels of data is presented. The system allows one to visualize multiple data channels easily while simultaneously observing the effects of filters on the data and to evaluate signal detection algorithms. The software is designed for a workstation environment; it will find application in a variety of applications where one needs to visualize multiple data channels simultaneously. This time-domain analysis tool (TDAT) is used for the design and evaluation of filters and detection algorithms for electroencephalogram (EEG) waveforms, and serves as a prototype of a paperless system to be used by electroencephalographers. The general software structure of the system is described, and many of the system's features are illustrated.<<ETX>>
A Closed Form Solution for Multiple-Input Spike Based Adaptive Filters

Neurons are point process systems, in the sense that the inputs and output which are spike trains can be treated as point processes. System identification of a point process system has been studied mostly with single input. However, multiple input is required in many applications such as liquid state machines or neural prosthetics. We propose a simple multiple-input spike based adaptive filter which is based on an integrate-and-fire neuron model. The optimal closed solution is derived, and the performance is analyzed with respect to noise in various parameters and measurement.
A kernel based approach to maximum entropy mappings

We discuss a kernel based method for learning maximum entropy mappings from exemplars. Information theoretic signal processing has been examined by many authors. The method presented is related to the approaches of Linsker (1988, 1990), Bell and Sejnowski (see Neural Computation, vol.7, p.1129-59, 1995), and Viola et al. (see Neural Information Processing Systems, vol.8, p.851-7, 1995). We discuss the use of this method for deriving maximum entropy mappings in an unsupervised fashion. Extensions to optimizing mutual information are possible. The result of our approach is that maximizing and minimizing entropy for differentiable nonlinear mappings such as a multilayer perceptron can be accomplished through simple local interactions of the data in the output space. We present empirical results from application of the method to the problem of blind separation of linearly mixed speech sources. We compare our empirical results to the method of Bell and Sejnowski.
A biphasic integrate-and-fire system

A neuronal recording system for brain-machine interfaces (BMI) based on asynchronous biphasic pulse coding is described. It demonstrates the first step in the development of a completely implanted wireless solution with a fully integrated circuit architecture. A recording experiment comparing, in parallel, a commercial recording system (Tucker-Davis Technology (TDT)) and the UF's custom solution (FWIRE) is set up to compare performance. The novel aspect of the UF system is that the analog signal is represented by an asynchronous pulse train, which provides a low-power, low-bandwidth, noise-resistant means for coding and transmission. Taking advantage of neural firing features, the pulse-based approach uses only 3K pulses/second to record a 25 kHz bandwidth signal from a hardware neural simulator.
Pulse based signal processing for systolic peak recognition

Ultra low power wearable wireless patient monitoring systems will be critical for continuous remote monitoring of patients and fast transmission of data to medical personnel for timely intervention. This paper proposes a pulse based methodology for photoplethysmogram (PPG) feature discrimination using the time-based integrate and fire (IF) sampler for continuous, portable pulse oximetry. The analog PPG is transformed into a sequence of time events where the time between two events represents a constant area under the analog PPG signal, with injective mapping between the two domains. A simple and robust method is formulated that directly estimates the systolic peak time from the output of the IF sampler. The proposed processing method can be implemented in hardware using simple combinatorial logic and is comparable to performance of existing digital signal processing methods in the literature.
A switch kernel width method of correntropy for channel estimation

Correntropy has been successfully applied in non-Gaussian signal processing, but the superior performance achieved is depends on appropriate selection of the kernel width. How to select a proper kernel width is a crucial problem in correntropy applications. In this paper, we propose an adaptive algorithm to update the kernel width, which is set at a maximum between the absolute value of instantaneous error divided by square root of 2 and a predetermined kernel width. The new algorithm involves no extra free parameters and keeps the simplicity and robustness of the original maximum correntropy criterion (MCC) algorithm. Simulation results confirm that the proposed algorithm can achieve excellent performance in channel estimation under impulsive noises.