nils-herrmann/clean_scirepeval huggingface dataset:
Insights Into the Robustness of Minimum Error Entropy Estimation

The minimum error entropy (MEE) is an important and highly effective optimization criterion in information theoretic learning (ITL). For regression problems, MEE aims at minimizing the entropy of the prediction error such that the estimated model preserves the information of the data generating system as much as possible. In many real world applications, the MEE estimator can outperform significantly the well-known minimum mean square error (MMSE) estimator and show strong robustness to noises especially when data are contaminated by non-Gaussian (multimodal, heavy tailed, discrete valued, and so on) noises. In this brief, we present some theoretical results on the robustness of MEE. For a one-parameter linear errors-in-variables (EIV) model and under some conditions, we derive a region that contains the MEE solution, which suggests that the MEE estimate can be very close to the true value of the unknown parameter even in presence of arbitrarily large outliers in both input and output variables. Theoretical prediction is verified by an illustrative example.
Kernel adaptive filtering subject to equality function constraints

Kernel adaptive filters (KAFs) are powerful tools for online nonlinear system modeling, which are direct extensions of traditional linear adaptive filters in kernel space, with growing linear-in-the-parameters (LIP) structure. However, like most other nonlinear adaptive filters, the KAFs are “black box” models where no prior information about the unknown nonlinear system is utilized. If some prior information is available, the “grey box” models may achieve improved performance. In this work, we consider the kernel adaptive filtering with prior information in terms of equality function constraints. A novel Mercer kernel, called the constrained Mercer kernel (CMK), is proposed. With this new kernel, we develop the kernel least mean square subject to equality function constraints (KLMS-EFC), which can satisfy the constraints perfectly while achieving significant performance improvement.
Stochastic blind equalization based on PDF fitting using Parzen estimator

This work presents a new blind equalization approach that aims to force the probability density function (pdf) at the equalizer output to match the known constellation pdf. Quadratic distance between pdf's is used as the cost function to be minimized. The proposed method relies on the Parzen window method to estimate the data pdf and is implemented by a stochastic gradient descent algorithm. The kernel size of the Parzen estimator allows a dual mode switch or a soft switch between blind and decision-directed equalization. The proposed method converges faster than the constant modulus algorithm (CMA) working at the symbol rate, with a similar computational burden, and reduces the residual error of the CMA in multilevel modulations at the same time. A comparison with the most common blind techniques is presented.
Quantifying neuronal importance in value-based Brain-Machine Interfaces

Brain-Machine Interfaces (BMI) actively learn to control a prosthetic device using a relatively small number of neurons. Once the task is successfully learned, what information can be extracted from the BMI control mapping? Most BMIs reconstruct trajectories and ascertaining neural contributions is possible via sensitivity analysis of the input-output relationship. Value-based (a subset of goal-based) BMIs judge possible actions to find a best action at each time step and the sequence of selected actions will form a trajectory. Here, we expand the sensitivity analysis of trajectory-based BMI such that it applies to value-based BMI. Our finding that only a subset of recorded neurons contributes most to prosthetic control agrees with prior BMI research. Additionally, we find some specialization in neurons, i.e. certain neurons contribute to a subset of actions while other neurons contribute to different actions. Finally, we discuss implications of this metric and areas for future improvement.
A robust point matching algorithm for non-rigid registration using the Cauchy-Schwarz divergence

In this paper, we describe an algorithm that provides both rigid and non-rigid point-set registration. The point sets are represented as probability density functions and the registration problem is treated as distribution alignment. Using the PDFs instead of the points provides a more robust way of dealing with outliers and noise, and it mitigates the need to establish a correspondence between the points in the two sets. The algorithm operates on the distance between the two PDFs to recover the spatial transformation function needed to register the two point sets. The distance measure used is the Cauchy-Schwarz divergence. The algorithm is robust to noise and outliers, and performswell in varying degrees of transformations and noise.
Echo cancellation by global optimization of Kautz filters using an information theoretic criterion

In practical settings, the echo cancellation problem generally requires the adaptation of an IIR filter using some optimality criterion. This brings two problems: direct adaptation of numerator and denominator polynomial coefficients of IIR filters might result in unstable systems and/or the optimization might result in a suboptimal local minimum of the criterion. These two issues are addressed in this paper. To resolve the first problem, orthogonal Kautz filters are utilized for their stability is easily controlled through the pole locations. The second problem is addressed by employing an information theoretic optimality criterion, which has a parameter that is annealed to ensure global optimization.
Noise reduction in state space using the focused gamma neural network

In this paper we utilize the gamma neural model to improve the signal to noise ratio (SNR) of broadband signals corrupted by white noise. The projection of a noisy signal onto the signal subspace can not remove the noise in the subspace. A focus gamma network, when trained as a nonlinear predictor of the projected trajectory, reduces this noise further. The property of adaptive memory depth of the gamma model is utilized to decide when to stop the training of the network. The preliminary results show that the SNR can be improved significantly, preserving the broadband signal spectrum.<<ETX>>
A parallel hardware architecture for information-theoretic adaptive filtering

Information-theoretic cost functions such as minimization of the error entropy (MEE) can extract more structure from the error signal, yielding better results in many realistic problems. However, adaptive filters (AFs) using MEE methods are more computationally intensive when compared to conventional, mean-squared error (MSE) methods employed in the well-known, least mean squares (LMS) algorithm. This paper presents a novel, parallel hardware architecture for MEE adaptive filtering. The design has been implemented and evaluated in realtime on one of the servers of the Novo-G machine in the NSF CHREC Center at the University of Florida, believed to be the most powerful reconfigurable supercomputer in academia. By pipelining the design and parallelizing independent computations within the algorithm, our proposed hardware architecture successfully achieves a speedup of 5800 on one FPGA, 23200 on one quad-FPGA board, and 46400 on two quad-FPGA boards, as compared to the same algorithm running in software (optimized C program) on a single CPU core. Just as important, our results show that this reconfigurable design does not lose precision while converging to the optimum solution in the same number of steps as the software version. As a result, our approach makes it possible for AFs using the MEE cost function to adapt in real-time for signals that require a sampling rate in excess of 400 kHz and thus can target a much wider range of applications.
Kernel adaptive filtering with confidence intervals

Since its introduction, kernel adaptive filtering (KAF) has attracted considerable attention in recent years. Its main advantages include universal nonlinear approximation using kernel methods, linearity with convex learning in the Reproducing Kernel Hilbert Space (RKHS), and online adaptation with moderate complexity. Among its applications, the kernel least mean square (KLMS) algorithm deserves particular attention due to its simplicity and effectiveness for learning complex systems. A major drawback of current implementations of KAF is the lack of a simple determination of the certainty of each estimate. In this paper, we present a novel kernel adaptive filtering architecture with confidence intervals. By introducing an auxiliary filter, the variance of each estimate can be computed using stochastic gradient descent in O(N). Results show that the proposed algorithm produces comparable estimates of the mean and the variance functions, using only a fraction of the computation associated with the Gaussian process (GP) prediction, and is more versatile in the cases of time-varying noise variance or heteroskedasticity.
An introduction to information theoretic learning

Learning from examples has been traditionally done with correlation or with the mean square error (MSE) criterion, in spite of the fact that learning is intrinsically related with the extraction of information from examples. The problem is that Shannon (1948) introduced the idea of information entropy which has a sound theoretical foundation but is not easy to implement in a learning from examples scenario. In this paper Renyi's entropy definition (1976) is used and integrated with a nonparametric estimator of the probability density function (Parzen window). The experimental results on blind source separation confirm the theory. Although the work is preliminary, the "information potential" method is rather general and will have many applications.
Information point set registration for shape recognition

This paper proposes a way of enhancing shape recognition through point set registration. Firstly, a modified version of shape context (SC) is developed, which is invariant to rigid transformation and flipping. With the point correspondence obtained by the modified SC, an affine transformation based on the maximum correntropy criterion (MCC) is performed on the query shape. This point set registration could be further refined by non-rigid morphing with the minimization of Cauchy-Schwarz divergence (DCS). Not only does this information theoretical learning (ITL) approach renders excellent registration result, but a new shape similarity measure can also be derived from the registration.
Information Theoretic Shape Matching

In this paper, we describe two related algorithms that provide both rigid and non-rigid point set registration with different computational complexity and accuracy. The first algorithm utilizes a nonlinear similarity measure known as correntropy. The measure combines second and high order moments in its decision statistic showing improvements especially in the presence of impulsive noise. The algorithm assumes that the correspondence between the point sets is known, which is determined with the surprise metric. The second algorithm mitigates the need to establish a correspondence by representing the point sets as probability density functions (PDF). The registration problem is then treated as a distribution alignment. The method utilizes the Cauchy-Schwarz divergence to measure the similarity/distance between the point sets and recover the spatial transformation function needed to register them. Both algorithms utilize information theoretic descriptors; however, correntropy works at the realizations level, whereas Cauchy-Schwarz divergence works at the PDF level. This allows correntropy to be less computationally expensive, and for correct correspondence, more accurate. The two algorithms are robust against noise and outliers and perform well under varying levels of distortion. They outperform several well-known and state-of-the-art methods for point set registration.
Functional Connectivity Dynamics Among Cortical Neurons: A Dependence Analysis

This paper quantifies and comparatively validates functional connectivity between neurons by measuring the statistical dependence between their firing rates. Based on statistical analysis of the pairwise functional connectivity, we estimate, exclusively from neural data, the neural assembly functional connectivity given a behavior task, which provides a quantifiable representation of the dynamic nature during the behavioral task. Because of the time scale of behavior (100-1000 ms), a statistical method that yields robust estimators for this small sample size is desirable. In this work, the temporal resolutions of four estimators of functional connectivity are compared on both simulated data and real neural ensemble recordings. The comparison highlights how the properties and assumptions of statistical-based and phase-based metrics affect the interpretation of connectivity. Simulation results show that mean square contingency (MSC) and mutual information (MI) create more robust quantification of functional connectivity under identical conditions than cross correlation (CC) and phase synchronization (PhS) when the sample size is 1 s. The results of the simulated analysis are extended to real neuronal recordings to assess the functional connectivity in monkey's cortex corresponding to three movement states in a food reaching task and construct the assembly graph given a movement state and the activation degree of a state-related assembly over time using the statistical test exclusively from neural data dependencies. The activation degree of a given state-related assembly reaches the peak repeatedly when the specific movement states occur, which also reveals the network of interactions among the neurons are key for the operation of a specific behavior.
Kernel-based adaptive learning improves accuracy of glucose predictive modelling in type 1 diabetes: A proof-of-concept study

This study aims at demonstrating the need for nonlinear recursive models to the identification and prediction of the dynamic glucose system in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by the Approximate Linear Dependency Kernel Recursive Least Squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. The method is evaluated on seven people with type 1 diabetes in free-living conditions, where a change in glycaemic dynamics is forced by increasing the level of physical activity in the middle of the observational period. The univariate input allows for short-term (≤30 min) predictions with KRLS-ALD reaching an average root mean square error of 15.22±5.95 mgdL−1 and an average time lag of 17.14±2.67 min for an horizon of 30 min. Its performance is considerably better than that of time-invariant (regularized) linear regression models.
Effective insect recognition using a stacked autoencoder with maximum correntropy criterion

Throughout the history, insects had been intimately connected to humanity, in both positive and negative ways. Insects play an important part in crop pollination, on the other hand, some of them spread diseases that kill millions of people every year. Effective control of harmful insects while having little impact to beneficial insects and environment is extremely important. Recently, an intelligent trap that uses laser sensors was presented to control the population of target insects. The device could record and analyze sensor signals when an insect passes through the trap and make quick decisions whether to catch it or not. The effectiveness of the trap relies on the correct choice of classification algorithm to perform the insect detection. In this paper, we propose to use a deep neural network with maximum correntropy criterion (MCC) for reliable classification of insects in real-time. Experimental results show that, deep networks are effective for learning stable features from brief insect passage signals. By replacing the mean square error cost with MCC, the robustness of autoencoders against noise is improved significantly and robust features could be learned. The method is tested on five species of insects and a total of 5325 passages. High classification accuracy of 92.1% is achieved. Compared with previously applied methods, better classification performance is obtained using only 10% of the computation time. Therefore, our method is efficient and reliable for online insect detection.
Irregular Tree-Structured Bayesian Network for image segmentation

Unsupervised image segmentation algorithms rely heavily on a probabilistic smoothing prior to enforce local homogeneity in the segmentation results. The tree-structured prior [1, 2, 3] is one such prior which allows important multi-scale spatial correlations that exist in natural images to be captured. Two main types of tree structure prior have been previously proposed: 1) fixed quadtree structure , which suffers from “blockiness” in the segmentation results and 2) flexible tree structure [2, 3] which can adapt its structure to the natural object boundary but at a significant computational cost. This paper presents a novel probabilistic unsupervised image segmentation framework called Irregular Tree-Structured Bayesian Networks (ITSBN) which introduces the notion of irregular tree structure that combines the merits of the two previous approaches. As in [2, 3], more natural object boundaries can be modeled in our framework since a tree is learned for each input image. Our method, however, does not update the adaptive structure at every iteration which drastically reduces the computation required. We derive a time-efficient exact inference algorithm based on a sum-product framework using factor graphs . Furthermore, a novel methodology for the evaluation of unsupervised image segmentation is proposed. By integrating non-parametric density estimation techniques with the traditional precision-recall framework, the proposed method is more robust to boundary inconsistency due to human subjects.
Generalized correlation function: definition, properties, and application to blind equalization

With an abundance of tools based on kernel methods and information theoretic learning, a void still exists in incorporating both the time structure and the statistical distribution of the time series in the same functional measure. In this paper, a new generalized correlation measure is developed that includes the information of both the distribution and that of the time structure of a stochastic process. It is shown how this measure can be interpreted from a kernel method as well as from an information theoretic learning points of view, demonstrating some relevant properties. To underscore the effectiveness of the new measure, a simple blind equalization problem is considered using a coded signal.
Intraoperative focus localization system based on spatio-temporal ECoG analysis

Surgical therapy has been shown to provide significant relief io a number of eligible patients with othenvise intractable focal epilepsy. In the interest of these patients, the neurosurgeon may try to “tailor” the resection of cerebral cortex according to intraoperative findings regarding the location of the epileptic focus. This enhanced focus localization is achieved by reading the electrocorticogram (ECoG) obtiined from an may of electrodes  The system described here has been designed as a tool to facilitate the localization of the epiIeptic focus using the signals obtained from the ECoG array.
Analysis and Design of Echo State Networks

The design of echo state network (ESN) parameters relies on the selection of the maximum eigenvalue of the linearized system around zero (spectral radius). However, this procedure does not quantify in a systematic manner the performance of the ESN in terms of approximation error. This article presents a functional space approximation framework to better understand the operation of ESNs and proposes an information-theoretic metric, the average entropy of echo states, to assess the richness of the ESN dynamics. Furthermore, it provides an interpretation of the ESN dynamics rooted in system theory as families of coupled linearized systems whose poles move according to the input signal dynamics. With this interpretation, a design methodology for functional approximation is put forward where ESNs are designed with uniform pole distributions covering the frequency spectrum to abide by the richness metric, irrespective of the spectral radius. A single bias parameter at the ESN input, adapted with the modeling error, configures the ESN spectral radius to the input-output joint space. Function approximation examples compare the proposed design methodology versus the conventional design.
Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation.
SOM-based similarity index measure: quantifying interactions between multivariate structures

This work addresses the issue of quantifying asymmetric functional relationships between signals. We specifically consider a previously proposed similarity index that is conceptually powerful, yet computationally very expensive. The complexity increases with the square of the number of samples in the signals. In order to counter this difficulty, a self-organizing map that is trained to model the statistical distribution of the signals of interest is introduced in the similarity index evaluation procedure. The SOM based technique is equally accurate, but computationally less expensive compared to the conventional measure. These results are demonstrated by comparing the original and SOM-based similarity index approaches on synthetic chaotic signal and real EEG signal mixtures.
Blinking Artifact Removal in Cognitive EEG Data Using ICA

Eye blinking artifacts present serious problems for electroencephalographic (EEG) interpretation and analysis. In this study, we apply independent component analysis (ICA) to eye blinking artifact removal from cognitive EEG recordings. Due to the specific design of the experiment, the eye blinks almost always co-occur with the event-related potentials (ERP), which creates problems for ICA. We introduced another data set of spontaneous blink and combined it with single-trial ERP data. Our results show that ICA on the combined data set gives separation that makes more sense and makes it easier for EEG interpretation and analysis