nils-herrmann/clean_scirepeval huggingface dataset:
Plenary lectures [9 abstracts]

These keynote speeches discuss the following: Advances (Innovations) in Neurotechnology; Implantable Microsystems for the Management of Trauma; Emerging trends for decentralized e-health services in smart cities; Wellness, Disease and Public Health Informatics: Multidimensional Global Threats with Local Impact; An Overview of M-Health Medical Video Communication Systems; Somatosensory Brain Machine Interfaces; Brain on a Chip: From Patterns to Circuits with Information Transfer; Systems Genetics: Experimental and Computational Challenges; Challenges and Opportunities in Cardiovascular Health Informatics.
Sparse kernel recursive least squares using L1 regularization and a fixed-point sub-iteration

A new kernel adaptive filtering (KAF) algorithm, namely the sparse kernel recursive least squares (SKRLS), is derived by adding a ℓ1-norm penalty on the center coefficients to the least squares (LS) cost (i.e. the sum of the squared errors). In each iteration, the center coefficients are updated by a fixed-point sub-iteration. Compared with the original KRLS algorithm, the proposed algorithm can produce a much sparser network, in which many coefficients are negligibly small. A much more compact structure can thus be achieved by pruning these negligible centers. Simulation results show that the SKRLS performs very well, yielding a very sparse network while preserving a desirable performance.
Theories, Analysis, and Bounds of the Finite-Support Approximation for the Inverses of Mixing-Phase FIR Systems

Inverse-system approximation using finite-impulse responses (FIR) is essential to a broad area of signal-processing applications. The conventional Wiener filtering techniques based on the least-square approach cannot provide an analytical framework simultaneously governing two crucial problems, namely, the selection of model order and the evaluation of asymptotical error bounds. In fact, the square approximation error induced from the FIR realization of a linear time-invariant system is quite complicated, specifically for those system transfer functions possessing repeated zeros with large multiplicities. Therefore, in this paper, we establish an isomorphism to characterize the z-transform pairs. In this mathematical paradigm, we elaborate the problem of approximating an inverse system or filter with an infinite number of coefficients by an FIR filter and derive the new L 1 and L 2 approximation-error bounds between the actual inverse filter and the corresponding approximated FIR. Our new theories, analysis, and bounds can be utilized to quantify the appropriate model order for the inverse-system approximation that is often needed for signal processing, control, communications, etc.
Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training

Recently, we have proposed the minimum error entropy (MEE) criterion as an information theoretic alternative to the widely used mean square error criterion in supervised adaptive system training. For this purpose, we have formulated a nonparametric estimator for Renyi's entropy that employs Parzen windowing. Mathematical investigation of the proposed entropy estimator revealed interesting insights about the process of information theoretical learning. This new estimator and the associated criteria have been applied to the supervised and unsupervised training of adaptive systems in a wide range of problems successfully. In this paper, we analyze the structure of the MEE performance surface around the optimal solution, and we derive the upper bound for the step size in adaptive linear neuron (ADALINE) training with the steepest descent algorithm using MEE. In addition, the effects of the entropy order and the kernel size in Parzen windowing on the shape of the performance surface and the eigenvalues of the Hessian at and around the optimal solution are investigated. Conclusions from the theoretical analyses are illustrated through numerical examples.
Kernel adaptive filtering with maximum correntropy criterion

Kernel adaptive filters have drawn increasing attention due to their advantages such as universal nonlinear approximation with universal kernels, linearity and convexity in Reproducing Kernel Hilbert Space (RKHS). Among them, the kernel least mean square (KLMS) algorithm deserves particular attention because of its simplicity and sequential learning approach. Similar to most conventional adaptive filtering algorithms, the KLMS adopts the mean square error (MSE) as the adaptation cost. However, the mere second-order statistics is often not suitable for nonlinear and non-Gaussian situations. Therefore, various non-MSE criteria, which involve higher-order statistics, have received an increasing interest. Recently, the correntropy, as an alternative of MSE, has been successfully used in nonlinear and non-Gaussian signal processing and machine learning domains. This fact motivates us in this paper to develop a new kernel adaptive algorithm, called the kernel maximum correntropy (KMC), which combines the advantages of the KLMS and maximum correntropy criterion (MCC). We also study its convergence and self-regularization properties by using the energy conservation relation. The superior performance of the new algorithm has been demonstrated by simulation experiments in the noisy frequency doubling problem.
On initial convergence behavior of the kernel least mean square algorithm

The mean square convergence of the kernel least mean square (KLMS) algorithm has been studied in a recent paper [B. Chen, S. Zhao, P. Zhu,  Mean square convergence analysis of the kernel least mean square algorithm, Signal Processing, vol. 92, pp. 2624-2632, 2012]. In this paper, we continue this study and focus mainly on the initial convergence behavior. Two measures of the convergence performance are considered, namely the weight error power (WEP) and excess mean square error (EMSE). The analytical expressions of the initial decreases of the WEP and EMSE are derived, and several interesting facts about the initial convergence are presented. An illustration example is given to support our observation.
Towards closed-loop brain-machine experiments across wide-area networks

Experiments for the online closed-loop control of neural prosthetics require feedback within 100ms. In a typical neurophysiology laboratory with local computing machines, a majority of this time is spent on acquiring and analyzing the neural signals and a minority (i.e. less than a millisecond) is actual data transfer among machines on local- or campus-area networks. However, the local computing machines may not offer the computational resources necessary for running complex algorithms or scenarios that have been recently proposed. While scientists can take advantage of remote computing resource providers, wide-area networks present much larger latencies that can affect an online experiment. This work presents a split modeling approach that allows the execution of a controller on the neurophysiology resource and the execution of computationally intensive modeling and adaptation algorithms on a remote datacenter, even with the inevitable network latency. Simulation results are presented to quantify how the accuracy of the controller is affected by the split modeling approach in the presence of delays, and to demonstrate that scientists can take advantage of remotely available massive resources.
Decoding stimuli from multi-source neural responses

Spike trains and local field potentials (LFPs) are two different manifestations of neural activity recorded simultaneously from the same electrode array and contain complementary information of stimuli or behaviors. This paper proposes a tensor product kernel based decoder, which allows modeling the sample from different sources individually and mapping them onto the same reproducing kernel Hilbert space (RKHS) defined by the tensor product of the individual kernels for each source, where linear regression is conducted to identify the nonlinear mapping from the multi-type neural responses to the stimuli. The decoding results of the rat sensory stimulation experiment show that the tensor-product-kernel-based decoder outperforms the decoders with either single-type neural activities.
Kernel LMS

In this paper a nonlinear adaptive algorithm based on a kernel space least mean squares (LMS) approach is presented. With most of the neural network based methods for time series modeling it is difficult to implement a sample-by-sample adaptation method. This puts a serious limitation on the applicability of adaptive nonlinear filters in many optimal signal processing and communication applications where data arrives sequentially. This paper shows that the kernel LMS algorithm provides a computational simple and an effective algorithm to train nonlinear systems for system modeling without the need for regularization, without convergence to local minima and without the need for a separate book of data as a training set.
Measures of Entropy From Data Using Infinitely Divisible Kernels

Information theory provides principled ways to analyze different inference and learning problems, such as hypothesis testing, clustering, dimensionality reduction, classification, and so forth. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, poses a challenging estimation problem that often leads to strong simplifications, such as Gaussian models, or the use of plug in density estimators that are restricted to certain representation of the data. In this paper, a framework to nonparametrically obtain measures of entropy directly from data using operators in reproducing kernel Hilbert spaces defined by infinitely divisible kernels is presented. The entropy functionals, which bear resemblance with quantum entropies, are defined on positive definite matrices and satisfy similar axioms to those of Renyi's definition of entropy. Convergence of the proposed estimators follows from concentration results on the difference between the ordered spectrum of the Gram matrices and the integral operators associated to the population quantities. In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data. Moreover, estimators of kernel-based conditional entropy and mutual information are also defined. Numerical experiments on independence tests compare favorably with state-of-the-art.
Recursive Complex Blind Source Separation via Eigendecomposition of Cumulant Matrices

Under the assumptions of non-Gaussian, non-stationary, or non-white independent sources, linear blind source separation can be formulated as a generalized eigenvalue decomposition problem. Here we provide an elegant method of doing this online, instead of waiting for a sufficiently large batch of data. This is done through a recursive generalized eigendecomposition algorithm that tracks the optimal solution, which is obtained using all the data observed. The algorithms proposed in this paper follow the well-known recursive least squares (RLS) algorithm in nature.
Flight dynamics modeling and recognition using finite state machine for automatic insect recognition

In this paper, we propose a novel state-based approach to insect flight dynamics modeling and automatic insect recognition (AIR). An emerging technology uses noninvasive laser sensors to capture flight information of flying insects in situ. However current computational entomology methods treat these optical signals as static patterns, rather than time series. We propose to evaluate each insect passage as a nonstationary process involving a sequence of pseudo-acoustic frames and model the short-term flight dynamics using a kernel adaptive autoregressive-moving-average (KAARMA) network. Since flight behavior is both nonlinear and nonstationary in nature, dynamic modeling provides a general framework that fully exploits the transitional and contextual information. Using the kernel adaptive ARMA algorithm and spatial clustering, each flight passage is defined to be an ordered sequence of states in a discretized spatiotemporal space. The state transition trajectories are used to construct a finite state machine (FSM) recognizer. The computational efficiency of the automaton classifier eliminates the need to perform numeric computations and enables realtime performance with far-reaching implications in entomology, agriculture, and disease control and prevention. Results show multiclass KAARMA classifier outperforms the state-of-the-art AIR methods, using support vector machine (SVM), deep learning autoencoder, and batch learning, in identifying Zika vector mosquito Aedes aegypti among five species of flying insects, while using significantly more efficient data representation. The extracted FSM from trained KAARMA network retained competitive performance compared to the current computationally expensive implementations.
A conditional distribution function based approach to design nonparametric tests of independence and conditional independence

Measures of independence and conditional independence are two important statistical concepts that have found profound applications in engineering such as in feature selection and causality detection, respectively. Therefore, designing efficient ways, typically nonparametric, to estimate these measures has been an active research area in the last decade. In this paper, we propose a novel framework to test (conditional) independence, using the concept of conditional distribution function. Although, estimating conditional distribution function is a difficult task on its own, we show that the proposed measures can be estimated efficiently and actually can be expressed as the Frobenius norm of a matrix. We compare the proposed methods with other state-of-the-art techniques and show that they yield very promising results.
Extended Kalman filter using a kernel recursive least squares observer

In this paper, a novel methodology is proposed to solve the state estimation problem combining the extended Kalman filter (EKF) with a kernel recursive least squares (KRLS) algorithm (EKF-KRLS). The EKF algorithm estimates hidden states in the input space, while the KRLS algorithm estimates the measurement model. The algorithm works well without knowing the linear or nonlinear measurement model. We apply this algorithm to vehicle tracking, and compare the performances with traditional Kalman filter, EKF and KRLS algorithms. Results demonstrate that the performance of the EKF-KRLS algorithm outperforms these existing algorithms. Especially when nonlinear measurement functions are applied, the advantage of the EKF-KRLS algorithm is very obvious.
Quantized Kernel Recursive Least Squares Algorithm

In a recent paper, we developed a novel quantized kernel least mean square algorithm, in which the input space is quantized (partitioned into smaller regions) and the network size is upper bounded by the quantization codebook size (number of the regions). In this paper, we propose the quantized kernel least squares regression, and derive the optimal solution. By incorporating a simple online vector quantization method, we derive a recursive algorithm to update the solution, namely the quantized kernel recursive least squares algorithm. The good performance of the new algorithm is demonstrated by Monte Carlo simulations.
Survival Information Potential: A New Criterion for Adaptive System Training

Recently, the information potential (IP) of order α, defined as the argument of the log in the α -order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance.
Neural Decoding with Kernel-Based Metric Learning

In studies of the nervous system, the choice of metric for the neural responses is a pivotal assumption. For instance, a well-suited distance metric enables us to gauge the similarity of neural responses to various stimuli and assess the variability of responses to a repeated stimulus—exploratory steps in understanding how the stimuli are encoded neurally. Here we introduce an approach where the metric is tuned for a particular neural decoding task. Neural spike train metrics have been used to quantify the information content carried by the timing of action potentials. While a number of metrics for individual neurons exist, a method to optimally combine single-neuron metrics into multineuron, or population-based, metrics is lacking. We pose the problem of optimizing multineuron metrics and other metrics using centered alignment, a kernel-based dependence measure. The approach is demonstrated on invasively recorded neural data consisting of both spike trains and local field potentials. The experimental paradigm consists of decoding the location of tactile stimulation on the forepaws of anesthetized rats. We show that the optimized metrics highlight the distinguishing dimensions of the neural response, significantly increase the decoding accuracy, and improve nonlinear dimensionality reduction methods for exploratory neural analysis.
Variable Selection: A Statistical Dependence Perspective

Measures of statistical dependence such as the correlation coefficient and mutual information have been widely used in variable selection. The use of correlation has been inspired by the concept of regression whereas the use of mutual information has been largely motivated by information theory. In a statistical sense, however, the concept of dependence is much broader, and extends beyond correlation and mutual information. In this paper, we explore the fundamental notion of statistical dependence in the context of variable selection. In particular, we discuss the properties of dependence as proposed by Renyi, and evaluate their significance in the variable selection context. We, also, explore a measure of dependence that satisfies most of these desired properties, and discuss its applicability as a substitute for correlation coefficient and mutual information. Finally, we compare these measures of dependence to select important variables for regression with real world data.
Minimum error entropy Luenberger observer

In this paper, we apply the information-theoretic learning (ITL) technique to the extended Luenberger observer. Instead of prespecifying the globally stable observer gains for nonlinear dynamic systems, we propose minimizing the entropy of the error between the measurement and the estimated output to update the observer gains. A stochastic gradient-based algorithm is presented and the performance of the entropy observer is demonstrated on linear and nonlinear dynamic systems. We also point out that this approach leads to the introduction of kernel methods into state estimation.
Vector-quantization by density matching in the minimum Kullback-Leibler divergence sense

Representation of a large set of high-dimensional data is a fundamental problem in many applications such as communications and biomedical systems. The problem has been tackled by encoding the data with a compact set of code-vectors called processing elements. In this study, we propose a vector quantization technique that encodes the information in the data using concepts derived from information theoretic learning. The algorithm minimizes a cost function based on the Kullback-Liebler divergence to match the distribution of the processing elements with the distribution of the data. The performance of this algorithm is demonstrated on synthetic data as well as on an edge-image of a face. Comparisons are provided with some of the existing algorithms such as LBG and SOM.
A relation between Hebbian and MSE learning

Traditionally, adaptive learning systems are classified into two distinct paradigms-supervised and unsupervised learning. Although a lot of results have been published in these two learning paradigms, the relations between them have been seldom investigated. We focus on the relationship between the two kinds of learning and show that in a linear network the supervised learning with mean square error (MSE) criterion is equivalent to the basic anti-Hebbian learning rule when the desired signal is a zero mean random noise independent of the input. At least for this case there is a simple relationship between the two apparent different learning paradigms.
Repairing lesions via kernel adaptive inverse control in a biomimetic model of sensorimotor cortex

In this paper we propose a kernel adaptive filtering (KAF) approach to repairing lesions via microstimulation in a biomimetic spiking neural network of sensorimotor cortex. The fundamental challenge of designing neuroprosthetics and brain machine interfaces (BMIs) is the decoding of electrical activity of neurons and behavior. For injured or damaged brain, intracranial stimulation has the potential to modulate neural activity to match meaningful and natural response or behavior. In order to optimize the microstimulation sequences, we construct an inverse model of the target system. However, to obtain sufficient learning data, the neural system must be stimulated or probed extensively. For real brains, this is especially challenging and often unfeasible. Here, we demonstrate that by applying KAF to a biomimetic brain and realistic virtual musculoskeletal model, we can repair simulated lesion and drive a virtual arm to perform the correct motor task.
A two-processing element adaptable linear oscillating recurrent system with single-weight plasticity

A simple recursive linear two-processing element adaptable oscillator is developed and demonstrated. Simple harmonic motion and its mathematical description is the foundation of this study. State equations for an undamped spring-mass system are converted to a discrete time system, followed by eigenvalue analysis to convert the four-degree-of-freedom system to a recursive network with plasticity in one variable. The oscillator is initialized to a frequency in the neighborhood of the desired frequency, and tuned by using resilient backpropagation modified for backpropagation through time. It is shown that the network can track frequencies in a spectrum of operation as defined by the sampling frequency, given the network is initialized to frequencies in the neighborhood of those of interest.
Matched pdf-based blind equalization

In this paper, a new blind equalization algorithm for multilevel modulations is proposed. It is based on maximizing the correlation between the probability density function (pdf) of the signal at the output of the equalizer and the desired pdf. The algorithm employs the Parzen window method to estimate the pdf of the squared modulus of the equalizer output. A stochastic gradient-based algorithm is used to maximize the correlation between this pdf and the pdf of the corresponding modulation. The proposed algorithm shows an excellent performance when compared with conventional adaptive blind algorithms, such as CMA, in quadrature amplitude modulation (QAM) schemes.