nils-herrmann/clean_scirepeval huggingface dataset:
Marine animal classification using combined CNN and hand-designed image features

Digital imagery and video have been widely used in many undersea applications. Online automated labeling of marine animals in such video clips comprises of three major steps: detection and tracking, feature extraction and classification. The latter two aspects are the focus of this paper. Feature extracted from convolutional neural network (CNN) is tested on two real-world marine animal datasets (Taiwan sea fish and Monterey Bay Aquarium Research Institute (MBARI) benthic animal), and yields better classification results than existing approaches. Appropriate combination of CNN and hand-designed features can achieve even higher accuracy than applying CNN alone. The group feature selection scheme, which is a modified version of the minimal-redundancy-maximal-relevance (mRMR) algorithm, serves as the criterion for selecting an optimal set of hand-designed features. Performance of CNN and hand-designed features are further examined for images with lowered quality that emulates bad lighting condition in water.
Prerequesites for symbiotic brain-machine interfaces

Recent advancements in the neuroscience and engineering of Brain-Machine Interfaces are providing a blueprint for how new co-adaptive designs based on reinforcement learning change the nature of a user's ability to accomplish tasks that were not possible using static methodologies. By designing adaptive controls and artificial intelligence into the neural interface, computers can become active assistants in goal-directed behavior and further enhance human performance. This paper presents a set of minimal prerequisites that enable a cooperative symbiosis and dialogue between biological and artificial systems.
Extended Kernel Recursive Least Squares Algorithm

This paper presents a kernelized version of the extended recursive least squares (EX-KRLS) algorithm which implements for the first time a general linear state model in reproducing kernel Hilbert spaces (RKHS), or equivalently a general nonlinear state model in the input space. The center piece of this development is a reformulation of the well known extended recursive least squares (EX-RLS) algorithm in RKHS which only requires inner product operations between input vectors, thus enabling the application of the kernel property (commonly known as the kernel trick). The first part of the paper presents a set of theorems that shows the generality of the approach. The EX-KRLS is preferable to 1) a standard kernel recursive least squares (KRLS) in applications that require tracking the state-vector of general linear state-space models in the kernel space, or 2) an EX-RLS when the application requires a nonlinear observation and state models. The second part of the paper compares the EX-KRLS in nonlinear Rayleigh multipath channel tracking and in Lorenz system modeling problem. We show that the proposed algorithm is able to outperform the standard KRLS and EX-RLS in both simulations.
Kernel width adaptation in information theoretic cost functions

This paper presents an algorithm for online adaptation of the kernel width parameter in information theoretic cost functions used for adaptive system training. Training algorithms which optimize information theoretic quantities like entropy involve choosing a kernel size for their sample estimators. The kernel size essentially dictates the nature of the performance surface of the cost function over which the system parameters adapt. This, in turn, governs factors like speed of adaptation and presence of local minima. We show results of using the Minimum Error Entropy (MEE) criterion with the proposed adaptive kernel algorithm for training a time delay neural network. Our simulations show that having an adaptive kernel width results in faster convergence of parameters as compared to having fixed values.
Context Dependent Encoding Using Convolutional Dynamic Networks

Perception of sensory signals is strongly influenced by their context, both in space and time. In this paper, we propose a novel hierarchical model, called convolutional dynamic networks, that effectively utilizes this contextual information, while inferring the representations of the visual inputs. We build this model based on a predictive coding framework and use the idea of empirical priors to incorporate recurrent and top-down connections. These connections endow the model with contextual information coming from temporal as well as abstract knowledge from higher layers. To perform inference efficiently in this hierarchical model, we rely on a novel scheme based on a smoothing proximal gradient method. When trained on unlabeled video sequences, the model learns a hierarchy of stable attractors, representing low-level to high-level parts of the objects. We demonstrate that the model effectively utilizes contextual information to produce robust and stable representations for object recognition in video sequences, even in case of highly corrupted inputs.
Neural control of the NASA Langley 16-foot transonic tunnel

Experimental results of controlling the Mach number in a transonic wind tunnel with a system of artificial neural networks are presented. Kohonen self-organizing maps are used to cluster the local tunnel dynamics and thereby predict the Mach number response to candidate control input sequences. The sequence minimizing the predicted error between the desired and actual Mach number is applied to the tunnel fan drive system. Comparison is made to gain scheduled automatic control currently in use.
A nonparametric information theoretic approach for change detection in time series

This paper presents an online nonparametric methodology based on the Kernel Least Mean Square (KLMS) algorithm and the surprise criterion, which is based on an information theoretic framework. Surprise quantifies the amount of information a datum contains given a known system state, and can be estimated online using Gaussian Process Theory. Based on this concept, we use the KLMS algorithm together with surprise criterion to detect regime change in nonstationary time series. We test the methodology on a synthesized chaotic time series to illustrate this criterion. The results show that surprise criterion is better than the conventional segmentation based on the error criterion.
A Normalized Minimum Error Entropy Stochastic Algorithm

We propose in this paper the normalized minimum error entropy (NMEE). Following the same rational that lead to the normalized LMS, the weight update adjustment for minimum error entropy (MEE) is constrained by the principle of minimum disturbance. Unexpectedly, we obtained an algorithm that not only is insensitive to the power of the input, but is also faster than the MEE for the same misadjustment, and also that is less sensitive to the kernel size. We explain these results analytically, and through system identification simulations
Correntropy: A Localized Similarity Measure

The measure of similarity normally utilized in statistical signal processing is based on second order moments. In this paper, we reveal the probabilistic meaning of correntropy as a new localized similarity measure based on information theoretic learning (ITL) and kernel methods. As such it has vastly different properties when compared with mean square error (MSE) that can be very useful in nonlinear, non-Gaussian signal processing. Two examples are presented to illustrate the technique.
An overview of optimal signal processing methodologies for brain machine interfaces

We are part of a multi-university consortium headed by Duke University with the goal of developing Brain Machine Interfaces (BMIs). Hundreds of electrodes are surgically implanted in the motor cortex of monkeys and live sessions of the behaving animals are simultaneous recorded with the position of the hand in 3D space. Our goal is to derive optimal models to translate the spike train data into the hand position, using a “black box” system identification framework. This task poses a new set of problems for optimal signal processing because of the sheer dimensionality of the input and its discrete nature (spike trains) compounded with sampling noise eventual nonlinearity and nonstationarity. In this presentation we will review our approaches to develop optimal input-output models based on HMMs, Kalman and Wiener filters and recurrent neural networks. Furthermore, we will quantify the performance of the recurrent neural network and from the trained model, infer the importance of the neurons and how the mapping was achieved.
Magnitude spectral estimation via Poisson moments with application to speech recognition

We propose to use the Gamma filter as a continuous time spectral feature extractor for the preprocessing of speech signals. The Gamma filter is a simple analog structure which can be implemented as a cascade of identical first order low-pass filters. The filter generates the Poisson moments of the input signal at its taps. These moments carry spectral information about the recent history of the input signal and in return they can be used to construct a time-frequency representation alternative to the conventional methods of the short-term Fourier transform, cepstrum, etc. The appeal of the proposed method comes from the fact that in the analog domain the Poisson moments are readily available as a continuous time electrical signal and can be physically measured rather than computed offline by a digital computer. With this convenience, the speed of the discrete time processor following the preprocessor is independent of the highest frequency of the input signal, but is constrained by the stationarity interval of the signal. The moments can be directly fed into artificial neural networks (ANNs) for tasks like classification and identification of time-varying signals like speech.
An energy study on IR-UWB transmitter using integration-and-fire modulation

The integrate-and-fire model of a biological neuron is an amplitude to time encoding in the spacing between action potentials (spikes). In principle, this encoding can be used to modulate signals in an Impulse Radio Ultra Wide-Band (IR-UWB) transmitter suitable for Wireless Sensor Networks (WSN). This paper presents a system level study on power efficiency using MATLAB/Simulink to evaluate the required energy for an IR-UWB Transmitter using integrate-and-fire encoding technique. Also, a simple but clear comparison with common systems utilizing Nyquist rate Analog-to-Digital Converters (ADC) is presented. This study has been carried out on a band-limited random Gaussian signal and the results show that IR-IF transmitter consumes roughly seven times less energy than a digital UWB transmitter; moreover, in the proposed architecture the need for power hungry ADC is relaxed.
A Pitch Detector Based on a Generalized Correlation Function

This paper proposes a novel pitch determination algorithm (PDA) based on the newly introduced concept of a generalized correlation function called correntropy. Correntropy is a positive definite kernel function which implicitly transforms the original signal into a high-dimensional reproducing kernel Hilbert space (RKHS) in a nonlinear way, and calculates very efficiently the generalized correlation in that RKHS. By incorporating the kernel function, correntropy is able to utilize higher order statistics to enhance the resolution of pitch estimation. The proposed PDA computes the summary of correntropy functions from the outputs of an equivalent rectangular bandwidth (ERB) filter bank. We present simulations on pitch determination for a single vowel, double vowels, and a benchmark database test. Simulations show that correntropy exhibits much better resolution than conventional autocorrelation in pitch determination and outperforms other PDAs in the benchmark database test.
Sparsity aware minimum error entropy algorithms

Sparse estimation has received a lot of attention due to its broad applicability. In sparse channel estimation, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through sparse adaptive filters. In previous studies, most works use the mean square error (MSE) based cost to develop sparse filters, which is rational under the assumption of Gaussian distributions. However, Gaussian assumption does not always hold in real-world environments. To address this issue, we incorporate in this work l1-norm and reweighted l1-norm into the minimum error entropy (MEE) criterion to develop new sparse adaptive filters, which may perform much better than the MSE based methods especially in non-Gaussian situations, since the error entropy can capture higher-order statistics of the errors . Furthermore, a new approximator of l0-norm based on the Correntropy Induced Metric (CIM) is also used as a sparsity penalty term (SPT). Simulation results show the excellent performance of the proposed algorithms.
Error whitening criterion for linear filter estimation

Mean square error (MSE) has been the most widely used tool to solve the linear filter estimation or system identification problem. However, MSE gives biased results when the input signals are noisy. This paper presents a novel error whitening criterion (EWC) to tackle the problem of linear system identification in the presence of additive white disturbances. We would motivate the theory behind the new criterion and derive an online stochastic gradient algorithm based on EWC. Convergence proof of the stochastic gradient algorithm is derived making mild assumptions. Simulation results show the effectiveness of this criterion. We compare its performance with MSE as well as the powerful total least squares method.
Towards a unification of information theoretic learning and kernel methods

In this paper, we discuss an intriguing relationship between information theoretic learning (ITL), based on Parzen window density estimation, and kernel-based learning algorithms. We show that some of the widely used ITL cost functions, when estimated by the Parzen method, can be expressed in terms of inner products in a kernel feature space defined by a Mercer kernel, where the Mercer kernel, in fact, is the Parzen window. This link gives a theoretical criterion for the selection of the Mercer kernel, based on density estimation. Also, we show that the support vector machine (SVM), as an example of a well-known kernel-based learning algorithm, can be examined in an information theoretic framework, using weighted Parzen windows for density estimation
Fast error whitening algorithms for system identification and control

Linear system identification with noisy inputs is a critical problem in signal processing and control. Conventional techniques based on the mean squared-error (MSE) criterion can at best provide a biased estimate of the unknown system being modeled. Recently, we proposed a new criterion called the error whitening criterion (EWC) to solve the problem of linear parameter estimation in the presence of additive white noise. In this paper, we present a fixed-point type algorithm with O(N/sup 2/) complexity for EWC, called the recursive error whitening (REW) algorithm. We would also show that the EWC solution could be solved using the computational principles of total least squares (TLS). A novel EWC-TLS algorithm with O(N/sup 2/) complexity is derived. We will then apply the EWC methods for adaptive inverse control and show the superiority over existing methods.
Nonlinear channel equalization using multilayer perceptrons with information-theoretic criterion

The minimum error entropy criterion was recently suggested in adaptive system training as an alternative to the mean-square-error criterion, and it was shown to produce better results in many tasks. The authors apply a multilayer perceptron scheme trained with this information theoretic criterion to the problem of nonlinear channel equalization. In our simulations, we use a realistic nonlinear channel model, which is encountered when practical power amplifiers are used in the transmitter. The bandwidth-efficient 16-QAM scheme, which uses a dispersed constellation, is assumed.
An efficient rank-deficient computation of the Principle of Relevant Information

One of the main difficulties in computing information theoretic learning (ITL) estimators is the computational complexity that grows quadratically with data. Considerable amount of work has been done on computation of low rank approximations of Gram matrices without accessing all their elements. In this paper we discuss how these techniques can be applied to reduce computational complexity of Principle of Relevant Information (PRI). This particular objective function involves estimators of Renyi's second order entropy and cross-entropy and their gradients, therefore posing a technical challenge for implementation in a realistic scenario. Moreover, we introduce a simple modification to the Nyström method motivated by the idea that our estimator must perform accurately only for certain vectors not for all possible cases. We show some results on how this rank deficient decompositions allow the application of the PRI on moderately large datasets.
Kernel classifier with Correntropy loss

Classification can be seen as a mapping problem where some function of xn predicts the expectation of a class variable yn. This paper uses kernel methods for the prediction of class variable, together with a recently proposed cost function for classification, called Correntropy-loss (C-loss) function. C-Loss is a non-convex loss function based on a similarity measure called correntropy and is known to closely approximate the ideal 0-1 loss function for classification. This paper shows via experimental results that, by replacing the cost function - Mean Square Error (MSE) in a conventional kernel based functional mapping, by a non-convex loss function C-Loss, a non-overfitting, and hence, a better classifier can be obtained. Since gradient descent can still be used with the C-loss and the kernel mapper, the classifier can be easily trained without performance penalty, compared to the SVM, which makes the approach very practical.
Linear Predictive Analysis for Targeting the Basal Ganglia in Deep Brain Stimulation Surgeries

Intra-operative automated recognition of deep brain stimulation (DBS) targets from microelectrode recordings would improve the safety, efficiency, standardization, and accuracy of the surgical procedure. Our approach to the cellular classification problem is from a speech recognition perspective where linear predictive coefficient (LPC) analysis is used to model segments of thalamic and subthalamic nucleus cellular activity. We then cluster the linear prediction coefficients for three Parkinson's disease patients and develop discriminant surfaces with an artificial neural network to generate the target classes. The methods presented here yielded a significant separation of the cell types within a two-dimensional prediction coefficient data space. The results indicate that LPC analysis for DBS targeting warrants additional study for a larger variety of deep brain structures and patients
Self organizing maps with the correntropy induced metric

The similarity measure popularly used in Kohonen's self organizing maps and several of its other variants is the mean square error (MSE). It is shown that this leads to, in information theoretic sense, a suboptimal solution of distributing the centers of the map. Here we show that using a similarity measure called the correntropy induced metric (CIM) can lead to a solution with better magnification of the input density. It provides an insight into how the type of the kernel effects the mapping and also under what condition is using SOM with CIM (SOM-CIM) can perform better than SOM with MSE. We also show that the use of this in clustering and data visualization can provide better results.
A neural implementation of interpolation with a family of kernels

A paradigm for interpolating images based on a family of kernels is presented. Each kernel is "tuned" to specific image characteristics and contains the information responsible for the local creation of missing detail. This interpolation process (1) exploits the correlation that exists in the local structure of images via a self-organizing feature map (SOFM) and (2) establishes an optimal set of linear associative memories (LAMs) from the homologous neighborhoods of a set of low and high resolution image counterparts. Each LAM creates members of the family of interpolation kernels. We compare the performance of this technique with the commonly used bilinear and spline interpolation methods and demonstrate its ability to generalize well.
A Reproducing Kernel Hilbert Space Framework for Information-Theoretic Learning

This paper provides a functional analysis perspective of information-theoretic learning (ITL) by defining bottom-up a reproducing kernel Hilbert space (RKHS) uniquely determined by the symmetric nonnegative definite kernel function known as the cross-information potential (CIP). The CIP as an integral of the product of two probability density functions characterizes similarity between two stochastic functions. We prove the existence of a one-to-one congruence mapping between the ITL RKHS and the Hilbert space spanned by square integrable probability density functions. Therefore, all the statistical descriptors in the original information-theoretic learning formulation can be rewritten as algebraic computations on deterministic functional vectors in the ITL RKHS, instead of limiting the functional view to the estimators as is commonly done in kernel methods. A connection between the ITL RKHS and kernel approaches interested in quantifying the statistics of the projected data is also established.
Blind separation of convolutive mixtures

Reverberant signals recorded by multiple microphones can be described as sums of sources convolved with different parameters. Blind source separation of this unknown linear system can be transformed to a set of instantaneous mixtures for every frequency band. In each frequency band, we may use the simultaneous diagonalization algorithms to separate the sources. In addition to our previous simultaneous diagonalization to minimize the Frobenius norm, we now propose another set of efficient simultaneous diagonalization algorithms based on Hadamard's inequality to make the source separation feasible in the frequency domain.
Learning joint features for color and depth images with Convolutional Neural Networks for object classification

In this paper we investigate the advantages of learning representations of color plus depth images (Red-Blue-Green-Depth, RGB-D) over color only images (RGB) for computer vision. Specifically, we investigate the advantages on the task of object recognition. For this purpose, we applied the state-of-art deep convolutional neural networks (CNN) for classification of images on the RGB-D dataset published by (Bo et al., 2011). We show that this approach provides better results than those that use separate features for color and depth. Also, we probe the resulting CNN to gain intuition about how filters for depth and color channels iterate to generate useful features.
Information Theoretic Clustering

Clustering is an important topic in pattern recognition. Since only the structure of the data dictates the grouping (unsupervised learning), information theory is an obvious criteria to establish the clustering rule. The paper describes a novel valley seeking clustering algorithm using an information theoretic measure to estimate the cost of partitioning the data set. The information theoretic criteria developed here evolved from a Renyi entropy estimator (A. Renyi, 1960) that was proposed recently and has been successfully applied to other machine learning applications (J.C. Principe et al., 2000). An improved version of the k-change algorithm is used in optimization because of the stepwise nature of the cost function and existence of local minima. Even when applied to nonlinearly separable data, the new algorithm performs well, and was able to find nonlinear boundaries between clusters. The algorithm is also applied to the segmentation of magnetic resonance imaging data (MRI) with very promising results.