nils-herrmann/clean_scirepeval huggingface dataset:
Fast feedforward non-parametric deep learning network with automatic feature extraction

In this paper, a new type of feedforward non-parametric deep learning network with automatic feature extraction is proposed. The proposed network is based on human-understandable local aggregations extracted directly from the images. There is no need for any feature selection and parameter tuning. The proposed network involves nonlinear transformation, segmentation operations to select the most distinctive features from the training images and builds RBF neurons based on them to perform classification with no weights to train. The design of the proposed network is very efficient (computation and time wise) and produces highly accurate classification results. Moreover, the training process is parallelizable, and the time consumption can be further reduced with more processors involved. Numerical examples demonstrate the high performance and very short training process of the proposed network for different applications.
Optimization in companion search spaces: the case of cross-entropy and the Levenberg-Marquardt algorithm

We present a new learning algorithm for the supervised training of multilayer perceptions for classification that is significantly faster than any previously known method. Like existing methods, the algorithm assumes a multilayer perceptron with a normalized exponential (softmax) output trained under a cross-entropy criterion. However, this output-criteria pairing turns out to have poor properties for existing optimization methods (backpropagation and its second order extensions) because second-order expansion of the network weights about the optimal solution is not a good approximation. The proposed algorithm overcomes this limitation by defining a new search space for which a second-order expansion is valid and such that the optimal solution in the new space coincides with the original criterion. This allows the application of the Levenberg-Marquardt search procedure to the cross-entropy criterion, which was previously thought applicable only to a mean square error criteria.
Some studies of European Portuguese nasal vowels using an articulatory synthesizer

In this paper we present some details of our work in the study of European Portuguese nasal vowels using an articulatory synthesizer. In the first part of the paper we describe some of the articulatory synthesizer characteristics motivated by the particularities of our object of study. In the second part we describe the realization of some experiments using the synthesizer. Experiments address the problems of: (1) static versus dynamic configurations in production of nasal vowels; (2) the relevance of the nostril radiated flux in the perception of nasality; and (3) the effect of nasal passage blocking in the perception of nasality. Results reported are preliminary due to the small number of tests performed.
Explicit versus implicit source estimation for blind multiple input single output system identification

Sparsely-activated time series are found in many physical systems. In these cases, the signals can be approximated by convolution of sparse sources with a set of shift-invariant filters. When there is access to only one sensor, such that there is a single observation signal, identifying the source signals appears to be an ill-posed problem, but for very sparse sources it is still possible to learn the system. We discuss analysis techniques for sparsely activated signals, which retrieve sparse sources given the filters, and identify conditions when algorithms based on independent component analysis (ICA) and sparse coding can blindly estimate filters from a single noisy time-series. Many qualitative results have been made for learning shift-invariant bases on natural signals, but for a thorough understanding of the effect of sparsity, we quantitatively analyze results on synthetic examples, comparing how ICA and shift-invariant sparse coding approaches perform for multiple-source blind system identification.
Exponential C-Loss for data fitting

As a robust measure of similarity, C-Loss can be successfully used for data fitting such as regression and classification, especially when data contain large outliers. In this paper, we propose a modified C-Loss function, called exponential C-Loss (EC-Loss), which is defined as an exponential function of the C-Loss. The EC-Loss inherits the robustness and smoothness of the C-Loss but may have a better performance surface that favors the usage of a gradient-based learning algorithm, particularly at a region far from the optimal solution. In order to avoid the flatness of the performance surface near the optimal solution and obtain a fast convergence speed during the overall adaptation process, we also propose a novel switching strategy between C-Loss and EC-Loss. A simple simulation example is presented to demonstrate the performance surface and desirable performance of the new method.
From adaptive linear to information filtering

Adaptive signal processing theory was born and has lived by exclusively exploiting the mean square error criterion. When we think of the goal of least squares without restrictions of Gaussianity, one has to wonder why an information theoretic error criterion is not utilized instead. After all, the goal of adaptive filtering should be to find the linear projection that best captures the information in the desired response. We summarize our efforts to extend adaptive linear filtering to information filtering. We review Renyi's (1987) entropy definition, Parzen (1967) windows and put them together in a framework to estimate entropy directly from samples (nonparametric). Once this criterion is developed we can train linear or nonlinear adaptive networks for entropy maximization or minimization. We present results on the properties of the Renyi's nonparametric entropy estimator, and show how it performs in chaotic time series prediction.
Diffusion least-mean squares over adaptive networks with dynamic topologies

The purpose of this paper is to study the performance of diffusion-based distributed adaptive algorithms when relaxing the static assumption on network topology. Adaptive networks with topologies that change across time are useful to model a wide class of real-time sensor networks. This includes topologies where the number of nodes is variable and links are dynamic. We propose two schemes to accommodate dynamic topologies. The first proceeds following a probabilistic mechanism to add or remove nodes at each point in time. The second allows edges to be re-assigned at each iteration. The suggested changes allow retaining fundamental characteristics of the sensor graph, like maximum and minimum node degrees, as their significance impacts the network's throughput and its resistance to failures of neighbors. Simulations were carried for different node collaboration settings and averaged over Monte Carlo runs. Results show that no significant deterioration in performance is observed despite changes in the network size and connectivity, with a gained affinity to accommodate real sensor network systems.
Tracking of multivariate time-variant systems based on on-line variable selection

Tracking time-variant systems has been of great interest in many engineering fields. Specifically, when system statistics change both in space (multivariate) and time with a short stationary regime, conventional adaptive algorithms suffer from the trade-off between convergence rate and accuracy. In this paper, we propose a tracking system consisting of a linear adaptive system accompanied by an on-line variable selection algorithm that is based on the least angle regression algorithm. This algorithm explicitly employs local (in time) correlation between the input and the output of an unknown system to select a subset of input variables at every time step. Therefore, it enables the multivariate adaptive filter to track the temporal changes of correlated variables. Simulations involving tracking of multi-channel time-variant systems demonstrate superior performance of the proposed approach when compared with the conventional methods
Recursive Renyi's entropy estimator for adaptive filtering

Recently we have proposed a recursive estimator for Renyi's quadratic entropy. This estimator can accurately converge the results for stationary signals or track the changing entropy of nonstationary signals. We demonstrate the application of the recursive entropy estimator to supervised and unsupervised training of linear and nonlinear adaptive systems. The simulations suggest a smooth and fast convergence to the optimal solution with a reduced complexity in the algorithm as compared to the batch training approach using the same entropy-based criteria. The presented approach also allows on-line information theoretic adaptation of model parameters.
Functional dependence in the human brain: A graph theoretical analysis

In this paper, we propose a graph-theoretical approach to reveal patterns of functional dependencies between different scalp regions. We start by computing pairwise measures of dependence from dense-array scalp electroencephalographic (EEG) recordings. The obtained dependence matrices are then averaged over trials and further statistically processed to provide more reliability. Graph structure information is subsequently extracted using several graph theoretical measures. Simple measures of node degree and clustering strength are shown to be useful to describe the global properties of the analyzed networks. More sophisticated measures, such as betweenness centrality and subgraph centrality tend to provide additional insight into the network structure, and therefore robustly discriminate two cognitive states. We further examine the connected components of the graph to identify the dependent functional regions. The approach supports dynamicity in that all suggested computations can be easily extended to different points in time, thus enabling to monitor dependence evolution and variability with time.
Transient model of EEG using Gini Index-based matching pursuit

We introduce a novel, transient model for the electroencephalogram (EEG) as the noisy addition of linear filters responding to trains of delta functions. We set the synthesis part as a parameter-tuning problem and obtain synthetic EEG-like data that visually resembles brain activity in the time and frequency domains. For the analysis counterpart, we use sparse approximation to decompose the signal in relevant events via Matching Pursuit. We improve this algorithm by incorporating the Gini Index as a stopping criteria; in this way, we promote sparse sources while, at the same time, eliminating one of the free parameters of Matching Pursuit. Results are presented using synthetic EEG and BCI competition data. Statistics of the model parameters are more informative and posses finer temporal resolution than classical methods such as Power Spectral Density (PSD) estimation.
Competitive principal component analysis for locally stationary time series

A new unsupervised algorithm is proposed that performs competitive principal component analysis (PCA) of a time series. A set of expert PCA networks compete, through the mixture of experts (MOE) formalism, on the basis of their ability to reconstruct the original signal. The resulting network finds an optimal projection of the input onto a reduced dimensional space as a function of the input and, hence, of time. As a byproduct, the time series is both segmented and identified according to stationary regions. Examples showing the performance of the algorithm are included.
Predicting visual attention using gamma kernels

Saliency measures are a popular way to predict visual attention. However, saliency is normally tested on sets of single resolution images that are unlike what the human vision system sees. We propose a new saliency measure based on convolving images with 2D gamma kernels which function as a comparison between a center and a surrounding neighborhood. The two parameters in the gamma kernel provide an ideal way to change the size of both the center and the surrounding neighborhood, which makes finding saliency at different scales simple and fast. We test the new saliency measure on both the CAT2000 database and the Toronto database and compare the results with other simple saliency methods. In addition, we test the methods on a foveated version of the Toronto database to test whether these methods perform well in a fixation system similar to the human vision system. Gamma saliency is shown to both perform better and compute faster than the competing methods in both the standard databases and the foveated version.
Interpreting spatial and temporal neural activity through a recurrent neural network brain-machine interface

We propose the use of optimized brain-machine interface (BMI) models for interpreting the spatial and temporal neural activity generated in motor tasks. In this study, a nonlinear dynamical neural network is trained to predict the hand position of primates from neural recordings in a reaching task paradigm. We first develop a method to reveal the role attributed by the model to the sampled motor, premotor, and parietal cortices in generating hand movements. Next, using the trained model weights, we derive a temporal sensitivity measure to asses how the model utilized the sampled cortices and neurons in real-time during BMI testing.
Period Estimation in Astronomical Time Series Using Slotted Correntropy

In this letter, we propose a method for period estimation in light curves from periodic variable stars using correntropy. Light curves are astronomical time series of stellar brightness over time, and are characterized as being noisy and unevenly sampled. We propose to use slotted time lags in order to estimate correntropy directly from irregularly sampled time series. A new information theoretic metric is proposed for discriminating among the peaks of the correntropy spectral density. The slotted correntropy method outperformed slotted correlation, string length, VarTools (Lomb-Scargle periodogram and Analysis of Variance), and SigSpec applications on a set of light curves drawn from the MACHO survey.
Reconstructed dynamics and chaotic signal modeling

A nonlinear AR model is derived from the reconstructed dynamics of a signal. The underlying system is assumed to be nonlinear, autonomous, and deterministic. In this formulation. The output error scheme is shown to be more suitable than the equation error scheme in network training. A method to incorporate the information of dynamical invariants in signal modeling is proposed. Using this global information, the authors are able to avoid the oscillation problem in training a network to model chaotic time series.<<ETX>>
Detection-interference dilemma for spectrum sensing in cognitive radio systems

Cognitive radio technique is proposed as a means of improving the efficiency of current spectrum allocation policy by allowing the unlicensed secondary users (SU) to access the unused spectrum resources (white band) of the licensed primary users (PU). This technique largely relies on the secondary users' capability to detect the unused spectrum band, thus avoiding interference to the primary users. Instead of viewing the spectrum sensing as merely a detection process, in this paper, we analyze the spectrum sensing strategies from the communication perspective, with system capacity and interference as two major system performance metrics. As a result, we then discover the dilemma between PU presence detection and the interference from the SU to the PU caused by the reciprocality of the wireless channel in large-scale fading sense. To solve this dilemma, we propose a new performance criterion called utility loss function to set the threshold in spectrum sensing. Additionally, the cooperation among spectrum sensing users is shown to effectively solve the dilemma.
Estimation and modeling of EEG amplitude-temporal characteristics using a marked point process approach.

We propose a novel interpretation of single channel Electroencephalogram (EEG) traces based on the transient nature of encoded processes in the brain. In particular, the proposed framework models EEG as the output of the noisy addition of temporal, reoccurring, transient patterns known as phasic events. This is not only neurophysiologically sound, but it also provides additional information that classical EEG analysis often disregards. Furthermore, by utilizing sparse decomposition techniques, it is possible to obtain amplitude and timing that is further modeled using estimation and fitting techniques. We model Brain-Computer Interfaces (BCI) competition data features as Gaussian Mixture Model (GMM) samples in order to show the potential of working in the joint space of the parameters. The results not only preserve the topographic discriminant behavior but also expand the realm of possible EEG analysis.
The Laplacian Classifier

We develop a novel classifier In a kernel feature space related to the eigenspectrum of the Laplacian data matrix. The classification cost function measures the angle between class mean vectors in the kernel feature space, and is derived from an information theoretic divergence measure using Parzen windowing. The classification rule is expressed in terms of a weighted kernel expansion. The weighting associated with a data point is inversely proportional to the probability density at that point, emphasizing the least probable regions. No optimization is needed to determine the weighting scheme, as opposed to the support vector machine. The connection to Parzen windowing also provides a theoretical criterion for kernel size selection, reducing the need for computationally demanding cross-validation. We show that the new classifier performs better than the Parzen window Bayes classifier, and in many cases comparable to the support vector machine, at a computationally lower cost.
A parameter-free kernel design based on cumulative distribution function for correntropy

This paper proposes a parameter-free kernel that is translation invariant and positive definite. The new kernel is based on the data cumulative distribution function (CDF) that provides all the statistical information about the observed samples. Without an explicit kernel size parameter, this novel kernel is used to define the autocorrentropy function, which is a generalized similarity measure, and spectral density estimator. Numerical examples show that the proposed method provides comparable performance to the existing Gaussian kernel with optimized kernel size.
A bio-amplifier with pulse output

A low-power fully integrated bioamplifier is presented that can amplify signals in the range from mHz to kHz while rejecting large DC offsets generated at the electrode-tissue interface. The novel aspect of this amplifier is that its analog output is represented by a series of pulses which provide a low-power, noise-resistant means for coding and transmission. The original analog signal can be reconstructed from the resulting pulse train with 13 bit precision at a remote site where power consumption is not so crucial. The fabricated analog amplifier exhibits a gain of 39.5 dB from 0.3 Hz to 5.4k Hz. The power consumption of the whole system is less than 300 /spl mu/W/channel from a 5-V supply. The fully integrated system was designed in the AMI 0.6 /spl mu/m CMOS process and it consumes 0.088 mm/sup 2//channel of chip area.
Advanced search algorithms for information-theoretic learning with kernel-based estimators

Recent publications have proposed various information-theoretic learning (ITL) criteria based on Renyi's quadratic entropy with nonparametric kernel-based density estimation as alternative performance metrics for both supervised and unsupervised adaptive system training. These metrics, based on entropy and mutual information, take into account higher order statistics unlike the mean-square error (MSE) criterion. The drawback of these information-based metrics is the increased computational complexity, which underscores the importance of efficient training algorithms. In this paper, we examine familiar advanced-parameter search algorithms and propose modifications to allow training of systems with these ITL criteria. The well known algorithms tailored here for ITL include various improved gradient-descent methods, conjugate gradient approaches, and the Levenberg-Marquardt (LM) algorithm. Sample problems and metrics are presented to illustrate the computational efficiency attained by employing the proposed algorithms.
Local Hammerstein modeling based on self-organizing map

This work presents a method to determine a local polynomial model from a finite number of measurements of the inputs and outputs for Hammerstein systems which are a zero-memory nonlinearity followed by a linear filter. Self-organizing map (SOM) is utilized to cluster the dynamics in the input-output joint space, where processing-elements (PEs) are extended with local models to enable the original algorithm to learn input-output relationships with reasonable accuracy. Moreover, in order to increase the approximation accuracy, local models are built by polynomial models instead of just linear models. The identification method is applied to two simulation examples of a discrete-time system and compared with other neural networks-based alternatives to demonstrate the performance and efficiency of the proposed technique.
A reconfigurable parallel FPGA accelerator for the adapt-then-combine diffusion LMS algorithm

The combination of diffusion strategies and least-mean-square (LMS) algorithm provides many advantages for adaptive-filter to solve distributed optimization, estimation and inference problems. However, suffering from high computation complexity, software implementation of diffusion LMS algorithm is unsuitable for real-time and portable applications. In order to extend its availability, we design a reconfigurable parallel FPG accelerator by exploring multiple dimensions of parallelism, including: parallel execution of agents state updating, data combining, data training and multi-stages pipeline to speedup the execution time. The accelerator for networks with various number of agents and different input dimensions is implemented. Results demonstrate that, it can achieve a speedup of three orders of magnitude at 100Mhz compared with C implementation for a 32-nodes network with 16-dimensional input-data.
A spatiotemporal vector quantizer for missing sample reconstruction

In this paper we present the application of a spatiotemporal memory motivated by reaction diffusion mechanisms for missing sample reconstruction. The model is a spatiotemporal vector quantizer trained to learn and recall sequences that have a temporal order. This vector quantization scheme has an interesting property of creating time varying Voronoi cells that not only cluster feature vectors based on spatial proximity but also can provide information of the next anticipating cluster with a certain probability. This can be exploited to develop a scheme for predicting a temporal sequence. Unlike conventional prediction models which estimate the desired sample based on a linear combination of a few past samples and local statistics, this scheme employs a nonlinear memory structure to predict samples iteratively. The method is interesting in that it can predict samples missing in a burst.
A local linear modeling paradigm with a modified counterpropagation network

The counter-propagation neural network (CPN) was selected to investigate the modeling problems because it integrates both supervised and unsupervised learning. The network is taught to have clusters that are described by codebook vectors in the training phase. The basic CPN algorithm is modified incorporating the local linear models (LLMs) to provide functional mappings and identify potentially nonlinear plants. The objective is a reduction of the approximation error in a CPN. In this framework the quantization error in the output space serves as a basis for the LLMs in the output space. The performance of the proposed algorithms is tested on the nonlinear dynamic system and shows the influence of this modification on the system identification quality.