nils-herrmann/clean_scirepeval huggingface dataset:
Sequential Bayesian Decoding with a Population of Neurons

Population coding is a simplified model of distributed information processing in the brain. This study investigates the performance and implementation of a sequential Bayesian decoding (SBD) paradigm in the framework of population coding. In the first step of decoding, when no prior knowledge is available, maximum likelihood inference is used; the result forms the prior knowledge of stimulus for the second step of decoding. Estimates are propagated sequentially to apply maximum a posteriori (MAP) decoding in which prior knowledge for any step is taken from estimates from the previous step. Not only do we analyze the performance of SBD, obtaining the optimal form of prior knowledge that achieves the best estimation result, but we also investigate its possible biological realization, in the sense that all operations are performed by the dynamics of a recurrent network. In order to achieve MAP, a crucial point is to identify a mechanism that propagates prior knowledge. We find that this could be achieved by short-term adaptation of network weights according to the Hebbian learning rule. Simulation results on both constant and time-varying stimulus support the analysis.
Subspaces of text discrimination with application to biological literature

This paper is about the application of statistical pattern recognition techniques to the classification of text with the objective of retrieving documents relevant for the construction of gene networks. We start from the usual practice of representing a document, electronically available abstracts of scientific papers in this case, as a high dimensional vector of term of occurrences. We consider the problem of retrieving documents corresponding to the metabolic pathway of the organism yeast, Saccharomyces Cerevisiae, using a trained classifier as filter. We use support vector machines (SVMs) as classifiers and compare techniques for reducing the dimensionality of the problem: latent semantic kernels (LSK) and sequential forward selection (SFS). In order to deal with the issue of having only a small set of accurately labelled documents, we used the approach of transductive inference. In this case, LSK leads to a subspace formed as a linear combination of features (terms in the lexicon) while SFS selects a subset of the dimension. We find, for this problem, that the discriminant information appears to lie in a subspace, which is very small in dimensionality compared to that of the original formulation. By matching against the gene ontology (GO) database, we further find that the selection process (SFS) picks out the discriminant terms that are of biological significance for this problem.
Uncertainty in geometric computations

Contributors and Participants. Preface. 1. Affine Intervals in a CSG Geometric Modeller A. Bowyer, et al. 2. Fast and Reliable Plotting of Implicit Curves K. Buehler. 3. Data Assimilation with Sequential Gaussian Processes L. Csato, et al. 4. Conformal Geometry, Euclidean Space and Geometric Algebra C. Doran, et al. 5. Towards the Robust Intersection of Implicit Quadrics L. Dupont, et al. 6. Computational Geometry and Uncertainty A.R. Forrest. 7. Geometric Uncertainty in Solid Modeling P.J. Malraison, W.A. Denker. 8. Reliable Geometric Computations with Algebraic Primitives and Predicates M. Foskey, et al. 9. Feature Localization Error in 3D Computer Vision D.D. Morris, T. Kanade. 10. Bayesian Analysis of Computer Model Outputs J. Oakley, A. O'Hagan. 11. b. T. Poggio, et al. 12. Affine Arithmetic and Bernstein Hull Methods for Algebraic Curve Drawing H. Shou, et al. 13. Local Polynomial Metrics for kappa Nearest Neighbor Classifiers R.R. Snapp. 14. Visualisation of Incomplete Data Using Class Information Constraints Y. Sun, et al. 15. Towards Videorealistic Synthetic Visual Speech B. Theobald, et al. 16. Properties of the Companion Matrix Resultant for Bernstein Polynomials J.R. Winkler. 17. Computation with a Number of Neurons S. Wu, D. Chen. Index.
Comparison of Four Machine Learning Methods for Predicting PM 10 Concentrations in Helsink

Machine learning methods can offer a practical alternative to deterministic and statistical methods for predicting air pollution concentrations. However, for a given data set, it is often not clear beforehand which machine learning method will yield the best prediction performance. This study compares the variable selection and prediction performance of four machine-learning methods of different complexity: logistic regression, decision tree, multivariate adaptive regression splines and neural network. The methods are applied to the task of predicting the exceedance of the European PM10 daily average objective of 50 μg m −3 for a station in Helsinki, Finland. Our study shows that some predictors were selected by all models but that the different models also picked different variables. The performance of three of the four methods investigated was very similar, however, performance of the decision tree method was significantly inferior. Performance was sensitive to the learning sample size and time period used.
Comparison of Four Machine Learning Methods for Predicting PM10 Concentrations in Helsinki, Finland

Machine learning methods can offer a practicalalternative to deterministic and statistical methods forpredicting air pollution concentrations. However, for agiven data set, it is often not clear beforehand whichmachine learning method will yield the best predictionperformance. This study compares the variable selection andprediction performance of four machine-learning methods ofdifferent complexity: logistic regression, decision tree,multivariate adaptive regression splines and neuralnetwork. The methods are applied to the task of predictingthe exceedance of the European PM10 daily averageobjective of 50 μg m-3 for a station in Helsinki,Finland. Our study shows that some predictors were selectedby all models but that the different models also pickeddifferent variables. The performance of three of the fourmethods investigated was very similar, however, performanceof the decision tree method was significantly inferior.Performance was sensitive to the learning sample size andtime period used.
Speech enhancement using a Bayesian evidence approach

We consider the enhancement of speech corrupted by additive white Gaussian noise. In a Bayesian inference framework, maximum a posteriori (MAP) estimation of the signal is performed, along the lines developed by Lim & Oppenheim (1978). The speech enhancement problem is treated as a signal estimation problem, whose aim is to obtain a MAP estimate of the clean speech signal, given the noisy observations. The novelty of our approach, over previously reported work, is that we relate the variance of the additive noise and the gain of the autoregressive (AR) process to hyperparameters in a hierarchical Bayesian framework. These hyperparameters are computed from the noisy speech data to maximize the denominator in Bayes formula, also known as the evidence. The resulting Bayesian scheme is capable of performing speech enhancement from the noisy data without the need for silence detection. Experimental results are presented for stationary and slowly varying additive white Gaussian noise. The Bayesian scheme is also compared to the Lim and Oppenheim system, and the spectral subtraction method.
Clinical validation of an artificial neural network trained to identify acute allograft rejection in liver transplant recipients

Artificial neural networks (ANNs) are techniques of nonlinear data modeling that have been studied in a wide variety of medical applications. An ANN was developed to assist in the diagnosis of acute rejection in liver transplant recipients. We investigated the diagnostic accuracy of this ANN on a new data set of patients from the same hospital. In addition, we compared the diagnostic accuracy of the ANN with that of the individual input variables (alanine aminotransferase [ALT] and bilirubin levels and day posttransplantation). Clinical and biochemical data were collected retrospectively for 124 consecutive liver transplantations (117 patients) over the first 3 months after transplantation. Diagnostic accuracy was calculated using receiver operating characteristic (ROC) curve analysis. The ANN differentiated rejection from rejection‐free episodes in the new data set over the first 3 months posttransplantation with an area under the ROC curve of 0.902 and sensitivity and specificity of 80.0% and 90.1% at the optimum decision threshold, respectively. The ANN was significantly more specific than ALT or bilirubin level or day posttransplantation at their corresponding optimum decision thresholds (P < .0001). Peak ANN output occurred 1 day earlier than peak values for either ALT or bilirubin (P < .005). The diagnostic accuracy of the ANN was greater than that of any of the individual variables that had been used as inputs. It would be a useful adjunct to conventional liver function tests for monitoring liver transplant recipients in the early postoperative period.
Data-dependent kernels in svm classification of speech patterns

Support Vector Machines (SVMs) have recently proved to be powerful pattern classi cation tools with a strong connection to statistical learning theory. One of the hurdles to using SVMs in speech recognition, and a crucial aspect of SVM design in general, is the choice of the kernel function for non-separable data, and the setting of its parameters. This is often based on experience or a potentially costly search. This paper gives some experimental justi cation for the Fisher kernels proposed in ; kernels are obtained and their extra regularisation and use of labelled and unlabelled data discussed. Fisher kernels are derived from generative probability models of the data, and are a rststep to implementing kernels for variable length sequences.
Sequential Monte Carlo Methods to Train Neural Network Models

We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimization strategy that allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear, and nongaussian signal processing. We show how the new algorithm outperforms extended Kalman filter training on several problems. In particular, we address the problem of pricing option contracts, traded in financial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices.
Dynamic Learning with the EM Algorithm for Neural Networks

In this paper, we derive an EM algorithm for nonlinear state space models. We use it to estimate jointly the neural network weights, the model uncertainty and the noise in the data. In the E-step we apply a forward-backward Rauch-Tung-Striebel smoother to compute the network weights. For the M-step, we derive expressions to compute the model uncertainty and the measurement noise. We find that the method is intrinsically very powerful, simple and stable.
Hierarchical Bayesian Models for Regularization in Sequential Learning

We show that a hierarchical Bayesian modeling approach allows us to perform regularization in sequential learning. We identify three inference levels within this hierarchy: model selection, parameter estimation, and noise estimation. In environments where data arrive sequentially, techniques such as cross validation to achieve regularization or model selection are not possible. The Bayesian approach, with extended Kalman filtering at the parameter estimation level, allows for regularization within a minimum variance framework. A multilayer perceptron is used to generate the extended Kalman filter nonlinear measurements mapping. We describe several algorithms at the noise estimation level that allow us to implement on-line regularization. We also show the theoretical links between adaptive noise estimation in extended Kalman filtering, multiple adaptive learning rates, and multiple smoothing regularization coefficients.
Matched filter design for diphone subspace models

Considering the perceptual importance of phonetic transitions as minimal contextual variant units, this paper addresses the problem by modelling explicitly interphone dynamics covered in diphones. Subspace projections based on a time-constrained PCA (TC-PCA) are developed which focus on the temporal evolution. They reveal characteristic trajectories present in a low-dimensional spectral representation facilitating robust parameter estimation and simultaneously optimise the discriminant information. A matched filter design is applied to a multiple hypotheses rescoring scheme which enables operating in very low-dimensional parameter space. Using such multiple hypotheses paradigm the complementary information effectiveness of modelling explicitly inter-phone dynamics covered in diphones can be shown using the TIMIT database, resulting in improved phone error rates.
Neuro-computing versus linear statistical techniques applied to liver transplant monitoring: a comparative study

Explores the potential for the application of neurocomputing in on-line monitoring in the liver transplantation domain. It extends the authors' previously documented work to provide both an assessment of the performance gains achievable by incorporating temporal and dynamical information about the measurements made on a patient as well as presenting a novel computerized clinical decision aid for this domain. A comparison of the performance of linear and nonlinear classification system is made and used to motivate the final selection of the diagnostic inputs.
Speech Modelling Using Subspace and EM Techniques

The speech waveform can be modelled as a piecewise-stationary linear stochastic state space system, and its parameters can be estimated using an expectation-maximisation (EM) algorithm. One problem is the initialisation of the EM algorithm. Standard initialisation schemes can lead to poor formant trajectories. But these trajectories however are important for vowel intelligibility. The aim of this paper is to investigate the suitability of subspace identification methods to initialise EM. 
 
The paper compares the subspace state space system identification (4SID) method with the EM algorithm. The 4SID and EM methods are similar in that they both estimate a state sequence (but using Kalman filters and Kalman smoothers respectively), and then estimate parameters (but using least-squares and maximum likelihood respectively). The similarity of 4SID and EM motivates the use of 4SID to initialise EM. Also, 4SID is non-iterative and requires no initialisation, whereas EM is iterative and requires initialisation. However 4SID is sub-optimal compared to EM in a probabilistic sense. During experiments on real speech, 4SID methods compare favourably with conventional initialisation techniques. They produce smoother formant trajectories, have greater frequency resolution, and produce higher likelihoods.
Hybrid sequential Monte Carlo/Kalman methods to train neural networks in non-stationary environments

We propose a novel sequential algorithm for training neural networks in non-stationary environments. The approach is based on a Monte Carlo method known as the sampling-importance resampling simulation algorithm. We derive our algorithm using a Bayesian framework, which allows us to learn the probability density functions of the network weights and outputs. Consequently, it is possible to compute various statistical estimates including centroids, modes, confidence intervals and kurtosis. The algorithm performs a global search for minima in parameter space by monitoring the errors and gradients at several points in the error surface. This global optimisation strategy is shown to perform better than local optimisation paradigms such as the extended Kalman filter.
Sequential Bayesian computation of logistic regression models

The extended Kalman filter (EKF) algorithm for identification of a state space model is shown to be a sensible tool in estimating a logistic regression model sequentially. A Gaussian probability density over the parameters of the logistic model is propagated on a sample by sample basis. Two other approaches, the Laplace approximation and the variational approximation are compared with the state space formulation. Features of the latter approach, such as the possibility of inferring noise levels by maximising the "innovation probability" are indicated. Experimental illustrations of these ideas on a synthetic problem and two real world problems are discussed.
Diphone subspace models for phone-based HMM complementation

Considering the perceptual importance of phonetic transitions as minimal contextual variant units, this paper addresses the problem by modelling explicitly interphone dynamics covered in diphones. Subspace projections based on a time-constrained PCA (TC-PCA) are developed which focus on the temporal evolution. They reveal characteristic trajectories present in a lowdimensional spectral representation facilitating robust parameter estimation and simultaneously optimise the discriminant information. The applied multiple hypotheses rescoring scheme enables operating in very low-dimensional parameter space. Using such multiple hypotheses paradigm the complementary information e ectiveness of modelling explicitly inter-phone dynamics covered in diphones can be shown using the TIMIT database, resulting in improved phone error rates.
Why I am not a non-Bayesian? [Medical diagnostics application]

Recently there has been much interest, and some hype, on the use of Bayesian methods in problems of machine learning and inference. While it is true that the Bayesian methodology offers a nice way of specifying uncertainties about a problem domain and carrying them through the modelling process to any inference one makes, there are very few convincing practical demonstrations of it. This is particularly the case when Bayesian techniques are coupled with powerful nonlinear function approximators such as neural networks. Since nonlinear models induce complicated probability densities, approximations become necessary. It is not clear how much of the elegance of the Bayesian framework is lost in the presence of these approximations. The alternative approach is to represent the probability densities in a nonparametric way, by resorting to Markov Chain Monte Carlo techniques. Here too, algorithmic as well as computational limitations exist. However, I have recently worked on some problems in which useful algorithmic advantages can be achieved using Bayesian techniques. This talk is a review of the above concepts with an emphasis on their application to two problems in medical diagnostics: (a) predicting adverse outcome in pregnancy, and (b) monitoring liver transplant patients for rejection of transplanted organ.
Diphone multi-trajectory subspace models

We report on the extension of capturing speech transitions embedded in diphones using trajectory models. The slowly varying dynamics of spectral trajectories carry much discriminant information that is very crudely modelled by traditional approaches such as HMMs. We improved our methodology of explicitly capturing the trajectory of short time spectral parameter vectors introducing multi-trajectory concepts in a probabilistic framework. Optimal subspace selection is presented which finds the most discriminant plane for classification. Using the E-set from the TIMIT database results suggest that discriminant information is preserved in the subspace.
Speech Modelling: Models, Parameter Estimation and Its Application to Speech Enhancement

In this report system identiication techniques are applied to speech. The purpose is to compare diierent models and diierent parameter estimation techniques on both a theoretical and an empirical basis. Results are then used in the practical problem of speech enhancement in additive white Gaussian noise. Two model families are identiied: polynomial and state space models. These are compared within a common state space framework, which makes explicit the assumptions and constraints of diierent models regarding process noise, observation noise, input-output delays and initial conditions. State space models are then cast in block matrix form because this representation is used in subspace state space system identiication (4SID) methods. Next, three common parameter estimation techniques are compared: prediction error minimisation (PEM), instrumental variables (IV) and 4SID. These models and parameter estimation techniques are compared through experiments on real clean and noisy speech data, and evaluated in terms of their prediction errors, spectrograms and perceptual quality of the one-step-ahead predicted waveform. Finally, the best models are used to initialise Kalman lters which are used to lter noisy speech (where noise is white, additive and Gaussian) in the speech enhancement problem. In general, the results are that modelling accuracy is improved by using the glottal waveform, a more general noise model and non-zero initial conditions. This is evident by reduced model prediction errors and better noise model spectrograms. Voiced speech can be more accurately modelled than non-voiced speech. Regarding parameter estimation techniques, PEM gives smallest prediction errors, then 4SID then IV. 4SID methods have advantages, for example the one-step-ahead predicted waveform does not seem to suuer from musical noise like PEM methods. Other advantages include numerical stability and the use of a frequency-weighted balanced state space basis, which allows model order to be reduced in a simpler and better manner. In addition, PEM and 4SID weight modelling errors diierently in the frequency domain. PEM, 4SID and IV methods can be used to initialise a Kalman lter, which can be applied to speech enhancement.
Support vector machines: a tutorial overview and critical appraisal

There has been much interest in the use of support vector machines (SVM) as an approach to high performance pattern classification. In the linearly separable case, SVMs attempt to position a class boundary so that the margin from the nearest example is maximised. This criterion can be implemented by solving a quadratic programming problem, and the solution turns out to be one in which the class boundary may be expressed as a linear combination of a subset of the training data (the support vectors). The elegance of the QP formulation, and the relationship between control of complexity in this formulation and Vapnik-Chervonenkis dimensions are seen as prime attractions of the SVM method. A related idea in high performance pattern classification is that of boosting multiple classifiers. The author shows that the standard SVM formulation is not robust to noise and explains the performance of boosting algorithms by reference to receiver operating characteristics curves.
Subspace techniques in speech enhancement

This paper concerns speech enhancement. Speech is first modelled and then model parameters used to design a Kalman smoother to reduce the background noise. The 4SID (subspace state space system identification), DOA (direction-of-arrival) and polynomial techniques are described within a common subspace state space framework, and assumptions for each technique stated. It is shown that 4SID methods make least assumptions about the data, provide detailed spectra for the vocal tract, and can be used effectively for speech enhancement when combined with a Kalman smoother. An iterative technique is also possible.