nils-herrmann/clean_scirepeval huggingface dataset:
Representative Sampling for Text Classification Using Support Vector Machines

In order to reduce human efforts, there has been increasing interest in applying active learning for training text classifiers. This paper describes a straightforward active learning heuristic, representative sampling, which explores the clustering structure of 'uncertain' documents and identifies the representative samples to query the user opinions, for the purpose of speeding up the convergence of Support Vector Machine (SVM) classifiers. Compared with other active learning algorithms, the proposed representative sampling explicitly addresses the problem of selecting more than one unlabeled documents. In an empirical study we compared representative sampling both with random sampling and with SVM active learning. The results demonstrated that representative sampling offers excellent learning performance with fewer labeled documents and thus can reduce human efforts in text classification tasks.
Assessment of proximal finger joint inflammation in patients with rheumatoid arthritis, using a novel laser-based imaging technique.

To evaluate a newly developed laser-based imaging technique for the study of soft tissue changes and acute inflammatory processes of the proximal interphalangeal (PIP) joints in patients with rheumatoid arthritis (RA).



A novel imaging device was developed which allows the transillumination of PIP joints using laser light in the near-infrared wavelength range. In a first clinical followup study, a total of 72 PIP joints of 22 patients with RA and 64 PIP joints of 8 healthy controls were examined both clinically and with the new laser device. At baseline and at followup after a mean of 6 weeks, clinical signs of synovitis, the joint circumference, and the degree of pain were assessed for each PIP joint in order to determine the clinical degree of inflammation. Different features were extracted from the laser images and evaluated by a neural network.



At baseline, 72 PIP joints in the RA patients showed clinical signs of inflammation. At followup, 45 PIP joints showed clinical improvement, 13 showed steady active inflammation, and 14 showed deterioration compared with the first visit. None of the 64 PIP joints in the healthy individuals showed any signs of synovitis. The inflammatory status of 60 of the 72 RA joints examined was classified correctly by laser examination and joint circumference determination, giving a sensitivity of 80%, a specificity of 89%, and an accuracy of 83% in detecting inflammatory changes in affected joints. Laser data and joint circumference determination of healthy joints at followup resulted in an accuracy of 85% in reproducing the image.



The new laser-based imaging technique allows the transillumination of PIP joints and gives information about the inflammatory status of the joint after processing through a neural network. Our data indicate that laser imaging may provide additional information in the early diagnosis of an inflammatory joint process and may prove particularly useful in assessing acute joint inflammation at followup.
A Nonlinear State Space Model for the Blood Glucose Metabolism of a Diabetic (Ein nichtlineares Zustandsraummodell für den Blutglukosemetabolismus eines Diabetikers)

The blood glucose metabolism of a diabetic is a complex nonlinear process closely linked to a number of internal factors which are not easily accessible to measurement. Based on accessible information – such as occasional blood glucose measurements and information about food intake and physical exercise – the system appears highly stochastic and the quantity of main interest, the blood glucose concentration, is very difficult to model and to predict. In this paper we describe a stochastic nonlinear state space model for modeling the blood glucose concentration of a diabetic patient. The model structure is based on physiological prior knowledge and the main nonlinearities are modeled using artificial neural networks. Offline training of the model is performed using a newly developed Monte-Carlo generalized EM (expectation maximization) algorithm. Online prediction is performed using particle filters. Our experimental results show that our approach provides better prediction results than a number of competing approaches.
Knowledge Acquisition and Automated Generation of Bayesian Networks for a Medical Dialogue and Advisory System

Probabilistic models such as Bayesian networks [6] are well suited for medical decision support and are the basis of many successful applications [1],[3],[4],[8],[9],[10]. Bayesian networks provide a rigorous and efficient framework for inference, i.e. for calculating the probability of each stochastic variable given a set of observations. However, knowledge acquisition and generation of the network are still demanding tasks when large medical domains have to be modelled.
A learning vector quantization algorithm for probabilistic models

In classification problems, it is preferred to attack the discrimination problem directly rather than indirectly by first estimating the class densities and by then estimating the discrimination function from the generative models through Bayes's rule. Sometimes, however, it is convenient to express the models as probabilistic models, since they are generative in nature and can handle the representation of high-dimensional data like time-series. In this paper, we derive a discriminative training procedure based on Learning Vector Quantization (LVQ) where the codebook is expressed in terms of probabilistic models. The likelihood-based distance measure is justified using the Kullback-Leibler distance. In updating the winner unit, a gradient learning step is taken with regard to the parameters of the probabilistic model. The method essentially departs from a prototypical representation and incorporates learning in the parameter space of generative models. As an illustration, we present experiments in the fraud detection domain, where models of calling behavior are used to classify mobile phone subscribers to normal and fraudulent users. This is an extension of our earlier work in clustering probabilistic models with the Self-Organizing Map (SOM) algorithm to the classification domain.
Mixtures of Gaussian Processes

We introduce the mixture of Gaussian processes (MGP) model which is useful for applications in which the optimal bandwidth of a map is input dependent. The MGP is derived from the mixture of experts model and can also be used for modeling general conditional probability densities. We discuss how Gaussian processes - in particular in form of Gaussian process classification, the support vector machine and the MGP model--can be used for quantifying the dependencies in graphical models.
A Bayesian Committee Machine

The Bayesian committee machine (BCM) is a novel approach to combining estimators that were trained on different data sets. Although the BCM can be applied to the combination of any kind of estimators, the main foci are gaussian process regression and related systems such as regularization networks and smoothing splines for which the degrees of freedom increase with the number of training data. Somewhat surprisingly, we find that the performance of the BCM improves if several test points are queried at the same time and is optimal if the number of test points is at least as large as the degrees of freedom of the estimator. The BCM also provides a new solution for on-line learning with potential applications to data mining. We apply the BCM to systems with fixed basis functions and discuss its relationship to gaussian process regression. Finally, we show how the ideas behind the BCM can be applied in a non-Bayesian setting to extend the input-dependent combination of estimators.
Procede et dispositif pour la modelisation d'un systeme

L'invention concerne un procede et un dispositif pour la determination d'au moins un modele partiel faisant partie d'un modele d'un systeme. Selon l'invention, un etat du systeme est decrit a l'aide de grandeurs d'etat, au moins une de ces grandeurs d'etat etant une grandeur d'etat discrete. Plusieurs ensembles de valeurs de ces grandeurs d'etat sont determines. A partir des ensembles de valeurs, une distribution de probabilite concernant les grandeurs d'etat est determinee. A l'aide des ensembles de donnees et de la distribution de probabilite concernant les grandeurs d'etat, et a l'aide d'un procede d'apprentissage statistique, le modele partiel du systeme est determine, ce modele partiel decrivant le systeme conformement a la distribution de probabilite relative aux grandeurs d'etat.
The generalized Bayesian committee machine

In this paper we introduce the Generalized Bayesian Committee Machine (GBCM) for applications with large data sets. In particular, the GBCM can be used in the context of kernel based systems such as smoothing splines, kriging, regularization networks and Gaussian process regression which —for computational reasons— are otherwise limited to rather small data sets. The GBCM provides a novel and principled way of combining estimators trained for regression, classification, the prediction of counts, the prediction of lifetimes and other applications which can be derived from the exponential family of distributions. We describe an online version of the GBCM which only requires one pass through the data set and only requires the storage of a matrix of the dimension of the number of query or test points. After training, the prediction at additional test points only requires resources dependent on the number of query points but is independent of the number of training data. We confirm the good scaling behavior using real and experimental data sets.
A hidden Markov model for metric and event-based data

The question of data representation is central to any data analysis problem. Ideally, the representation should faithfully describe the domain to be analyzed and in addition, the model used should be able to process such a representation. In practice, however, the modeler must often compromise how the problem is described, since the class of possible representations is constrained by the model. This problem may be circumvented by extending conventional models to handle more unconventional data representations. These data are often found in industrial environments and especially in telecommunications. In this paper, we consider an extension of hidden Markov models (HMM) for modeling data streams, which switch between metric and event-based representations. In a HMM, the representation of the observed data is constrained by the emission probability density. Since this density can not change its representation once it is fixed, modeling data streams involving different types of data semantics can be difficult. In the extension introduced in this paper, an additional data semantics variable is introduced, which is conditional on the hidden variable. Furthermore, data itself is conditioned on its semantics, which enables correct interpretation of the observed data. We briefly review the essentials of HMMs and present our extended architecture. We proceed by introducing inference and learning rules for the extension. As an application, we present a HMM for user profiling in mobile communications networks, where the data exhibits switching behavior.
Dynamic Neural Regression Models

We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network' s parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by deriving approximate posterior means with the importance sampling method. Furthermore, we replace the commonly used Gaussian noise assumption in the neural regression model by a more flexible noise model based on the Student t-density. Since the t-density can be interpreted as being an infinite mixture of Gaussians, hyperparameters such as the degrees of freedom of the t-density can be learned from the data based on an online EM-type algorithm. We show experimentally that our novel methods outperform state-of-the art neural network online learning algorithms like the extended Kalman filter method for both, situations with standard Gaussian noise terms and situations with measurement outliers.
Robust Neural Network Regression for Offline and Online Learning

We replace the commonly used Gaussian noise model in nonlinear regression by a more flexible noise model based on the Student-t- distribution. The degrees of freedom of the t-distribution can be chosen such that as special cases either the Gaussian distribution or the Cauchy distribution are realized. The latter is commonly used in robust regression. Since the t-distribution can be interpreted as being an infinite mixture of Gaussians, parameters and hyperparameters such as the degrees of freedom of the t-distribution can be learned from the data based on an EM-learning algorithm. We show that modeling using the t-distribution leads to improved predictors on real world data sets. In particular, if outliers are present, the t-distribution is superior to the Gaussian noise model. In effect, by adapting the degrees of freedom, the system can "learn" to distinguish between outliers and non-outliers. Especially for online learning tasks, one is interested in avoiding inappropriate weight changes due to measurement outliers to maintain stable online learning capability. We show experimentally that using the t-distribution as a noise model leads to stable online learning algorithms and outperforms state-of-the art online learning methods like the extended Kalman filter algorithm.
Model-independent mean-field theory as a local method for approximate propagation of information

We present a systematic approach to mean-field theory (MFT) in a general probabilistic setting without assuming a particular model. The mean-field equations derived here may serve as a local, and thus very simple, method for approximate inference in probabilistic models such as Boltzmann machines or Bayesian networks. Our approach is 'model-independent' in the sense that we do not assume a particular type of dependences; in a Bayesian network, for example, we allow arbitrary tables to specify conditional dependences. In general, there are multiple solutions to the mean-field equations. We show that improved estimates can be obtained by forming a weighted mixture of the multiple mean-field solutions. Simple approximate expressions for the mixture weights are given. The general formalism derived so far is evaluated for the special case of Bayesian networks. The benefits of taking into account multiple solutions are demonstrated by using MFT for inference in a small and in a very large Bayesian network. The results are compared with the exact results.
Mixture Approximations to Bayesian Networks

Structure and parameters in a Bayesian network uniquely specify the probability distribution of the modeled domain. The locality of both structure and probabilistic information are the great benefits of Bayesian networks and require the modeler to only specify local information. On the other hand this locality of information might prevent the modeler -and even more any other personfrom obtaining a general overview of the important relationships within the domain. The goal of the work presented in this paper is to provide an "alternativen view on the knowledge encoded in a Bayesian network which might sometimes be very helpful for providing insights into the underlying domain. The basic idea is to calculate a mixture approximation to the probability distribution represented by the Bayesian network. The mixture component densities can be thought of as representing typical scenarios implied by the Bayesian model, providing intuition about the basic relationships. As an additional benefit, performing inference in the approximate model is very simple and intuitive and can provide additional insights. The computational complexity for the calculation of the mixture approximations critically depends on the measure which defines the distance between the probability distribution represented by the Bayesian network and the approximate distribution. Both the KL-divergence and the backward KL-divergence lead to inefficient algorithms. Incidentally, the latter is used in recent work on mixtures of mean field solutions to which the work presented here is closely related. We show, however, that using a mean squared error cost function leads to update equations which can be solved using the junction tree algorithm. We conclude that the mean squared error cost function can be used for Bayesian networks in which inference based on the junction tree is tractable. For large networks, however, one may have to rely on mean field approximations.
Model-independent mean-field theory as a local method for approximate propagation of information.

We present a systematic approach to mean-field theory (MFT) in a general probabilistic setting without assuming a particular model. The mean-field equations derived here may serve as a local, and thus very simple, method for approximate inference in probabilistic models such as Boltzmann machines or Bayesian networks. Our approach is 'model-independent' in the sense that we do not assume a particular type of dependences; in a Bayesian network, for example, we allow arbitrary tables to specify conditional dependences. In general, there are multiple solutions to the mean-field equations. We show that improved estimates can be obtained by forming a weighted mixture of the multiple mean-field solutions. Simple approximate expressions for the mixture weights are given. The general formalism derived so far is evaluated for the special case of Bayesian networks. The benefits of taking into account multiple solutions are demonstrated by using MFT for inference in a small and in a very large Bayesian network. The results are compared with the exact results.
Robust neural network online learning in time-variant regression models

We consider robust online learning in time-variant neural network regression models. Using a state space representation for the neural network's weight evolution in time we derive weight estimates by maximizing posterior modes via the Fisher scoring algorithm. By taking the family of densities as the output error cost function we get a robust error measure suitable for handling additive outliers. Fisher scoring was implemented using a forward backward pass of fixed length through the data set for every time step resulting in so-called online smoothing algorithms. Furthermore, we derive an EM-type algorithm for approximate maximum likelihood estimation of unknown hyperparameters. Our experiments show that online posterior mode weight smoothing outperforms standard online methods like online backpropagation and extended Kalman filtering, both for Gaussian measurements and non-Gaussian measurements with additive outliers.
Neural-network models for the blood glucose metabolism of a diabetic

We study the application of neural networks to modeling the blood glucose metabolism of a diabetic. In particular we consider recurrent neural networks and time series convolution neural networks which we compare to linear models and to nonlinear compartment models. We include a linear error model to take into account the uncertainty in the system and for handling missing blood glucose observations. Our results indicate that best performance can be achieved by the combination of the recurrent neural network and the linear error model.
Mean field inference in a general probabilistic setting

We present a systematic, model-independent formulation of mean eld theory (MFT) as an inference method in probabilistic models. \Model-independent" means that we do not assume a particular type of dependency among the variables of a domain but instead work in a general probabilistic setting. In a Bayesian network, for example, you may use arbitrary tables to specify conditional dependencies and thus run MFT in any Bayesian network. Furthermore, the general mean eld equations derived here shed a light on the essence of MFT. MFT can be interpreted as a local iteration scheme which relaxes in a consistent state (a solution of the mean eld equations). Iterating the mean eld equations means propagating information through the network. In general, however , there are multiple solutions to the mean eld equations. We show that improved approximations can be obtained by forming a weighted mixture of the multiple mean eld solutions. Simple approximate expressions for the mixture weights are given. The bene-ts of taking into account multiple solutions are demonstrated by using MFT for inference in a small Bayesian network representing a medical domain. Thereby it turns out that every solution of the mean eld equations can be interpreted as a `disease scenario'.
A Self-Organizing Map for Clustering Probabilistic Models

We present a general framework for self-organizing maps, which store probabilistic models in map units. We introduce the negative log probability of the data sample as the error function and motivate its use by showing its correspondence to the Kullback-Leibler distance between the unknown true distribution of data and our empirical models. We present a general winner search procedure based on this probability measure and an update step based on its gradients. As an application, we derive the learning rules for a particular probabilistic model that is used in user profiling in mobile communications network. Due to the constrained nature of the parameters of our probabilistic model, we introduce a new parameter space, in which the gradient update step is performed. In the experiments, we show clustering of user profiles using calling data involving normal users of mobile phones and users that are known to be victims of fraud. Finally, we discuss further applications of the approach.
Nonlinear Time-Series Prediction with Missing and Noisy Data

We derive solutions for the problem of missing and noisy data in nonlinear time-series prediction from a probabilistic point of view. We discuss different approximations to the solutionsin particular, approximations that require either stochastic simulation or the substitution of a single estimate for the missing data. We show experimentally that commonly used heuristics can lead to suboptimal solutions. We show how error bars for the predictions can be derived and how our results can be applied to K-step prediction. We verify our solutions using two chaotic time series and the sunspot data set. In particular, we show that for K-step prediction, stochastic simulation is superior to simply iterating the predictor.
Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates

We apply the idea of averaging ensembles of estimators to probability density estimation. In particular, we use Gaussian mixture models which are important components in many neural-network applications. We investigate the performance of averaging using three data sets. For comparison, we employ two traditional regularization approaches, i.e., a maximum penalized likelihood approach and a Bayesian approach. In the maximum penalized likelihood approach we use penalty functions derived from conjugate Bayesian priors such that an expectation maximization (EM) algorithm can be used for training. In all experiments, the maximum penalized likelihood approach and averaging improved performance considerably if compared to a maximum likelihood approach. In two of the experiments, the maximum penalized likelihood approach outperformed averaging. In one experiment averaging was clearly superior. Our conclusion is that maximum penalized likelihood gives good results if the penalty term in the cost function is appropriate for the particular problem. If this is not the case, averaging is superior since it shows greater robustness by not relying on any particular prior assumption. The Bayesian approach worked very well on a low-dimensional toy problem but failed to give good performance in higher dimensional problems.