nils-herrmann/clean_scirepeval huggingface dataset:
Pattern Recognition

We present an approach to discretizing multivariate continuous data while learning the structure of a graphical model. We derive a joint scoring function from the principle of predictive accuracy, which inherently ensures the optimal trade-off between goodness of fit and model complexity including the number of discretization levels. Using the socalled finest grid implied by the data, our scoring function depends only on the number of data points in the various discretization levels (independent of the metric used in the continuous space). Our experiments with artificial data as well as with gene expression data show that discretization plays a crucial role regarding the resulting network structure.
Semi-supervised Kernel Regression Using Whitened Function Classes

The use of non-orthonormal basis functions in ridge regression leads to an often undesired non-isotropic prior in function space. In this study, we investigate an alternative regularization technique that results in an implicit whitening of the basis functions by penalizing directions in function space with a large prior variance. The regularization term is computed from unlabelled input data that characterizes the input distribution. Tests on two datasets using polynomial basis functions showed an improved average performance compared to standard ridge regression.
Analysis of Some Methods for Reduced Rank Gaussian Process Regression

While there is strong motivation for using Gaussian Processes (GPs) due to their excellent performance in regression and classification problems, their computational complexity makes them impractical when the size of the training set exceeds a few thousand cases. This has motivated the recent proliferation of a number of cost-effective approximations to GPs, both for classification and for regression. In this paper we analyze one popular approximation to GPs for regression: the reduced rank approximation. While generally GPs are equivalent to infinite linear models, we show that Reduced Rank Gaussian Processes (RRGPs) are equivalent to finite sparse linear models. We also introduce the concept of degenerate GPs and show that they correspond to inappropriate priors. We show how to modify the RRGP to prevent it from being degenerate at test time. Training RRGPs consists both in learning the covariance function hyperparameters and the support set. We propose a method for learning hyperparameters for a given support set. We also review the Sparse Greedy GP (SGGP) approximation (Smola and Bartlett, 2001), which is a way of learning the support set for given hyperparameters based on approximating the posterior. We propose an alternative method to the SGGP that has better generalization capabilities. Finally we make experiments to compare the different ways of training a RRGP. We provide some Matlab code for learning RRGPs.
Propagation of uncertainty in Bayesian kernel models - application to multiple-step ahead forecasting

The object of Bayesian modelling is predictive distribution, which, in a forecasting scenario, enables evaluation of forecasted values and their uncertainties. We focus on reliably estimating the predictive mean and variance of forecasted values using Bayesian kernel based models such as the Gaussian process and the relevance vector machine. We derive novel analytic expressions for the predictive mean and variance for Gaussian kernel shapes under the assumption of a Gaussian input distribution in the static case, and of a recursive Gaussian predictive density in iterative forecasting. The capability of the method is demonstrated for forecasting of time-series and compared to approximate methods.
Gaussian Processes in Reinforcement Learning

We exploit some useful properties of Gaussian process (GP) regression models for reinforcement learning in continuous state spaces and discrete time. We demonstrate how the GP model allows evaluation of the value function in closed form. The resulting policy iteration algorithm is demonstrated on a simple problem with a two dimensional state space. Further, we speculate that the intrinsic ability of GP models to characterise distributions of functions would allow the method to capture entire distributions over future values instead of merely their expectation, which has traditionally been the focus of much of reinforcement learning.
Prediction at an Uncertain Input for Gaussian Processes and Relevance Vector Machines Application to Multiple-Step Ahead Time-Series Forecasting

We consider in this report non-linear models that map an input D-dimensional column vector x into a single dimensional output f(x). The non-linear mapping f(·) is implemented by means of a Gaussian process (GP) or a Relevance Vector Machine (RVM), see for example [Rasmussen, 1996] and [Tipping, 2001]. We are given a training data set D = {xi, yi}i=1 where the target yi relates to the input xi through yi = f(xi) + (1)
Perfusion quantification using Gaussian process deconvolution

The quantification of perfusion using dynamic susceptibility contrast MRI (DSC‐MRI) requires deconvolution to obtain the residual impulse response function (IRF). In this work, a method using the Gaussian process for deconvolution (GPD) is proposed. The fact that the IRF is smooth is incorporated as a constraint in the method. The GPD method, which automatically estimates the noise level in each voxel, has the advantage that model parameters are optimized automatically. The GPD is compared to singular value decomposition (SVD) using a common threshold for the singular values, and to SVD using a threshold optimized according to the noise level in each voxel. The comparison is carried out using artificial data as well as data from healthy volunteers. It is shown that GPD is comparable to SVD with a variable optimized threshold when determining the maximum of the IRF, which is directly related to the perfusion. GPD provides a better estimate of the entire IRF. As the signal‐to‐noise ratio (SNR) increases or the time resolution of the measurements increases, GPD is shown to be superior to SVD. This is also found for large distribution volumes.  48:351–361, 2002.
Gaussian Process priors with Uncertain Inputs: Multiple-Step-Ahead Prediction

We consider the problem of multi-step ahead prediction in time series analysis using the non-parametric Gaussian process model. k-step ahead forecasting of a discrete-time nonlinear dynamic system can be performed by doing repeated one-step ahead predictions. For a state-space model of the form yt = f(yt 1 ;:::;y t L), the prediction of y at time t + k is based on the estimates ^ yt+k 1 ;:::; ^ yt+k L of the previous outputs. We show how, using an analytical Gaussian approximation, we can formally incorporate the uncertainty about intermediate regressor values, thus updating the uncertainty on the current prediction. In this framework, the problem is that of predicting responses at a random input and we compare the Gaussian approximation to the Monte-Carlo numerical approximation of the predictive distribution. The approach is illustrated on a simulated non-linear dynamic example, as well as on a simple one-dimensional static example.
Bayesian Modelling of fMRI lime Series

We present a Hidden Markov Model (HMM) for inferring the hidden psychological state (or neural activity) during single trial fMRI activation experiments with blocked task paradigms. Inference is based on Bayesian methodology, using a combination of analytical and a variety of Markov Chain Monte Carlo (MCMC) sampling techniques. The advantage of this method is that detection of short time learning effects between repeated trials is possible since inference is based only on single trial experiments.
Evaluation of gaussian processes and other methods for non-linear regression

This thesis develops two Bayesian learning methods relying on Gaussian processes and a rigorous statistical approach for evaluating such methods. In these experimental designs the sources of uncertainty in the estimated generalisation performances due to both variation in training and test sets are accounted for. The framework allows for estimation of generalisation performance as well as statistical tests of significance for pairwise comparisons. Two experimental designs are recommended and supported by the DELVE software environment. 
Two new non-parametric Bayesian learning methods relying on Gaussian process priors over functions are developed. These priors are controlled by hyperparameters which set the characteristic length scale for each input dimension. In the simplest method, these parameters are fit from the data using optimization. In the second, fully Bayesian method, a Markov chain Monte Carlo technique is used to integrate over the hyperparameters. One advantage of these Gaussian process methods is that the priors and hyperparameters of the trained models are easy to interpret. 
The Gaussian process methods are benchmarked against several other methods, on regression tasks using both real data and data generated from realistic simulations. The experiments show that small datasets are unsuitable for benchmarking purposes because the uncertainties in performance measurements are large. A second set of experiments provide strong evidence that the bagging procedure is advantageous for the Multivariate Adaptive Regression Splines (MARS) method. 
The simulated datasets have controlled characteristics which make them useful for understanding the relationship between properties of the dataset and the performance of different methods. The dependency of the performance on available computation time is also investigated. It is shown that a Bayesian approach to learning in multi-layer perceptron neural networks achieves better performance than the commonly used early stopping procedure, even for reasonably short amounts of computation time. The Gaussian process methods are shown to consistently outperform the more conventional methods.
Design goals and solutions for display of hyperspectral images

Design goals and solutions are proposed for the display of hyperspectral imagery on tristimulus displays. The requirements of a hyperspectral visualization depend on the task. We focus on creating consistent representations of hyperspectral data that can facilitate understanding and analysis of hyperspectral scenes, and may be used in conjunction with task-specific visualizations. Fixed linear spectral weighting envelopes are given which create natural looking imagery where hue, brightness, saturation and white-point have meanings consistent with the human visual system interpretation of natural scenes. For AVIRIS images, hue interpretation of water and vegetation is also preserved. The proposed designs avoid the pre-attentive distractions of PCA imagery, and provide comparable spectral and edge discriminability.
Simulating the effect of illumination using color transformations

We investigate design and estimation issues for using the standard color management profile architecture for general custom image enhancement. Color management profiles are a flexible architecture for describing a mapping from an original colorspace to a new colorspace. We investigate use of this same architecture for describing color enhancements that could be defined by a non-technical user using samples of the mapping, just as color management is based on samples of a mapping between an original colorspace and a new colorspace. As an example enhancement, we work with photos of the 24 color patch Macbeth chart under different illuminations, with the goal of defining transformations that would take, for example, a studio D65 image and reproduce it as though it had been taken during a particular sunset. The color management profile architecture includes a look-up-table and interpolation. We concentrate on the estimation of the look-up-table points from minimal number of color enhancement samples (comparing interpolative and extrapolative statistical learning techniques), and evaluate the feasibility of using the color management architecture for custom enhancement definitions.
Custom color enhancements by statistical learning

We consider the problem of automatically learning color enhancements from a small set of sample color pairs, and then describing the enhancement by a three-dimensional look-up-table that can be stored and implemented as an ICC profile. We propose a new method for automatically learning a neighborhood for local statistical learning methods such as local linear regression, and show that this leads to relatively accurate descriptions of the desired color transformation and results in images that appear smooth and have natural depth of detail. In a previous work we showed that learning arbitrary color enhancements could result in colored specular highlights, causing images to look unnatural. We show that this can be solved by adding a null sample that maps white to white.
Segmenting for wavelet compression

We propose a new approach to segmenting mixed raster content images for compression by wavelet encoders and binary compressors. Though wavelets are efficient at modeling edges in natural images, text and sharp edges can be more effectively represented by a binary mask than with wavelets. A common approach to mixed raster content documents is to segment images from text, and separately compress images and a binary mask that defines the segmentation. We propose that the segmentation be done jointly with a projection-onto-convex-sets (POCS) smoothing, with the design goal of segmented images that have a minimal number of nonzero wavelet coefficients. Our goal is a document segmentation that is easy to compress. Knowing that the image encoder will be a lossy wavelet coder, we approximate the above goal as minimizing the number of nonzero wavelet coefficients. We attempt to achieve this goal by jointly choosing the mask and data-filling.
Design goals and solutions for display of hyperspectral images

Design goals and solutions are proposed for the display of hyperspectral imagery on tristimulus displays. The requirements of a hyperspectral visualization depend on the task. We focus on creating consistent representations of hyperspectral data that can facilitate understanding and analysis of hyperspectral scenes, and may be used in conjunction with task-specific visualizations. Fixed linear spectral weighting envelopes are given, creating natural-looking imagery where hue, brightness, saturation, and whitepoint have meanings consistent with the human visual system interpretation of natural scenes. For Airborne Visible/Infrared Imaging Spectrometer images, hue interpretation of water and vegetation is also preserved. The proposed designs avoid the preattentive distractions of principal component analysis imagery, and appear to provide comparable or enhanced spectral and edge discriminability.
Inverting color transforms

We consider experimental methods for creating regular grids for applications such as color management where the grids must be estimated from non-grid samples. To estimate the regular grid, we propose applying a generalization of linear interpolation, called linear interpolation with maximum entropy (LIME). Evaluating different estimation methods for this problem is difficult and does not correspond to the standard statistical learning paradigm of using iid training and test sets in order to compare algorithms. In this paper we consider the experimental issues and propose considering the end goal of the regular grid in evaluating an estimated grid's value. Preliminary experimental results compare LIME, traditional linear interpolation, linear regression and ridge regression.
Halftoning on the wavelet domain

Wavelet representations of images are increasingly important as more image processing functions are shown to be advantageously executed in the wavelet domain. Images may be inverse halftoned, compressed, denoised, and enhanced in the wavelet domain. In conjunction with other wavelet processing, it would be efficient to halftone directly from the wavelet domain. In this paper we demonstrate how to perform error diffusion in the wavelet domain. The wavelet coefficients are modified by a normalization factor and re-arranged. Then, traditional feed-forward raster scan error diffusion is performed and quality halftones are shown to result. Error diffusing in the wavelet domain is noted to be non-causal with respect to the pixels, and thus the method is not reproducible by feed-forward raster scan error diffusion of pixels. It is shown that the wavelet halftones preserve the average value of the input for constant patches. The resulting halftones may appear smoother in smooth regions and sharper at edges than the corresponding pixel-domain halftones. Disadvantages may include a greater susceptibility to moire and false contouring. Error diffusion is a two-dimensional sigma-delta modulation, and the ideas presented may also be useful for one-dimensional sigma-delta modulation applications.
Analysis and classification of internal pipeline images

Recently developed optical inspection tools provide images from the inside of natural gas pipelines to monitor pipeline integrity. The vast amount of data generated prohibits human inspection of the resulting images. We designed an image processing and classification method to identify ab- normal events. Non-overlapping image blocks are classified into twelve categories: normal, black line, grinder marks, magnetic flux leakage inspector marks, single dots, small black corrosion dots, osmosis blisters, corrosion dots, longitudinal weld, field joint, cavity at a weld and longitudinal weld too close to field joints. Results compare different types of statistical classifiers. Features extracted from the pipeline image are designed to mimic the features humans use to identify the different classes. Difficulties include the large number of classes, the uneven costs associated with different errors, and training on a limited amount of expert classified data. Classification results show this to be a useful tool for pipeline monitoring.
Two-stage color palettization for error diffusion

Image-adaptive color palettization chooses a decreased number of colors to represent an image. Palettization is one way to decrease storage and memory requirements for low-end displays. Palettization is generally approached as a clustering problem, where one attempts to find the k palette colors that minimize the average distortion for all the colors in an image. This would be the optimal approach if the image was to be displayed with each pixel quantized to the closest palette color. However, to improve the image quality the palettization may be followed by error diffusion. In this work, we propose a two-stage palettization where the first stage finds some m << k clusters, and the second stage chooses palette points that cover the spread of each of the M clusters. After error diffusion, this method leads to better image quality at less computational cost and with faster display speed than full k-means palettization.
Robust speech recognition using wavelet coefficient features

We propose a new vein of feature vectors for robust speech recognition that use denoised wavelet coefficients; greater robustness to unexpected additive noise or spectrum distortions begins with more robust acoustic features. The use of wavelet coefficients is motivated by human acoustic process modelling and by the ability of wavelet coefficients to capture important time and frequency features. Wavelet denoising accentuates the most salient information about the speech signal and adds robustness. We show encouraging results using denoised cosine packet features on small-scale experiments with the TIMIT database, its NTIMIT counterpart, and low-pass filter distortions.
Color conversions using maximum entropy estimation

We propose a new estimation method using the maximum entropy principle and show that it is successfully used for the three-dimensional interpolation step involved in many color conversions. Color conversions are a key part of color management systems, and many device characterizations, especially for printers, rely on multidimensional interpolation to perform the conversion. Our method is a linear interpolation that is not limited in the number of sample points used to estimate new color values. We find a unique solution to the underdetermined inverse matrix problem by invoking the maximum entropy principle. We compare our approach to the standard technique of tetrahedral interpolation and demonstrate that more accurate and more robust approximations may result.
Vector color filter array demosaicing

Single-sensor digital cameras spatially sample the incoming image using a color filter array (CFA). Consequently, each pixel only contains a single color value. In order to reconstruct the original full-color image, a demosaicing step must be performed which interpolates the missing colors at each pixel. Goals in CFA demosaicing include color fidelity, spatial resolution, no false colors, no jagged edges, and computational practicality. Most demosaicing algorithms do well for color fidelity, but there is often a trade-off between a sharp image and the so-called 'zipper effect' or jagged edge look. We propose a novel demosaicing algorithm called vector demosaicing that interpolates missing colors jointly by selecting the color vector that minimizes the sum of distances to the surrounding pixels. The selected color vector is a vector median of the surrounding pixels. The vector median forms an 'average', but preserves sharp edges. We will discuss the theory behind our approach and show experimentally how the theoretical advantages manifest themselves to improve edge resolution while retaining smoothness. Computational complexity is shown to be possible quite low, and we discuss how different approximations may affect the output.
Block color quantization: a new method for color halftoning

An important halftoning problem faced in the design of many color printers and copiers is to represent a 24-bit color image by a small number of preset output colors, generally at a higher spatial resolution. We propose dividing the input and output image into corresponding small blocks, calculating each input block's color average, and then determining a set of the printing colors for the corresponding output block to best render the input block's average color. Our method exploits the higher spatial resolution of the printing process and yields constrained local color optimality. Artifacts common with error diffusion methods do not occur. In an experimental comparison to vector color error diffusion halftoning, block color quantization yields superior rendering of graphics. For natural images, block color quantization may occasionally generate false contours, but due to the lack of texture artifacts, block color quantization halftones may be preferable to error diffusion halftones.
Estimating Predictive Variances with Kernel Ridge Regression

In many regression tasks, in addition to an accurate estimate of the conditional mean of the target distribution, an indication of the predictive uncertainty is also required. There are two principal sources of this uncertainty: the noise process contaminating the data and the uncertainty in estimating the model parameters based on a limited sample of training data. Both of them can be summarised in the predictive variance which can then be used to give confidence intervals. In this paper, we present various schemes for providing predictive variances for kernel ridge regression, especially in the case of a heteroscedastic regression, where the variance of the noise process contaminating the data is a smooth function of the explanatory variables. The use of leave-one-out cross-validation is shown to eliminate the bias inherent in estimates of the predictive variance. Results obtained on all three regression tasks comprising the predictive uncertainty challenge demonstrate the value of this approach.