nils-herrmann/clean_scirepeval huggingface dataset:
Incremental peA Based Face Recognition

In the real world, learning is often expected to be a continuous process, which is capable of incorporat­ ing new facts into the past experience. However, cur­ rently many typical face recognition methods, such as eigellface and Fisherface, have only focused on non-incremental IC<J.rning tasks, where the learnill g stops once the training set has been duly processed. In this paper, we present a PCA-based algorithm for
Learning with Idealized Kernels

The kernel function plays a central role in kernel methods. Existing methods typically fix the functional form of the kernel in advance and then only adapt the associated kernel parameters based on empirical data. In this paper, we consider the problem of adapting the kernel so that it becomes more similar to the so-called ideal kernel. We formulate this as a distance metric learning problem that searches for a suitable linear transform (feature weighting) in the kernel-induced feature space. This formulation is applicable even when the training set can only provide examples of similar and dissimilar pairs, but not explicit class label information. Computationally, this leads to a local-optima-free quadratic programming problem, with the number of variables independent of the number of features. Performance of this method is evaluated on classification and clustering tasks on both toy and real-world data sets.
Eigenvoice Speaker Adaptation via Composite Kernel PCA

Eigenvoice speaker adaptation has been shown to be effective when only a small amount of adaptation data is available. At the heart of the method is principal component analysis (PCA) employed to find the most important eigenvoices. In this paper, we postulate that nonlinear PCA, in particular kernel PCA, may be even more effective. One major challenge is to map the feature-space eigenvoices back to the observation space so that the state observation likelihoods can be computed during the estimation of eigenvoice weights and subsequent decoding. Our solution is to compute kernel PCA using composite kernels, and we will call our new method kernel eigenvoice speaker adaptation. On the TIDIGITS corpus, we found that compared with a speaker-independent model, our kernel eigenvoice adaptation method can reduce the word error rate by 28-33% while the standard eigenvoice approach can only match the performance of the speaker-independent model.
Parametric Distance Metric Learning with Label Information

Distance-based methods in pattern recognition and machine learning have to rely on a similarity or dissimilarity measure between patterns in the input space. For many applications, Euclidean distance in the input space is not a good choice and hence more complicated distance metrics have to be used. In this paper, we propose a parametric method for metric learning based on class label information. We first define a dissimilarity measure that can be proved to be metric. It has the favorable property that between-class dissimilarity is always larger than within-class dissimilarity. We then perform parametric learning to find a regression mapping from the input space to a feature space, such that the dissimilarity between patterns in the input space is approximated by the Euclidean distance between points in the feature space. Parametric learning is performed using the iterative majorization algorithm. Experimental results on real-world benchmark data sets show that this approach is promising.
Probabilistic Kernel Matrix Learning with a Mixture Model of Kernels

This paper addresses the kernel matrix learning problem in kernel methods. We model the kernel matrix as a random positive definite matrix following the Wishart distribution, with the parameter matrix of the Wishart distribution represented as a linear combination of mutually independent matrices with their own Wishart distributions. This defines a probabilistic mixture model of kernels that can be represented as a hierarchical model involving three levels, relating the target kernel matrix, the parameter kernel matrix, and the hyperparameter kernel matrices of the mixture components. Since in most cases a linear combination of Wishart matrices is no longer Wishart, we propose an approximation method by employing a new Wishart matrix to approximate the parameter matrix through preserving the first and second moments of the linear combination of Wishart matrices. Based on the Wishart matrix estimated, kernel matrix learning is then solved by an expectation-maximization (EM) learning algorithm to infer the missing data of the kernel matrix and the unknown parameter matrix of the distribution. Moreover, we study the kernel matrix learning problem in the context of classification problems with the use of a kernel nearest neighbor classifier. Classification experiments on several benchmark data sets show promising results. Furthermore, our method has opened up a possible direction for addressing the kernel
Distance metric learning with kernels

In this paper, we propose a feature weighting method that works in both the input space and the kernel-induced feature space. It assumes only the availability of similari ty (dissimilarity) information, and the number of parameters in the transformation does not depend on the number of features . Besides feature weighting, it can also be regarded as perfor ming nonparametric kernel adaptation. Experimental results on both toy and real-world datasets show promising results.
Eigenvoice Speaker Adaptation via Composite Kernel Principal Component Analysis

Eigenvoice speaker adaptation has been shown to be effective when only a small amount of adaptation data is available. At the heart of the method is principal component analysis (PCA) employed to find the most important eigenvoices. In this paper, we postulate that nonlinear PCA, in particular kernel PCA, may be even more effective. One major challenge is to map the feature-space eigenvoices back to the observation space so that the state observation likelihoods can be computed during the estimation of eigenvoice weights and subsequent decoding. Our solution is to compute kernel PCA using composite kernels, and we will call our new method kernel eigenvoice speaker adaptation. On the TIDIGITS corpus, we found that compared with a speaker-independent model, our kernel eigenvoice adaptation method can reduce the word error rate by 28–33% while the standard eigenvoice approach can only match the performance of the speaker-independent model.
Linear dependency between ε and the input noise in ε-support vector regression

In using the /spl epsi/-support vector regression (/spl epsi/-SVR) algorithm, one has to decide a suitable value for the insensitivity parameter /spl epsi/. Smola et al. considered its "optimal" choice by studying the statistical efficiency in a location parameter estimation problem. While they successfully predicted a linear scaling between the optimal /spl epsi/ and the noise in the data, their theoretically optimal value does not have a close match with its experimentally observed counterpart in the case of Gaussian noise. In this paper, we attempt to better explain their experimental results by studying the regression problem itself. Our resultant predicted choice of /spl epsi/ is much closer to the experimentally observed optimal value, while again demonstrating a linear trend with the input noise.
Incremental eigen decomposition

Eigen decomposition is a central mathematical tool in many pattern recognition and machine learning techniques. However, it becomes computationally infeasible in the pres ence of a large set of high-dimensional samples. Moreover, in hig hly dynamic domains with a continual supply of new samples, an incremental approach that keeps on updating the eigen decom position will be more desirable than the traditional approach of simply re-computing the decomposition from scratch. In this paper, by using a method for updating singular value decompositions, we propose a procedure to obtain an approximate eigen decomposition by processing the samples in chunks or sequentially. On applying this to principal component anal ysis for image denoising, experimental results show that this restr icted form of updating can still achieve comparable performance as the batch version.
Fusing Images with Multiple Focuses Using Support Vector Machines

Optical lenses, particularly those with long focal lengths, suffer from the problem of limited depth of field. Consequently, it is often difficult to obtain good focus for all the objects in the scene. One approach to address this problem is by performing image fusion, i.e., several pictures with different focus points are combined to a single image. This paper proposes a multifocus image fusion method based on the discrete wavelet frame transform and support vector machines. Experimental results show that the proposed method outperforms the conventional approach based on the discrete wavelet transform and maximum selection rule, particularly when there is slight camera/object movement or mis-registration of the source images.
A novel distance-based classifier using convolution kernels and Euclidean embeddings

Distance-based classification methods such as the nearest-neighbor and k-nearest-neighbor classifiers have to rely on a metric or distance measure between points in the input space. For many applications, Euclidean distance in the input space is not a good choice and hence more complicated distance measures have to be used. In this paper, we propose a novel kernel-based method that achieves Euclidean embedding by ensuring that the feature space is always Euclidean. Thus, Euclidean distance-based classification methods can be applied in the feature space. Unlike typical kernels which correspond to a nonlinear mapping from the input space, our kernel function corresponds to a nonlinear mapping from the joint space incorporating both the input and class label spaces. The kernel function, which can be seen as a convolution kernel formed from the tensor product kernel and the direct sum kernel, tends to increase the separability between classes. We have applied our new classification method to some face recognition benchmark datasets. Our method is significantly faster than other face recognition methods and yet it can deliver comparable classification accuracy.
A novel family of subspace methods---protoface and its kernel version

In this paper, we present a novel feature extraction called the protoface. While the Eigenface is based on principal component analysis and the Fisherface on Fisher’s linear discriminant analysis, the protoface only requires decomposing the covariance matrix of the prototypes that can better describe whole observations. It is thus more computationally efficient especially when the number of the observations is large. Moreover, instead of using the method of multivariate analysis of variance, protoface relies on the analysis of distance. It is thus more appropriate in situations where the assumptions of normality of the observations are violated. Besides, by using the kernel trick, we also obtain a kernel protoface in the feature space defined by the kernel. Finally, we apply our methods to face recognition, and experimental results on both the AT&T and Yale databases are reported.
Improving de-noising by coefficient de-noising and dyadic wavelet transform

Soft thresholding has been a standard wavelet de-noising procedure in many signal and image processing applications. Theoretically it is also almost optimal in the sense of nearly achieving the minimax mean-squared error. Inspired by this property, the paper proposes the addition of coefficient de-noising before soft thresholding. This extra step serves to reduce noise in the empirical wavelet coefficients at each scale, and can be shown to yield a lower mean-squared error Moreover we advocate the use of the translation-invariant dyadic wavelet transform, together with an approximate self-dual wavelet, instead of the discrete wavelet transform (DWT) in performing denoising. Experiments show that the proposed method improves the signal-to-noise ratios of the de-noised signals. Moreover the de-noised signals do not have artifacts typically associated with DWT-based methods.
Applying the Bayesian Evidence Framework to \nu -Support Vector Regression

Following previous successes on applying the Bayesian evidence framework to support vector classifiers and the Ɛ-support vector regression algorithm, in this paper we extend the evidence framework also to the ν-support vector regression (ν-SVR) algorithm. We show that ν-SVR training implies a prior on the size of the Ɛ-tube that is dependent on the number of training patterns. Besides, this prior has properties that are in line with the error-regulating behavior of ν. Under the evidence framework, standard ν-SVR training can then be regarded as performing level one inference, while levels two and three allow automatic adjustments of the regularization and kernel parameters respectively, without the need of a validation set. Furthermore, this Bayesian extension allows computation of the prediction intervals, taking uncertainties of both the weight parameter and the Ɛ-tube width into account. Performance of this method is illustrated on both synthetic and real-world data sets.
Linear Dependency betweenand the Input Noise in -Support Vector Regression

In using the � -support vector regression (� -SVR) algorithm, one has to decide on a suitable value of the insensitivity parameter � . Smola et al. (6) determined its "optimal" choice based on maximizing the statistical efficiency of a location parameter estimator. While they successfully predicted a linear scaling between the optimaland the noise in the data, the value of the theoretically optimaldoes not have a close match with its experimentally observed counterpart. In this paper, we attempt to better explain the experimental results there, by analyzing a toy problem with a closer setting to the � -SVR. Our resultant predicted choice ofis much closer to the experimentally observed value, while still demonstrating a linear trend with the data noise.
Linear Dependency between epsilon and the Input Noise in epsilon-Support Vector Regression

In using the Ɛ-support vector regression (Ɛ-SVR) algorithm, one has to decide on a suitable value of the insensitivity parameter Ɛ. Smola et al.  determined its "optimal" choice based on maximizing the statistical efficiency of a location parameter estimator. While they successfully predicted a linear scaling between the optimal Ɛ and the noise in the data, the value of the theoretically optimal Ɛ does not have a close match with its experimentally observed counterpart. In this paper, we attempt to better explain the experimental results there, by analyzing a toy problem with a closer setting to the Ɛ-SVR. Our resultant predicted choice of Ɛ is much closer to the experimentally observed value, while still demonstrating a linear trend with the data noise.
Rival penalized competitive learning for model-based sequence clustering

We propose a model-based, competitive learning procedure for the clustering of variable-length sequences. Hidden Markov models (HMMs) are used as representations for the cluster centers, and rival penalized competitive learning (RPCL), originally developed for domains with static, fixed-dimensional features, is extended. State merging operations are also incorporated to favor the discovery of smaller HMMs. Simulation results show that our extended version of RPCL can produce a more accurate cluster structure than k-means clustering.
Mining customer preference ratings for product recommendation using the support vector machine and the latent class model

As Internet commerce becomes more popular, customers' preferences on various products can now be readily acquired on-line via various e-commerce systems. Properly mining this extracted data can generate useful knowledge for providing personalized product recommendation services. In general, recommender systems use two complementary techniques. Content-based systems match customer interests with products attributes, while collaborative filtering systems utilize preference ratings from other customers. In this paper, we address some problems faced by these two systems, and study how machine learning techniques, namely the support vector machine and the latent class model, can be used to alleviated them.
The evidence framework applied to support vector machines

In this paper, we show that training of the support vector machine (SVM) can be interpreted as performing the level 1 inference of MacKay's evidence framework.We further on show that levels 2 and 3 of the evidence framework can also be applied to SVMs. This integration allows automatic adjustment of the regularization parameter and the kernel parameter to their near-optimal values. Moreover, it opens up a wealth of Bayesian tools for use with SVMs. Performance of this method is evaluated on both synthetic and real-world data sets.
Moderating the outputs of support vector machine classifiers

In this paper, we extend the use of moderated outputs to the support vector machine (SVM) by making use of a relationship between SVM and the evidence framework. The moderated output is more in line with the Bayesian idea that the posterior weight distribution should be taken into account upon prediction, and it also alleviates the usual tendency of assigning overly high confidence to the estimated class memberships of the test patterns. Moreover, the moderated output derived here can be taken as an approximation to the posterior class probability. Hence, meaningful rejection thresholds can be assigned and outputs from several networks can be directly compared. Experimental results on both artificial and real-world data are also discussed.
Support vector mixture for classification and regression problems

We study the incorporation of the support vector machine (SVM) into the (hierarchical) mixture of experts model to form a support vector mixture. We show that, in both classification and regression problems, the use of a support vector mixture leads to quadratic programming (QP) problems that are very similar to those for a SVM, with no increase in the dimensionality of the QP problems. Moreover, a support vector mixture, besides allowing for the use of different experts in different regions of the input space, also supports easy combination of different architectures such as polynomial networks and radial basis function networks.
Objective functions for training new hidden units in constructive neural networks

In this paper, we study a number of objective functions for training new hidden units in constructive algorithms for multilayer feedforward networks. The aim is to derive a class of objective functions the computation of which and the corresponding weight updates can be done in O(N) time, where N is the number of training patterns. Moreover, even though input weight freezing is applied during the process for computational efficiency, the convergence property of the constructive algorithms using these objective functions is still preserved. We also propose a few computational tricks that can be used to improve the optimization of the objective functions under practical situations. Their relative performance in a set of two-dimensional regression problems is also discussed.
Constructive algorithms for structure learning in feedforward neural networks for regression problems

In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state-space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm, and the network architecture, is then presented.
Bayesian Regularization in Constructive Neural Networks

In this paper, we study the incorporation of Bayesian regularization into constructive neural networks. The degree of regularization is automatically controlled in the Bayesian inference framework and hence does not require manual setting. Simulation shows that regularization, with input training using a full Bayesian approach, produces networks with better generalization performance and lower susceptibility to over-fitting as the network size increases. Regularization with input training under MacKay's evidence framework, however, does not produce significant improvement on the problems tested.
Use of bias term in projection pursuit learning improves approximation and convergence properties

In a regression problem, one is given a multidimensional random vector X, the components of which are called predictor variables, and a random variable, Y, called response. A regression surface describes a general relationship between X and Y. A nonparametric regression technique that has been successfully applied to high-dimensional data is projection pursuit regression (PPR). The regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors. Projection pursuit learning (PPL) formulates PPR using a 2-layer feedforward neural network. The smoothers in PPR are nonparametric, whereas those in PPL are based on Hermite functions of some predefined highest order R. We demonstrate that PPL networks in the original form do not have the universal approximation property for any finite R, and thus cannot converge to the desired function even with an arbitrarily large number of hidden units. But, by including a bias term in each linear projection of the predictor variables, PPL networks can regain these capabilities, independent of the exact choice of R. Experimentally, it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units, improves the generalization performance, and makes it less sensitive to the setting of R. Finally, we apply PPL to chaotic time series prediction, and obtain superior results compared with the cascade-correlation architecture.
Efficient cross-validation for feedforward neural networks

Studies the use of cross-validation for estimating the prediction risk of feedforward neural networks. In particular, the problem of variability due to the choice of random initial weights for learning is addressed. The authors demonstrate that nonlinear cross-validation may not be able to prevent the network from falling into the "wrong" perturbed local minimum. A modified approach that reduces the problem to a linear problem is proposed. It is more efficient and does not suffer from the local minimum problem. Simulation results for two regression problems are discussed.
A Computer Vision System on a Chip: a case study from the automotive domain

The automotive market puts strict and often conflicting requirements on computer vision systems. On the one hand the algorithms require considerable computing power to work reliably in real-time and under a wide range of lighting conditions. On the other hand, the cost must be kept low, the package size must be small and the power consumption must be low. In addition, automotive qualified parts must be used both to withstand the harsh operating environment and to guarantee long product life. To meet all these conflicting requirements Mobileye developed the EyeQ, a complete ’system on a chip’ (SoC) which has the computing power to support a variety of applications such as lane, vehicle and pedestrian detection. This paper describes the process of designing an ASIC to support a family of vision algorithms.