nils-herrmann/clean_scirepeval huggingface dataset:
Pattern de-noising based on support vector data description

The SVDD (support vector data description) is one of the most well-known one-class support vector learning methods, in which one tries the strategy of utilizing balls defined on the feature space in order to distinguish a set of normal data from all other possible abnormal objects. The major concern of this paper is to extend the main idea of the SVDD for the problem of pattern de-noising. Combining the projection onto the spherical decision boundary resulting from the SVDD together with a solver for the pre-image problem, we propose a new method for pattern de-noising. In the proposed method, we first solve the SVDD for the training data, then for each noisy test pattern, perform de-noising by projecting its feature vector onto the decision boundary on the feature space, and finally find the location of the de-noised pattern by obtaining the pre-image of the projection. The applicability of the proposed method is illustrated via an example dealing with noisy handwritten digits.
Data-dependent kernels for high-dimensional data classification

For high-dimensional data classification problems such as face recognition, one of the most efficient classifiers is the nearest neighbor (NN) classifier. What mostly affects the NN classification performance is the feature extracted by some methods. And the kernel method is one of the efficient methods for extracting features. However, the selection of kernel parameters is still difficult. In this paper, we propose a so-called data dependent kernel (DDK) which is defined by generalizing the Gaussian kernel. Also an efficient and practical method is presented to calculate the DDK parameters. Moreover, one DDK based on subspaces is given to improve the recognition performance. Experiments show that the proposed DDK can achieve promising classification performance in face recognition and SPECT heart diagnosis.
Accurate and Low-cost Location Estimation Using Kernels

We present a novel method for indoor-location estimation using a vector-space model based on signals received from a wireless client. Our aim is to obtain an accurate mapping between the signal space and the physical space without incurring too much human calibration effort. This problem has traditionally been tackled through probabilistic models trained on manually labeled data, which are expensive to obtain. In this paper, we present a novel approach to building a mapping between the signalvector space and the physical location space using kernel canonical correlation analysis (KCCA). Its training requires much less human labor. Moreover, unlike traditional location-estimation systems that treat grid points as independent and discrete target classes during training, we use the physical location as a continuous feedback to build a similarity mapping using KCCA. We test our algorithm in a 802.11 wireless LAN environment, and demonstrate the advantage of our method in both accuracy and its ability to utilize a much smaller set of labeled training data than previous methods.
Kernel relevant component analysis for distance metric learning

Defining a good distance measure between patterns is of crucial importance in many classification and clustering algorithms. Recently, relevant component analysis (RCA) is proposed which offers a simple yet powerful method to learn this distance metric. However, it is confined to linear transforms in the input space. In this paper, we show that RCA can also be kernelized, which then results in significant improvements when nonlinearities are needed. Moreover, it becomes applicable to distance metric learning for structured objects that have no natural vectorial representation. Besides, it can be used in an incremental setting. Performance of this kernel method is evaluated on both toy and real-world data sets with encouraging results.
Very Large SVM Training using Core Vector Machines

Standard SVM training has O(m) time and O(m) space complexities, where m is the training set size. In this paper, we scale up kernel methods by exploiting the “approximateness” in practical SVM implementations. We formulate many kernel methods as equivalent minimum enclosing ball problems in computational geometry, and then obtain provably approximately optimal solutions efficiently with the use of core-sets. Our proposed Core Vector Machine (CVM) algorithm has a time complexity that is linear in m and a space complexity that is independent of m. Experiments on large toy and real-world data sets demonstrate that the CVM is much faster and can handle much larger data sets than existing scaleup methods. In particular, on our PC with only 512M RAM, the CVM with Gaussian kernel can process the checkerboard data set with 1 million points in less than 13 seconds.
Position estimation for wireless sensor networks

In wireless sensor networks, estimating nodal positions is important for routing efficiency and location-based services. Traditional techniques based on precise measurements are often expensive and power-inefficient, while approaches based on landmarks often require bandwidth-inefficient flooding and hence are not scalable for large networks. In this paper, we propose and investigate a cost-effective and distributed algorithm to accurately estimate nodal positions for wireless sensor networks. In our algorithm, a node only needs to identify and exchange information with a certain number of neighbors (around 30) in its proximity in order to estimate its relative nodal position accurately. For location-identification, only a small number of nodes (around 10) are needed to have additional GPS capabilities to accurately estimate the absolute position of every node in the network. Our algorithm is shown to have fast convergence with low estimation error, even for large networks
Core Vector Machines: Fast SVM Training on Very Large Data Sets

Standard SVM training has O(m3) time and O(m2) space complexities, where m is the training set size. It is thus computationally infeasible on very large data sets. By observing that practical SVM implementations only approximate the optimal solution by an iterative strategy, we scale up kernel methods by exploiting such "approximateness" in this paper. We first show that many kernel methods can be equivalently formulated as minimum enclosing ball (MEB) problems in computational geometry. Then, by adopting an efficient approximate MEB algorithm, we obtain provably approximately optimal solutions with the idea of core sets. Our proposed Core Vector Machine (CVM) algorithm can be used with nonlinear kernels and has a time complexity that is linear in m and a space complexity that is independent of m. Experiments on large toy and real-world data sets demonstrate that the CVM is as accurate as existing SVM implementations, but is much faster and can handle much larger data sets than existing scale-up methods. For example, CVM with the Gaussian kernel produces superior results on the KDDCUP-99 intrusion detection data, which has about five million training patterns, in only 1.4 seconds on a 3.2GHz Pentium--4 PC.
Towards end-to-end privacy control in the outsourcing of marketing activities: a web service integration solution

With the recent adoption of marketing activities outsourcing, there have been increasing demands and concerns for privacy control. The traditional approach of a bulk transmission of the customers' information to a marketing company cannot meet such demands, especially in the finance and healthcare businesses. Therefore, we propose a layered architecture and a development methodology for end-to-end privacy control over the export of each individual customer's records through a Web services platform, according to the corresponding enterprise's privacy control policies. A Web services system, with up-dated security and privacy facilities, can provide a suitable interoperation platform for required application-to-application interactions over the Internet. We further develop a conceptual model and an interaction protocol to send only the required part of a customer's records at a time. We illustrate our approach for end-to-end privacy control with a tele-marketing case study and show how the software of the outsourced call center can be integrated effectively with the Web services of a bank to protect privacy.
Applying neighborhood consistency for fast clustering and kernel density estimation

Nearest neighborhood consistency is an important concept in statistical pattern recognition, which underlies the well-known k-nearest neighbor method. In this paper, we combine this idea with kernel density estimation based clustering, and derive the fast mean shift algorithm (FMS). FMS greatly reduces the complexity of feature space analysis, resulting satisfactory precision of classification. More importantly, we show that with FMS algorithm, we are in fact relying on a conceptually novel approach of density estimation, the fast kernel density estimation (FKDE) for clustering. The FKDE combines smooth and non-smooth estimators and thus inherits advantages from both. Asymptotic analysis reveals the approximation of the FKDE to standard kernel density estimator. Data clustering and image segmentation experiments demonstrate the efficiency of FMS.
Page segmentation using mathematical morphology

In this paper we present a page segmentation algorithm based on mathematical morphology. Usually, text and picture regions in a document image are different in size and continuity. Making use of these distinctive differences, a set of morphology operations are applied to segment document images. The whole algorithm contains mainly two stages: coarse text extraction and post processing. Experimental results show that this algorithm performances effectively on regular document image. It is also robust to irregular images, such as degraded images and images whose picture and text are intertwined or skewed.
Kernel eigenvoice speaker adaptation

Eigenvoice-based methods have been shown to be effective for fast speaker adaptation when only a small amount of adaptation data, say, less than 10 s, is available. At the heart of the method is principal component analysis (PCA) employed to find the most important eigenvoices. In this paper, we postulate that nonlinear PCA using kernel methods may be even more effective. The eigenvoices thus derived will be called kernel eigenvoices (KEV), and we will call our new adaptation method kernel eigenvoice speaker adaptation. However, unlike the standard eigenvoice (EV) method, an adapted speaker model found by the kernel eigenvoice method resides in the high-dimensional kernel-induced feature space, which, in general, cannot be mapped back to an exact preimage in the input speaker supervector space. Consequently, it is not clear how to obtain the constituent Gaussians of the adapted model that are needed for the computation of state observation likelihoods during the estimation of eigenvoice weights and subsequent decoding. Our solution is the use of composite kernels in such a way that state observation likelihoods can be computed using only kernel functions without the need of a speaker-adapted model in the input supervector space. In this paper, we investigate two different composite kernels for KEV adaptation: direct sum kernel and tensor product kernel. In an evaluation on the TIDIGITS task, it is found that KEV speaker adaptation using both forms of composite Gaussian kernels are equally effective, and they outperform a speaker-independent model and adapted models found by EV, MAP, or MLLR adaptation using 2.1 and 4.1 s of speech. For example, with 2.1 s of adaptation data, KEV adaptation outperforms the speaker-independent model by 27.5%, whereas EV, MAP, or MLLR adaptation are not effective at all.
Bayesian Inference on Principal Component Analysis Using Reversible Jump Markov Chain Monte Carlo

Based on the probabilistic reformulation of principal component analysis (PCA), we consider the problem of determining the number of principal components as a model selection problem. We present a hierarchical model for probabilistic PCA and construct a Bayesian inference method for this model using reversible jump Markov chain Monte Carlo (MCMC). By regarding each principal component as a point in a one-dimensional space and employing only birth-death moves in our reversible jump methodology, our proposed method is simple and capable of automatically determining the number of principal components and estimating the parameters simultaneously under the same disciplined framework. Simulation experiments are performed to demonstrate the effectiveness of our MCMC method.
Signal De-Noising by Improving Soft Thresholding on the Dyadic Wavelet Transform

Soft thresholding method has been a standard procedure in signal de-noising. Theoretically, it is also almost optimal in the sense of minimax mean-squared error. This paper shows that by combining coefficient de-noising and soft thresholding, a lower bound of mean-squared-error can be achieved. Furthermore, the translation-invariant (TI) dyadic wavelet transform is used instead of DWT, which can avoid the artifacts caused by the non-TI reconstruction. Experiments show that the proposed method improves the signal-to-noise ratios of the de-noised signals. Moreover, the de-noised signals have smooth and nice visual appearance.
Bayesian inference for transductive learning of kernel matrix using the Tanner-Wong data augmentation algorithm

In kernel methods, an interesting recent development seeks to learn a good kernel from empirical data automatically. In this paper, by regarding the transductive learning of the kernel matrix as a missing data problem, we propose a Bayesian hierarchical model for the problem and devise the Tanner-Wong data augmentation algorithm for making inference on the model. The Tanner-Wong algorithm is closely related to Gibbs sampling, and it also bears a strong resemblance to the expectation-maximization (EM) algorithm. For an efficient implementation, we propose a simplified Bayesian hierarchical model and the corresponding Tanner-Wong algorithm. We express the relationship between the kernel on the input space and the kernel on the output space as a symmetric-definite generalized eigenproblem. Based on this eigenproblem, an efficient approach to choosing the base kernel matrices is presented. The effectiveness of our Bayesian model with the Tanner-Wong algorithm is demonstrated through some classification experiments showing promising results.
Convexity , Surrogate Functions and Iterative Optimization in Multi-class Logistic Regression Models

In this paper, we propose a family of surrogate maximization (SM) algorithms for multi-class logistic regression models (also called conditional exponential models). An SM algorithm aims at turning an otherwise intractable maximization problem into a tractable one by iterating two steps. The S-step computes a tractable surrogate function to substitute the original objective function, and the M-step seeks to maximize this surrogate function. We apply SM algorithms to logistic regression models, leading to the standard SM, generalized SM, gradient SM, and quadratic SM algorithms. Compared with Newton’s method, these SM algorithms dramatically save computational costs when either the dimensionality or number of data samples is huge. Finally, we demonstrate the efficacy of these SM algorithms and compare their empirical performance on text categorization.
Using kernel PCA to improve eigenvoice speaker adaptation

Eigenvoice-based methods have been shown to be effective for fast speaker adaptation when only a small amount of adaptation data is available. Conventionally, these methods employ linear principal component analysis (PCA) to find the most important eigenvoices. Recently, in what we called kernel eigenvoice (KEV) speaker adaptation, we suggested the use of kernel PCA to compute the eigenvoices so as to exploit possible nonlinearity in the data. The major challenge is that unlike the standard eigenvoice (EV) method, an adapted speaker model found by KEV adaptation resides in the high-dimensional kernel-induced feature space; it is not clear how to obtain the constituent Gaussians of the adapted model that are needed for the computation of state observation likelihoods during the estimation of eigenvoice weights and subsequent decoding. Our solution is the use of composite kernels in such a way that state observation likelihoods can be computed using only kernel functions. In an evaluation on the TIDIGITS task using less than 10s of adaptation speech, it is found that KEV speaker adaptation using composite Gaussian kernels outperforms a speaker-independent model and adapted models found by EV, MAP, or MLLR adaptation using 2.1s and 4.1s of speech.
Surrogate maximization/minimization algorithms for AdaBoost and the logistic regression model

Surrogate maximization (or minimization) (SM) algorithms are a family of algorithm that can be regarded as a generalization of expectation-maximization (EM) algorithms. There are three major approaches to the construction of surrogate function, all relying on the convexity of some function. In this paper, we solve the boosting problem by proposing SM algorithms for the corresponding optimization problem. Specifically, for AdaBoost, we derive an SM algorithm that can be shown to be identical to the algorithm proposed by Collins et al. (2002) based on Bregman distance. More importantly, for LogitBoost (or logistic boosting), we use several methods to construct different surrogate functions which result in different SM algorithms. By combining multiple methods, we are able to derive an SM algorithm that is also the same as an algorithm derived by Collins et al. (2002). Our approach based on SM algorithms is much simpler and convergence results follow naturally.
A study of various composite kernels for kernel eigenvoice speaker adaptation

Eigenvoice-based methods have been shown to be effective for fast speaker adaptation when the amount of adaptation data is small, say, less than 10 seconds. In traditional eigenvoice (EV) speaker adaptation, linear principal component analysis (PCA) is used to derive the eigenvoices. Recently, we proposed that eigenvoices found by nonlinear kernel PCA could be more effective, and the eigenvoices thus derived were called kernel eigenvoices (KEV). One of our novelties is the use of composite kernel that makes it possible to compute state observation likelihoods via kernel functions. We investigate two different composite kernels: direct sum kernel and tensor product kernel for KEV adaptation. In an evaluation on the TIDIGITS task, it is found that KEV speaker adaptations using either form of composite kernel are equally effective, and they outperform a speaker-independent model and the adapted models from EV, MAP, or MLLR adaptation using 2.1s and 4.1s of speech. For example, with 2.1s of adaptation data, KEV adaptation outperforms the speaker-independent model by 27.5%, whereas EV, MAP, and MLLR adaptations are not effective at all.
Incremental PCA based face recognition

In the real world, learning is often expected to be a continuous process, which is capable of incorporating new facts into the past experience. However, currently many typical face recognition methods, such as eigenface and Fisherface, have only focused on non-incremental learning tasks, where the learning stops once the training set has been duly processed. In this paper, we present a PCA-based algorithm for face recognition, which takes the incremental learning in account. This method can update the principal subspace without simply re-computing the eigen decomposition from scratch.
Speedup of kernel eigenvoice speaker adaptation by embedded kernel PCA

Recently, we proposed an improvement to the eigenvoice (EV) speaker adaptation called kernel eigenvoice (KEV) speaker adaptation. In KEV adaptation, eigenvoices are computed using kernel PCA, and a new speaker’s adapted model is implicitly computed in the kernel-induced feature space. Due to many online kernel evaluations, both adaptation and subsequent recognition of KEV adaptation are slower than EV adaptation. In this paper, we eliminate all online kernel computations by finding an approximate pre-image of the implicit adapted model found by KEV adaptation. Furthermore, the two steps of finding the implicit adapted model and its approximate pre-image are integrated by embedding the kernel PCA procedure in our new embedded kernel eigenvoice (eKEV) speaker adaptation method. When tested in an TIDIGITS task with less than 10s of adaptation speech, eKEV adaptation obtained a speedup of 6–14 times in adaptation and 136 times in recognition over KEV adaptation with 12–13% relative improvement in recognition accuracy.
Scaling up support vector data description by using core-sets

Support vector data description (SVDD) is a powerful kernel method that has been commonly used for novelty detection. While its quadratic programming formulation has the important computational advantage of avoiding the problem of local minimum, this has a runtime complexity of O(N/sup 3/), where N is the number of training patterns. It thus becomes prohibitive when the data set is large. Inspired from the use of core-sets in approximating the minimum enclosing ball problem in computational geometry, we propose An approximation method that allows SVDD to scale better to larger data sets. Most importantly, the proposed method has a running time that is only linear in N. Experimental results on two large real-world data sets demonstrate that the proposed method can handle data sets that are much larger than those that can be handled by standard SVDD packages, while its approximate solution still attains equally good, or sometimes even better, novelty detection performance.
Fusing images with different focuses using support vector machines

Many vision-related processing tasks, such as edge detection, image segmentation and stereo matching, can be performed more easily when all objects in the scene are in good focus. However, in practice, this may not be always feasible as optical lenses, especially those with long focal lengths, only have a limited depth of field. One common approach to recover an everywhere-in-focus image is to use wavelet-based image fusion. First, several source images with different focuses of the same scene are taken and processed with the discrete wavelet transform (DWT). Among these wavelet decompositions, the wavelet coefficient with the largest magnitude is selected at each pixel location. Finally, the fused image can be recovered by performing the inverse DWT. In this paper, we improve this fusion procedure by applying the discrete wavelet frame transform (DWFT) and the support vector machines (SVM). Unlike DWT, DWFT yields a translation-invariant signal representation. Using features extracted from the DWFT coefficients, a SVM is trained to select the source image that has the best focus at each pixel location, and the corresponding DWFT coefficients are then incorporated into the composite wavelet representation. Experimental results show that the proposed method outperforms the traditional approach both visually and quantitatively.
The pre-image problem in kernel methods

In this paper, we address the problem of finding the pre-image of a feature vector in the feature space induced by a kernel. This is of central importance in some kernel applications, such as on using kernel principal component analysis (PCA) for image denoising. Unlike the traditional method in which relies on nonlinear optimization, our proposed method directly finds the location of the pre-image based on distance constraints in the feature space. It is noniterative, involves only linear algebra and does not suffer from numerical instability or local minimum problems. Evaluations on performing kernel PCA and kernel clustering on the USPS data set show much improved performance.