improved estimation of the disturbance variance in a linear regression model abstract in this paper we consider estimation of the disturbance variance in a linear regression model a new class of improved estimators is obtained by extending results dating to stein 1964 these estimators dominate the ordinary least squares estimator under squared error loss

a particle swarm optimization based simultaneous learning framework for clustering and classification a particle swarm optimization based simultaneous learning framework for clustering and classification psoslcc is proposed in this paper firstly an improved particle swarm optimization pso is used to partition the training samples the number of clusters must be given in advance an automatic clustering algorithm rather than the trial and error is adopted to find the proper number of clusters and a set of clustering centers is obtained to form classification mechanism secondly in order to exploit more useful local information and get a better optimizing result a global factor is introduced to the update strategy update strategy of particle in pso psoslcc has been extensively compared with fuzzy relational classifier frc vector quantization and learning vector quantization vq lvq3 and radial basis function neural network rbfnn a simultaneous learning framework for clustering and classification scc over several real life datasets the experimental results indicate that the proposed algorithm not only greatly reduces the time complexity but also obtains better classification accuracy for most datasets used in this paper moreover psoslcc is applied to a real world application namely texture image segmentation with a good performance obtained which shows that the proposed algorithm has a potential of classifying the problems with large scale a particle swarm optimization based simultaneous learning framework for clustering and classification psoslcc is proposed the simultaneous frame consists of two parts clustering section and classification section an automatic clustering algorithm is used to find the proper number of clusters an improved particle swarm optimization with a global factor is used in the training phase the performance of psoslcc has been extensively compared with four state of the art classification algorithms over a test suit of datasets and texture image segmentation

fuzzy granular gravitational clustering algorithm for multivariate data abstract a new method for finding fuzzy information granules from multivariate data through a gravitational inspired clustering algorithm is proposed in this paper the proposed algorithm incorporates the theory of granular computing which adapts the cluster size with respect to the context of the given data via an inspiration in newton s law of universal gravitation both conditions of clustering similar data and adapting to the size of each granule are achieved this paper compares the fuzzy granular gravitational clustering algorithm fggca against other clustering techniques on two grounds classification accuracy and clustering validity indices e g rand fm davies bouldin dunn homogeneity and separation the fggca is tested with multiple benchmark classification datasets such as iris wine seeds and glass identification

genetic training instance selection in multiobjective evolutionary fuzzy systems a coevolutionary approach when dealing with datasets that are characterized by a large number of instances multiobjective evolutionary learning moel of fuzzy rule based systems frbss suffers from high computational costs mainly because of the fitness evaluation the use of a reduced set of representative instances in place of the overall training set ts would considerably lessen the computational effort even though a large number of papers have proposed instance selection approaches mainly in classification problems how this selection should be performed especially in the context of regression is still an open issue in this paper we tackle the instance selection problem in the framework of moel of frbss through a coevolutionary approach in the execution of the moel periodically a single objective genetic algorithm soga evolves a population of reduced tss the soga aims to maximize a purposely defined index which measures how much the pareto fronts computed by using respectively the reduced ts and the overall ts are close to each other the closer the fronts the more the reduced ts is representative of the overall ts during the execution of the moel the rule base and the membership function parameters of the fuzzy sets are concurrently learned by maximizing the accuracy and minimizing the complexity we tested our approach on 12 large datasets we adopted reduced tss composed of 5 10 and 20 of the overall ts using nonparametric statistical tests we verified that with 10 and 20 of the overall ts the pareto front approximations that are generated by our coevolutionary approach are comparable with the ones generated by applying the moel with the overall ts although the coevolution allows us to save up to 86 36 of the execution time in addition the analysis of the behavior of three representative solutions on the test set highlights that the use of the reduced tss does not affect the generalization capabilities of the generated frbss

feature evaluation and selection with cooperative game theory recent years various information theoretic based measurements have been proposed to remove redundant features from high dimensional data set as many as possible however most traditional information theoretic based selectors will ignore some features which have strong discriminatory power as a group but are weak as individuals to cope with this problem this paper introduces a cooperative game theory based framework to evaluate the power of each feature the power can be served as a metric of the importance of each feature according to the intricate and intrinsic interrelation among features then a general filter feature selection scheme is presented based on the introduced framework to handle the feature selection problem to verify the effectiveness of our method experimental comparisons with several other existing feature selection methods on fifteen uci data sets are carried out using four typical classifiers the results show that the proposed algorithm achieves better results than other methods in most cases

on using prototype reduction schemes to optimize kernel based nonlinear subspace methods abstract the subspace method of pattern recognition is a classification technique in which pattern classes are specified in terms of linear subspaces spanned by their respective class based basis vectors to overcome the limitations of the linear methods kernel based nonlinear subspace kns methods have been recently proposed in the literature in kns the kernel principal component analysis kpca has been employed to get principal components not in an input space but in a high dimensional space where the components of the space are nonlinearly related to the input variables the length of projections onto the basis vectors in the kpca are computed using a kernel matrix k whose dimension is equivalent to the number of sample data points clearly this is problematic especially for large data sets in this paper we suggest a computationally superior mechanism to solve the problem rather than define the matrix k with the whole data set and compute the principal components we propose that the data be reduced into a smaller representative subset using a prototype reduction scheme prs since a prs has the capability of extracting vectors that satisfactorily represent the global distribution structure we demonstrate that data points which are ineffective in the classification can be eliminated to obtain a reduced kernel matrix k without degrading the performance our experimental results demonstrate that the proposed mechanism dramatically reduces the computation time without sacrificing the classification accuracy for samples involving real life data sets as well as artificial data sets the results especially demonstrate the computational advantage for large data sets such as those involved in data mining and text categorization applications

on using prototype reduction schemes to enhance the computation of volume based inter class overlap measures in most pattern recognition pr applications it is advantageous if the accuracy or error rate of the classifier can be evaluated or bounded prior to testing it in a real life setting it is also well known that if the two class conditional distributions have a large overlapping volume almost all the available work on overlapping of classes deals with the case when there are only two classes the classification accuracy is poor this is because if we intend to use the classification accuracy as a criterion for evaluating a pr system the points within the overlapping volume tend to lead to maximal misclassification unfortunately the computation of the indices which quantify the overlapping volume is expensive in this vein we propose a strategy of using a prototype reduction scheme prs to approximately but quickly compute the latter in this paper we demonstrate first of all that this is an extremely expedient proposition indeed we show that by completely discarding we are not aware of any reported scheme which discards irrelevant sample training points and which simultaneously attains to an almost comparable accuracy the points not included by the prs we can obtain a reduced set of sample points using which in turn the measures for the overlapping volume can be computed the value of the corresponding figures is comparable to those obtained with the original training set i e the one which considers all the data points even though the computations required to obtain the prototypes and the corresponding measures are significantly less the proposed method has been rigorously tested on artificial and real life datasets and the results obtained are in our opinion quite impressive sometimes faster by two orders of magnitude

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

locally linear reconstruction for instance based learning instance based learning ibl so called memory based reasoning mbr is a commonly used non parametric learning algorithm k nearest neighbor k nn learning is the most popular realization of ibl due to its usability and adaptability k nn has been successfully applied to a wide range of applications however in practice one has to set important model parameters only empirically the number of neighbors k and weights to those neighbors in this paper we propose structured ways to set these parameters based on locally linear reconstruction llr we then employed sequential minimal optimization smo for solving quadratic programming step involved in llr for classification to reduce the computational complexity experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation not only did llr outperform the conventional weight allocation methods without much additional computational cost but also llr was found to be robust to the change of k

selecting critical patterns based on local geometrical and statistical information pattern selection methods have been traditionally developed with a dependency on a specific classifier in contrast this paper presents a method that selects critical patterns deemed to carry essential information applicable to train those types of classifiers which require spatial information of the training data set critical patterns include those edge patterns that define the boundary and those border patterns that separate classes the proposed method selects patterns from a new perspective primarily based on their location in input space it determines class edge patterns with the assistance of the approximated tangent hyperplane of a class surface it also identifies border patterns between classes using local probability the proposed method is evaluated on benchmark problems using popular classifiers including multilayer perceptrons radial basis functions support vector machines and nearest neighbors the proposed approach is also compared with four state of the art approaches and it is shown to provide similar but more consistent accuracy from a reduced data set experimental results demonstrate that it selects patterns sufficient to represent class boundary and to preserve the decision surface

pre testing for linear restrictions in a regression model with spherically symmetric disturbances abstract in this paper we derive the exact risk under quadratic loss of pre test estimators of the prediction vector and of the error variance of a linear regression model with spherically symmetric disturbances the pre test in question is one of the validity of a set of exact linear restrictions on the model s coefficient vector we demonstrate how the known results for the model with normal disturbances can be extended to this broader case we also show that the critical value of unity results in a minimum of the risk of the pre test estimator of the error variance to illustrate the results we assume multivariate student t regression disturbances and numerically evaluate the derived expressions

generalized fuzzy c means clustering strategies using l sub p norm distances fuzzy c means fcm is a useful clustering technique modifications of fcm using l sub 1 norm distances increase robustness to outliers object and relational data versions of fcm clustering are defined for the more general case where the l sub p norm p spl ges 1 or semi norm 0 0 in order to facilitate the empirical examination of the object data models both object and relational approaches are included in a numerical study

estimation of the error variance after a preliminary test of homogeneity in a regression model with spherically symmetric disturbances abstract in this paper we consider the risk under quadratic loss of an estimator of the error variance after a pre test for homogeneity of the variances in the two sample linear regression model we investigate the effects on risk of assuming normal disturbances when in fact the error distribution is spherically symmetric we also broaden the standard assumption that the never pool variance estimators are based on the least squares principle to illustrate the results numerically we consider two families of variance mixing distributions one of which results in multivariate student t disturbances our results show that in some situations we should always pre test even if the error variances are equal and we provide the optimal test critical value the evaluations also show that using the least squares technique to form the never pool estimators may not always be the preferred strategy

on the distributional properties of model selection criteria abstract it is commonly accepted that statistical modeling should follow the parsimony principle namely that simple models should be given priority whenever possible but little quantitative knowledge is known concerning the amount of penalty for complexity regarded as allowable we try to understand the parsimony principle in the context of model selection in particular the generalized final prediction error criterion is considered and we argue that the penalty term should be chosen between 1 5 and 5 for most practical situations applying our results to the cross validation criterion we obtain insights into how the partition of data should be done we also discuss the small sample performance of our methods

information based objective functions for active data selection learning can be made more efficient if we can actively select particularly salient data points within a bayesian learning framework objective functions are discussed that measure the expected informativeness of candidate measurements three alternative specifications of what we want to gain information about lead to three different criteria for data selection all these criteria depend on the assumption that the hypothesis space is correct which may prove to be their main weakness

bifurcation analysis of a speed gradient continuum traffic flow model abstract a bifurcation analysis approach is presented based on the macroscopic traffic flow model this method can be used to describe and predict the nonlinear traffic phenomena on the highway from a system global stability perspective based on a recently proposed speed gradient continuum traffic flow model the types and stabilities of the equilibrium solutions are discussed and the existence of hopf bifurcation and saddle node bifurcation is proved then various bifurcations such as hopf bifurcation saddle node bifurcation limit point bifurcation of cycles cusp bifurcation and bogdanov takens bifurcation are found and the traffic flow behaviors at some of them are analyzed when the hopf bifurcation is selected as the starting point of density temporal evolution it may help to explain the stop and go traffic phenomena

a note on pooling variances abstract the estimation of variance in the normal distribution is studied under statistical inference based on conditional specification when there are two samples available for estimating the variance and it is not certain whether the two samples are from the same population the experimenter usually uses a test to resolve the uncertainty when the test is not significant the samples are pooled to obtain a pooled estimator otherwise the individual sample variance is used the bias and mean squared error of such a preliminary test estimator are studied it is shown that the preliminary test estimator has a smaller mean squared error than the usual unbiased estimator when the level of significance for the preliminary test is appropriately chosen

optimal critical values of pre tests when estimating the regression error variance analytical findings under a general loss structure this paper re visits the problem of estimating the regression error variance in a linear multiple regression model after preliminary hypothesis tests for either linear restrictions on the coefficients or homogeneity of variances there is an extensive literature that discusses these problems particularly in terms of the sampling properties of the pre test estimators using various loss functions as the basis for risk analysis in this paper a unified framework for analysing the risk properties of these estimators is developed under a general class of loss structures that incorporates virtually all first order differentiable losses particular consideration is given to the choice of critical values for the pre tests analytical results indicate that an level substantially higher than those normally used may be appropriate for optimal risk properties under a wide range of loss functions the paper also generalizes some known analytical results in the pre test literature and proves other results only previously shown numerically

how much better are better estimators of a normal variance abstract this article considers the estimation problem of a normal variance 2 on the basis of a random sample x 1 x n under quadratic loss when the mean is unknown the best equivariant estimator 0 xj xbar 2 n 1 is known to be inadmissible but the extent to which this inadmissibility phenomenon is serious has not previously been considered herein the mean squared error of the minimax and admissible estimator due to brewster and zidek 1974 is evaluted and an explicit formula for that estimator is given it is shown that this risk function has maximum at 0 which is the mode of the corresponding generalized prior density locally optimal shrinkage scale equivariant estimators are introduced and their risks are calculated for several values of the sample size n it is observed that the brewster zidek estimator has risk function close to that of locally optimal minimax shrinkage estimators but that the latter cannot give more than 4 relative improvement on the traditional proced

optimizing the kernel in the empirical feature space in this paper we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space an euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved employing a data dependent kernel we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space it is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by cristianini extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data and leads to a substantial sometimes significant improvement in the performance of various data classification algorithms

nonlinear analysis of the optimal velocity difference model with reaction time delay abstract we investigate the effect of reaction time delay of drivers upon traffic flow analytically and numerically we extend the optimal velocity difference model by taking reaction time delay of drivers into account the stability condition is obtained the burgers korteweg de vries kdv and modified korteweg de vries mkdv equations are derived in the stable metastable and unstable regions the results show that the increase of reaction time delay of drivers leads to the unrealistic traffic phenomena and unstable traffic flow by adjusting the coefficient of the velocity and optimal velocity difference the unrealistic traffic phenomena and unstable traffic flow can disappear

frequentist model average estimators the traditional use of model selection methods in practice is to proceed as if the final selected model had been chosen in advance without acknowledging the additional uncertainty introduced by model selection this often means underreporting of variability and too optimistic confidence intervals we build a general large sample likelihood apparatus in which limiting distributions and risk properties of estimators post selection as well as of model average estimators are precisely described also explicitly taking modeling bias into account this allows a drastic reduction in complexity as competing model averaging schemes may be developed discussed and compared inside a statistical prototype experiment where only a few crucial quantities matter in particular we offer a frequentist view on bayesian model averaging methods and give a link to generalized ridge estimators our work also leads to new model selection criteria the methods are illustrated with real data applications

vector quantization technique for nonparametric classifier design an effective data reduction technique based on vector quantization is introduced for nonparametric classifier design two new nonparametric classifiers are developed and their performance is evaluated using various examples the new methods maintain a classification accuracy that is competitive with that of classical methods but at the same time yields very high data reduction rates

adaptive relevance matrices in learning vector quantization we propose a new matrix learning scheme to extend relevance learning vector quantization rlvq an efficient prototype based classification algorithm toward a general adaptive metric by introducing a full matrix of relevance factors in the distance measure correlations between different features and their importance for the classification scheme can be taken into account and automated and general metric adaptation takes place during training in comparison to the weighted euclidean metric used in rlvq and its variations a full matrix is more powerful to represent the internal structure of the data appropriately large margin generalization bounds can be transferred to this case leading to bounds that are independent of the input dimensionality this also holds for local metrics attached to each prototype which corresponds to piecewise quadratic decision boundaries the algorithm is tested in comparison to alternative learning vector quantization schemes using an artificial data set a benchmark multiclass problem from the vci repository and a problem from bioinformatics the recognition of splice sites for c elegans

bayesian model averaging for linear regression models abstract we consider the problem of accounting for model uncertainty in linear regression models conditioning on a single selected model ignores model uncertainty and thus leads to the underestimation of uncertainty when making inferences about quantities of interest a bayesian solution to this problem involves averaging over all possible models i e combinations of predictors when making inferences about quantities of interest this approach is often not practical in this article we offer two alternative approaches first we describe an ad hoc procedure occam s window which indicates a small set of models over which a model average can be computed second we describe a markov chain monte carlo approach that directly approximates the exact solution in the presence of model uncertainty both of these model averaging procedures provide better predictive performance than any single model that might reasonably have been selected in the extreme case where there are many candidate predictors but

the exact general formulae for the moments and the mse dominance of the stein rule and positive part stein rule estimators abstract in this paper we derive the general formulae for the moments of a linear functional of the stein rule and positive part stein rule estimators i e h bsr and h bpsr in a different way from phillips 1984 the mse dominance of h bpsr over h bsr and a sufficient condition for h bsr to dominate a linear functional of the ols estimator h b are shown using the formulae for the moments we evaluate numerically the first four moments of bsr and bpsr and examine the bias variance mse skewness and kurtosis

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

steady state solutions of hydrodynamic traffic models we investigate steady state solutions of hydrodynamic traffic models in the absence of any intrinsic inhomogeneity on roads such as on ramps it is shown that typical hydrodynamic models possess seven different types of inhomogeneous steady state solutions the seven solutions include those that have been reported previously only for microscopic models the characteristic properties of wide jam such as moving velocity of its spatiotemporal pattern and or out flux from wide jam are shown to be uniquely determined and thus independent of initial conditions of dynamic evolution topological considerations suggest that all of the solutions should be common to a wide class of traffic models the results are discussed in connection with the universality conjecture for traffic models also the prevalence of the limit cycle solution in a recent study of a microscopic model is explained in this approach

three new instance selection methods based on local sets the local set is the largest hypersphere centered on an instance such that it does not contain instances from any other class due to its geometrical nature this structure can be very helpful for distance based classification such as classification based on the nearest neighbor rule this paper is focused on instance selection for nearest neighbor classification which in short aims to reduce the number of instances in the training set without affecting the classification accuracy three instance selection methods based on local sets which follow different and complementary strategies are proposed in an experimental study involving 26 known databases they are compared with 11 of the most successful state of the art methods in standard and noisy environments to evaluate their performances two complementary approaches are applied the pareto dominance relation and the technique for order preference by similarity to ideal solution the results achieved by the proposals reveal that they are among the most effective methods in this field highlightswe propose three selection strategies with different accuracy reduction tradeoff we assess them on 26 known databases with more than 1000 instances each one the results are compared with those of 11 successful state of the art methods according to different criteria the new methods are always among the top performers

on pooling disturbance variances when the goal is testing restrictions on regression coefficients abstract in this paper we examine the problem of pooling variances from the viewpoint of testing linear restrictions on regression coefficients that is we examine the sampling performance of the two stage test which consists of the pre test for equality of disturbance variances followed by the main test for linear restrictions on regression coefficients it is shown that the bias in the size of the two stage test may not be important and the two stage test can be more powerful than the usual test which does not conduct the pre test if the size of the pre test is chosen appropriately

a memetic algorithm for evolutionary prototype selection a scaling up approach prototype selection problem consists of reducing the size of databases by removing samples that are considered noisy or not influential on nearest neighbour classification tasks evolutionary algorithms have been used recently for prototype selection showing good results however due to the complexity of this problem when the size of the databases increases the behaviour of evolutionary algorithms could deteriorate considerably because of a lack of convergence this additional problem is known as the scaling up problem memetic algorithms are approaches for heuristic searches in optimization problems that combine a population based algorithm with a local search in this paper we propose a model of memetic algorithm that incorporates an ad hoc local search specifically designed for optimizing the properties of prototype selection problem with the aim of tackling the scaling up problem in order to check its performance we have carried out an empirical study including a comparison between our proposal and previous evolutionary and non evolutionary approaches studied in the literature the results have been contrasted with the use of non parametric statistical procedures and show that our approach outperforms previously studied methods especially when the database scales up

bayes error estimation using parzen and k nn procedures the use of k nearest neighbor k nn and parzen density estimates to obtain estimates of the bayes error is investigated under limited design set conditions by drawing analogies between the k nn and parzen procedures new procedures are suggested and experimental results are given which indicate that these procedures yield a significant improvement over the conventional k nn and parzen procedures we show that by varying the decision threshold many of the biases associated with the k nn or parzen density estimates may be compensated and successful error estimation may be performed in spite of these biases experimental results are given which demonstrate the effect of kernel size and shape parzen the size of k k nn and the number of samples in the design set

on using prototype reduction schemes to optimize locally linear reconstruction methods this paper concerns the use of prototype reduction schemes prs to optimize the computations involved in typical k nearest neighbor k nn rules these rules have been successfully used for decades in statistical pattern recognition pr 1 15 applications and are particularly effective for density estimation classification and regression because of the known error bounds that they possess for a given data point of unknown identity the k nn possesses the phenomenon that it combines the information about the samples from a priori target classes values of selected neighbors to predict the target class of the tested sample or to estimate the density function value of the given queried sample recently an implementation of the k nn named as the locally linear reconstruction llr 2 has been proposed the salient and brilliant feature of the latter is that by invoking a quadratic optimization process it is capable of systematically setting model parameters such as the number of neighbors specified by the parameter k and the weights however the llr takes more time than other conventional methods when it has to be applied to classification tasks to overcome this problem we propose a strategy of using a prs to efficiently compute the optimization problem in this paper we demonstrate first of all that by completely discarding the points not included by the prs we can obtain a reduced set of sample points using which in turn the quadratic optimization problem can be computed far more expediently the values of the corresponding indices are comparable to those obtained with the original training set i e the one which considers all the data points even though the computations required to obtain the prototypes and the corresponding classification accuracies are noticeably less the proposed method has been tested on artificial and real life data sets and the results obtained are very promising and could have potential in pr applications

testing linear restrictions on coefficients in a linear regression model with proxy variables and spherically symmetric disturbances abstract in this paper we consider the power function of the classical f test for linear restrictions on the coefficients in a linear regression model with spherically symmetric disturbances when proxies are used in the place of unobservable regressors to illustrate the results we numerically evaluate the power function for two families of variance mixing distributions in a simple univariate regression model our results show the effects on the power function of the degree of nonnormality and of the correlation between the omitted and the proxy variable

estimating the error variance in regression after a preliminary test of restrictions on the coefficients abstract maximum likelihood estimation of the variance of the disturbances in a linear regression is considered in the context of exact linear restrictions on the coefficient vector the exact risk function for the associated preliminary test estimator of this variance is derived and evaluated numerically on the basis of relative quadratic loss our results indicate that in practical situations the risk of the pre test estimator differs only slightly from that of the naive estimator which ignores the restrictions without testing applying the coefficient restrictions without testing their validity is not recommended

a novel radial basis function neural network for discriminant analysis a novel radial basis function neural network for discriminant analysis is presented in this paper in contrast to many other researches this work focuses on the exploitation of the weight structure of radial basis function neural networks using the bayesian method it is expected that the performance of a radial basis function neural network with a well explored weight structure can be improved as the weight structure of a radial basis function neural network is commonly unknown the bayesian method is therefore used in this paper to study this a priori structure two weight structures are investigated in this study i e a single gaussian structure and a two gaussian structure an expectation maximization learning algorithm is used to estimate the weights the simulation results showed that the proposed radial basis function neural network with a weight structure of two gaussians outperformed the other algorithms

ifs coco instance and feature selection based on cooperative coevolution with nearest neighbor rule feature and instance selection are two effective data reduction processes which can be applied to classification tasks obtaining promising results although both processes are defined separately it is possible to apply them simultaneously this paper proposes an evolutionary model to perform feature and instance selection in nearest neighbor classification it is based on cooperative coevolution which has been applied to many computational problems with great success the proposed approach is compared with a wide range of evolutionary feature and instance selection methods for classification the results contrasted through non parametric statistical tests show that our model outperforms previously proposed evolutionary approaches for performing data reduction processes in combination with the nearest neighbor rule

correntropy based feature selection using binary projection most feature selection algorithms based on information theoretic learning itl adopt ranking process or greedy search as their searching strategies the former selects features individually so that it ignores feature interaction and dependencies the latter heavily relies on the search paths as only one path will be explored with no possible back track in addition both strategies typically lead to heuristic algorithms to cope with these problems this article proposes a novel feature selection framework based on correntropy in itl namely correntropy based feature selection using binary projection bpfs our framework selects features by projecting the original high dimensional data to a low dimensional space through a special binary projection matrix the formulated objective function aims at maximizing the correntropy between selected features and class labels and this function can be efficiently optimized via standard mathematical tools we apply the half quadratic method to optimize the objective function in an iterative manner where each iteration reduces to an assignment subproblem which can be highly efficiently solved with some off the shelf toolboxes comparative experiments on six real world datasets indicate that our framework is effective and efficient

simultaneous clustering and classification over cluster structure representation two main tasks in pattern recognition area are clustering and classification owing to their different goals traditionally these two tasks are treated separately however when label information is available such separate treatment can not fully explore data information first classification is not favored by the data cluster structure second clustering is not guided by valuable label information third the relationship of clusters and classes is not revealed contrary to this separate learning treatment simultaneous learning clustering and classification could benefit each other and overcomes these problems recently a simultaneous learning framework scc was proposed through modeling p class cluster classification and clustering mechanism in scc depend only on cluster centroids however it produces severely nonlinear objective thus has to use a heuristic searching method modified particle swarm optimization to find the optimal solution but it is very slow further modeling p class cluster makes scc hard to incorporate semi supervised settings in this paper we propose an alternative framework sc 3sr for simultaneous learning besides a classifier derived on the original data another classifier on the newly formed cluster structure representation is derived as well through this classifier the clustering learning is guided by the label and classification learning is also favored by cluster structure of data the final objective is continuously differentiable for which some principled optimization algorithms with convergence guaranteed exist as a result our algorithm is much faster than scc further we generalize this framework to semisupervised situation with the idea of manifold regularization and propose semisc 3sr algorithm our experiments demonstrate the effectiveness of both sc 3sr and semisc 3sr

fuzzy clustering with supervision this study is concerned with clustering carried out in presence of labeled patterns an objective of this optimization is to reconcile between the structure residing in data and being primarily discovered by the underlying clustering mechanism and the labels of the patterns forming such structure in this sense one can consider the supervised fuzzy clustering to be a framework of preliminary data analysis providing with a thorough insight into the structure of the data and supporting the ensuing design of detailed classifiers the proposed method augments the standard fuzzy c means algorithm by extending the original objective function by the supervision component labeled patterns experimental results illustrate the approach and discuss the use of this type of clustering in vector quantization

validity index for crisp and fuzzy clusters abstract in this article a cluster validity index and its fuzzification is described which can provide a measure of goodness of clustering on different partitions of a data set the maximum value of this index called the pbm index across the hierarchy provides the best partitioning the index is defined as a product of three factors maximization of which ensures the formation of a small number of compact clusters with large separation between at least two clusters we have used both the k means and the expectation maximization algorithms as underlying crisp clustering techniques for fuzzy clustering we have utilized the well known fuzzy c means algorithm results demonstrating the superiority of the pbm index in appropriately determining the number of clusters as compared to three other well known measures the davies bouldin index dunn s index and the xie beni index are provided for several artificial and real life data sets

simultaneous estimation of poisson means under entropy loss on caracterise des estimateurs lineaires admissibles de parametres de poisson multiples sous une perte d entropie on donne les estimateurs dominant certains des estimateurs standards

on the robustness of lm lr and w tests in regression models in this paper we show that in the small sample case while the lr test is robust the lm and w tests are not robust when the errors are student t for the large sample however all three tests are found to be robust a variation of the w test is also considered and it turns out to be nonrobust the implications of assuming the errors to be distributed as multivariate student t rather than multivariate normal are also discussed it is found that not all of evans and savin s 4 results carry through when the errors are student t

a simultaneous learning framework for clustering and classification traditional pattern recognition generally involves two tasks unsupervised clustering and supervised classification when class information is available fusing the advantages of both clustering learning and classification learning into a single framework is an important problem worthy of study to date most algorithms generally treat clustering learning and classification learning in a sequential or two step manner i e first execute clustering learning to explore structures in data and then perform classification learning on top of the obtained structural information however such sequential algorithms cannot always guarantee the simultaneous optimality for both clustering and classification learning in fact the clustering learning in these algorithms just aids the subsequent classification learning and does not benefit from the latter to overcome this problem a simultaneous learning framework for clustering and classification scc is presented in this paper scc aims to achieve three goals 1 acquiring the robust classification and clustering simultaneously 2 designing an effective and transparent classification mechanism 3 revealing the underlying relationship between clusters and classes to this end with the bayesian theory and the cluster posterior probabilities of classes we define a single objective function to which the clustering process is directly embedded by optimizing this objective function the effective and robust clustering and classification results are achieved simultaneously experimental results on both synthetic and real life datasets show that scc achieves promising classification and clustering results at one time

inference after variable selection in linear regression models summary we explore the impact of variable selection on statistical inferences in linear regression models in particular the generalized final prediction error criterion of shibata 1984 is considered and it is found among other things that inferences on the regression coefficients are impaired by the variable selection procedure most notably the size of the nominal confidence sets tend to be inflated if they are derived based on the selected model on the other hand variable selection does not seem to have much impact on the inferences for the error variance our results complement those obtained by potscher 1991 in which testing procedures are used for variable selection

prototype selection for dissimilarity based classifiers a conventional way to discriminate between objects represented by dissimilarities is the nearest neighbor method a more efficient and sometimes a more accurate solution is offered by other dissimilarity based classifiers they construct a decision rule based on the entire training set but they need just a small set of prototypes the so called representation set as a reference for classifying new objects such alternative approaches may be especially advantageous for non euclidean or even non metric dissimilarities the choice of a proper representation set for dissimilarity based classifiers is not yet fully investigated it appears that a random selection may work well in this paper a number of experiments has been conducted on various metric and non metric dissimilarity representations and prototype selection methods several procedures like traditional feature selection methods here effectively searching for prototypes mode seeking and linear programming are compared to the random selection in general we find out that systematic approaches lead to better results than the random selection especially for a small number of prototypes although there is no single winner as it depends on data characteristics the k centres works well in general for two class problems an important observation is that our dissimilarity based discrimination functions relying on significantly reduced prototype sets 3 10 of the training objects offer a similar or much better classification accuracy than the best k nn rule on the entire training set this may be reached for multi class data as well however such problems are more difficult

optimal reference subset selection for nearest neighbor classification by tabu search this paper presents an approach to select the optimal reference subset ors for nearest neighbor classifier the optimal reference subset which has minimum sample size and satisfies a certain resubstitution error rate threshold is obtained through a tabu search ts algorithm when the error rate threshold is set to zero the algorithm obtains a near minimal consistent subset of a given training set while the threshold is set to a small appropriate value the obtained reference subset may have reasonably good generalization capacity a neighborhood exploration method and an aspiration criterion are proposed to improve the efficiency of ts experimental results based on a number of typical data sets are presented and analyzed to illustrate the benefits of the proposed method the performances of the result consistent and non consistent reference subsets are evaluated

regularization in matrix relevance learning in this paper we present a regularization technique to extend recently proposed matrix learning schemes in learning vector quantization lvq these learning algorithms extend the concept of adaptive distance measures in lvq to the use of relevance matrices in general metric learning can display a tendency towards oversimplification in the course of training an overly pronounced elimination of dimensions in feature space can have negative effects on the performance and may lead to instabilities in the training we focus on matrix learning in generalized lvq glvq extending the cost function by an appropriate regularization term prevents the unfavorable behavior and can help to improve the generalization ability the approach is first tested and illustrated in terms of artificial model data furthermore we apply the scheme to benchmark classification data sets from the uci repository of machine learning we demonstrate the usefulness of regularization also in the case of rank limited relevance matrices i e matrix learning with an implicit low dimensional representation of the data

exponential stability of large amplitude traveling fronts for quasi linear relaxation systems with diffusion a b s t r a c t this paper is concerned with the stability of traveling front solutions for 2 2 quasi linear relaxation systems with small diffusion rate by applying geometric singular perturbation method special evans function estimates detailed spectral analysis and c0 semigroup theories we prove that all the non degenerate waves for semi linear relaxation systems are locally exponentially stable in some exponentially weighted spaces we also obtain the linear exponential stability of the non degenerate waves for quasi linear relaxation systems where the wave strengths can be large

bias of nearest neighbor error estimates the bias of the finite sample nearest neighbor nn error from its asymptotic value is examined expressions are obtained which relate the bias of the nn and 2 nn errors to sample size dimensionality metric and distributions these expressions isolate the effect of sample size from that of the distributions giving an explicit relation showing how the bias changes as the sample size is increased experimental results are given which suggest that the expressions accurately predict the bias it is shown that when the dimensionality of the data is high it may not be possible to estimate the asymptotic error simply by increasing the sample size a new procedure is suggested to alleviate this problem this procedure involves measuring the mean nn errors at several sample sizes and using our derived relationship between the bias and the sample size to extrapolate an estimate of the asymptotic nn error the results are extended to the multiclass problem the choice of an optimal metric to minimize the bias is also discussed

variance estimation in a model with gaussian sub models this article considers the problem of estimating the dispersion parameter in a gaussian model that is intermediate between a model where the mean parameter is fully known fixed and a model where the mean parameter is completely unknown one of the goals is to understand the implications of the two step process of first selecting a model among a finite number of submodels then estimating a parameter of interest after the model selection but using the same sample data the estimators are classified into global two step and weighted estimators whereas the global type estimators ignore the model space structure the two step estimators explore the structure adaptively and can be related to pretest estimators and the weighted estimators are motivated by the bayesian approach their performances are compared theoretically and through simulations using their risk functions based on a scale invariant quadratic loss function it is shown that in the variance estimation problem efficiency gains arise by ex

discriminatively regularized least squares classification over the past decades regularization theory is widely applied in various areas of machine learning to derive a large family of novel algorithms traditionally regularization focuses on smoothing only and does not fully utilize the underlying discriminative knowledge which is vital for classification in this paper we propose a novel regularization algorithm in the least squares sense called discriminatively regularized least squares classification drlsc method which is specifically designed for classification inspired by several new geometrically motivated methods drlsc directly embeds the discriminative information as well as the local geometry of the samples into the regularization term so that it can explore as much underlying knowledge inside the samples as possible and aim to maximize the margins between the samples of different classes in each local area furthermore by embedding equality type constraints in the formulation the solutions of drlsc can follow from solving a set of linear equations and the framework naturally contains multi class problems experiments on both toy and real world problems demonstrate that drlsc is often superior in classification performance to the classical regularization algorithms including regularization networks support vector machines and some of the recent studied manifold regularization techniques

the optimal size of a preliminary test of linear restrictions in a misspecified regression model abstract when the choice of estimator for the coefficients in a linear regression model is determined by the outcome of a prior test of the validity of restrictions on the model it is well known that a minimax risk regret criterion leads to the simple rule that the optimal critical value for the preliminary test is approximately two in value regardless of the degrees of freedom we show that this result no longer holds in the likely event that relevant regressors are excluded from the model at the outset

quantum behaved discrete multi objective particle swarm optimization for complex network clustering abstract complex network research has attracted lots of attention in both academic community and various application fields complex network clustering as one of the key issues in complex network explores the internal organization of the nodes in a complex network the discrete particle swarm optimization strategy has been successfully proposed for network clustering while the existing method works with weak robust in this paper we model the task of complex network clustering as a multi objective optimization problem and solve the problem with the quantum mechanism based particle swarm optimization algorithm which is a parallel algorithm to our knowledge this is the first attempt to apply the quantum mechanism based discrete particle swarm optimization algorithm into network clustering in addition the non dominant sorting selection operation is employed for individual replacement consequently a quantum behaved discrete multi objective particle swarm optimization algorithm is proposed for complex network clustering the experimental results demonstrate that the proposed algorithm performs effectively and achieves competitive performance with the state of the art approaches on the extension of girvan and newman benchmarks and real world networks especially on large scale networks

the nearest subclass classifier a compromise between the nearest mean and nearest neighbor classifier we present the nearest subclass classifier nsc which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier the algorithm is based on the maximum variance cluster algorithm and as such it belongs to the class of prototype based classifiers the variance constraint parameter of the cluster algorithm serves to regularize the classifier that is to prevent overfitting with a low variance constraint value the classifier turns into the nearest neighbor classifier and with a high variance parameter it becomes the nearest mean classifier with the respective properties in other words the number of prototypes ranges from the whole training set to only one per class in the experiments we compared the nsc with regard to its performance and data set compression ratio to several other prototype based methods on several data sets the nsc performed similarly to the k nearest neighbor classifier which is a well established classifier in many domains also concerning storage requirements and classification speed the nsc has favorable properties so it gives a good compromise between classification performance and efficiency

an introduction to kernel based learning algorithms this paper provides an introduction to support vector machines kernel fisher discriminant analysis and kernel principal component analysis as examples for successful kernel based learning methods we first give a short background about vapnik chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations we illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and dna analysis

learning simultaneous adaptive clustering and classification via moea clustering learning and classification learning are two major tasks in pattern recognition the traditional hybrid clustering and classification algorithms handle them in a sequential way rather than a simultaneous way fortunately multiobjective optimization provides a way to solve this problem in this paper an algorithm that learns simultaneous clustering and classification adaptively via multiobjective evolutionary algorithm is proposed the main idea of this paper is to optimize two objective functions which represent fuzzy cluster connectedness and classification error rate to achieve the goal of simultaneous learning firstly we adopt a graph based representation scheme to encode so that it can generate a set of solutions with different number of clusters in a single run then the relationship between clustering and classification is built via the bayesian theory during the optimization process the quality of clustering and classification is measured by the objective functions and the feedback drawn from both aspects is used to guide the mutation at last a set of nondominated solutions are generated from which the final pareto optimal solution is selected by using adjusted rand index the results on synthetic datasets and real life datasets demonstrate the rationality and effectiveness of the proposed algorithm furthermore we apply the proposed algorithm to image segmentation including texture images and synthetic aperture radar images the experimental results show the superiority of the proposed algorithm compared with other five algorithms highlightsa simultaneous adaptive clustering and classification learning via moea is proposed new clustering objective function is designed and two objective functions complement with each other the number of clusters can be determined adaptively during the learning process the drawback from clustering classification learning is used to guide the search we extend it to texture image and sar image segmentation to show its effectiveness

on using prototype reduction schemes to optimize dissimilarity based classification the aim of this paper is to present a strategy by which a new philosophy for pattern classification namely that pertaining to dissimilarity based classifiers dbcs can be efficiently implemented this methodology proposed by duin and his co authors see refs experiments with a featureless approach to pattern recognition pattern recognition lett 18 1997 1159 1166 relational discriminant analysis pattern recognition lett 20 1999 1175 1181 dissimilarity representations allow for buillding good classifiers pattern recognition lett 23 2002 943 956 dissimilarity representations in pattern recognition concepts theory and applications ph d thesis delft university of technology delft the netherlands 2005 prototype selection for dissimilarity based classifiers pattern recognition 39 2006 189 208 is a way of defining classifiers between the classes and is not based on the feature measurements of the individual patterns but rather on a suitable dissimilarity measure between them the advantage of this methodology is that since it does not operate on the class conditional distributions the accuracy can exceed the bayes error bound the problem with this strategy is however the need to compute store and process the inter pattern dissimilarities for all the training samples and thus the accuracy of the classifier designed in the dissimilarity space is dependent on the methods used to achieve this in this paper we suggest a novel strategy to enhance the computation for all families of dbcs rather than compute store and process the dbc based on the entire data set we advocate that the training set be first reduced into a smaller representative subset also rather than determine this subset on the basis of random selection or clustering etc we advocate the use of a prototype reduction scheme prs whose output yields the points to be utilized by the dbc the rationale for this is explained in the paper apart from utilizing prss in the paper we also propose simultaneously employing the mahalanobis distance as the dissimilarity measurement criterion to increase the dbcs classification accuracy our experimental results demonstrate that the proposed mechanism increases the classification accuracy when compared with the conventional approaches for samples involving real life as well as artificial data sets even though the resulting dissimilarity criterion is not symmetric

regularized margin based conditional log likelihood loss for prototype learning the classification performance of nearest prototype classifiers largely relies on the prototype learning algorithm the minimum classification error mce method and the soft nearest prototype classifier snpc method are two important algorithms using misclassification loss this paper proposes a new prototype learning algorithm based on the conditional log likelihood loss cll which is based on the discriminative model called log likelihood of margin logm a regularization term is added to avoid over fitting in training as well as to maximize the hypothesis margin the cll in the logm algorithm is a convex function of margin and so shows better convergence than the mce in addition we show the effects of distance metric learning with both prototype dependent weighting and prototype independent weighting our empirical study on the benchmark datasets demonstrates that the logm algorithm yields higher classification accuracies than the mce generalized learning vector quantization glvq soft nearest prototype classifier snpc and the robust soft learning vector quantization rslvq and moreover the logm with prototype dependent weighting achieves comparable accuracies to the support vector machine svm classifier

model selection and accounting for model uncertainty in graphical models using occam s window abstract we consider the problem of model selection and accounting for model uncertainty in high dimensional contingency tables motivated by expert system applications the approach most used currently is a stepwise strategy guided by tests based on approximate asymptotic p values leading to the selection of a single model inference is then conditional on the selected model the sampling properties of such a strategy are complex and the failure to take account of model uncertainty leads to underestimation of uncertainty about quantities of interest in principle a panacea is provided by the standard bayesian formalism that averages the posterior distributions of the quantity of interest under each of the models weighted by their posterior model probabilities furthermore this approach is optimal in the sense of maximizing predictive ability but this has not been used in practice because computing the posterior model probabilities is hard and the number of models is very large often greater than 1

asymptotic level density for a class of vector quantization processes it is shown that for a class of vector quantization processes related to neural modeling that the asymptotic density q x of the quantization levels in one dimension in terms of the input signal distribution p x is a power law q x c p x sup alpha where the exponent alpha depends on the number n of neighbors on each side of a unit and is given by alpha 2 3 1 3n sup 2 3 n 1 sup 2 the asymptotic level density is calculated and monte carlo simulations are presented

enhancing prototype reduction schemes with lvq3 type algorithms abstract various prototype reduction schemes have been reported in the literature foremost among these are the prototypes for nearest neighbor pnn the vector quantization vq and the support vector machines svm methods in this paper we shall show that these schemes can be enhanced by the introduction of a post processing phase that is related but not identical to the lvq3 process although the post processing with lvq3 has been reported for the som and the basic vq methods in this paper we shall show that an analogous philosophy can be used in conjunction with the svm and pnn rules our essential modification to lvq3 first entails a partitioning of the respective training sets into two sets called the placement set and the optimizing set which are instrumental in determining the lvq3 parameters such a partitioning is novel to the literature our experimental results demonstrate that the proposed enhancement yields the best reported prototype condensation scheme to date for both artificial data sets and for samples involving real life data sets

asymptotic quantization error of continuous signals and the quantization dimension extensions of the limiting qnanfizafion error formula of bennet are proved these are of the form d s k n f n beta b where n is the number of output levels d s k n f is the s th moment of the metric distance between quantizer input and output beta b 0 k s beta is the signal space dimension and f is the signal distribution if a suitably well behaved k dimensional signal density f x exists b b s k int f rho x dx 1 rho rho k s k and b s k does not depend on f for k 1 s 2 this reduces to bennett s formula if f is the cantor distribution on 0 1 0 and this k equals the fractal dimension of the cantor set 12 13 random quantization optimal quantization in the presence of an output information constraint and quantization noise in high dimensional spaces are also investigated

on generalized ridge regression estimators under collinearity and balanced loss in regression analysis ridge estimators are often used to alleviate the problem of multicollinearity ridge estimators have traditionally been evaluated using the risk under quadratic loss criterion which places sole emphasis on estimators precision here we consider the balanced loss function a zellner in s s gupta j o berger eds statistical decision theory and related topics vol v springer new york 1994 p 377 which incorporates a measure for the goodness of fit of the model as well as estimation precision by adopting this loss we derive and numerically evaluate the risks of the feasible generalized ridge and the almost unbiased feasible generalized ridge estimators we show that in the case of severe multicollinearity the feasible generalized ridge estimator often produces the greatest risk reductions even if a relatively heavy weight is given to goodness of fit in the balanced loss function

bayesian estimation and prediction using asymmetric loss functions abstract estimators and predictors that are optimal relative to varian s asymmetric linex loss function are derived for a number of well known models their risk functions and bayes risks are derived and compared with those of usual estimators and predictors it is shown that some usual estimators for example a scalar sample mean or a scalar least squares regression coefficient estimator are inadmissible relative to asymmetric linex loss by providing alternative estimators that dominate them uniformly in terms of risk

a bootstrap technique for nearest neighbor classifier design a bootstrap technique for nearest neighbor classifier design is proposed our primary interest in designing a classifier is in small training sample size situations conventional bootstrapping techniques sample the training samples with replacement on the other hand our technique generates bootstrap samples by locally combining original training samples the nearest neighbor classifier is designed on the bootstrap samples and is tested on the test samples independent of training samples the performance of the proposed classifier is demonstrated on three artificial data sets and one real data set experimental results show that the nearest neighbor classifier designed on the bootstrap samples outperforms the conventional k nn classifiers as well as the edited 1 nn classifiers particularly in high dimensions

kolmogorov s contributions to mathematical statistics in this paper kolmogorov s work in mathematical statistics is reviewed the main areas under discussion are the idea of sufficiency linear models and unbiased estimators the relationship of kolmogorov s contributions with modern statistical theory in particular with bayesian analysis is analyzed

density waves in a traffic flow model with reaction time delay abstract density waves are investigated analytically and numerically in the optimal velocity model with reaction time delay of drivers the stability condition of this model is obtained by using the linear stability theory the results show that the decrease of reaction time delay of drivers leads to the stabilization of traffic flow the burgers korteweg de vries kdv and modified korteweg de vries mkdv equations are derived to describe the density waves in the stable metastable and unstable regions respectively the triangular shock waves soliton waves and kink antikink waves appearing respectively in the three distinct regions are derived to describe the traffic jams the numerical simulations are given

on utilizing search methods to select subspace dimensions for kernel based nonlinear subspace classifiers in kernel based nonlinear subspace kns methods the subspace dimensions have a strong influence on the performance of the subspace classifier in order to get a high classification accuracy a large dimension is generally required however if the chosen subspace dimension is too large it leads to a low performance due to the overlapping of the resultant subspaces and if it is too small it increases the classification error due to the poor resulting approximation the most common approach is of an ad hoc nature which selects the dimensions based on the so called cumulative proportion computed from the kernel matrix for each class we propose a new method of systematically and efficiently selecting optimal or near optimal subspace dimensions for kns classifiers using a search strategy and a heuristic function termed the overlapping criterion the rationale for this function has been motivated in the body of the paper the task of selecting optimal subspace dimensions is reduced to find the best ones from a given problem domain solution space using this criterion as a heuristic function thus the search space can be pruned to very efficiently find the best solution our experimental results demonstrate that the proposed mechanism selects the dimensions efficiently without sacrificing the classification accuracy

input space versus feature space in kernel based methods this paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector sv kernel functions we first discuss the geometry of feature space in particular we review what is known about the shape of the image of input space under the feature space map and how this influences the capacity of sv methods following this we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel using the example of the class of inhomogeneous polynomial kernels which are often used in sv pattern recognition we then discuss the connection between feature space and input space by dealing with the question of how one can given some vector in feature space find a preimage exact or approximate in input space we describe algorithms to tackle this issue and show their utility in two applications of kernel methods first we use it to reduce the computational complexity of sv decision functions second we combine it with the kernel pca algorithm thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real world data

a fast nearest neighbor search algorithm by filtration abstract classifying an unknown input is a fundamental problem in pattern recognition one standard method is finding its nearest neighbors in a reference set it would be very time consuming if computed feature by feature for all templates in the reference set this naive method is o nd where n is the number of templates in the reference set and d is the number of features or dimension for this reason we present a technique for quickly eliminating most templates from consideration as possible neighbors the remaining candidate templates are then evaluated feature by feature against the query vector we utilize frequencies of features as a pre processing to reduce query processing time burden our approach is simple to implement and achieves great speedup experimentally the most notable advantage of the new method over other existing techniques occurs where the number of features is large and the type of each feature is binary although it works for other type features we improved our ocr system at least twice without a threshold or faster with higher threshold value by using the new algorithm

loss functions for loss estimation on introduit une classe de fonctions de scores propres qui combinent l erreur dans un probleme de decision et la precision de la regle de decision

soft nearest prototype classification we propose a new method for the construction of nearest prototype classifiers which is based on a gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization lvq the algorithm performs a gradient descent on a cost function minimizing the classification error on the training set we investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task results show 1 that annealing in the dispersion parameter of the gaussian kernels improves classification accuracy 2 that classification results are better than those obtained with standard learning vector quantization lvq 2 1 lvq 3 for equal numbers of prototypes and 3 that annealing of the width parameter improved the classification capability additionally the principled approach provides an explanation of a number of features of the heuristic lvq methods

inadmissibility of the stein rule estimator under the balanced loss function in this paper the risk of the stein rule sr estimator is compared with that of the improved stein rule estimator using the stein variance srsv estimator under the balanced loss function it is shown that the srsv estimator dominates the sr estimator even if the loss function is extended to the blf also our numerical results show that the srsv estimator dominates the ols estimator when the weight of precision of estimation is larger than about half and vice versa