cubic hermite interpolation with minimal derivative oscillation in this paper a new optimal cubic hermite interpolation method is presented the method is to optimize the derivative of the interpolant the diagonally dominant property of the obtained system of normal equations and the error bound are better than some of the existing cubic interpolants for parametric curve design the vector valued interpolation method is given some numerical examples are provided to illustrate the satisfactory shape of the interpolation curves

svm decision boundary based discriminative subspace induction we study the problem of linear dimension reduction for classification with a focus on sufficient dimension reduction i e finding subspaces without loss of discrimination power first we formulate the concept of sufficient subspace for classification in parallel terms as for regression then we present a new method to estimate the smallest sufficient subspace based on an improvement of decision boundary analysis dba the main idea is to combine dba with support vector machines svm to overcome the inherent difficulty of dba in small sample size situations while keeping dba s estimation simplicity the compact representation of svm boundary results in a significant gain in both speed and accuracy over previous dba implementations alternatively this technique can be viewed as a way to reduce the run time complexity of svm itself comparative experiments on one simulated and four real world benchmark datasets highlight the superior performance of the proposed approach

svm based feature extraction for face recognition the primary goal of linear discriminant analysis lda in face feature extraction is to find an effective subspace for identity discrimination the introduction of kernel trick has extended the lda to nonlinear decision hypersurface however there remained inherent limitations for the nonlinear lda to deal with physical applications under complex environmental factors these limitations include the use of a common covariance function among each class and the limited dimensionality inherent to the definition of the between class scatter since these problems are inherently caused by the definition of the fisher s criterion itself they may not be solvable under the conventional lda framework this paper proposes to adopt a margin based between class scatter and a regularization process to resolve the issue essentially we redesign the between class scatter matrix based on the svm margins to facilitate an effective and reliable feature extraction this is followed by a regularization of the within class scatter matrix extensive empirical experiments are performed to compare the proposed method with several other variants of the lda method using the feret ar and cmu pie databases

parametric representation of fuzzy numbers and application to fuzzy calculus we present several models to obtain simple parametric representations of the fuzzy numbers or intervals based on the use of piecewise monotonic functions of different forms the representations have the advantage of allowing flexible and easy to control shapes of the fuzzy numbers we use the standard cuts setting but also the membership functions are obtained immediately and can be used directly to obtain error controlled approximations of the fuzzy calculus in terms of a finite set of parameters the general setting is the hermite type interpolation where the values and the slopes of the monotonic interpolators are given by appropriate parameters and the overall errors of the fuzzy computations can be controlled within a prefixed tolerance by eventually augmenting the total number of pieces and of the parameters by which the results are obtained the representations are designed to model the lower and the upper extremal values of each cut compact intervals of the fuzzy numbers and are able to produce almost any possible configuration differentiable continuous or with a finite number of discontinuity points by using parametric monotonic functions of different types we show applications in the standard fuzzy calculus and we stress the generality and the applicability of the proposed representation to a large class of problems including the numerical solution of fuzzy differential equations the fuzzy linear regression and the stochastic extensions of the fuzzy mathematics the proposed model is called the lower upper representation and we denote the associated fuzzy numbers or intervals as lu fuzzy

feature extraction based on the bhattacharyya distance in this paper we present a feature extraction method by utilizing an error estimation equation based on the bhattacharyya distance we propose to use classification errors in the transformed feature space which are estimated using the error estimation equation as a criterion for feature extraction the construction of linear transformation for feature extraction is conducted using an iterative gradient descent algorithm so that the estimated classification error is minimized due to the ability to predict error it is possible to determine the minimum number of features required for classification experimental results show that the proposed feature extraction method compares favorably with conventional methods

adaptive linear dimensionality reduction for classification abstract dimensionality reduction is the representation of high dimensional patterns in a low dimensional subspace based on a transformation which optimizes a specified criterion in the subspace for pattern classification the ideal criteria is the minimum achievable classification error the bayes error under strict assumptions of the pattern distribution the bayes error can be analytically expressed we use this as a starting point to develop an adaptive algorithm that computes a linear transformation based on the minimization of a cost function that approximates the bayes error in the subspace using kernel estimators we then relax the assumptions and extend the algorithm to more general pattern distributions our simulations with three synthetic and one real data set indicate that the proposed algorithm substantially outperforms fisher s linear discriminant

on extensions to fisher s linear discriminant function this correspondence describes extensions to fisher s linear discriminant function which allow both differences in class means and covariances to be systematically included in a process for feature reduction it is shown how the fukunaga koontz transform can be combined with fisher s method to allow a reduction of feature space from many dimensions to two performance is seen to be superior in general to the foley sammon method the technique is developed to show how a new radius vector or pair of radius vectors can be combined with fisher s vector to produce a classifier with even more power of discrimination illustrations of the technique show that good discrimination can be obtained even if there is considerable overlap of classes in any one projection

a novel svm nda model for classification with an application to face recognition support vector machine svm is a powerful classification methodology where the support vectors fully describe the decision surface by incorporating local information on the other hand nonparametric discriminant analysis nda is an improvement over lda where the normality assumption is relaxed nda also detects the dominant normal directions to the decision plane this paper introduces a novel svm nda model which can be viewed as an extension to the svm by incorporating some partially global information especially discriminatory information in the normal direction to the decision boundary this can also be considered as an extension to the nda where the support vectors improve the choice of k nearest neighbors on the decision boundary by incorporating local information being an extension to both svm and nda it can deal with heteroscedastic and non normal data it also avoids the small sample size problem moreover it can be reduced to the classical svm model so that existing softwares can be used a kernel extension of the model called ksvm knda is also proposed to deal with nonlinear problems we have carried an extensive comparison of the svm nda to the lda svm heteroscedastic lda hlda nda and the combined svm and lda on artificial real and face recognition data sets results for ksvm knda have also been presented these comparisons demonstrate the advantages and superiority of our proposed model

shape preserving piecewise rational interpolation with higher order continuity abstract a united form of the classical hermite interpolation and shape preserving interpolation is presented in this paper the presented interpolation method provides higher order continuous shape preserving interpolation splines the given interpolants are explicit piecewise rational expressions without solving a linear or nonlinear system of consistency equations by setting parameter values the interpolation curve can be generated by choosing the classical piecewise hermite interpolation polynomials or the presented piecewise rational expressions for monotonicity preserving and convexity preserving interpolation the appropriate values of a parameter are given on each subinterval numerical examples indicate that the given method produces visually pleasing curves

convexity preserving piecewise rational quartic interpolation a family of piecewise rational quartic interpolants is given identified uniquely by the value of a tension parameter lambda i each interpolant of the family can be c 2 spline without solving a linear or nonlinear system of consistency equations for the derivative values at the knots the interpolant can preserve the local convexity concavity properties of the given data a proper choice of lambda i to guarantee shape preservation is given a convergence analysis establishes an error bound in terms of lambda i and shows that o h 3 accuracy is obtained for c 2 continuity several examples are supplied to support the practical value of the method

convexity preserving approximation by univariate cubic splines a convexity preserving approximation method is presented similar to the cubic spline interpolation the given approximation function is univariate cubic spline with c 2 continuity a very simple computing method is described the approximation order and polynomial reproducing property of this convexity preserving approximant are discussed the computing method for interpolating a small quantity of data is given some numerical examples are given to show the effectivity

linear classifier design in the weight space abstract in this paper we propose a linear classifier design method in the weight space a linear classifier is completely determined by a weight vector to design a linear classifier is equivalent to finding a weight vector when there are a number of training samples each training sample represents a plane in the weight space on one side of the plane the training sample is correctly classified while it is incorrectly classified on the other side thus finding the optimal linear classifier can be formulated as finding a subspace that provides the best classification accuracy in this paper we have developed an algorithm to find such optimal subspaces in the weight space although we could not find the optimal linear classifier due to prohibitive computational complexity the proposed design method produced noticeable improvements in some cases experimental results show that the proposed linear classifier performed better than or equivalent to existing linear classifiers

metastability and low lying spectra in reversible markov chains we study a large class of reversible markov chains with discrete state space and transition matrix pn we define the notion of a set of metastable points as a subset of the state space n such that i this set is reached from any point x n without return to x with probability at least bn while ii for any two points x y in the metastable set the probability t 1x y to reach y from x without return to x is smaller than an 1 bn under some additional non degeneracy assumption we show that in such a situation

shape preserving approximations by polynomials and splines a review of shape preserving approximation methods and algorithms for approxi mating univariate functions or discrete data is given the notion of shape refers to the geometrical behavior of a function s or approximant s graph and usually includes positivity monotonicity and or convexity but in the recent literature the broader concept of shape also includes symmetry gen eralized convexity unimodality lipschitz property possessing peaks or discontinuities etc special stress is put on shape preserving interpolation methods by polynomials and splines of course this text has no pretensions to be complete geywords shape preserving approximation approximation of univariate functions approxima tion of discrete data shape preserving interpolation polynomial spline positivity monotonicity convexity generalized convexity unimodality lipschitz property

two realizations of a general feature extraction framework abstract a general feature extraction framework is proposed as an extension of conventional linear discriminant analysis two nonlinear feature extraction algorithms based on this framework are investigated the first is a kernel function feature extraction kffe algorithm a disturbance term is introduced to regularize the algorithm moreover it is revealed that some existing nonlinear feature extraction algorithms are the special cases of this kffe algorithm the second feature extraction algorithm mean std 1 norm feature extraction algorithm is also derived from the framework experiments based on both synthetic and real data are presented to demonstrate the performance of both feature extraction algorithms

shape preserving piecewise rational interpolant with quartic numerator and quadratic denominator an explicit representation of a piecewise rational interpolant with quartic numerator and quadratic denominator is presented for positivity data monotone data and convex data the shape preserving properties of the interpolant are given the interpolant is c 2 continuous spline with a shape parameter w i on each subinterval the values of w i to guarantee shape preservation are estimated a convergence analysis establishes an error bound in terms of w i and shows that the interpolant is o h 2 or o h 3 accurate several examples are supplied to support the practical value of the given interpolation method

a linear feature extraction for multiclass classification problems based on class mean and covariance discriminant information a parametric linear feature extraction method is proposed for multiclass classification the skeleton of the proposed method consists of two types of schemes that are complementary to each other with regard to the discriminant information used the approximate pairwise accuracy criterion apac and the common mean feature extraction cmfe are chosen to exploit the discriminant information about class mean and about class covariance respectively choosing apac rather than the linear discriminant analysis lda can also resolve the problem of overemphasized large distances introduced by lda while maintaining other decent properties of lda to alleviate the suboptimum problem caused by a direct cascading of the two different types of schemes there should be a mechanism for sorting and merging features based on their effectiveness usage of a sample based classification error estimation for evaluation of effectiveness of features usually costs a lot of computational time therefore we develop a fast spanning tree based parametric classification accuracy estimator as an intermediary for the apac and cmfe combination the entire framework is parametric based this avoids paying a costly price in computation which normally happens to the sample based approach our experiments have shown that the proposed method can achieve a satisfactory performance on real data as well as simulated data

a class discriminality measure based on feature space partitioning this paper presents a new class discriminability measure based on an adaptive partitioning of the feature space according to the available class samples it is intended to be used as a criterion in a classifier independent feature selection procedure the partitioning is performed according to a binary splitting rule and appropriate stopping criteria results from several tests withc gaussian and non gaussian multidimensional and multicalss computer generated samples were very similar to those obtained using a bayes error criterion function i e the optimal feature subsets selected by both criterion functions were the same the main advantage of the new measure is that it is computationally efficient

accurate c 2 rational interpolants in tension a family of rational interpolants is defined for a set of real data points x i f i 1 leqq i leqq n with x 1 x 2 cdots x n each interpolant of the family is a c 2 spline with a cubic numerator and a quadratic denominator which is identified uniquely by the value of a positive tension parameter c 0 the solution constructed is nonparametric a value of c 0 close to zero will produce piecewise linear behaviour while a large value of c 0 will give approximately the cubic spline solution the method used arises from a study of other rational functional forms for effecting monotonic or convex interpolation a convergence analysis establishes an error bound in terms of c 0 and shows that o h 4 accuracy is obtained when exact boundary derivatives are prescribed several examples are supplied to support the practical value of the method

robust relief feature weighting margin maximization and fuzzy optimization a latest advance in relief feature weighting techniques is that the iterative procedure of relief can be approximately expressed as a margin maximization problem and therefore its distinctive properties can be investigated with the help of optimization theory being motivated by this advance the relief feature weighting algorithm is investigated for the first time within a fuzzy optimization framework a new margin based objective function that incorporates three fuzzy concepts namely fuzzy difference measure fuzzy feature weighting and fuzzy instance force coefficient is introduced by the application of fuzzy optimization to this new margin based objective function several useful theoretical results are derived based upon which a set of robust relief feature weighting algorithms are proposed for two class data multiclass data and then online data as demonstrated by extensive experiments in synthetic datasets the university of california at irvine uci benchmark datasets cancer gene expression datasets and face image datasets the proposed algorithms were found to be competitive with the state of the art algorithms and robust for datasets with noise and or outliers

hyperbolic splines and nonlinear distortion using mobius transformations we consider how to interpolate a set of monotone increasing data points x sub k y sub k so that the interpolant is a smooth invertible function and that the interpolant and its inverse have the same simple functional form we use our method to cancel jammer induced distortion in a nonlinear channel

toward a fundamental theory of optimal feature selection part i several authors have studied the problem of dimensionality reduction or feature selection using statistical distance measures e g the chernoff coefficient bhattacharyya distance i divergence and j divergence because they generally felt that direct use of the probability of classification error expression was either computationally or mathematically intractable we show that for the difficult problem of testing one weakly stationary gaussian stochastic process against another when the mean vectors are similar and the covariance matrices patterns differ the probability of error expression may be dealt with directly using a combination of classical methods and distribution function theory the results offer a new and accurate finite dimensionality information theoretic strategy to feature selection and are shown by use of examples to be superior to the well known kadota shepp approach which employs distance measures and asymptotics in its formulation the present part i deals with the theory part ii deals with the implementation of a computer based real time pattern classifier which takes into account a realistic quasi stationarity of the patterns

feature extraction based on decision boundaries a novel approach to feature extraction for classification based directly on the decision boundaries is proposed it is shown how discriminantly redundant features and discriminantly informative features are related to decision boundaries a procedure to extract discriminantly informative features based on a decision boundary is proposed the proposed feature extraction algorithm has several desirable properties 1 it predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem and 2 it finds the necessary feature vectors the proposed algorithm does not deteriorate under the circumstances of equal class means or equal class covariances as some previous algorithms do experiments show that the performance of the proposed algorithm compares favorably with those of previous algorithms

on an extended fisher criterion for feature selection this correspondence considers the extraction of features as a task of linear transformation of an initial pattern space into a new space optimal with respect to discriminating the data a solution of the feature extraction problem is given for two multivariate normal distributed pattern classes using an extended fisher criterion as the distance measure the introduced distance measure consists of two terms the first term estimates the distance between classes upon the difference of mean vectors of classes and the second one upon the difference of class covariance matrices the proposed method is compared to some of the more popular alternative methods fukunaga koontz method and foley sammon method

evaluation of weighted fisher criteria for large category dimensionality reduction in application to chinese handwriting recognition to improve the class separability of fisher linear discriminant analysis fda for large category problems we investigate the weighted fisher criterion wfc by integrating weighting functions for dimensionality reduction the objective of wfc is to maximize the sum of weighted distances of all class pairs by setting larger weights for the most confusable classes wfc can improve the class separation while the solution remains an eigen decomposition problem we evaluate five weighting functions in three different weighting spaces in a typical large category problem of handwritten chinese character recognition the weighting functions include four based on existing methods namely fda approximate pairwise accuracy criterion apac power function pow confused distance maximization cdm and a new one based on k nearest neighbors knn all the weighting functions can be calculated in the original feature space low dimensional space or fractional space our experiments on a 3 755 class chinese handwriting database demonstrate that wfc can improve the classification accuracy significantly compared to fda among the weighting functions the knn method in the original space is the most competitive model which achieves significantly higher classification accuracy and has a low computational complexity to further improve the performance we propose a nonparametric extension of the knn method from the class level to the sample level the sample level knn sknn method is shown to outperform significantly other methods in chinese handwriting recognition such as the locally linear discriminant analysis llda neighbor class linear discriminant analysis nclda and heteroscedastic linear discriminant analysis hlda