image thresholding based on the em algorithm and the generalized gaussian distribution in this paper a novel parametric and global image histogram thresholding method is presented it is based on the estimation of the statistical parameters of object and background classes by the expectation maximization em algorithm under the assumption that these two classes follow a generalized gaussian gg distribution the adoption of such a statistical model as an alternative to the more common gaussian model is motivated by its attractive capability to approximate a broad variety of statistical behaviors with a small number of parameters since the quality of the solution provided by the iterative em algorithm is strongly affected by initial conditions which if inappropriately set may lead to unreliable estimation a robust initialization strategy based on genetic algorithms gas is proposed experimental results obtained on simulated and real images confirm the effectiveness of the proposed method

discriminant analysis by gaussian mixtures fisher rao linear discriminant analysis lda is a valuable tool for multigroup classification lda is equivalent to maximum likelihood classification assuming gaussian distributions for each class in this paper we fit gaussian mixtures to each class to facilitate effective classification in non normal settings especially when the classes are clustered low dimensional views are an important by product of lda our new techniques inherit this feature we can control the within class spread of the subclass centres relative to the between class spread our technique for fitting these models permits a natural blend with nonparametric versions of lda

learning gaussian mixture models with entropy based criteria in this paper we address the problem of estimating the parameters of gaussian mixture models although the expectation maximization em algorithm yields the maximum likelihood ml solution its sensitivity to the selection of the starting parameters is well known and it may converge to the boundary of the parameter space furthermore the resulting mixture depends on the number of selected components but the optimal number of kernels may be unknown beforehand we introduce the use of the entropy of the probability density function pdf associated to each kernel to measure the quality of a given mixture model with a fixed number of kernels we propose two methods to approximate the entropy of each kernel and a modification of the classical em algorithm in order to find the optimum number of components of the mixture moreover we use two stopping criteria a novel global mixture entropy based criterion called gaussianity deficiency gd and a minimum description length mdl principle based one our algorithm called entropy based em ebem starts with a unique kernel and performs only splitting by selecting the worst kernel attending to gd we have successfully tested it in probability density estimation pattern classification and color image segmentation experimental results improve the ones of other state of the art model order selection methods

flexible discriminant analysis by optimal scoring abstract fisher s linear discriminant analysis is a valuable tool for multigroup classification with a large number of predictors one can find a reduced number of discriminant coordinate functions that are optimal for separating the groups with two such functions one can produce a classification map that partitions the reduced space into regions that are identified with group membership and the decision boundaries are linear this article is about richer nonlinear classification schemes linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups in this paper we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method in this way any multiresponse regression technique such as mars or neural networks can be postprocessed to improve its classification performance

svm decision boundary based discriminative subspace induction we study the problem of linear dimension reduction for classification with a focus on sufficient dimension reduction i e finding subspaces without loss of discrimination power first we formulate the concept of sufficient subspace for classification in parallel terms as for regression then we present a new method to estimate the smallest sufficient subspace based on an improvement of decision boundary analysis dba the main idea is to combine dba with support vector machines svm to overcome the inherent difficulty of dba in small sample size situations while keeping dba s estimation simplicity the compact representation of svm boundary results in a significant gain in both speed and accuracy over previous dba implementations alternatively this technique can be viewed as a way to reduce the run time complexity of svm itself comparative experiments on one simulated and four real world benchmark datasets highlight the superior performance of the proposed approach

class conditional density estimation using mixtures with constrained component sharing we propose a generative mixture model classifier that allows for the class conditional densities to be represented by mixtures having certain subsets of their components shared or common among classes we argue that when the total number of mixture components is kept fixed the most efficient classification model is obtained by appropriately determining the sharing of components among class conditional densities in order to discover such an efficient model a training method is derived based on the em algorithm that automatically adjusts component sharing we provide experimental results with good classification performance

stable and efficient multiple smoothing parameter estimation for generalized additive models representation of generalized additive models gam s using penalized regression splines allows gam s to be employed in a straightforward manner using penalized regression methods not only is inference facilitated by this approach but it is also possible to integrate model selection in the form of smoothing parameter selection into model fitting in a computationally efficient manner using well founded criteria such as generalized cross validation the current fitting and smoothing parameter selection methods for such models are usually effective but do not provide the level of numerical stability to which users of linear regression packages for example are accustomed in particular the existing methods cannot deal adequately with numerical rank deficiency of the gam fitting problem and it is not straightforward to produce methods that can do so given that the degree of rank deficiency can be smoothing parameter dependent in addition models with the potential flexibility of gam s can also present

prepotential formulation of su 3 lattice gauge theory the su 3 lattice gauge theory is reformulated in terms of su 3 prepotential harmonic oscillators this reformulation has enlarged su 3 u 1 u 1 gauge invariance under which the prepotential operators transform like matter fields the hilbert space of su 3 lattice gauge theory is shown to be equivalent to the hilbert space of the prepotential formulation satisfying certain color invariant sp 2 r constraints the su 3 irreducible prepotential operators which solve these sp 2 r constraints are used to construct su 3 gauge invariant hilbert spaces at every lattice site in terms of su 3 gauge invariant vertex operators the electric fields and the link operators are reconstructed in terms of these su 3 irreducible prepotential operators we show that all the su 3 mandelstam constraints become local and take a very simple form within this approach we also discuss the construction of all possible linearly independent su 3 loop states which solve the mandelstam constraints the techniques can be easily generalized to su n

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

the distribution function of the effective field of the ising spin glass on the bethe lattice for the coordination number z 4 5 6 abstract the integral equation for the effective fields of the spin glass is solved analytically for the j ising model on the bether lattice with coordination number z 4 5 and 6 and symmetric distribution functions are obtained the paramagnetic distribution spin glass distribution with three delta functions and spin glass distributions with a continuous distribution are obtained the effect of the truncation of the algebraic equation for solving the integral equation is discussed

estimating central subspaces via inverse third moments summary modern graphical tools have enhanced our ability to learn many things from data directly in recent years dimension reduction has proven to be an effective tool for generating low dimensional summary plots without appreciable loss of information some well known inverse regression methods for dimension reduction such as sliced inverse regression li 1991 and sliced average variance estimation cook weisberg 1991 have been developed to estimate summary plots for regression and discriminant analysis in this paper we suggest a new method that makes use of inverse third moments this method can find structure beyond that found by sliced inverse regression and sliced average variance estimation particularly regression mixtures illustrative examples are presented

structural minimax probability machine minimax probability machine mpm is an interesting discriminative classifier based on generative prior knowledge it can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification the structural information of data is an effective way to represent prior knowledge and has been found to be vital for designing classifiers in real world problems however mpm only considers the prior probability distribution of each class with a given mean and covariance matrix which does not efficiently exploit the structural information of data in this paper we use two finite mixture models to capture the structural information of the data from binary classification for each subdistribution in a finite mixture model only its mean and covariance matrix are assumed to be known based on the finite mixture models we propose a structural mpm smpm smpm can be solved effectively by a sequence of the second order cone programming problems moreover we extend a linear model of smpm to a nonlinear model by exploiting kernelization techniques we also show that the smpm can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi min margin machine under certain special conditions experimental results on both synthetic and real world data sets demonstrate the effectiveness of smpm

model enforcement a unified feature transformation framework for classification and recognition bayesian classifiers rely on models of the a priori and class conditional feature distributions the classifier is trained by optimizing these models to best represent features observed in a training corpus according to certain criterion in many problems of interest the true class conditional feature probability density function pdf is not a member of the set of pdfs the classifier can represent previous research has shown that the effect of this problem may be reduced either by improving the models or by transforming the features used in the classifier this paper addresses this model mismatch problem in statistical identification classification and recognition systems we formulate the problem as the problem of minimizing the relative entropy which is also known as the kullback leibler distance between the true conditional pdf and the hypothesized probabilistic model based on this formulation we provide a computationally efficient solution to the problem based on volume preserving maps existing linear transform designs are shown to be special cases of the proposed solution using this result we propose the symplectic maximum likelihood transform smlt which is a nonlinear volume preserving extension of the maximum likelihood linear transform mllt this approach has many applications in statistical modeling classification and recognition we apply it to the maximum likelihood estimation mle of the joint pdf of order statistics and show a significant increase in the likelihood for the same number of parameters we provide also phoneme recognition experiments that show recognition accuracy improvement compared with using the baseline mel frequency cepstrum coefficient mfcc features or using mllt we present an iterative algorithm to jointly estimate the parameters of the symplectic map and the probabilistic model for both applications

regionalization of multiscale spatial processes by using a criterion for spatial aggregation error summary the modifiable areal unit problem and the ecological fallacy are known problems that occur when modelling multiscale spatial processes we investigate how these forms of spatial aggregation error can guide a regionalization over a spatial domain of interest by regionalization we mean a specification of geographies that define the spatial support for areal data this topic has been studied vigorously by geographers but has been given less attention by spatial statisticians thus we propose a criterion for spatial aggregation error which we minimize to obtain an optimal regionalization to define the criterion we draw a connection between spatial aggregation error and a new multiscale representation of the karhunen loeve expansion this relationship between the criterion for spatial aggregation error and the multiscale karhunen loeve expansion leads to illuminating theoretical developments including connections between spatial aggregation error squared prediction error spatial variance and a novel extension of obled creutin eigenfunctions the effectiveness of our approach is demonstrated through an analysis of two data sets one using the american community survey and one related to environmental ocean winds

triplet markov fields for the classification of complex structure data we address the issue of classifying complex data we focus on three main sources of complexity namely the high dimensionality of the observed data the dependencies between these observations and the general nature of the noise model underlying their distribution we investigate the recent triplet markov fields and propose new models in this class designed for such data and in particular allowing very general noise models in addition our models can handle the inclusion of a learning step in a consistent way so that they can be used in a supervised framework one advantage of our models is that whatever the initial complexity of the noise model parameter estimation can be carried out using state of the art bayesian clustering techniques under the usual simplifying assumptions as generative models they can be seen as an alternative in the supervised case to discriminative conditional random fields identifiability issues underlying the models in the nonsupervised case are discussed while the models performance is illustrated on simulated and real data exhibiting the mentioned various sources of complexity

averaging maximum penalized likelihood and bayesian estimation for improving gaussian mixture probability density estimates we apply the idea of averaging ensembles of estimators to probability density estimation in particular we use gaussian mixture models which are important components in many neural network applications we investigate the performance of averaging using three data sets for comparison we employ two traditional regularization approaches i e a maximum penalized likelihood approach and a bayesian approach in the maximum penalized likelihood approach we use penalty functions derived from conjugate bayesian priors such that an expectation maximization em algorithm can be used for training in all experiments the maximum penalized likelihood approach and averaging improved performance considerably if compared to a maximum likelihood approach in two of the experiments the maximum penalized likelihood approach outperformed averaging in one experiment averaging was clearly superior our conclusion is that maximum penalized likelihood gives good results if the penalty term in the cost function is appropriate for the particular problem if this is not the case averaging is superior since it shows greater robustness by not relying on any particular prior assumption the bayesian approach worked very well on a low dimensional toy problem but failed to give good performance in higher dimensional problems

multiscale multigranular statistical image segmentation we consider a problem in image segmentation in which the goal is to determine and label a relatively small number of homogeneous subregions in an image scene based on multivariate pixelwise measurements motivated by current challenges in the field of remote sensing land cover characterization we introduce a framework that allows for adaptive choice of both the spatial resolution of subregions and the categorical granularity of labels our framework is based on a class of models that we call mixlets a blending of recursive dyadic partitions and finite mixture models the first component of these models allows for the sparse representation of a spatial structure at multiple resolutions the second component provides a natural mechanism for capturing the varying degrees of mixing of pure categories that accompany the use of different resolutions and for relating these to a user specified hierarchy of labels at multiple granularities in a straightforward manner a segmentation is produced in our framewor

a region based segmentation method for multichannel image data abstract segmentation has become a widely used tool in modern image analysis typical methods are highly nonlinear and often ad hoc and none so far have proved accessible to detailed statistical analysis even by asymptotic approximations can such methods be any good this article describes a segmentation procedure for multichannel image data and attempts to develop an understanding of its statistical performance characteristics in terms of nonparametric regression the procedure is based on a recursive merging algorithm defined via a nested sequence of discretizations of the image domain a cross validation rule with a near neighbor block replacement strategy is proposed for selecting the final segmentation model idealized numerical simulation experiments are used to evaluate the rate of convergence of the mean square error of estimation and also to study the efficiency of the cross validation technique interestingly the results show that the rate of convergence tends to decrease as the degree of smo

classification trees with unbiased multiway splits two univariate split methods and one linear combination split method are proposed for the construction of classification trees with multiway splits examples are given where the trees are more compact and hence easier to interpret than binary trees a major strength of the univariate split methods is that they have negligible bias in variable selection both when the variables differ in the number of splits they offer and when they differ in the number of missing values this is an advantage because inferences from the tree structures can be adversely affected by selection bias the new methods are shown to be highly competitive in terms of computational speed and classification accuracy of future observations

wedgelets nearly minimax estimation of edges we study a simple horizon model for the problem of recovering an image from noisy data in this model the image has an edge with fi holder regularity adopting the viewpoint of computational harmonic analysis we develop an overcomplete collection of atoms called wedgelets dyadically organized indicator functions with a variety of locations scales and orientations the wedgelet representation provides nearly optimal representations of objects in the horizon model as measured by minimax description length we show how to rapidly compute a wedgelet approximation to noisy data by flnding a special edgelet decorated recursive partition which minimizes a complexity penalized sum of squares this estimate using su cient sub pixel resolution achieves nearly the minimax mean squared error in the horizon model in fact the method is adaptive in the sense that it achieves nearly the minimax risk for any value of the unknown degree of regularity of the horizon 1 fi 2 wedgelet analysis and de noising may be used successfully outside the horizon model we study images modelled as indicators of star shaped sets with smooth bound aries and show that complexity penalized wedgelet partitioning achieves nearly the minimax risk in that setting also

unsupervised learning of finite mixture models this paper proposes an unsupervised algorithm for learning a finite mixture model from multivariate data the adjective unsupervised is justified by two properties of the algorithm 1 it is capable of selecting the number of components and 2 unlike the standard expectation maximization em algorithm it does not require careful initialization the proposed method also avoids another drawback of em for mixture fitting the possibility of convergence toward a singular estimate at the boundary of the parameter space the novelty of our approach is that we do not use a model selection criterion to choose one among a set of preestimated candidate models instead we seamlessly integrate estimation and model selection in a single algorithm our technique can be applied to any type of parametric mixture model for which it is possible to write an em algorithm in this paper we illustrate it with experiments involving gaussian mixtures these experiments testify for the good performance of our approach

graph partitioning and dilute spin glasses the minimum cost solution the authors calculate the ground state cost function energy for graph bipartitioning and spin glasses on networks with fixed finite valences allowing a continuous distribution for the effective field compared with previous calculations using a field distribution in integral multiples of the coupling strength j the ground state energy is higher by less than one per cent although the distribution itself is altered drastically

discriminative components of data a simple probabilistic model is introduced to generalize classical linear discriminant analysis lda in finding components that are informative of or relevant for data classes the components maximize the predictability of the class distribution which is asymptotically equivalent to 1 maximizing mutual information with the classes and 2 finding principal components in the so called learning or fisher metrics the fisher metric measures only distances that are relevant to the classes that is distances that cause changes in the class distribution the components have applications in data exploration visualization and dimensionality reduction in empirical experiments the method outperformed in addition to more classical methods a renyi entropy based alternative while having essentially equivalent computational cost

harmonic oscillator pre potentials in su 2 lattice gauge theory we write the su 2 lattice gauge theory hamiltonian in d dimensions in terms of pre potentials which are su 2 fundamental doublets of harmonic oscillators the hamiltonian in terms of pre potentials has su 2 u 1 local gauge invariance these pre potentials enable us to solve the su 2 gauss law and characterize the su 2 gauge invariant hilbert space in terms of a set of integers we discuss the consequences of the additional u 1 gauge invariance the extension to su n lattice gauge theory is discussed

cart and best ortho basis a connection we study what we call dyadic cart a method of nonparametric regression which constructs a recursive partition by optimizing a complexity penalized sum of squares where the optimization is over all recursive partitions arising from midpoint splits we show that the method is adaptive to unknown degrees of anisotropic smoothness specifically consider the anisotropic smoothness classes of nikol skii consisting of bivariate functions f xl x2 whose finite difference of distance h in direction i is bounded in lp norm by ch8i i 1 2 we show that dyadic cart with an appropriate complexity penalty parameter a u2 const log n is within logarithmic terms of minimax over every anisotropic smoothness class 0 c x 0 81 82 1 the proof shows that dyadic cart is identical to a certain adaptive best ortho basis algorithm based on the library of all anisotropic haar bases then it applies empirical basis selection ideas of donoho and johnstone the basis empirically selected by dyadic cart is shown to be nearly as good as a basis ideally adapted to the underlying f the risk of estimation in an ideally adapted anisotropic haar basis is shown to be comparable to the minimax risk over anisotropic smoothness classes underlying the success of this argument is harmonic analysis of anisotropic smoothness classes we show that for each anisotropic smoothness class there is an anisotropic haar basis which is a best orthogonal basis for representing that smoothness class the basis is optimal not just within the library of anisotropic haar bases but among all orthogonal bases of l2 0 1 2

a new dirichlet process for mining dynamic patterns in functional data this paper proposes a novel model for mining patterns including dynamic function clustering segmentation and forecasting values of dependent variables simultaneously the proposed dependent dirichlet process piecewise regression mixture ddpprm model is capable of handling dynamic nature of data by detecting evolving clusters at each time step this evolution manifests dynamically in three states of clusters newly created existing and transient states of clusters the model is also able to generate clusters over time infinitely it is capable of learning the optimal number of clusters rather than using the fixed predefined clusters however other clustering methods such as fuzzy c regression model and piecewise regression mixture technique support none of those capabilities the proposed model is also capable of showing regime changes and segmenting functions in regression time series problems a two step gibbs sampling method is utilized for assigning data to clusters expectation maximization method is used for finding the optimal values of parameters of functions and probability distributions the model is validated by using some numerical experiments and calculating three validity indexes as well as mean square error of the model the results indicate that the proposed method outperforms other clustering segmentation and forecasting models in literature

mixtures of shifted asymmetriclaplace distributions a mixture of shifted asymmetric laplace distributions is introduced and used for clustering and classification a variant of the em algorithm is developed for parameter estimation by exploiting the relationship with the generalized inverse gaussian distribution this approach is mathematically elegant and relatively computationally straightforward our novel mixture modelling approach is demonstrated on both simulated and real data to illustrate clustering and classification applications in these analyses our mixture of shifted asymmetric laplace distributions performs favourably when compared to the popular gaussian approach this work which marks an important step in the non gaussian model based clustering and classification direction concludes with discussion as well as suggestions for future work

fluid dynamic models for bhattacharyya based discriminant analysis classical discriminant analysis attempts to discover a low dimensional subspace where class label information is maximally preserved under projection canonical methods for estimating the subspace optimize an information theoretic criterion that measures the separation between the class conditional distributions unfortunately direct optimization of the information theoretic criteria is generally non convex and intractable in high dimensional spaces in this work we propose a novel tractable algorithm for discriminant analysis that considers the class conditional densities as interacting fluids in the high dimensional embedding space we use the bhattacharyya criterion as a potential function that generates forces between the interacting fluids and derive a computationally tractable method for finding the low dimensional subspace that optimally constrains the resulting fluid flow we show that this model properly reduces to the optimal solution for homoscedastic data as well as for heteroscedastic gaussian distributions with equal means we also extend this model to discover optimal filters for discriminating gaussian processes and provide experimental results and comparisons on a number of datasets

randomized mixture models for probability density approximation and estimation abstract neural networks nns with random weights are an interesting alternative to conventional nns that are used more for data modeling the random vector functional link rvfl network is an established and theoretically well grounded randomized learner a key theoretical result for rvfl networks is that they provide universal approximation for continuous maps in expectation with respect to the square integral norm we specialize and modify this result and show that rfvl networks can provide functional approximations that converge in kullback leibler divergence when the target function is a probability density function expanding on the approximation results we demonstrate the rfvl networks lead to a simple randomized mixture model mm construction for density estimation from sample data an expectation maximization em algorithm is derived for the maximum likelihood estimation of our randomized mm the em algorithm is proved to be globally convergent and the maximum likelihood estimator is proved to be consistent a set of simulation studies is given to provide empirical evidence towards our approximation and density estimation results

multiclass linear discriminant analysis with ultrahigh dimensional features within the framework of fisher s discriminant analysis we propose a multiclass classification method which embeds variable screening for ultrahigh dimensional predictors leveraging interfeature correlations we show that the proposed linear classifier recovers informative features with probability tending to one and can asymptotically achieve a zero misclassification rate we evaluate the finite sample performance of the method via extensive simulations and use this method to classify posttransplantation rejection types based on patients gene expressions

discriminant analysis through a semiparametric model summary we consider a semiparametric generalisation of normal theory discriminant analysis the semiparametric model assumes that after unspecified univariate monotone transformations the class distributions are multivariate normal we introduce an estimation procedure based on the distribution quantiles in which the parameters of the semiparametric model are estimated directly without estimating the nonparametric transformations the procedure is computationally fast and the estimation accuracy is shown to have the usual parametric rate the relationship between the method and more general nonparametric discriminant analysis is discussed the semiparametric specification of the class densities is a submodel of the nonparametric log density functional analysis of variance model in which the main effects are completely nonparametric but the interaction terms are specified semiparametrically simulations and real examples are used to illustrate the procedure

a concave pairwise fusion approach to subgroup analysis abstractan important step in developing individualized treatment strategies is correct identification of subgroups of a heterogeneous population to allow specific treatment for each subgroup this article considers the problem using samples drawn from a population consisting of subgroups with different mean values along with certain covariates we propose a penalized approach for subgroup analysis based on a regression model in which heterogeneity is driven by unobserved latent factors and thus can be represented by using subject specific intercepts we apply concave penalty functions to pairwise differences of the intercepts this procedure automatically divides the observations into subgroups to implement the proposed approach we develop an alternating direction method of multipliers algorithm with concave penalties and demonstrate its convergence we also establish the theoretical properties of our proposed estimator and determine the order requirement of the minimal difference of signals between g

regularized parameter estimation in high dimensional gaussian mixture models finite gaussian mixture models are widely used in statistics thanks to their great flexibility however parameter estimation for gaussian mixture models with high dimensionality can be challenging because of the large number of parameters that need to be estimated in this letter we propose a penalized likelihood estimator to address this difficulty the type penalty we impose on the inverse covariance matrices encourages sparsity on its entries and therefore helps to reduce the effective dimensionality of the problem we show that the proposed estimate can be efficiently computed using an expectation maximization algorithm to illustrate the practical merits of the proposed method we consider its applications in model based clustering and mixture discriminant analysis numerical experiments with both simulated and real data show that the new method is a valuable tool for high dimensional data analysis

bankruptcy analysis with self organizing maps in learning metrics we introduce a method for deriving a metric locally based on the fisher information matrix into the data space a self organizing map som is computed in the new metric to explore financial statements of enterprises the metric measures local distances in terms of changes in the distribution of an auxiliary random variable that reflects what is important in the data in this paper the variable indicates bankruptcy within the next few years the conditional density of the auxiliary variable is first estimated and the change in the estimate resulting from local displacements in the primary data space is measured using the fisher information matrix when a self organizing map is computed in the new metric it still visualizes the data space in a topology preserving fashion but represents the local directions in which the probability of bankruptcy changes the most

bilinear analysis for kernel selection and nonlinear feature extraction this paper presents a unified criterion fisher kernel criterion fkc for feature extraction and recognition this new criterion is intended to extract the most discriminant features in different nonlinear spaces and then fuse these features under a unified measurement thus fkc can simultaneously achieve nonlinear discriminant analysis and kernel selection in addition we present an efficient algorithm fisher kernel analysis fka which utilizes the bilinear analysis to optimize the new criterion this fka algorithm can alleviate the ill posed problem existed in traditional kernel discriminant analysis kda and usually has no singularity problem the effectiveness of our proposed algorithm is validated by a series of face recognition experiments on several different databases

natural discriminant analysis using interactive potts models natural discriminant analysis based on interactive potts models is developed in this work a generative model composed of piece wise multivariate gaussian distributions is used to characterize the input space exploring the embedded clustering and mixing structures and developing proper internal representations of input parameters the maximization of a log likelihood function measuring the fitness of all input parameters to the generative model and the minimization of a design cost summing up square errors between posterior outputs and desired outputs constitutes a mathematical framework for discriminant analysis we apply a hybrid of the mean field annealing and the gradient descent methods to the optimization of this framework and obtain multiple sets of interactive dynamics which realize coupled potts models for discriminant analysis the new learning process is a whole process of component analysis clustering analysis and labeling analysis its major improvement compared to the radial basis function and the support vector machine is described by using some artificial examples and a real world application to breast cancer diagnosis

generalized t statistic for two group classification summary in the classic discriminant model of two multivariate normal distributions with equal variance matrices the linear discriminant function is optimal both in terms of the log likelihood ratio and in terms of maximizing the standardized difference the t statistic between the means of the two distributions in a typical case control study normality may be sensible for the control sample but heterogeneity and uncertainty in diagnosis may suggest that a more flexible model is needed for the cases we generalize the t statistic approach by finding the linear function which maximizes a standardized difference but with data from one of the groups the cases filtered by a possibly nonlinear function u we study conditions for consistency of the method and find the function u which is optimal in the sense of asymptotic efficiency optimality may also extend to other measures of discriminatory efficiency such as the area under the receiver operating characteristic curve the optimal function u depends on a scalar probability density function which can be estimated non parametrically using a standard numerical algorithm a lasso like version for variable selection is implemented by adding regularization to the generalized t statistic two microarray data sets in the study of asthma and various cancers are used as motivating examples

mixture of experts classification using a hierarchical mixture model a three level hierarchical mixture model for classification is presented that models the following data generation process 1 the data are generated by a finite number of sources clusters and 2 the generation mechanism of each source assumes the existence of individual internal class labeled sources subclusters of the external cluster the model estimates the posterior probability of class membership similar to a mixture of experts classifier in order to learn the parameters of the model we have developed a general training approach based on maximum likelihood that results in two efficient training algorithms compared to other classification mixture models the proposed hierarchical model exhibits several advantages and provides improved classification performance as indicated by the experimental results

on different facets of regularization theory this review provides a comprehensive understanding of regularization theory from different perspectives emphasizing smoothness and simplicity principles using the tools of operator theory and fourier analysis it is shown that the solution of the classical tikhonov regularization problem can be derived from the regularized functional defined by a linear differential integral operator in the spatial fourier domain state of the art research relevant to the regularization theory is reviewed covering occam s razor minimum length description bayesian theory pruning algorithms informational entropy theory statistical learning theory and equivalent regularization the universal principle of regularization in terms of kolmogorov complexity is discussed finally some prospective studies on regularization theory and beyond are suggested

quantile based classifiers classification with small samples of high dimensional data is important in many application areas quantile classifiers are distance based classifiers that require a single parameter regardless of the dimension and classify observations according to a sum of weighted componentwise distances of the components of an observation to the within class quantiles an optimal percentage for the quantiles can be chosen by minimizing the misclassification error in the training sample it is shown that this choice is consistent for the classification rule with the asymptotically optimal quantile and that under some assumptions as the number of variables goes to infinity the probability of correct classification converges to unity the effect of skewness of the distributions of the predictor variables is discussed the optimal quantile classifier gives low misclassification rates in a comprehensive simulation study and in a real data application

the free energy at t 0 of the spin glass in the case of the continuous distribution of the effective field on the bethe lattice for z 4 5 6 abstract we evaluated the free energy at t 0 of the j random ising model on the bethe lattice of the coordination number z 4 5 6 in the case where the distribution function of the effective field has a continuous distribution by using the method of the integral equation of the effective field

geometric mean for subspace selection subspace selection approaches are powerful tools in pattern classification and data visualization one of the most important subspace approaches is the linear dimensionality reduction step in the fisher s linear discriminant analysis flda which has been successfully employed in many fields such as biometrics bioinformatics and multimedia information management however the linear dimensionality reduction step in flda has a critical drawback for a classification task with c classes if the dimension of the projected subspace is strictly lower than c 1 the projection to a subspace tends to merge those classes which are close together in the original feature space if separate classes are sampled from gaussian distributions all with identical covariance matrices then the linear dimensionality reduction step in flda maximizes the mean value of the kullback leibler kl divergences between different classes based on this viewpoint the geometric mean for subspace selection is studied in this paper three criteria are analyzed 1 maximization of the geometric mean of the kl divergences 2 maximization of the geometric mean of the normalized kl divergences and 3 the combination of 1 and 2 preliminary experimental results based on synthetic data uci machine learning repository and handwriting digits show that the third criterion is a potential discriminative subspace selection method which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in flda and its several representative extensions

functional linear discriminant analysis for irregularly sampled curves we introduce a technique for extending the classical method of linear discriminant analysis lda to data sets where the predictor variables are curves or functions this procedure which we call functional linear discriminant analysis flda is particularly useful when only fragments of the curves are observed all the techniques associated with lda can be extended for use with flda in particular flda can be used to produce classifications on new test curves give an estimate of the discriminant function between classes and provide a one or two dimensional pictorial representation of a set of curves we also extend this procedure to provide generalizations of quadratic and regularized discriminant analysis

classifiability based discriminatory projection pursuit fisher s linear discriminant fld is one of the most widely used linear feature extraction method especially in many visual computation tasks based on the analysis on several limitations of the traditional fld this paper attempts to propose a new computational paradigm for discriminative linear feature extraction named classifiability based discriminatory projection pursuit cdpp which is different from the traditional fld and its variants there are two steps in the proposed cdpp one is the construction of a candidate projection set cps and the other is the pursuit of discriminatory projections specifically in the former step candidate projections are generated by using the nearest between class boundary samples while the latter is efficiently achieved by classifiability based adaboost learning from the cps we show that the new projection pursuit paradigm not only does not suffer from the limitations of the traditional fld but also inherits good generalizability from the boundary attribute of candidate projections extensive experiments on both synthetic and real datasets validate the effectiveness of cdpp for discriminative linear feature extraction

on the relevance of linear discriminative features linear discriminant analysis lda has been widely used to extract linear features for classification in real applications the usefulness of the extracted features usually needs to be confirmed using an error rate of classification embedded in a classifier little attention has been paid to whether and how discriminative features themselves can be interpreted as indicators of usefulness we refer to this as relevance i e the capability of discriminative features to characterize the contribution of the original variables to classification we approach the relevance by considering how it could be lost while extracting optimal discriminative features then the discrepancy between the relevance and optimality of discriminative features is shown to originate from the angle between the space spanned by eigenvectors of the within class scattering matrix and the primary space in which the original variables reside in particular for a given dataset the larger the angle the less evident is the relevance discovered from optimal discriminative features furthermore the relevance and optimality are regarded as two constraint conditions or a tradeoff in order to extract relevant discriminative features at last a simulated experiment is used to show how the relevance is lost when the angle is changed experimental results based on both usps handwritten digitals and pie face databases show that a maximum margin criterion is a reasonable compromise between the relevance and optimality since it approximates the averaged class margin using euclidean distance measured in the primary space

generalization performance of fisher linear discriminant based on markov sampling fisher linear discriminant fld is a well known method for dimensionality reduction and classification that projects high dimensional data onto a low dimensional space where the data achieves maximum class separability the previous works describing the generalization ability of fld have usually been based on the assumption of independent and identically distributed i i d samples in this paper we go far beyond this classical framework by studying the generalization ability of fld based on markov sampling we first establish the bounds on the generalization performance of fld based on uniformly ergodic markov chain u e m c samples and prove that fld based on u e m c samples is consistent by following the enlightening idea from markov chain monto carlo methods we also introduce a markov sampling algorithm for fld to generate u e m c samples from a given data of finite size through simulation studies and numerical studies on benchmark repository using fld we find that fld based on u e m c samples generated by markov sampling can provide smaller misclassification rates compared to i i d samples

enhanced iterative projection for subclass discriminant analysis under em alike framework mixture discriminant analysis mda and subclass discriminant analysis sda belong to the supervised classification approaches they have advantage over the standard linear discriminant analysis lda in large sample size problems since both of them divide the samples in each class into subclasses which keep locality but lda does not however since the current mda and sda algorithms perform subclass division in just one step in the original data space before solving the generalized eigenvalue problem two problems are exposed 1 they ignore the relation among classes since subclass division is performed in each isolated class 2 they cannot guarantee good performance of classifiers in the transformed space because locality in the original data space may not be kept in the transformed space to address these problems this paper presents a new approach for subclass division based on k means clustering in the projected space class by class using the iterative steps under em alike framework experiments are performed on the artificial data set the uci machine learning data sets the cenparmi handwritten numeral database the nust603 handwritten chinese character database and the terrain cover database extensive experimental results demonstrate the performance advantages of the proposed method highlightswe focus on the classification of the large sample size problems we propose a new em alike iterative projection approach emipa for subclass division our proposed approach outperforms the traditional mda and sda our approach operates subclass division and eigenvector seeking class by class our proposed approach takes only a bit more time than mda and sda

su n irreducible schwinger bosons we construct su n irreducible schwinger bosons satisfying certain u n 1 constraints which implement the symmetries of su n young tableaues as a result all su n irreducible representations are simple monomials of n 1 types of su n irreducible schwinger bosons further we show that these representations are free of multiplicity problems thus all su n representations are made as simple as su 2

multiclass linear dimension reduction by weighted pairwise fisher criteria we derive a class of computationally inexpensive linear dimension reduction criteria by introducing a weighted variant of the well known k class fisher criterion associated with linear discriminant analysis lda it can be seen that lda weights contributions of individual class pairs according to the euclidean distance of the respective class means we generalize upon lda by introducing a different weighting function

bayesian multiscale models for poisson processes abstract i introduce a class of bayesian multiscale models bmsm s for one dimensional inhomogeneous poisson processes the focus is on estimating the discretized intensity function underlying the process unlike the usual transform based approach at the heart of most wavelet based methods for gaussian data these bmsm s are constructed using recursive dyadic partitions rdp s within an entirely likelihood based framework each rdp may be associated with a binary tree and a new multiscale prior distribution is introduced for the unknown intensity through the placement of mixture distributions at each of the nodes of the tree the concept of model mixing is then applied to a complete collection of such trees in addition to allowing for the inclusion of full location scale information in the model this last step also is fundamental both in inducing stationarity in the prior distribution and in enabling a given intensity function to be approximated at the resolution of the data under squared error lo

on the relation between discriminant analysis and mutual information for supervised linear feature extraction this paper provides a unifying view of three discriminant linear feature extraction methods linear discriminant analysis heteroscedastic discriminant analysis and maximization of mutual information we propose a model independent reformulation of the criteria related to these three methods that stresses their similarities and elucidates their differences based on assumptions for the probability distribution of the classification data we obtain sufficient conditions under which two or more of the above criteria coincide it is shown that these conditions also suffice for bayes optimality of the criteria our approach results in an information theoretic derivation of linear discriminant analysis and heteroscedastic discriminant analysis finally regarding linear discriminant analysis we discuss its relation to multidimensional independent component analysis and derive suboptimality bounds based on information theory

characterization of a family of algorithms for generalized discriminant analysis on undersampled problems a generalized discriminant analysis based on a new optimization criterion is presented the criterion extends the optimization criteria of the classical linear discriminant analysis lda when the scatter matrices are singular an efficient algorithm for the new optimization problem is presented the solutions to the proposed criterion form a family of algorithms for generalized lda which can be characterized in a closed form we study two specific algorithms namely uncorrelated lda ulda and orthogonal lda olda ulda was previously proposed for feature extraction and dimension reduction whereas olda is a novel algorithm proposed in this paper the features in the reduced space of ulda are uncorrelated while the discriminant vectors of olda are orthogonal to each other we have conducted a comparative study on a variety of real world data sets to evaluate ulda and olda in terms of classification accuracy

trust tech based expectation maximization for learning finite mixture models the expectation maximization em algorithm is widely used for learning finite mixture models despite its greedy nature most popular model based clustering techniques might yield poor clusters if the parameters are not initialized properly to reduce the sensitivity of initial points a novel algorithm for learning mixture models from multivariate data is introduced in this paper the proposed algorithm takes advantage of trust tech transformation under stability retaining equilibria characterization to compute neighborhood local maxima on the likelihood surface using stability regions basically our method coalesces the advantages of the traditional em with that of the dynamic and geometric characteristics of the stability regions of the corresponding nonlinear dynamical system of the log likelihood function two phases namely the em phase and the stability region phase are repeated alternatively in the parameter space to achieve local maxima with improved likelihood values the em phase obtains the local maximum of the likelihood function and the stability region phase helps to escape out of the local maximum by moving toward the neighboring stability regions though applied to gaussian mixtures in this paper our technique can be easily generalized to any other parametric finite mixture model the algorithm has been tested on both synthetic and real data sets and the improvements in the performance compared to other approaches are demonstrated the robustness with respect to initialization is also illustrated experimentally

canonical correlation analysis when the data are curves summary it is not immediately straightforward to extend canonical correlation analysis to the context of functional data analysis where the data are themselves curves or functions the obvious approach breaks down and it is necessary to use a method involving smoothing in some way such a method is introduced and discussed with reference to a data set on human gait the breakdown of the unsmoothed method is illustrated in a practical context and is demonstrated theoretically a consistency theorem for the smoothed method is proved in an increasing number of problems in a wide range of fields the data observed are not the univariate or multivariate observations of classical statistics but are func tions observed continuously ramsay and dalzell 1991 give the name functional data analysis to the analysis of data of this kind in most cases the observations will be functions of time or a closely related variable but there are clearly applications where the functions are surfaces observed over two or three dimensional space the motivating example for the present paper is a set of data on children s gait collected by the motion analysis laboratory at children s hospital san diego california see olshen et al 1989 for full details for each of a number of children several angles made by the child s joints knee hip etc are observed during the child s gait cycle one aim of the study is to gain understanding of the gait cycle of a normal child to make comparisons with children suffering from walking diffi culties these data motivated rice and silverman 1991 to discuss the extension of principal component analysis to the functional setting and to explain how smoothing can be incorporated into the analysis in a natural way in a substantial complemen tary paper ramsay and dalzell 1991 discuss different approaches to regression and principal component analysis in functional data analysis illustrating their work by a meteorological example the insights of functional data analysis may be helpful in chemometrics for instance a standard chemometric problem is the analysis of spectra observed in chromatography and these are to all intents and purposes functional observations another obvious area of potential relevance is the analysis of growth curve data the focus in the present paper will be on canonical correlation analysis cca in the gait example one might ask how variability in the knee angle cycle is related to

face recognition by sparse discriminant analysis via joint l2 1 norm minimization abstract recently joint feature selection and subspace learning which can perform feature selection and subspace learning simultaneously is proposed and has encouraging ability on face recognition in the literature a framework of utilizing l 2 1 norm penalty term has also been presented but some important algorithms cannot be covered such as fisher linear discriminant analysis and sparse discriminant analysis therefore in this paper we add l 2 1 norm penalty term on flda and propose a feasible solution by transforming its nonlinear model into linear regression type in addition we modify the optimization model of sda by replacing elastic net with l 2 1 norm penalty term and present its optimization method experiments on three standard face databases illustrate flda and sda via l 2 1 norm penalty term can significantly improve their recognition performance and obtain inspiring results with low computation cost and for low dimension feature

coherent states for su 3 we define coherent states for su 3 using six bosonic creation and annihilation operators these coherent states are explicitly characterized by six complex numbers with constraints for the completely symmetric representations n 0 and 0 m only three of the bosonic operators are required for mixed representations n m all six operators are required the coherent states provide a resolution of identity satisfy the continuity property and possess a variety of group theoretic properties we introduce an explicit parametrization of the group su 3 and the corresponding integration measure finally we discuss the path integral formalism for a problem in which the hamiltonian is a function of su 3 operators at each site

semiparametric mixed models for increment averaged data with application to carbon sequestration in agricultural soils adoption of conservation tillage practice in agriculture offers the potential to mitigate greenhouse gas emissions studies comparing conservation tillage methods to traditional tillage pair fields under the two management systems and obtain soil core samples from each treatment cores are divided into multiple increments and matching increments from one or more cores are aggregated and analyzed for carbon stock these data represent not the actual value at a specific depth but rather the total or average over a depth increment a semiparametric mixed model is developed for such increment averaged data the model uses parametric fixed effects to represent covariate effects random effects to capture correlation within studies and an integrated smooth function to describe effects of depth the depth function is specified as an additive model estimated with penalized splines using standard mixed model software smoothing parameters are automatically selected using restricted maximum likelihood the meth

zero temperature hysteresis in the random field ising model on a bethe lattice we consider the single spin flip dynamics of the random field ising model on a bethe lattice at zero temperature in the presence of a uniform external field we determine the average magnetization as the external field is varied from to by setting up the self consistent field equations which we show are exact in this case the qualitative behaviour of magnetization as a function of the external field unexpectedly depends on the coordination number z of the bethe lattice for z 3 with a gaussian distribution of the quenched random fields we find no jump in magnetization for any non zero strength of disorder for for weak disorder the magnetization shows a jump discontinuity as a function of the external uniform field which disappears for a larger variance of the quenched field we determine exactly the critical point separating smooth hysteresis curves from those with a jump we have checked our results by monte carlo simulations of the model on three and four coordinated random graphs which for large system sizes give the same results as on the bethe lattice but avoid surface effects altogether

learning gradients predictive models that infer geometry and statistical dependence the problems of dimension reduction and inference of statistical dependence are addressed by the modeling framework of learning gradients the models we propose hold for euclidean spaces as well as the manifold setting the central quantity in this approach is an estimate of the gradient of the regression or classification function two quadratic forms are constructed from gradient estimates the gradient outer product and gradient based diffusion maps the first quantity can be used for supervised dimension reduction on manifolds as well as inference of a graphical model encoding dependencies that are predictive of a response variable the second quantity can be used for nonlinear projections that incorporate both the geometric structure of the manifold as well as variation of the response variable on the manifold we relate the gradient outer product to standard statistical quantities such as covariances and provide a simple and precise comparison of a variety of supervised dimensionality reduction methods we provide rates of convergence for both inference of informative directions as well as inference of a graphical model of variable dependencies

oscillator like unitary representations of non compact groups with a jordan structure and the non compact groups of supergravity we give a general bosonic construction of oscillator like unitary irreducible representations uir of non compact groups whose coset spaces with respect to their maximal compact subgroups are hermitian symmetric with the exception of e7 7 they include all the non compact invariance groups of extended supergravity theories in four dimensions these representations have the remarkable property that each uir is uniquely determined by an irreducible representation of the maximal compact subgroup we study the connection between our construction the hermitian symmetric spaces and the tits koecher construction of the lie algebras of corresponding groups we then give the bosonic construction of the lie algebra ofe7 7 in su 8 so 8 and u 7 bases and study its properties application of our method toe7 7 leads to reducible unitary representations

penalized discriminant analysis fisher s linear discriminant analysis lda is a popular data analytic tool for studying the relationship between a set of predictors and a categorical response in this paper we describe a penalized version of lda it is designed for situations in which there are many highly correlated predictors such as those obtained by discretizing a function or the grey scale values of the pixels in a series of images in cases such as these it is natural efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients both for improved prediction performance and interpretability we cast the classification problem into a regression framework via optimal scoring using this our proposal facilitates the use of any penalized regression technique in the classification setting the technique is illustrated with examples in speech recognition and handwritten character recognition

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

multiscale likelihood analysis and complexity penalized estimation we describe here a framework for a certain class of multiscale likelihood factorizations wherein in analogy to a wavelet decomposition of an l 2 function a given likelihood function has an alternative representation as a product of conditional densities reflecting information in both the data and the parameter vector localized in position and scale the framework is developed as a set of sufficient conditions for the existence of such factorizations formulated in analogy to those underlying a standard multiresolution analysis for wavelets and hence can be viewed as a multiresolution analysis for likelihoods we then consider the use of these factorizations in the task of nonparametric complexity penalized likelihood estimation we study the risk properties of certain thresholding and partitioning estimators and demonstrate their adaptivity and near optimality in a minimax sense over a broad range of function spaces based on squared hellinger distance as a loss function in particular our results provide an illustration of how properties of classical wavelet based estimators can be obtained in a single unified framework that includes models for continuous count and categorical data types

su n coherent states we generalize schwinger boson representation of su 2 algebra to su n and define coherent states of su n using 2 2n 1 1 bosonic harmonic oscillator creation and annihilation operators we give an explicit construction of all n 1 casimirs of su n in terms of these creation and annihilation operators the su n coherent states belonging to any irreducible representations of su n are labeled by the eigenvalues of the casimir operators and are characterized by n 1 complex orthonormal vectors describing the su n manifold the coherent states provide a resolution of identity satisfy the continuity property and possess a variety of group theoretic properties

image segmentation using hidden markov gauss mixture models image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing we develop a multiclass image segmentation method using hidden markov gauss mixture models hmgmms and provide examples of segmentation of aerial images and textures hmgmms incorporate supervised learning fitting the observation probability distribution given each class by a gauss mixture estimated using vector quantization with a minimum discrimination information mdi distortion we formulate the image segmentation problem using a maximum a posteriori criteria and find the hidden states that maximize the posterior density given the observation we estimate both the hidden markov parameter and hidden states using a stochastic expectation maximization algorithm our results demonstrate that hmgmm provides better classification in terms of bayes risk and spatial homogeneity of the classified objects than do several popular methods including classification and regression trees learning vector quantization causal hidden markov models hmms and multiresolution hmms the computational load of hmgmm is similar to that of the causal hmm

transformation invariant clustering using the em algorithm clustering is a simple effective way to derive useful representations of data such as images and videos clustering explains the input as one of several prototypes plus noise in situations where each input has been randomly transformed e g by translation rotation and shearing in images and videos clustering techniques tend to extract cluster centers that account for variations in the input due to transformations instead of more interesting and potentially useful structure for example if images from a video sequence of a person walking across a cluttered background are clustered it would be more useful for the different clusters to represent different poses and expressions instead of different positions of the person and different configurations of the background clutter we describe a way to add transformation invariance to mixture models by approximating the nonlinear transformation manifold by a discrete set of points we show how the expectation maximization algorithm can be used to jointly learn clusters while at the same time inferring the transformation associated with each input we compare this technique with other methods for filtering noisy images obtained from a scanning electron microscope clustering images from videos of faces into different categories of identification and pose and removing foreground obstructions from video we also demonstrate that the new technique is quite insensitive to initial conditions and works better than standard techniques even when the standard techniques are provided with extra data

sparse discriminant analysis we consider the problem of performing interpretable classification in the high dimensional setting in which the number of features is very large and the number of observations is limited this setting has been studied extensively in the chemometrics literature and more recently has become commonplace in biological and medical applications in this setting a traditional approach involves performing feature selection before classification we propose sparse discriminant analysis a method for performing linear discriminant analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously sparse discriminant analysis is based on the optimal scoring interpretation of linear discriminant analysis and can be extended to perform sparse discrimination via mixtures of gaussians if boundaries between classes are nonlinear or if subgroups are present within each class our proposal also provides low dimensional views of the discriminative directions

model based learning using a mixture of mixtures of gaussian and uniform distributions we introduce a mixture model whereby each mixture component is itself a mixture of a multivariate gaussian distribution and a multivariate uniform distribution although this model could be used for model based clustering model based unsupervised learning or model based classification model based semi supervised learning we focus on the more general model based classification framework in this setting we fit our mixture models to data where some of the observations have known group memberships and the goal is to predict the memberships of observations with unknown labels we also present a density estimation example a generalized expectation maximization algorithm is used to estimate the parameters and thereby give classifications in this mixture of mixtures model to simplify the model and the associated parameter estimation we suggest holding some parameters fixed this leads to the introduction of more parsimonious models a simulation study is performed to illustrate how the model allows for bursts of probability and locally higher tails two further simulation studies illustrate how the model performs on data simulated from multivariate gaussian distributions and on data from multivariate t distributions this novel approach is also applied to real data and the performance of our approach under the various restrictions is discussed

smoothers and the cp generalized maximum likelihood and extended exponential criteria nonparametric regression often called smoothing is a widely used data analysis method the use of a smoother requires the choice of a smoothing parameter that by balancing fidelity and roughness controls how much smoothing is done two popular selection criteria for choosing the smoothing parameter are cp and generalized maximum likelihood gml each of these has its own problems forcp the problem is its high variability whereas for gml the problem is its potentially large bias by studying the geometry of selection criteria we give an intuitive explanation of the strength and weakness of cp and gml the geometry then motivates a new selection method the extended exponential ee criterion which combines the strength of cp and gml but mitigates their weaknesses in terms of variability bias and undersmoothing