detecting the presence of mixing with multiscale maximum likelihood a test of homogeneity tries to decide whether observations come from a single distribution or from a mixture of several distributions a powerful theory has been developed for the case where the component distributions are members of an exponential family when no parametric assumptions are appropriate the standard approach is to test for bimodality which is known not to be very sensitive for detecting heterogeneity to develop a more sensitive procedure this article builds on an approach employed in sampling literature and models the component distributions as logarithmically concave densities it is shown how this leads to a special semiparametric model in which the homogeneity problem is equivalent to testing whether a parameter c equals zero versus the alternative that c 0 this setup leads naturally to a novel multiscale maximum likelihood procedure where the multiscale character reflects the desirable property of adaptivity to the unknown value of the parameter c under the alternative to en

least squares twin bounded support vector machines based on l1 norm distance metric for classification abstract in this paper we construct a least squares version of the recently proposed twin bounded support vector machine tbsvm for binary classification as a valid classification tool tbsvm attempts to seek two non parallel planes that can be produced by solving a pair of quadratic programming problems qpps but this is time consuming here we solve two systems of linear equations rather than two qpps to avoid this deficiency furthermore the distance in least squares tbsvm lstbsvm is measured by l2 norm but l1 norm distance is usually regarded as an alternative to l2 norm to improve model robustness in the presence of outliers inspired by the advantages of least squares twin support vector machine lstwsvm tbsvm and l1 norm distance we propose a lstbsvm based on l1 norm distance metric for binary classification termed as l1 lstbsvm which is specially designed for suppressing the negative effect of outliers and improving computational efficiency in large datasets then we design a powerful iterative algorithm to solve the l1 norm optimal problems and it is easy to implement and its convergence to an optimum solution is theoretically ensured finally the feasibility and effectiveness of l1 lstbsvm are validated by extensive experimental results on both uci datasets and artificial datasets

principal component analysis based on l1 norm maximization a method of principal component analysis pca based on a new l1 norm optimization technique is proposed unlike conventional pca which is based on l2 norm the proposed method is robust to outliers because it utilizes l1 norm which is less sensitive to outliers it is invariant to rotations as well the proposed l1 norm optimization technique is intuitive simple and easy to implement it is also proven to find a locally maximal solution the proposed method is applied to several datasets and the performances are compared with those of other conventional methods

a connection between state space and doubly coprime fractional representations explicit formulas for a doubly coprime fractional representation of the transfer function of a lumped lti system are given in terms of a stabilizable and detectable state space realization of the transfer function these formulas allow existing computational algorithms to be utilized for the purpose of computing doubly coprime factorizations several additional implications of this result are briefly discussed as are some immediate extensions and future research directions

order selection in finite mixture models with a nonsmooth penalty order selection is a fundamental and challenging problem in the application of finite mixture models in this article we develop a new penalized likelihood approach the new method modified smoothly clipped absolute deviation mscad deviates from information based methods such as akaike information criterion aic and bayesian information criterion bic by introducing two penalty functions that depend on the mixing proportions and the component parameters it is consistent at estimating both the order of the mixture model and the mixing distribution simulations show that mscad has much better performance than a number of existing methods two real data examples are examined to illustrate the performance of mscad

testing the order of a model using locally conic parametrization population mixtures and stationary arma processes in this paper we address the problem of testing hypotheses using the likelihood ratio test statistic in nonidentifiable models with application to model selection in situations where the parametrization for the larger model leads to nonidentifiability in the smaller model we give two major applications the case where the number of populations has to be tested in a mixture and the case of stationary arma p q processes where the order p q has to be tested we give the asymptotic distribution for the likelihood ratio test statistic when testing the order of the model in the case of order selection for armas the asymptotic distribution is invariant with respect to the parameters generating the process a locally conic parametrization is a key tool in deriving the limiting distributions it allows one to discover the deep similarity between the two problems

recursive projection twin support vector machine via within class variance minimization in this paper a novel binary classifier coined projection twin support vector machine ptsvm is proposed the idea is to seek two projection directions one for each class such that the projected samples of one class are well separated from those of the other class in its respective subspace in order to further boost performance a recursive algorithm for ptsvm is proposed to generate more than one projection axis for each class to overcome the singularity problem principal component analysis pca is utilized to transform the data in the original space into a low dimensional subspace wherein the optimization problem of ptsvm is convex and can be solved efficiently the experimental results on several uci benchmark data sets and usps digit database show the feasibility and effectiveness of the proposed method

moment based oscillation properties of mixture models consider finite mixture models of the form g x q f f x dq where f is a parametric density and q is a discrete probability measure an important and difficult statistical problem concerns the determination of the number of support points usually known as components of q from a sample of observations from g for an important class of exponential family models we have the following result if p has more than p components and q is an appropriately chosen p component approximation of p then g x p g x q demonstrates a prescribed sign change behavior as does the corresponding difference in the distribution functions these strong structural properties have implications for diagnostic plots for the number of components in a finite mixture

nonparallel hyperplane support vector machine for binary classification problems in this paper we propose a nonparallel hyperplane support vector machine nhsvm for binary classification problems our proposed nhsvm is formulated by clustering the training points according to the similarity between classes it constructs two nonparallel hyperplanes simultaneously by solving a single quadratic programming problem and is consistent between its predicting and training processes an essential difference that distinguishes it from other nonparallel svms this proposed nhsvm has been analyzed theoretically and implemented experimentally the results of experiments conducted using it on both artificial and publicly available benchmark datasets confirm its feasibility and efficacy especially for cross planes datasets and datasets with heteroscedastic noise

linear discriminant analysis based on l1 norm maximization linear discriminant analysis lda is a well known dimensionality reduction technique which is widely used for many purposes however conventional lda is sensitive to outliers because its objective function is based on the distance criterion using l2 norm this paper proposes a simple but effective robust lda version based on l1 norm maximization which learns a set of local optimal projection vectors by maximizing the ratio of the l1 norm based between class dispersion and the l1 norm based within class dispersion the proposed method is theoretically proved to be feasible and robust to outliers while overcoming the singular problem of the within class scatter matrix for conventional lda experiments on artificial datasets standard classification datasets and three popular image databases demonstrate the efficacy of the proposed method

homogeneity tests against central mixture alternatives abstract a test of homogeneity is derived for a general sampling model the alternative hypothesis is a mixture over a parameter of the sampling model centered at the null hypothesis the test statistic is derived through a score test its functional form is independent of the particular mixing distribution suppose yi i 1 n are independent random variables with respective probability density or mass functions fi yi i under the null hypothesis suppose all i homogeneous and equal to the common value 0 under the alternative hypothesis suppose the i behave as random samples taken from a distribution with mean 0 and finite third moment the score statistic for testing these hypotheses rejects the null for large values of where is the maximum likelihood estimate of 0 under the null hypothesis and when the sample size n is large s is normally distributed under both the null hypothesis and a sequence of alternative hypotheses in which the variance of the mixing distribution tends

hypothesis test for normal mixture models the em approach normal mixture distributions are arguably the most important mixture models and also the most technically challenging the likelihood function of the normal mixture model is unbounded based on a set of random samples unless an artificial bound is placed on its component variance parameter moreover the model is not strongly identifiable so it is hard to differentiate between over dispersion caused by the presence of a mixture and that caused by a large variance and it has infinite fisher information with respect to mixing proportions there has been extensive research on finite normal mixture models but much of it addresses merely consistency of the point estimation or useful practical procedures and many results require undesirable restrictions on the parameter space we show that an em test for homogeneity is effective at overcoming many challenges in the context of finite normal mixtures we find that the limiting distribution of the em test is a simple function of the 0 5 2 0 5 2 and 2 distributions when the mixing variances are equal but unknown and the 2 when variances are unequal and unknown simulations show that the limiting distributions approximate the finite sample distribution satisfactorily two genetic examples are used to illustrate the application of the em test

a constrained formulation of maximum likelihood estimation for normal mixture distributions in this context the normal densities are sometimes referred to as component densities the log likelihood function l zy corresponding to a random sample xl xnj is defined by l ty ek 1 10g pm xk y the first problem associated with maximum likelihood estimation arises from the unboundedness of l qy on r day 1969 a global maximum likelihood estimate always fails to exist in addition the unboundedness of l y causes failures of optimization algorithms of both the em redner and walker 1984 and quasi newton fowlkes 1979 types in spite of the unboundedness of l 4y statistical theory kiefer 1978 guarantees that a particular local maximizer of l y is strongly consistent and asymptotically efficient several local maximizers can exist for a given sample and the other major maximum likelihood difficulty is in determining when the correct one has been found day 1969 noted that spurious maximizers corresponding to parameter points having some component standard deviations very small relative to others are generated by any small number of sample points grouped sufficiently close together the spurious maximizers like the unboundedness of l zy can create difflculties when using the em or quasi newton algorithms the constrained maximum likelihood formulation presented in the next section avoids the unconstrained problems of singularities and spurious maximizers of the type noted by day section 3 contains the statement and proof of a consistency result for the corresponding constrained estimator

twin support vector machines for pattern classification we propose twin svm a binary svm classifier that determines two nonparallel planes by solving two related svm type problems each of which is smaller than in a conventional svm the twin svm formulation is in the spirit of proximal svms via generalized eigenvalues on several benchmark data sets twin svm is not only fast but shows good generalization twin svm is also useful for automatically discovering two dimensional projections of the data

fitting the erlang mixture model to data via a gem cmm algorithm abstract the erlang mixture model with common scale parameter is flexible and analytically tractable as such it is a useful model to fit insurance loss data and to calculate quantities of interest for insurance risk management in this paper we propose a generalized expectation maximization gem algorithm along with a clusterized method of moments cmm to estimate the model parameters the gem algorithm not only estimates the mixing weights and scale parameter of the model but also estimates the shape parameters of the model using a local search method the cmm method enables to produce quality initial estimates for the gem algorithm as a result the proposed approach provides an efficient algorithm that can fit the model to the body and the tail of truncated and censored loss data well and converges fast we examine the performance of the proposed approach through several simulation studies and apply it to fit the erlang mixture model to two real loss data sets

selection of mixed copula model via penalized likelihood a fundamental issue of applying a copula method in applications is how to choose an appropriate copula function for a given problem in this article we address this issue by proposing a new copula selection approach via penalized likelihood plus a shrinkage operator the proposed method selects an appropriate copula function and estimates the related parameters simultaneously we establish the asymptotic properties of the proposed penalized likelihood estimator including the rate of convergence and asymptotic normality and abnormality particularly when the true coefficient parameters may be on the boundary of the parameter space and the dependence parameters are in an unidentified subset of the parameter space we show that the limiting distribution for boundary parameter estimator is half normal and the penalized likelihood estimator for unidentified parameter converges to an arbitrary value finally monte carlo simulation studies are carried out to illustrate the finite sample performance of the pro

a note on overdispersed exponential families abstract the issue of creating overdispersion in a given one parameter one dimensional exponential family by extending it to a two parameter exponential family with the same support is considered an easily verifiable sufficient condition for this is derived it is shown that a large class of families satisfy this condition and that this class includes efron s 1986 and lindsay s 1986 family as special cases this class is also closely related to jorgensen s 1988 exponential dispersion models ump unbiased tests for testing overdispersion are exhibited and it is shown that in this context cox s overdispersion test is a special case a graphical display is developed to select a family of the general class which should be used with a given set of data to overdisperse a given target one parameter family two real data illustrations are given kr

nonparametric mle under two closed capture recapture models with heterogeneity we conduct nonparametric maximum likelihood estimation under two common heterogeneous closed population capture recapture models our models specify mixture models as did previous researchers models which have a common generating distribution say f for the capture probabilities using lindsay and roeder s 1992 journal of the american statistical association 87 785 794 mixture model results and the em algorithm a nonparametric maximum likelihood estimator mle of f for any specified population size n is obtained then the nonparametric mle of the n f pair and thus for n is determined perhaps most importantly since our mle pair maximizes the likelihood under the entire nonparametric probability model it provides an excellent foundation for estimating properties of estimators conducting a goodness of fit test and performing a likelihood ratio test these are illustrated in the paper

testing the order of a model this paper deals with order identification for nested models in the i i d framework we study the asymptotic efficiency of two generalized likelihood ratio tests of the order they are based on two estimators which are proved to be strongly consistent a version of stein s lemma yields an optimal underestimation error exponent the lemma also implies that the overestimation error exponent is necessarily trivial our tests admit non trivial underestimation error exponents the optimal underestimation error exponent is achieved in some situations the overestimation error can decay exponentially fast with respect to a positive power of the number of observations these results are proved under mild assumptions by relating the underestimation resp overestimation error to large resp moderate deviations of the log likelihood process in particular it is not necessary that the classical cramer condition be satisfied namely the log densities are not required to admit every exponential moment three benchmark examples with specific difficulties location mixture of normal distributions abrupt changes and various regressions are detailed so as to illustrate the generality of our results

inference for subgroup analysis with a structured logistic normal mixture model in this article we propose a statistical model for the purpose of identifying a subgroup that has an enhanced treatment effect as well as the variables that are predictive of the subgroup membership the need for such subgroup identification arises in clinical trials and in market segmentation analysis by using a structured logistic normal mixture model our proposed framework enables us to perform a confirmatory statistical test for the existence of subgroups and at the same time to construct predictive scores for the subgroup membership the inferential procedure proposed in the article is built on the recent literature on hypothesis testing for gaussian mixtures but the structured logistic normal mixture model enjoys some distinctive properties that are unavailable to the simpler gaussian mixture models with the bootstrap approximations the proposed tests are shown to be powerful and equally importantly insensitive to the choice of tuning parameters as an illustration we analyze a dataset fr

testing for a finite mixture model with two components summary we consider a finite mixture model with k components and a kernel distribution from a general one parameter family the problem of testing the hypothesis k 2 versus k 3 is studied there has been no general statistical testing procedure for this problem we propose a modified likelihood ratio statistic where under the null and the alternative hypotheses the estimates of the parameters are obtained from a modified likelihood function it is shown that estimators of the support points are consistent the asymptotic null distribution of the modified likelihood ratio test proposed is derived and found to be relatively simple and easily applied simulation studies for the asymptotic modified likelihood ratio test based on finite mixture models with normal binomial and poisson kernels suggest that the test proposed performs well simulation studies are also conducted for a bootstrap method with normal kernels an example involving foetal movement data from a medical study illustrates the testing procedure

semiparametric estimation in the rasch model and related exponential response models including a simple latent class model for item analysis abstract the rasch model for item analysis is an important member of the class of exponential response models in which the number of nuisance parameters increases with the number of subjects leading to the failure of the usual likelihood methodology both conditional likelihood methods and mixture model techniques have been used to circumvent these problems in this article we show that these seemingly unrelated analyses are in fact closely linked to each other despite dramatic structural differences between the classes of models implied by each approach we show that the finite mixture model for j dichotomous items having t latent classes gives the same estimates of item parameters as conditional likelihood on a set whose probability approaches one if t j 1 2 unconditional maximum likelihood estimators for the finite mixture model can be viewed as keifer wolfowitz estimators for the random effects version of the rasch model latent class versions of the model are especially attractive when t is

the geometry of mixture likelihoods a general theory in this paper certain fundamental properties of the maximum likelihood estimator of a mixing distribution are shown to be geometric properties of the likelihood set the existence support size likelihood equations and uniqueness of the estimator are revealed to be directly related to the properties of the convex hull of the likelihood set and the support hyperplanes of that hull it is shown using geometric techniques that the estimator exists under quite general conditions with a support size no larger than the number of distinct observations analysis of the convex dual of the likelihood set leads to a dual maximization problem a convergent algorithm is described the defining equations for the estimator are compared with the usual parametric likelihood equations for finite mixtures sufficient conditions for uniqueness are given part ii will deal with a special theory for exponential family mixtures

improvements on twin support vector machines for classification problems the generalized eigenvalue proximal support vector machine gepsvm and twin support vector machine twsvm are regarded as milestones in the development of the powerful svms as they use the nonparallel hyperplane classifiers in this brief we propose an improved version named twin bounded support vector machines tbsvm based on twsvm the significant advantage of our tbsvm over twsvm is that the structural risk minimization principle is implemented by introducing the regularization term this embodies the marrow of statistical learning theory so this modification can improve the performance of classification in addition the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure experimental results show the effectiveness of our method in both computation time and classification accuracy and therefore confirm the above conclusion further

model selection in high dimensions a quadratic risk based approach we propose a general class of risk measures which can be used for data based evaluation of parametric models the loss function is defined as the generalized quadratic distance between the true density and the model proposed these distances are characterized by a simple quadratic form structure that is adaptable through the choice of a non negative definite kernel and a bandwidth parameter using asymptotic results for the quadratic distances we build a quick to compute approximation for the risk function its derivation is analogous to the akaike information criterion but unlike the akaike information criterion the quadratic risk is a global comparison tool the method does not require resampling which is a great advantage when point estimators are expensive to compute the method is illustrated by using the problem of selecting the number of components in a mixture model where it is shown that by using an appropriate kernel the method is computationally straightforward in arbitrarily high data dimensions in this same context it is shown that the method has some clear advantages over the akaike information criterion and bayesian information criterion

testing the order of a finite mixture the order is an important parameter in applications of finite mixture models yet designing a valid and easy to use statistical test for the order is challenging to date most results on hypothesis tests have focused on homogeneity a special case where the null model has order 1 in this work we designed an em test for the general problem of testing the null hypothesis of order m0 versus an alternative hypothesis of order larger than m0 for any positive integer m0 the null limiting distribution of the em test is a mixture of 2 distributions the weights in this mixture limiting distribution can be conveniently computed compared with related results the new result is obtained under much less strict requirements on the component distribution and the parameter space extensive simulation studies show that the limiting distributions closely match the finite sample distributions of the em test when m0 2 the new em test has more accurate type i errors and matches the power of the modified likelihood

an approach to simultaneous system design part 11 nonswitching gain and dynamic feedback compensation by algebraic geometric methods this paper studies structured uncertainty problems in feedback system design considers a compact parameterization of the space of linear dynamical systems and introduces base points and critical points as two algebraic geometric objects that have significance in sensitivity and robustness studies respectively using the nevanlinna pick interpolation theory the author obtains a necessary and sufficient condition for simultaneous stabilization of a structured one parameter family of plants a recent result due to kharitonov on the simultaneous stability of a parameterized family of polynomials leads to a sufficiency condition for simultaneous stabilization of a structured multiparameter family of plants furthermore the author considers simultaneous pole placement of an r tuple of plants as a means to arbitrarily tune the natural frequencies of a multimode linear dynamical system the concept of nondegenerate and twisted r tuples of plants is introduced as the pole placement problem is studie

empirical bayes analysis of a microarray experiment microarrays are a novel technology that facilitates the simultaneous measurement of thousands of gene expression levels a typical microarray experiment can produce millions of data points raising serious problems of data reduction and simultaneous inference we consider one such experiment in which oligonucleotide arrays were employed to assess the genetic effects of ionizing radiation on seven thousand human genes a simple nonparametric empirical bayes model is introduced which is used to guide the efficient reduction of the data to a single summary statistic per gene and also to make simultaneous inferences concerning which genes were affected by the radiation although our focus is on one specific experiment the proposed methods can be applied quite generally the empirical bayes inferences are closely related to the frequentist false discovery rate fdr criterion

asymptotics for likelihood ratio tests under loss of identifiability this paper describes the large sample properties of the likelihood ratio test statistic lrts when the parameters characterizing the true null distribution are not unique it is well known that the classical asymptotic theory for the likelihood ratio test does not apply to such problems and the lrts may not have the typical chi squared type limiting distribution this paper establishes a general quadratic approximation of the log likelihood ratio function in a hellinger neighborhood of the true density which is valid with or without loss of identifiability of the true distribution under suitable conditions the asymptotic null distribution of the lrts under loss of identifiability can be obtained by maximizing the quadratic form these results extend the work of chernoff and le cam in particular applications to testing the number of mixture components in finite mixture models are discussed

lagrangian support vector machines an implicit lagrangian for the dual of a simple reformulation of the standard quadratic program of a linear support vector machine is proposed this leads to the minimization of an unconstrained differentiable convex function in a space of dimensionality equal to the number of classified points this problem is solvable by an extremely simple linearly convergent lagrangian support vector machine lsvm algorithm lsvm requires the inversion at the outset of a single matrix of the order of the much smaller dimensionality of the original input space plus one the full algorithm is given in this paper in 11 lines of matlab code without any special optimization tools such as linear or quadratic programming solvers this lsvm code can be used as is to solve classification problems with millions of points for example 2 million points in 10 dimensional input space were classified by a linear surface in 82 minutes on a pentium iii 500 mhz notebook with 384 megabytes of memory and additional swap space and in 7 minutes on a 250 mhz ultrasparc ii processor with 2 gigabytes of memory other standard classification test problems were also solved nonlinear kernel classification can also be solved by lsvm although it does not scale up to very large problems it can handle any positive semidefinite kernel and is guaranteed to converge a short matlab code is also given for nonlinear kernels and tested on a number of problems

robust estimation of mixture complexity in many applications it is important to find the mixture with fewest number of components known as the mixture complexity that provides a satisfactory fit to the data this article focuses on developing an estimator of mixture complexity that is consistent when the form of component densities are unknown but are postulated to be members of some parametric family and is simultaneously robust against model misspecification we treat the estimation of mixture complexity as a model selection problem and construct an estimator of mixture complexity as a byproduct of minimizing a hellinger information criterion this estimator is shown to be consistent for any parametric family of mixtures when the model is correctly specified monte carlo simulations for a wide variety of normal mixtures show that our estimator is very competitive with several others in the literature in correctly identifying the true mixture complexity the basic construction being firmly rooted in the minimum hellinger distance approach

a graphical technique for determining the number of components in a mixture of normals abstract when a population is assumed to be composed of a finite number of subpopulations a natural model to choose is the finite mixture model it will often be the case however that the number of component distributions is unknown and must be estimated this problem can be difficult for instance the density of two mixed normals is not bimodal unless the means are separated by at least 2 standard deviations hence modality of the data per se can be an insensitive approach to component estimation we demonstrate that a mixture of two normals divided by a normal density having the same mean and variance as the mixed density is always bimodal this analytic result and other related results form the basis for a diagnostic and a test for the number of components in a mixture of normals the density is estimated using a kernel density estimator under the null hypothesis the proposed diagnostic can be approximated by a stationary gaussian process under the alternative hypothesis components in the mixtu

non finite fisher information and homogeneity an em approach summary even simple examples of finite mixture models can fail to fulfil the regularity conditions that are routinely assumed in standard parametric inference problems many methods have been investigated for testing for homogeneity in finite mixture models for example but all rely on regularity conditions including the finiteness of the fisher information and the space of the mixing parameter being a compact subset of some euclidean space very simple examples where such assumptions fail include mixtures of two geometric distributions and two exponential distributions and more generally mixture models in scale distribution families to overcome these difficulties we propose and study an em test statistic which has a simple limiting distribution for examples in this paper simulations show that the em test has accurate type i errors and is more efficient than existing methods when they are applicable a real example is also included

multiple change point analysis fast implementation and strong consistency one of the main challenges in identifying structural changes in stochastic processes is to carry out analysis for time series with dependency structure in a computationally tractable way another challenge is that the number of true change points is usually unknown requiring a suitable model selection criterion to arrive at informative conclusions to address the first challenge we model the data generating process as a segment wise autoregression which is composed of several segments time epochs each of which modeled by an autoregressive model we propose a multiwindow method that is both effective and efficient for discovering the structural changes the proposed approach was motivated by transforming a segment wise autoregression into a multivariate time series that is asymptotically segment wise independent and identically distributed to address the second challenge we derive theoretical guarantees for almost surely selecting the true number of change points of segment wise independent multivariate time series specifically under mild assumptions we show that a bayesian information criterion like criterion gives a strongly consistent selection of the optimal number of change points while an akaike information criterion like criterion cannot finally we demonstrate the theory and strength of the proposed algorithms by experiments on both synthetic and real world data including the eastern u s temperature data and the el nino data the experiment leads to some interesting discoveries about temporal variability of the summer time temperature over the eastern u s and about the most dominant factor of ocean influence on climate which were also discovered by environmental scientists

all controllers for the general h control problem lmi existence conditions and state space formulas this paper presents all controllers for the general h control problem with no assumptions on the plant matrices necessary and sufficient conditions for the existence of an h controller of any order are given in terms of three linear matrix inequalities lmis our existence conditions are equivalent to scherer s results but with a more elementary derivation furthermore we provide the set of all h controllers explicitly parametrized in the state space using the positive definite solutions to the lmis even under standard assumptions full rank etc our controller parametrization has an advantage over the q parametrization the freedom q a real rational stable transfer matrix with the h norm bounded above by a specified number is replaced by a constant matrix l of fixed dimension with a norm bound and the solutions x y to the lmis the inequality formulation converts the existence conditions to a convex feasibility problem and also the free matrix l and the pair x y define a finite dimensional design space as opposed to the infinite dimensional space associated with the q parametrization

a modified likelihood ratio test for homogeneity in finite mixture models summary testing for homogeneity in finite mixture models has been investigated by many researchers the asymptotic null distribution of the likelihood ratio test lrt is very complex and difficult to use in practice we propose a modified lrt for homogeneity in finite mixture models with a general parametric kernel distribution family the modified lrt has a x2 type of null limiting distribution and is asymptotically most powerful under local alternatives simulations show that it performs better than competing tests they also reveal that the limiting distribution with some adjustment can satisfactorily approximate the quantiles of the test statistic even for moderate sample sizes

testing for observation dependent regime switching in mixture autoregressive models abstract testing for regime switching when the regime switching probabilities are specified either as constants mixture models or are governed by a finite state markov chain markov switching models are long standing problems that have also attracted recent interest this paper considers testing for regime switching when the regime switching probabilities are time varying and depend on observed data observation dependent regime switching specifically we consider the likelihood ratio test for observation dependent regime switching in mixture autoregressive models the testing problem is highly nonstandard involving unidentified nuisance parameters under the null parameters on the boundary singular information matrices and higher order approximations of the log likelihood we derive the asymptotic null distribution of the likelihood ratio test statistic in a general mixture autoregressive setting using high level conditions that allow for various forms of dependence of the regime switching probabilities on past observations and we illustrate the theory using two particular mixture autoregressive models the likelihood ratio test has a nonstandard asymptotic distribution that can easily be simulated and monte carlo studies show the test to have good finite sample size and power properties

semiparametric maximum likelihood for measurement error model regression this paper presents an em algorithm for semiparametric likelihood analysis of linear generalized linear and nonlinear regression models with measurement errors in explanatory variables a structural model is used in which probability distributions are specified for a the response and b the measurement error a distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood for various types of extra information about the measurement error distribution the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of a and b if the true x were available simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption the approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution

residual diagnostics for mixture models abstract a sample is commonly modeled by a mixture distribution if the observations follow a common distribution but the parameter of interest differs between observations for example we observe the lengths but not the ages of a sample offish it may be reasonable to assume that length is normally distributed about an unknown mean that depends on the age of the fish provided there is more than one age class in the sample then the data are distributed as a mixture of normals in this article we assume that the data are a random sample from a mixture of exponential family distributions and that for each observation the parameter of interest is sampled independently from an unknown mixing distribution q the adequacy of a fitted mixture model can be assessed by examining residuals based on the ratio of the observed to expected fit residuals based on the homogeneity model in which q is a one point distribution display a convexity property when the data follow a mixture model this becomes the basis fo

optimal rate of convergence for finite mixture models in finite mixture models we establish the best possible rate of convergence for estimating the mixing distribution we find that the key for estimating the mixing distribution is the knowledge of the number of components in the mixture while a sqrt n consistent rate is achievable when the exact number of components is known the best possible rate is only n 1 4 when it is unknown under a strong identifiability condition it is shown that this rate is reached by some minimum distance estimators most commonly used models are found to satisfy the strong identifiability condition

testing homogeneity in a mixture distribution via the l2 distance between competing models ascertaining the number of components in a mixture distribution is an interesting and challenging problem for statisticians chen chen and kalbfleisch recently proposed a modified likelihood ratio test mlrt which is distribution free and locally most powerful asymptotically in this article we present a new method for testing whether a finite mixture distribution is homogeneous our method the d test is based on the l2 distance between a fitted homogeneous model and a fitted heterogeneous model for mixture components from standard parametric families the d test statistic has a closed form expression in terms of parameter estimators whereas likelihood ratio type test statistics do not the latter test statistics are nontrivial functions of both the parameter estimators and the full dataset the convergence rates of the d test statistic under a null hypothesis of homogeneity and an alternative hypothesis of heterogeneity are established the d test is shown to be competitive with the mlrt when th

multisurface proximal support vector machine classification via generalized eigenvalues a new approach to support vector machine svm classification is proposed wherein each of two data sets are proximal to one of two distinct planes that are not parallel to each other each plane is generated such that it is closest to one of the two data sets and as far as possible from the other data set each of the two nonparallel proximal planes is obtained by a single matlab command as the eigenvector corresponding to a smallest eigenvalue of a generalized eigenvalue problem classification by proximity to two distinct nonlinear surfaces generated by a nonlinear kernel also leads to two simple generalized eigenvalue problems the effectiveness of the proposed method is demonstrated by tests on simple examples as well as on a number of public data sets these examples show the advantages of the proposed approach in both computation time and test set correctness

model selection via multifold cross validation a natural extension of the simple leave one out cross validation cv method is to allow the deletion of more than one observations in this article several notions of the multifold cross validation mcv method have been discussed in the context of variable selection under a linear regression model we show that the delete d mcv criterion is asymptotically equivalent to the well known fpe criterion two computationally more feasible methods the r fold cross validation and the repeated learning testing criterion are also studied the performance of these criteria are compared with the simple leave one out cross validation method simulation results are obtained to gain some understanding on the small sample properties of these methods

testing the number of components in normal mixture regression models testing the number of components in finite normal mixture models is a long standing challenge because of its nonregularity this article studies likelihood based testing of the number of components in normal mixture regression models with heteroscedastic components we construct a likelihood based test of the null hypothesis of m0 components against the alternative hypothesis of m0 1 components for any m0 the null asymptotic distribution of the proposed modified em test statistic is the maximum of m0 random variables that can be easily simulated the simulations show that the proposed test has very good finite sample size and power properties supplementary materials for this article are available online

on the efficiency of score tests for homogeneity in two component parametric models for discrete data in many applications of two component mixture models for discrete data such as zero inflated models it is often of interest to conduct inferences for the mixing weights score tests derived from the marginal model that allows for negative mixing weights have been particularly useful for this purpose but the existing testing procedures often rely on restrictive assumptions such as the constancy of the mixing weights and typically ignore the structural constraints of the marginal model in this article we develop a score test of homogeneity that overcomes the limitations of existing procedures the technique is based on a decomposition of the mixing weights into terms that have an obvious statistical interpretation we exploit this decomposition to lay the foundation of the test simulation results show that the proposed covariate adjusted test statistic can greatly improve the efficiency over test statistics based on constant mixing weights a real life example in dental caries research is used to illustrate the methodology

minimum hellinger distance estimation for finite mixture models abstract minimum hellinger distance estimates are considered for finite mixture models when the exact forms of the component densities are unknown in detail but are thought to be close to members of some parametric family minimum hellinger distance estimates are asymptotically efficient if the data come from a member of the parametric family and are robust to certain departures from the parametric family a new algorithm is introduced that is similar to the em algorithm a specialized adaptive density estimate is also introduced standard measures of robustness are discussed some difficulties are noted the robustness and asymptotic efficiency of the estimators are illustrated using simulations

overdispersion diagnostics for generalized linear models abstract generalized linear models glm s are simple convenient models for count data but they assume that the variance is a specified function of the mean although overdispersed glm s allow more flexible mean variance relationships they are often not as simple to interpret nor as easy to fit as standard glm s this article introduces a convexity plot or c plot for short that detects overdispersion and relative variance curves and relative variance tests that help to understand the nature of the overdispersion convexity plots sometimes detect overdispersion better than score tests and relative variance curves and tests sometimes distinguish the source of the overdispersion better than score tests

inference on the order of a normal mixture finite normal mixture models are used in a wide range of applications hypothesis testing on the order of the normal mixture is an important yet unsolved problem existing procedures often lack a rigorous theoretical foundation many are also hard to implement numerically in this article we develop a new method to fill the void in this important area an effective expectation maximization em test is invented for testing the null hypothesis of arbitrary order m 0 under a finite normal mixture model for any positive integer m 0 2 the limiting distribution of the proposed test statistic is we also use a novel computer experiment to provide empirical formulas for the tuning parameter selection the finite sample performance of the test is examined through simulation studies real data examples are provided the procedure has been implemented in r code the p values for testing the null order of m 0 2 or m 0 3 can be calculated with a single command this article has supplementary materials avail

hidden markov models with applications in cell adhesion experiments estimation of the number of hidden states is challenging in hidden markov models motivated by the analysis of a specific type of cell adhesion experiments a new framework based on a hidden markov model and double penalized order selection is proposed the order selection procedure is shown to be consistent in estimating the number of states a modified expectation maximization algorithm is introduced to efficiently estimate parameters in the model simulations show that the proposed framework outperforms existing methods applications of the proposed methodology to real data demonstrate the accuracy of estimating receptor ligand bond lifetimes and waiting times which are essential in kinetic parameter estimation supplementary materials for this article are available online

unified maximum likelihood estimates for closed capture recapture models using mixtures summary agresti 1994 biometrics50 494 500 and norris and pollock 1996a biometrics52 639 649 suggested using methods of finite mixtures to partition the animals in a closed capture recapture experiment into two or more groups with relatively homogeneous capture probabilities this enabled them to fit the models mh mbh norris and pollock and mth agresti of otis et al 1978 wildlife monographs62 1 135 in this article finite mixture partitions of animals and or samples are used to give a unified linear logistic framework for fitting all eight models of otis et al by maximum likelihood likelihood ratio tests are available for model comparisons for many data sets a simple dichotomy of animals is enough to substantially correct for heterogeneity induced bias in the estimation of population size although there is the option of fitting more than two groups if the data warrant it

bayesian model selection in finite mixtures by marginal density decompositions we consider the problem of estimating the number of components d and the unknown mixing distribution in a finite mixture model in which d is bounded by some fixed finite number n our approach relies on the use of a prior over the space of mixing distributions with at most n components by decomposing the resulting marginal density under this prior we discover a weighted bayes factor method for consistently estimating d that can be implemented by an iid generalized weighted chinese restaurant gwcr monte carlo algorithm we also discuss a gibbs sampling method the blocked gibbs sampler for estimating d and also the mixing distribution we show that our resulting posterior is consistent and achieves the frequentist optimal op n 1 4 rate of estimation we compare the performance of the new gwcr model selection procedure with that of the akaike information criterion and the bayes information criterion implemented through an em algorithm applications of our methods to five real datasets and simulation

consistent estimation of mixture complexity the consistent estimation of mixture complexity is of fundamental importance in many applications of finite mixture models an enormous body of literature exists regarding the application computational issues and theoretical aspects of mixture models when the number of components is known but estimating the unknown number of components remains an area of intense research effort this article presents a semiparametric methodology yielding almost sure convergence of the estimated number of components to the true but unknown number of components the scope of application is vast as mixture models are routinely employed across the entire diverse application range of statistics including nearly all of the social and experimental sciences

hypothesis testing in mixture regression models summary we establish asymptotic theory for both the maximum likelihood and the maximum modified likelihood estimators in mixture regression models moreover under specific and reasonable conditions we show that the optimal convergence rate of n 1 4 for estimating the mixing distribution is achievable for both the maximum likelihood and the maximum modified likelihood estimators we also derive the asymptotic distributions of two log likelihood ratio test statistics for testing homogeneity and we propose a resampling procedure for approximating the p value simulation studies are conducted to investigate the empirical performance of the two test statistics finally two real data sets are analysed to illustrate the application of our theoretical results

information ratios for validating mixture analyses abstract determining the number of components in a mixture of distributions is an important but difficult problem this article introduces a procedure called minimum information ratio estimation and validation mirev which is based on a ratio of fisher information matrices the smallest eigenvalue of the information ratio matrix is used to determine the number of components a measure of uncertainty may be obtained using a bootstrap technique simulations illustrate the effectiveness of the procedure for mixtures of exponential families an expression for the observed information ratio matrix provides insight to the success of the procedure cluster analysis attempts to identify and characterize subpopulations believed to be present in a population a wide variety of methods are available including criterion optimization hierarchical methods and various heuristic methods criterion optimization techniques such as mixture analysis fuzzy clustering and partitioning methods are popular because they