tensor notation and cumulants of polynomials a modified and extended tensor notation is introduced that is sufficient to cover multivariate moments and cumulants as special cases using this notation two basic identities are given the first of these expresses generalized cumulants in terms of ordinary cumulants the second gives the joint cumulant generating function of any polynomial transformation in terms of the cumulants of the original variables three applications of the basic identities are given the first application is concerned with sample cumulants or k statistics the second to edgeworth series and the third to exponential family models

arbitrary norm support vector machines support vector machines svm are state of the art classifiers typically l2 norm or l1 norm is adopted as a regularization term in svms while other norm based svms for example the l0 norm svm or even the l norm svm are rarely seen in the literature the major reason is that l0 norm describes a discontinuous and nonconvex term leading to a combinatorially np hard optimization problem in this letter motivated by bayesian learning we propose a novel framework that can implement arbitrary norm based svms in polynomial time one significant feature of this framework is that only a sequence of sequential minimal optimization problems needs to be solved thus making it practical in many real applications the proposed framework is important in the sense that bayesian priors can be efficiently plugged into most learning methods without knowing the explicit form hence this builds a connection between bayesian learning and the kernel machines we derive the theoretical framework demonstrate how our approach works on the l0 norm svm as a typical example and perform a series of experiments to validate its advantages experimental results on nine benchmark data sets are very encouraging the implemented l0 norm is competitive with or even better than the standard l2 norm svm in terms of accuracy but with a reduced number of support vectors 9 46 of the number on average when compared with another sparse model the relevance vector machine our proposed algorithm also demonstrates better sparse properties with a training speed over seven times faster

saddlepoint approximations for estimating equations summary the saddlepoint method is used to approximate to the distribution of an estimator defined by an estimating equation two different approaches are available one of which is shown to be equivalent to the technique of field hampel 1982 two recent formulae for the tail probability due to lugannani rice 1980 and to robinson 1982 respectively which are uniformly accurate over the whole range of the estimator are compared numerically with the exact results and those computed by field hampel they are found to be of comparable accuracy while avoiding the use of numerical integration the most accurate is that of lugannani rice hampel 1974 introduced a new technique for approximating to the probability density of an estimator defined by an estimating equation it is an example of what he called small sample asymptotics where high accuracy is achieved for quite small sample sizes n even down to single figures in the original version it gave an approximation to the logarithmic derivative of the density function which was integrated numerically to get the density the distribution function was obtained by a second numerical integration which could then be used to renormalize both the density and the distribution function field hampel 1982 develop the technique in detail and compare its performance with that of other approximation methods as hampel had pointed out his approach is closely related to the saddlepoint method of daniels 1954 which was applied to sample means and ratios of means following the private communication referred to by field hampel 1982 p 31 it was realized that the first numerical integration was unnecessary and that hampel s approach could be shortened to give a direct approximation to the density which is in fact a saddlepoint approximation the purpose of the present paper is to extend the use of the saddlepoint method to estimating equations there are two distinct ways of doing this which lead to different approximations of similar accuracy one appears to be more convenient for approxi mating to tail probabilities by numerical integration and was used to compute the saddlepoint approximations quoted in field hampel s tables 1 and 2 the other gives the density directly in a form equivalent to their equation 4 3 which is then integrated

approximations of marginal tail probabilities and inference for scalar parameters summary in many situations inference for a scalar parameter in the presence of nuisance parameters requires integration of either a joint density of pivotal quantities or a joint posterior density for such inference accurate approximations of marginal tail probabilities are useful to avoid high dimensional integrals two tail probability approximations are developed in this paper numerical results given for conditional inference in location scale and linear regression models show the approximations to be generally accurate even for small sample sizes

finding maximum likelihood estimates of patterned covariance matrices by the em algorithm some patterned covariance matrices used to model multivariate normal data that do not have explicit maximum likelihood estimates can be viewed as submatrices of larger patterned covariance matrices that do have explicit maximum likelihood estimates in such cases the smaller covariance matrix can be viewed as the covariance matrix for observed variables and the larger covariance matrix can be viewed as the covariance matrix for both observed and missing variables the advantage of this perspective is that the em algorithm can be used to calculate the desired maximum likelihood estimates for the original problem two examples are presented

randomization distributions and saddlepoint approximations in generalized linear models various randomization distributions are shown to arise as conditional distributions in the setting of generalized linear models these conditional distributions admit saddlepoint approximations which can obviate the need to simulate the randomization distributions they approximate examples include multinominal and dirichlet bootstraps the jackknife and permutation distributions the double saddlepoint approximation in a regular exponential family is shown to be equivalent to conditional use of a single saddlepoint approximation when the exponential family admits to a cut

generalized monte carlo significance tests summary simple monte carlo significance testing has many applications particularly in the preliminary analysis of spatial data the method requires the value of the test statistic to be ranked among a random sample of values generated according to the null hypothesis however there are situations in which a sample of values can only be conveniently generated using a markov chain initiated by the observed data so that independence is violated this paper describes two methods that overcome the problem of dependence and allow exact tests to be carried out the methods are applied to the rasch model to the finite lattice ising model and to the testing of association between spatial processes power is discussed in a simple case

saddlepoint approximations for regression models summary this paper uses the techniques of saddlepoint or tilted exponential approximation to develop an approximation to the small sample distribution of estimators defined by a system of estimating equations when observations are independently but not identically distributed this allows for the explicit treatment of models with explanatory variables the approximation is applied to l1 regression and the accuracy of the approximation is demonstrated in a case in which the usual normal approximation is very poor an analysis of the example indicates the usefulness of a new variant of exponential tilting in calculating the transformation measure a quantity which plays an important role in assuring the accuracy of the approximation

berry esseen theorems for quadratic forms of gaussian stationary processes letx1 x2 be a gaussian stationary process with spectral densityf and let z j frac 1 sqrt t left sum limits l 1 t j 1 left x l x l j 1 ex l x l j 1 right right j 1 p the present paper gives a berry esseen type theorem for the joint distribution of z1 zp

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

nonsmooth analysis and frechet differentiability of m functionals abstract a necessary requirement for existence of the frechet derivative is that the defining psi function is uniformly bounded and this naturally excludes those nonrobust estimators such as the maximum likelihood estimator in normal parametric models in this paper the methods of nonsmooth analysis described in the book by f h clarke 1983 are introduced to the theory of statistical expansions and are used here in the proofs of weak continuity and frechet differentiability of m functionals subsequently the conditions for frechet differentiability given in clarke 1983 can be relaxed to include most popular m functionals additional keywords distribution functions m estimators robustness gross error sensitivity weak continuity asymptotic expansions asymptotic normality selection functional local uniqueness author

the isserlis matrix and its application to non decomposable graphical gaussian models the operations of matrix completion and isserlis matrix construction arise in the statistical analysis of graphical gaussian models in both the bayesian and frequentist approaches in this paper we study the properties of the isserlis matrix of the completion of a positive definite matrix and propose an edge set indexing notation which highlights the symmetry existing between and its isserlis matrix in this way well known properties of can be exploited to give an easy proof of certain asymptotic results for decomposable graphical gaussian models as well as to extend such results to the non decomposable case in particular we consider the asymptotic variance of maximum likelihood estimators the non informative jeffreys prior the laplace approximation to the bayes factor the asymptotic distribution of the maximum likelihood estimators and the asymptotic posterior distribution of the parameters in a bayesian conjugate analysis in dealing with distributions over non decomposable models an extension of the hyper markov property to models with non chordal graphs is required our proposed extension is justified by an analysis of the conditional independence structure of the sufficient statistic

practical use of higher order asymptotics for multiparameter exponential families recently developed asymptotics based on saddlepoint methods provide important practical methods for multiparameter exponential families especially in generalized linear models the aim here is to clarify and explore these attention is restricted to tests and confidence intervals regarding a single parametric function which can be represented as a natural parameter of a full rank exponential family excellent approximations to exact conditional inferences are often available in terms of simple adjustments to the signed square root of the likelihood ratio statistic

ignorable and informative designs in survey sampling inference summary the role of the sample selection mechanism in a model based approach to finite population inference is examined when the data analyst has only partial information on the sample design then a design which is ignorable when known fully may become informative conditions under which partially known designs can be ignored are established and examined for some standard designs the results are illustrated by an example used by scott 1977

the density of the maximum likelihood estimator in this paper we derive an expression for the exact density of the maximum likelihood estimator mle in the case where the mle is uniquely defined at each point in the sample space by the vanishing of the score vector

structural minimax probability machine minimax probability machine mpm is an interesting discriminative classifier based on generative prior knowledge it can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification the structural information of data is an effective way to represent prior knowledge and has been found to be vital for designing classifiers in real world problems however mpm only considers the prior probability distribution of each class with a given mean and covariance matrix which does not efficiently exploit the structural information of data in this paper we use two finite mixture models to capture the structural information of the data from binary classification for each subdistribution in a finite mixture model only its mean and covariance matrix are assumed to be known based on the finite mixture models we propose a structural mpm smpm smpm can be solved effectively by a sequence of the second order cone programming problems moreover we extend a linear model of smpm to a nonlinear model by exploiting kernelization techniques we also show that the smpm can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi min margin machine under certain special conditions experimental results on both synthetic and real world data sets demonstrate the effectiveness of smpm

on the bootstrap saddlepoint approximations we compare saddlepoint approximations to the exact distributions of a studentized mean and to its bootstrap approximation we show that on bounded sets these empirical saddlepoint approximations achieve second order relative errors uniformly we also consider the relative errors for larger deviations it follows that the studentized t bootstrap p value and the coverage of the bootstrap confidence interval have second order relative errors

interpolation rules suggested by asymptotic expansions summary we suggest a simple interpolation rule based on edgeworth and cornish fisher expansions which permits highly accurate numerical approximation in problems involving calculation of distributions or quantiles without requiring explicit calculation of terms in the expansions edgeworth expansions are often used to provide explicit corrections for asymptotic approxima tions as in recent work in testing and confidence procedures involving nonparametric methods pfanzagl 1979 hall 1983 abramovitch singh 1985 and in time series econometrics and related fields sargan 1976 phillips 1978 durbin 1980 taniguchi 1985 taniguchi maekawa 1990 in the present note we point out a useful application of edgeworth expansion methods in producing simple but accurate interpolation rules for deriving information from or for constructing statistical tables suppose the function f is tabulated for discrete values ni of its argument and that we seek the value off for an untabulated point say n we suggest that shouldf admit an edgeworth expansion in increasing powers of n 2 or n1 then interpolation should be among values of nwf nj or nf ni respectively rather than values of f ni this recommendation applies to approximations of both distribution functions and percentage points in the latter case our reference to edgeworth expansions should really be to cornish fisher expansions we should stress that our method does not require calculation of any terms of an edgeworth or cornish fisher expansion it uses only existing information available in statistical tables and knowledge that the edgeworth expansion exists this existence has been established for large classes of statistical problems see for example sargan 1975 phillips 1977 bhattacharya ghosh 1978 durbin 1980 and sargan satchell 1986 aside from advantages of greater accuracy compared with traditional interpolation rules our method allows linear and quadratic

approximately bayesian inference abstract consider statistical inference about a scalar parameter and suppose that information about that parameter is to be summarized by a system of interval estimates it is well known that methods of interval estimation that do not correspond to bayesian inference with respect to some prior distribution have some logical difficulties this article proposes a measure of how close a system of interval estimates is to being of bayesian form using this measure several non bayesian methods of interval estimation are analyzed these results are illustrated on several examples

the likelihood ratio approximation to the conditional distribution of the maximum likelihood estimator in the discrete case summary the likelihood ratio approximation also called barndorff nielsen s approximation and often denoted by p provides a highly accurate approximation to the conditional density of a maximum likelihood estimator 0 given an ancillary statistic in this paper the properties of p are considered for the case in which the underlying random variables have a lattice distribution and 0 has a discrete but not necessarily lattice distribution if 0 has a lattice distribution then p provides a valid approximation to the density of 0 with respect to counting measure if the distribution of 0 is non lattice then p is still a valid approximation to the conditional density of 0 however the dominating measure is no longer counting measure

asymptotic expansions for sums of weakly dependent random vectors it is shown that formal edgeworth expansions are valid for sums of weakly dependent random vectors the error of approximation has ordero n s 2 2 if the strong mixing coefficients in iii are decreasing at an exponential rate the above conditions can easily be checked and are often satisfied when the sequence of random vectors is a gaussian or a markov or an autoregressive process explicit formulas are given for the distribution of finite fourier transforms of a strictly stationary time series

saddlepoint approximations for short and long memory time series a frequency domain approach abstract saddlepoint techniques provide numerically accurate small sample approximations to the distribution of estimators and test statistics except for a few simple models these approximations are not available in the framework of stationary time series we contribute to fill this gap under short or long range serial dependence for gaussian and non gaussian processes we show how to derive and implement saddlepoint approximations for two relevant classes of frequency domain statistics ratio statistics and whittle s estimator we compare our new approximations to the ones obtained by the standard asymptotic theory and by two widely applied bootstrap methods the numerical exercises for whittle s estimator show that our approximations yield accuracy s improvements while preserving analytical tractability a real data example concludes the paper

exact and higher order properties of the mle in spatial autoregressive models with applications to inference abstract the quasi maximum likelihood estimator for the autoregressive parameter in a spatial autoregression usually cannot be written explicitly in terms of the data a rigorous analysis of the first order asymptotic properties of the estimator under some assumptions on the evolution of the spatial design matrix is available in lee 2004 but very little is known about its exact or higher order properties in this paper we first show that the exact cumulative distribution function of the estimator can under mild assumptions be written in terms of that of a particular quadratic form simple examples are used to illustrate important exact properties of the estimator that follow from this representation in general models a complete exact analysis is not possible but a higher order saddlepoint approximation is made available by the main result we use this approximation to construct confidence intervals for the autoregressive parameter coverage properties of the proposed confidence intervals are studied by monte carlo simulation and are found to be excellent in a variety of circumstances

estimation in generalized linear models with random effects summary a conceptually very simple but general algorithm for the estimation of the fixed effects random effects and components of dispersion in generalized linear models with random effects is proposed conditions are described under which the algorithm yields approximate maximum likelihood or quasi maximum likelihood estimates of the fixed effects and dispersion components and approximate empirical bayes estimates of the random effects the algorithm is applied to two data sets to illustrate the estimation of components of dispersion and the modelling of overdispersion

saddlepoint approximations for distributions of quadratic forms in normal variables reference stat article 1999 001doi 10 1093 biomet 86 4 929view record in web of science record created on 2006 04 21 modified on 2017 05 12

approximations for densities of sufficient estimators abstract a simple method of obtaining asymptotic expansions for the densities of sufficient estimators is described it is an extension of the one developed by o barndorff nielsen and d r cox 1979 for exponential families a series expansion in powers of n 1 is derived of which the first term has an error of order n 1 which can effectively be reduced to n 3 2 by renormalization the results obtained are similar to those given by h e daniels s 1954 saddlepoint method but the derivations are simpler a brief treatment of approximations to conditional densities is given theorems are proved which extend the validity of the multivariate edgeworth expansion to parametric families of densities of statistics which need not be standardized sums of independent and identically distributed vectors these extensions permit the treatment of problems arising in time series analysis the technique is used by j durbin 1980 to obtain approximations to the densities of partial serial correlation coefficients

properties of sufficiency and statistical tests 1 in a previous paper dealing with the importance of properties of sufficiency in the statistical theory of small samples attention was mainly confined to the theory of estimation in the present paper the structure of small sample tests whether these are related to problems of estimation and fiducial distributions or are of the nature of tests of goodness of fit is considered further

saddlepoint approximations in resampling methods summary saddlepoint approximations are shown to be easy to use and accurate in a variety of simple bootstrap and randomization applications examples include mean estimation ratio estimation two sample comparisons and autoregressive estimation

a theorem of validity for edgeworth expansisons on considere des equations autoregressives a variables exogenes et un terme de bruit blanc normal ce qui generalise les modeles dynamiques etudies precedemment

higher order improvements of the parametric bootstrap for long memory gaussian processes abstract this paper determines coverage probability errors of both delta method and parametric bootstrap confidence intervals cis for the covariance parameters of stationary long memory gaussian time series cis for the long memory parameter d 0 are included the results establish that the bootstrap provides higher order improvements over the delta method analogous results are given for tests the cis and tests are based on one or other of two approximate maximum likelihood estimators the first estimator solves the first order conditions with respect to the covariance parameters of a plug in log likelihood function that has the unknown mean replaced by the sample mean the second estimator does likewise for a plug in whittle log likelihood the magnitudes of the coverage probability errors for one sided bootstrap cis for covariance parameters for long memory time series are shown to be essentially the same as they are with iid data this occurs even though the mean of the time series cannot be estimated at the usual n 1 2 rate

efficient conditional tests for mixture experiments with applications to the birth and branching processes summary conditional tests which are efficient in the sense of being uniformly most powerful unbiased for any sample size and are also asymptotically efficient with reference to a relevant likelihood in the sense of maximizing asymptotic power at alternatives converging to the null hypothesis are derived for mixture experiments involving dependent observations as simple applications efficient tests in birth processes branching processes stable additive processes etc are derived

regression and time series model selection in small samples summary a bias correction to the akaike information criterion aic is derived for regression and autoregressive time series models the correction is of particular use when the sample size is small or when the number of fitted parameters is a moderate to large fraction of the sample size the corrected method called aicc is asymptotically efficient if the true model is infinite dimensional furthermore when the true model is of finite dimension aicc is found to provide better model order choices than any other asymptotically efficient method applications to nonstationary autoregressive and mixed autoregressive moving average time series models are also discussed

conjugate relation between loss functions and uncertainty sets in classification problems there are two main approaches to binary classification problems the loss function approach and the uncertainty set approach the loss function approach is widely used in real world data analysis statistical decision theory has been used to elucidate its properties such as statistical consistency conditional probabilities can also be estimated by using the minimum solution of the loss function in the uncertainty set approach an uncertainty set is defined for each binary label from training samples the best separating hyperplane between the two uncertainty sets is used as the decision function although the uncertainty set approach provides an intuitive understanding of learning algorithms its statistical properties have not been sufficiently studied in this paper we show that the uncertainty set is deeply connected with the convex conjugate of a loss function on the basis of the conjugate relation we propose a way of revising the uncertainty set approach so that it will have good statistical properties such as statistical consistency we also introduce statistical models corresponding to uncertainty sets in order to estimate conditional probabilities finally we present numerical experiments verifying that the learning with revised uncertainty sets improves the prediction accuracy

applications of second order cone programming in a second order cone program socp a linear function is minimized over the intersection of an affine set and the product of second order quadratic cones socps are nonlinear convex problems that include linear and convex quadratic programs as special cases but are less general than semidefinite programs sdps several efficient primaldual interior point methods for socp have been developed in the last few years after reviewing the basic theory of socps we describe general families of problems that tan be recast as socps these include robust linear programming and robust leastsquares problems problems involving sums or maxima of norms or with convex hyperbolic constraints we discuss a variety of engineering applications such as filter design antenna array weight design truss design and grasping forte optimization in robotics we describe an efficient primaldual interior point method for solving socps which shares many of the features of primaldual interior point methods for linear program

assessing the accuracy of normal approximations abstract the widespread applicability and use of normal approximations creates a need for methods for assessing their accuracy in an operational fashion in this article two new methods are proposed and exemplified these new methods are based on a comparison of the level curves of an exact likelihood with the level curves of the usual normal approximation to it and on a comparison of selected line integrals of an exact density and a normal approximation to it the operational usefulness of these methods is compared with the operational usefulness of two well known existing approaches to assessing the accuracy of approximations namely convergence rates and the curvature methods of bates and watts 1980 as an example the second method is applied to the lee geisser 1972 1975 approximation to the predictive distribution of dental measurements on girls at the ages of 8 10 12 and 14 years in this application the method shows that the 50 central predictive probability region has actual predicti

intrinsic autoregressions and related models on the two dimensional lattice summary stationary autoregressions on a two dimensional lattice are generalized to intrinsic models where only increments are assumed to be stationary prediction formulae and the asymptotic behaviour of the semivariogram are derived for parameter estimation we propose an approximate maximum likelihood estimator a generalization of whittle s estimator it is derived also for general intrinsic models

model checking with nonparametric curves summary the use of local likelihood curve estimates in model checking is described test statistics are based on deviance differences and are interpreted using nonparametric bootstrap monte carlo some theoretical aspects are discussed methods are illustrated for regression with discrete data

higher order infinitesimal robustness using the von mises expansion we study the higher order infinitesimal robustness of a general m functional and characterize its second order properties we show that second order robustness is equivalent to the boundedness of both the estimator s estimating function and its derivative with respect to the parameter it implies at the same time i variance robustness and ii robustness of higher order saddlepoint approximations to the estimator s finite sample density the proposed construction of second order robust m estimators is fairly general and potentially useful in a variety of relevant settings besides the theoretical contributions we discuss the main computational issues and provide an algorithm for the implementation of second order robust m estimators finally we illustrate our theory by monte carlo simulation and in a real data estimation of the maximal losses of nikkei 225 index returns our findings indicate that second order robust estimators can improve on other widely applied robus

practical small sample asymptotics for regression problems abstract saddlepoint approximations are derived for sums of independent but not necessarily identically distributed random variables along with generalizations to estimating equations and multivariate problems these results are particularly useful for accurately approximating the distribution of regression coefficients general formulas are given for the distribution of the coefficients arising out of a generalized linear model with both canonical and noncanonical link functions we illustrate the case of logistic regression with a real data example and show how the gibbs sampler may be used to obtain confidence sets for each regression parameter based on the saddlepoint approximation

hyperellipsoidal statistical classifications in a reproducing kernel hilbert space standard support vector machines svms have kernels based on the euclidean distance this brief extends standard svms to svms with kernels based on the mahalanobis distance the extended svms become a special case of the euclidean distance when the covariance matrix in a reproducing kernel hilbert space is degenerated to an identity the mahalanobis distance leads to hyperellipsoidal kernels and the euclidean distance results in hyperspherical ones in this brief the mahalanobis distance based kernel in a reproducing kernel hilbert space is developed systematically extensive experiments demonstrate that the hyperellipsoidal kernels slightly outperform the hyperspherical ones with fewer svs

a robust minimax approach to classification when constructing a classifier the probability of correct classification of future data points should be maximized we consider a binary classification problem where the mean and covariance matrix of each class are assumed to be known no further assumptions are made with respect to the class conditional distributions misclassification probabilities are then controlled in a worst case setting that is under all possible choices of class conditional densities with given mean and covariance matrix we minimize the worst case maximum probability of misclassification of future data points for a linear decision boundary this desideratum is translated in a very direct way into a convex second order cone optimization problem with complexity similar to a support vector machine problem the minimax problem can be interpreted geometrically as minimizing the maximum of the mahalanobis distances to the two classes we address the issue of robustness with respect to estimation errors in the means and covariances of the classes via a simple modification of the input data we also show how to exploit mercer kernels in this setting to obtain nonlinear decision boundaries yielding a classifier which proves to be competitive with current methods including support vector machines an important feature of this method is that a worst case bound on the probability of misclassification of future data is always obtained explicitly

general saddlepoint approximations with applications to l statistics abstract saddlepoint approximations are extended to general statistics the technique is applied to derive approximations to the density of linear combinations of order statistics including trimmed means a comparison with exact results shows the accuracy of these approximations even in very small sample sizes

on bartlett adjustments for approximate bayesian inference summary in wide generality the posterior distributions of the likelihood ratio statistic and the posterior ratio statistic are chi squared to error of order o n where n is sample size the error in the chi squared approximation can be reduced to order o n 2 by bartlett correction in this paper explicit formulae are derived for the bartlett adjustment factors of both statistics and the derivations are based on the tierney kass kadane 1989 asymptotic approximation for marginal posterior probability density functions the use of numerical differentiation to facilitate calculation of the bartlett adjustments is also described some applications are considered that concern inference about regression models from both complete and right censored data

biometrika centenary nonparametrics summary contributions in biometrika to nonparametric statistics are surveyed with particular emphasis on more recent work it is argued that even in the journal s very early years it had interests in smoothing methods for density estimation and in techniques for nonparametric regression in addition to its more obvious focus on classical nonparametric statistics such as ranking methods biometrika s early involvement with permutation testing has links to its contemporary work on bootstrap methods emphases on different types of nonparametric statistics have evolved over time however a rapidly increasing proportion of the journal s pages is devoted to papers that include some use of nonparametric ideas or methods

optimal inequalities in probability theory a convex optimization approach we propose a semidefinite optimization approach to the problem of deriving tight moment inequalities for p x in s for a set s defined by polynomial inequalities and a random vector x defined on omega subseteq cal r n that has a given collection of up to kth order moments in the univariate case we provide optimal bounds on p x in s when the first k moments of x are given as the solution of a semidefinite optimization problem in k 1 dimensions in the multivariate case if the sets s and omega are given by polynomial inequalities we obtain an improving sequence of bounds by solving semidefinite optimization problems of polynomial size in n for fixed k we characterize the complexity of the problem of deriving tight moment inequalities we show that it is np hard to find tight bounds for k geq 4 and omega cal r n and for k geq 2 and omega cal r n when the data in the problem is rational for k 1 and omega cal r n we show that we can find tight upper bounds by solving n convex optimization problems when the set s is convex and we provide a polynomial time algorithm when s and omega are unions of convex sets over which linear functions can be optimized efficiently for the case k 2 and omega cal r n we present an efficient algorithm for finding tight bounds when s is a union of convex sets over which convex quadratic functions can be optimized efficiently

empirical saddlepoint approximations for multivariate m estimators summary in this paper we investigate the use of the empirical distribution function in place of the underlying distribution function f to construct an empirical saddlepoint approximation to the density fn of a general multivariate m estimator we obtain an explicit form for the error term in the approximation investigate the effect of renormalizing the estimator carry out some numerical comparisons and discuss the regression problem the sampling distribution of a statistic is a basic tool for evaluating the properties of the statistic and for constructing inference procedures based on the statistic the density of the sampling distribution may be used to compute moments to develop approximations or to compute the distribution function and hence various tail prob abilities which can be used to construct tests or confidence intervals when the exact density is intractable it may be possible to approximate it by a normal density an edgeworth expansion or a saddlepoint approximation daniels 1954 the saddle point approximation is derived from an asymptotic expansion but is often very accurate even in quite small samples to highlight this property hampel 1973 described these and related techniques as small sample asymptotic approximations as can be seen from equation 3 later the main property of this approximation and its major advantage over edgeworth expansions is that it is always non negative and the relative error is uniformly of order n 1 for recent reviews see reid 1988 barndorff nielsen and cox 1989 and field and ronchetti 1990 the construction of these approximations typically requires knowledge of the underlying distribution however just as we can estimate the variance of a normal approxima tion or estimate the terms in an edgeworth expansion we can estimate the saddle point approximation in this paper we examine the properties of an estimate of the saddlepoint approximation our investigation is motivated by three considerations firstly the estimated approximation may be useful for comparing various estimators on a given data set if the sampling distribution is non normal at least in principle we need to compare the whole distribution and comparisons based on the estimated asymptotic variance alone may be misleading secondly the estimated approximation may be useful for evaluating the quality of simpler approximations including the normal approxima tion finally it may be useful for developing inference procedures methods like

on goodness of fit tests for the two parameter weibull distribution derived from the stabilized probability plot summary michael s statistic dsp based on the stabilized probability plot is considered for the assessment of goodness of fit of a two parameter weibull distribution this statistic is shown to have high power when the parameters are estimated using a procedure due to blom 1958 the correlation coefficient of the stabilized plot rsp is also proposed as a test statistic for this situation finally a family of transformations is obtained to reduce the null distribution of rsp to standard normality

conditional properties of likelihood based significance tests summary this paper considers the conditional properties of the unconditional significance tests based on standardized versions of the maximum likelihood estimate and the score statistic for the case of a simple null hypothesis and a one parameter model the properties of the tests are assessed by considering the conditional level of the test given the observed value of a statistic that is locally ancillary near the null model it is shown that for the test based on the maximum likelihood estimate the conditional level is equal to the nominal level of the test ignoring terms of order o n 1 and higher provided that the inverse of the observed fisher information is used as the variance estimate if the inverse of the expected fisher information is used the extent to which the conditional level of the test depends on the observed value of the locally ancillary statistic is shown to depend on the statistical curvature of the model as defined by efron 1975 hence a new interpretation of statistical curvature is given in terms of conditional inference similar results are established for the test based on the score statistic

the pdf projection theorem and the class specific method we present the theoretical foundation for optimal classification using class specific features and provide examples of its use a new probability density function pdf projection theorem makes it possible to project probability density functions from a low dimensional feature space back to the raw data space an m ary classifier is constructed by estimating the pdfs of class specific features then transforming each pdf back to the raw data space where they can be fairly compared although statistical sufficiency is not a requirement the classifier thus constructed becomes equivalent to the optimal bayes classifier if the features meet sufficiency requirements individually for each class this classifier is completely modular and avoids the dimensionality curse associated with large complex problems by recursive application of the projection theorem it is possible to analyze complex signal processing chains we apply the method to feature sets including linear functions of independent random variables cepstrum and mel cepstrum in addition we demonstrate how it is possible to automate the feature and model selection process by direct comparison of log likelihood values on the common raw data domain

a bayesian cart algorithm a stochastic search form of classification and regression tree cart analysis breiman et al 1984 is proposed motivated by a bayesian model an approximation to a probability distribution over the space of possible trees is explored using reversible jump markov chain monte carlo methods green 1995

on the relationship between empirical likelihood and empirical saddlepoint approximation for multivariate m estimators by comparing the expansions of the empirical log likelihood ratio and the empirical cumulant generating function calculated at the saddlepoint we investigate the relationship between empirical likelihood and empirical saddlepoint approximations this leads to a nonparametric approximation of the density of a multivariate m estimator based on the empirical likelihood and on the other hand it provides nonparametric confidence regions based on the empirical cumulant generating function some examples illustrate the use of the empirical likelihood in saddlepoint approximations and vice versa

robust multiple comparisons abstract the first order terms of the joint edgeworth expansion for the p studentized m estimates of location in a oneway layout are presented and the probabilities that generate the bonferroni maximum modulus and scheffe methods of multiple inference are approximated the validity robustness of these methods based on both classical estimators and a variety of m estimators will be compared for increasing numbers of simultaneous comparisons the bonferroni and maximum modulus methods are seen to be nonrobust when many comparisons are made scheffe s method is more stable these results are fairly independent of the choice of estimator monte carlo results indicate these expansion approximated probabilities accurately reflect qualitative behavior and give reasonable numerical values for moderately trimmed m estimators but are insufficiently accurate to generate critical points for heavily trimmed estimators

generalized risk zone selecting observations for classification in this paper we extend the risk zone concept by creating the generalized risk zone the generalized risk zone is a model independent scheme to select key observations in a sample set the observations belonging to the generalized risk zone have shown comparable in some experiments even better classification performance when compared to the use of the whole sample the main tool that allows this extension is the cauchy schwartz divergence used as a measure of dissimilarity between probability densities to overcome the setback concerning pdf s estimation we used the ideas provided by the information theoretic learning allowing the calculation to be performed on the available observations only we used the proposed methodology with learning vector quantization feedforward neural networks support vector machines and nearest neighbors

approximations of marginal tail probabilities for a class of smooth functions with applications to bayesian and conditional inference summary this paper presents an asymptotic approximation of marginal tail probabilities for a real valued function of a random vector where the function has continuous gradient that does not vanish at the mode of the joint density of the random vector this approximation has error 0 n 312 and improves upon a related standard normal approximation which has error 0 n 1 derivation involves the application of a tail probability formula given by diciccio field fraser 1990 to an approximation of a marginal density derived by tierney kass kadane 1989 the approximation can be applied for bayesian and conditional inference as well as for approximating sampling distributions and the accuracy of the approximation is illustrated through several numerical examples related to such applications in the context of conditional inference we develop refinements of the standard normal approximation to the distribution of two different signed root likelihood ratio statistics for a component of the natural parameter in exponential families

on two families of transformations to additivity for binary response data summary two families of power transformations for probabilities are introduced to model symmetric and asymmetric departures from the logistic model the aim is to provide in a comprehensive way a representation of alternative scales in terms of which to carry out the analysis of binary response data maximum likelihood methods are employed to estimate an appropriate transformation using glim tests for detection of symmetric and asymmetric departures are proposed

quasi likelihood regression models with missing covariates summary this paper presents methods to handle missing covariates when the quasi likelihood equations for the complete data are available our suggestion is to replace the functions of the missing data appearing in the quasi likelihood equation with their conditional means given the observed data or with unbiased predictors so that the resulting equation is unbiased we focus on two models one is a random effects model for count data where random effects are treated as missing covariates the other is the overdispersed binomial regression model with partially missing covariates we also investigate the efficiency of the proposed estimates relative to the maximum likelihood estimators

small sample confidence intervals abstract in this article we present a technique for constructing one or two sided confidence intervals which are second order correct in terms of coverage for either parametric or nonparametric problems the construction is valid in the presence of nuisance parameters the situation we consider is this there are p parameters and we want a confidence interval for some function of them possibly one of the parameters itself the p parameters are estimated by m estimates which means they are obtained as a solution of a system of equations maximum likelihood estimates are included as a special case the essential intermediate result given in equation 3 4 says that the estimated parameter of interest can be written as a mean up to order o p 1 n the representation 3 4 is attained when is a smooth function of the parameters and is the solution of a well behaved system of equations we avoid the use of pivots and strive to obtain accurate coverage confidence intervals are construc

maxi min margin machine learning large margin classifiers locally and globally in this paper we propose a novel large margin classifier called the maxi min margin machine m4 this model learns the decision boundary both locally and globally in comparison other large margin classifiers construct separating hyperplanes only either locally or globally for example a state of the art large margin classifier the support vector machine svm considers data only locally while another significant model the minimax probability machine mpm focuses on building the decision hyperplane exclusively based on the global information as a major contribution we show that svm yields the same solution as m4 when data satisfy certain conditions and mpm can be regarded as a relaxation model of m4 moreover based on our proposed local and global view of data another popular model the linear discriminant analysis can easily be interpreted and extended as well we describe the m4 model definition provide a geometrical interpretation present theoretical justifications and propose a practical sequential conic programming method to solve the optimization problem we also show how to exploit mercer kernels to extend m4 for nonlinear classifications furthermore we perform a series of evaluations on both synthetic data sets and real world benchmark data sets comparison with svm and mpm demonstrates the advantages of our new model

small sample asymptotic distributions of m estimators of location summary asymptotic formulae for the distribution of m estimators i e maximum likelihood type estimators of location including the arithmetic mean are derived which numerical studies show to give relative errors for densities and tail areas of the order of magnitude of 1 down to sample sizes 3 and 4 even in the extreme tails the paper is the continuation of earlier work by the second author and is also closely related to daniels s work on the saddlepoint approximation the method consists in expanding the derivative of the logarithm of the unstandardized density of the estimator in powers of 1 n at each point using recentring by means of conjugate distributions this method yields a unified point of view for the comparison of other asymptotic methods namely saddlepoint method edgeworth expansion and large deviations approach which are also compared numerically

the minimum error minimax probability machine we construct a distribution free bayes optimal classifier called the minimum error minimax probability machine mempm in a worst case setting i e under all possible choices of class conditional densities with a given mean and covariance matrix by assuming no specific distributions for the data our model is thus distinguished from traditional bayes optimal approaches where an assumption on the data distribution is a must this model is extended from the minimax probability machine mpm a recently proposed novel classifier and is demonstrated to be the general case of mpm moreover it includes another special case named the biased minimax probability machine which is appropriate for handling biased classification one appealing feature of mempm is that it contains an explicit performance indicator i e a lower bound on the worst case accuracy which is shown to be tighter than that of mpm we provide conditions under which the worst case bayes optimal classifier converges to the bayes optimal classifier we demonstrate how to apply a more general statistical framework to estimate model input parameters robustly we also show how to extend our model to nonlinear classification by exploiting kernelization techniques a series of experiments on both synthetic data sets and real world benchmark data sets validates our proposition and demonstrates the effectiveness of our model

importance sampling for bootstrap confidence intervals abstract the use of importance sampling methods to substantially reduce the amount of resampling necessary for the construction of nonparametric bootstrap confidence intervals is investigated the classical importance sampling method of hammersley and handscomb 1964 is modified to apply to the estimation of quantiles based on this method a resampling procedure is introduced that recenters the bootstrap distribution of a robust estimator of location to produce estimated confidence limits that are much more accurate for a given amount of resampling than those obtained by the usual bootstrap method efron 1982 the required recentering is accomplished with a suitable exponential tilting similar to that used in another connection by field and hampel 1982 this importance sampling procedure is used to produce bootstrap confidence intervals for location based on a class of estimators that includes symmetric m estimators these interval estimates are asymptotically optimal in a certain sense simulati

twin mahalanobis distance based support vector machines for pattern recognition twin support vector machines tsvms achieve fast training speed and good performance for data classification however tsvms do not take full advantage of the statistical information in data such as the covariance of each class of data this paper proposes a new twin mahalanobis distance based support vector machine tmsvm classifier in which two mahalanobis distance based kernels are constructed according to the covariance matrices of two classes of data for optimizing the nonparallel hyperplanes tmsvms have a special case of tsvms when the covariance matrices in a reproducing kernel hilbert space are degenerated to the identity ones tmsvms are suitable for many real problems especially for the case that the covariance matrices of two classes of data are obviously different the experimental results on several artificial and benchmark datasets indicate that tmsvms not only possess fast learning speed but also obtain better generalization than tsvms and other methods

regression analysis nonlinear or nonnormal simple and accurate p values from likelihood analysis we develop simple approximations for the p values to use with regression models having linear or nonlinear parameter structure and normal or nonnormal error distribution computer iteration then gives confidence intervals both frequentist and bayesian versions are given the approximations are derived from recent developments in likelihood analysis and have third order accuracy also for very small and medium sized samples the accuracy can typically be high the likelihood basis of the procedure seems to provide the grounds for this general accuracy examples are discussed and simulations record the distributional accuracy

biometrika centenary theory and general methodology note reprinted in biometrika one hundred years edited by d m titterington and d r cox oxford university press 11 50 reference stat article 2001 002doi 10 1093 biomet 88 1 13view record in web of science record created on 2006 04 21 modified on 2017 05 12