improved likelihood ratio tests on the von mises fisher distribution summary explicit expressions for the directed likelihood statistic r and two modified directed likelihood statistics r and rt are given for several standard hypotheses on concentrations in von misesfisher distributions and for several standard hypotheses on mean directions in von mises distributions

map estimation via agreement on trees message passing and linear programming we develop and analyze methods for computing provably optimal maximum a posteriori probability map configurations for a subclass of markov random fields defined on graphs with cycles by decomposing the original distribution into a convex combination of tree structured distributions we obtain an upper bound on the optimal value of the original problem i e the log probability of the map assignment in terms of the combined optimal values of the tree problems we prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common an important implication is that any such shared configuration must also be a map configuration for the original distribution next we develop two approaches to attempting to obtain tight upper bounds a a tree relaxed linear program lp which is derived from the lagrangian dual of the upper bounds and b a tree reweighted max product message passing algorithm that is related to but distinct from the max product algorithm in this way we establish a connection between a certain lp relaxation of the mode finding problem and a reweighted form of the max product min sum message passing algorithm

modified signed log likelihood ratio summary the signed log likelihood ratio r for a one dimensional interest parameter can be modified as r r r log u r so that r is asymptotically standard normally distributed with error of order o n 3 2 u being interpretable as a test statistic a new more direct derivation of this result is given which leads to a more explicit expression for u and provides a better handle on the error term the relation of the tail area approximation determined by r to a generalization of the lugannani rice approximation is discussed and the applicability of r to the analysis of residual variation is indicated

conditions on risk attitude for a single attribute for a decision problem having consequences described by a single attribute the task of determining a utility function can be facilitated by verifying that the decision maker s risk attitude satisfies a condition such as constant risk aversion we investigate a general class of conditions on risk attitude and show that a utility function for such a condition may exist only when the condition is of a special type next we discuss and interpret conditions of this special type then we define two conditions which imply that the decision maker s risk attitude satisfies a condition of this type and is represented by a generalized logarithmic utility function or a linear fractional utility function

coverage probability bias objective bayes and the likelihood principle summary we review objective bayes procedures based on both parametric and predictive coverage probability bias and explore the extent to which such procedures contravene the likelihood principle in the case of a scalar parameter the discussion encompasses choice of objective priors objective posterior probability statements and objective predictive probability statements we conclude with some remarks concerning the future development and implementation of objective priors based on small coverage probability bias

approximations of marginal tail probabilities and inference for scalar parameters summary in many situations inference for a scalar parameter in the presence of nuisance parameters requires integration of either a joint density of pivotal quantities or a joint posterior density for such inference accurate approximations of marginal tail probabilities are useful to avoid high dimensional integrals two tail probability approximations are developed in this paper numerical results given for conditional inference in location scale and linear regression models show the approximations to be generally accurate even for small sample sizes

understanding the role of noise in stochastic local search analysis and experiments stochastic local search sls algorithms have recently been proven to be among the best approaches to solving computationally hard problems sls algorithms typically have a number of parameters optimized empirically that characterize and determine their performance in this article we focus on the noise parameter the theoretical foundation of sls including an understanding of how to the optimal noise varies with problem difficulty is lagging compared to the strong empirical results obtained using these algorithms a purely empirical approach to understanding and optimizing sls noise as problem instances vary can be very computationally intensive to complement existing experimental results we formulate and analyze several markov chain models of sls in this article in particular we compute expected hitting times and show that they are rational functions for individual problem instances as well as their mixtures expected hitting time curves are analytical counterparts to noise response curves reported in the experimental literature hitting time analysis using polynomials and convex functions is also discussed in addition we present examples and experimental results illustrating the impact of varying noise probability on sls run time in experiments where most probable explanations in bayesian networks are computed we use synthetic problem instances as well as problem instances from applications we believe that our results provide an improved theoretical understanding of the role of noise in stochastic local search thereby providing a foundation for further progress in this area

conditional properties of unconditional parametric bootstrap procedures for inference in exponential families higher order inference about a scalar parameter in the presence of nuisance parameters can be achieved by bootstrapping in circumstances where the parameter of interest is a component of the canonical parameter in a full exponential family the optimal test which is approximated is a conditional one based on conditioning on the sufficient statistic for the nuisance parameter a bootstrap procedure that ignores the conditioning is shown to have desirable conditional properties in providing third order relative accuracy in approximation of p values associated with the optimal test in both continuous and discrete models the bootstrap approach is equivalent to third order analytical approaches and is demonstrated in a number of examples to give very accurate approximations even for very small sample sizes

controlled generation of hard and easy bayesian networks impact on maximal clique size in tree clustering this article presents and analyzes algorithms that systematically generate random bayesian networks of varying difficulty levels with respect to inference using tree clustering the results are relevant to research on efficient bayesian network inference such as computing a most probable explanation or belief updating since they allow controlled experimentation to determine the impact of improvements to inference algorithms the results are also relevant to research on machine learning of bayesian networks since they support controlled generation of a large number of data sets at a given difficulty level our generation algorithms called bpart and mpart support controlled but random construction of bipartite and multipartite bayesian networks the bayesian network parameters that we vary are the total number of nodes degree of connectivity the ratio of the number of non root nodes to the number of root nodes regularity of the underlying graph and characteristics of the conditional probability tables the main dependent parameter is the size of the maximal clique as generated by tree clustering this article presents extensive empirical analysis using the hugin tree clustering approach as well as theoretical analysis related to the random generation of bayesian networks using bpart and mpart

confidence intervals for a binomial proportion and asymptotic expansions p z 2n 1 2 p 1 p 1 2 is currently in near universal use we first show that the coverage properties of the wald interval are persistently poor and defy virtually all conventional wisdom we then proceed to a theoretical comparison of the standard interval and four additional alternative intervals by asymptotic expansions of their coverage probabilities and expected lengths the four additional interval methods we study in detail are the score test interval wilson the likelihood ratio test interval a jeffreys prior bayesian interval and an interval suggested by agresti and coull the asymptotic expansions for coverage show that the first three of these alternative methods have coverages that fluctuate about the nominal value while the agresti coull interval has a somewhat larger and more nearly conservative coverage function for the five interval methods we also investigate asymptotically their average coverage relative to distributions for p supported within 0 1 in terms of expected length asymptotic expansions show that the agresti coull interval is always the longest of these the remaining three are rather comparable and are shorter than the wald interval except for p near 0 or 1 these analytical calculations support and complement the findings and the recommendations in brown cai and dasgupta statist sci 2001 16 101 133

maximum likelihood estimation of a multi dimensional log concave density fn is shown to have smaller mean integrated squared error compared with kernel based methods even when we allow the use of a theoret ical optimal fixed bandwidth for the kernel estimator that would not be available in practice we also present a real data clustering example which shows that our methodology can be used in conjunction with the expectation maximization algorithm to fit finite mixtures of log concave densities

practical use of higher order asymptotics for multiparameter exponential families recently developed asymptotics based on saddlepoint methods provide important practical methods for multiparameter exponential families especially in generalized linear models the aim here is to clarify and explore these attention is restricted to tests and confidence intervals regarding a single parametric function which can be represented as a natural parameter of a full rank exponential family excellent approximations to exact conditional inferences are often available in terms of simple adjustments to the signed square root of the likelihood ratio statistic

on the density of minimum contrast estimators conditions for the existence of the density of a minimum contrast estimator in a parametric statistical family are given together with a formula for this density the formula is exact if multiple local minima cannot occur otherwise the formula is an exact expression for the point process of local minima of the contrast function although it is not in general feasible to compute the expression for the density the formula can be used as a basis for further expansion of the large deviation type when the estimate is sufficient either in the original model or after conditioning on an approximate or exact ancillary the formula simplifies drastically in particular it is shown how barndorff nielsen s formula for the density of the maximum likelihood estimator given an ancillary statistic is derived from the formula given here in this way the nature of barndorff nielsen s formula as an asymptotic approximation and its appearance as an exact formula for certain cases are demonstrated

stable and invariant adjusted directed likelihoods summary for the purpose of making inferences for a one dimensional interest parameter or constructing approximate complementary ancillaries or residuals the directed likelihood or signed square root of the likelihood ratio statistic can be adjusted so that the resulting modified directed likelihood is under ordinary repeated sampling approximately standard normal with error of o n 312 conditional on a suitable ancillary statistic and hence unconditionally in general suitable specification of the ancillary statistic may be difficult we introduce two adjusted directed likelihoods which are similar to the modified directed likelihood but do not require the specification of the ancillary statistic the error of the standard normal approximation to the distribution of these new adjusted directed likelihoods is o n 1 conditional on any reasonable ancillary statistic which is still an improvement over the unadjusted directed likelihoods

a bayesian approach to approximate conditional inference summary sweeting 1995 studies regular bayesian and frequentist approximations within a unified framework in the case of a single parameter and shows that higher order approximations to sampling distributions arise from their bayesian counterparts via an unsmoothing argument in the present paper we extend this programme to include formulae in approximate conditional inference in particular it is shown how bayesian arguments may be used to derive some formulae developed by barndorff nielsen 1980 1983 1986 the development proceeds in terms of likelihood roots

approximating maps for belief networks is np hard and other theorems abstract finding maximum a posteriori map assignments also called most probable explanations is an important problem on bayesian belief networks shimony has shown that finding maps is np hard in this paper we show that approximating maps with a constant ratio bound is also np hard in addition we examine the complexity of two related problems which have been mentioned in the literature we show that given the map for a belief network and evidence set or the family of maps if the optimal is not unique it remains np hard to find or approximate alternative next best explanations furthermore we show that given the map or maps for a belief network and an initial evidence set it is also np hard to find or approximate the map assignment for the same belief network with a modified evidence set that differs from the initial set by the addition or removal of even a single node assignment finally we show that our main result applies to networks with constrained in degree and out degree applies to randomized approximation and even still applies if the ratio bound instead of being constant is allowed to be a polynomial function of various aspects of the network topology

corrected confidence intervals after sequential testing with applications to survival analysis corrected confidence intervals are developed for an unknown parameter for data from a sequential experiment the corrections are obtained from asymptotic expansions for the distribution of the signed root transformation and they have a simple form the model does not require independent and identically distributed random variables or smooth stopping boundaries inference about the failure rate of an exponential distribution when data are subject to random censoring censoring by real time and optional stopping is used to illustrate the main results the accuracy of the method is confirmed through simulation studies

wald statistic for model order selection in superposition models a consistent model selection algorithm is presented for superimposed signal models the proposed method is motivated by the wald statistic and reduces the computational complexity of procedures based on the minimum description length mdl principle the procedure is suggested when a noncyclostationary signal model or short data length prevents use of covariance rank test for maximum model order k the procedure provides o k computational savings over an mdl test additionally a proof establishes the consistency of a least squares estimator using overparametrized models finite sample performance of the proposed model selection method is studied via monte carlo simulations for estimating the multipath delays and amplitudes of a chirp signal

objective bayes conditional inference and the signed root likelihood ratio statistic barndorffnielsen oe 1986 biometrika v73 p307 barndorff nielsen o e 1994 inference asymptotic datta g s 2004 probability matching diciccio tj 1993 j roy stat soc b met v55 p305 diciccio tj 1993 biometrika v80 p731 doi 10 1093 biomet 80 4 731 diciccio tj 1994 j roy stat soc b met v56 p397 diciccio tj 1994 scand j stat v21 p447 diciccio tj 2010 biometrika v97 p497 doi 10 1093 biomet asq002 hall p 1992 bootstrap edgeworth lawless jf 1982 stat models methods mukerjee r 1993 biometrika v80 p499 doi 10 1093 biomet 80 3 499 mukerjee r 1997 biometrika v84 p970 doi 10 1093 biomet 84 4 970 nicolaou a 1993 j roy stat soc b met v55 p377 peers hw 1965 j roy stat soc b v27 p9 tibshirani r 1989 biometrika v76 p604 doi 10 1093 biomet 76 3 604 tierney l 1986 j am stat assoc v81 p82 doi 10 2307 2287970 welch b l 1963 j r stat soc b v62 p159

comparisons of approximate confidence intervals for distributions used in life data analysis this article evaluates the accuracy of approximate confidence intervals for parameters and quantiles of the smallest extreme value and normal distributions the findings also apply to the weibull and the lognormal distributions the interval estimates are based on a the asymptotic normality of the maximum likelihood estimator b the asymptotic x 2 distribution of the likelihood ratio lr statistic c a mean and variance correction to the signed square roots of the lr statistic and d the bartlett correction to the lr statistic the extensive monte carlo results about true error probabilities and average lengths under various degrees of censoring show advantages of the lr based intervals for complete or moderately censored samples the mean and variance correction to the lr statistic gives nearly exact and symmetric error probabilities in small samples with heavy censoring the bartlett correction tends to give conservative error probabilities whereas the uncorrected lr interval is often antico

a differential approach to inference in bayesian networks we present a new approach to inference in bayesian networks which is based on representing the network using a polynomial and then retrieving answers to probabilistic queries by evaluating and differentiating the polynomial the network polynomial itself is exponential in size but we show how it can be computed efficiently using an arithmetic circuit that can be evaluated and differentiated in time and space linear in the circuit size the proposed framework for inference subsumes one of the most influential methods for inference in bayesian networks known as the tree clustering or jointree method which provides a deeper understanding of this classical method and lifts its desirable characteristics to a much more general setting we discuss some theoretical and practical implications of this subsumption

a simple method for the adjustment of profile likelihoods summary we propose a simple adjustment for profile likelihoods the aim of the adjustment is to alleviate some of the problems inherent in the use of profile likelihoods such as bias inconsistency and overoptimistic variance estimates the adjustment is applied to the profile loglikelihood score function at each parameter value so that its mean is zero and its variance is the negative expected derivative matrix of the adjusted score function for cases in which explicit calculation of the adjustments is difficult we give two methods to simplify their computation an automatic simulation method that requires as input only the profile loglikelihood and its first few derivatives first order asymptotic expressions some examples are provided and a comparison is made with the conditional profile log likelihood of cox and reid

likelihood ratio statistics based on an integrated likelihood an integrated likelihood depends only on the parameter of interest and the data so it can be used as a standard likelihood function for likelihood based inference in this paper the higher order asymptotic properties of the signed integrated likelihood ratio statistic for a scalar parameter of interest are considered these results are used to construct a modified integrated likelihood ratio statistic and to suggest a class of prior densities to use in forming the integrated likelihood the properties of the integrated likelihood ratio statistic are compared to those of the standard likelihood ratio statistic several examples show that the integrated likelihood ratio statistic can be a useful alternative to the standard likelihood ratio statistic

bootstrap confidence intervals for a class of parametric problems summary we consider the following class of problems having observed a multivariate normal data vector y with unknown mean vector q covariance matrix the identity find an approximate confidence interval for 0 t q a real valued function of q a simple geometric construction is given which leads to highly accurate solutions this construction shows that the standard approximation based on maximum likelihood theory 04 cz00 can be quite misleading when 0 is nonlinear in q we discuss bootstrap based confidence intervals which remove most of the error in the standard approximation at the expense of considerably more calculation the bootstrap intervals are invariant under transformation of both y and i and so they automatically produce accurate solutions in problems which can be transformed to multivariate normality without requiring knowledge of the normalizing transformation

mean loglikelihood and higher order approximations summary higher order approximations to p values can be obtained from the log likelihood function and a reparameterization that can be viewed as a canonical parameter in an exponential family approximation to the model this approach clarifies the connection between skovgaard 1996 and fraser et al 1999a and shows that the skovgaard approximation can be obtained directly using the mean log likelihood function

confidence intervals of effect size in randomized comparative parallel group studies we have presented a new likelihood based approach for constructing confidence intervals of effect size that are applicable to small samples we also conduct a simulation study to compare the coverage probability of the new likelihood based method with other three methods proposed by hedges and olkin and kraemer and paik simulation studies show that the confidence interval generated by the modified signed log likelihood ratio method possesses essentially exact coverage probabilities even for small samples although the coverage probabilities are consistently but slightly less than the nominal level the methods are also applied to two examples

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

understanding the scalability of bayesian network inference using clique tree growth curves one of the main approaches to performing computation in bayesian networks bns is clique tree clustering and propagation the clique tree approach consists of propagation in a clique tree compiled from a bn and while it was introduced in the 1980s there is still a lack of understanding of how clique tree computation time depends on variations in bn size and structure in this article we improve this understanding by developing an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from bns specifically i the ratio of the number of a bn s non root nodes to the number of root nodes and ii the expected number of moral edges in their moral graphs analytically we partition the set of cliques in a clique tree into different sets and introduce a growth curve for the total size of each set for the special case of bipartite bns there are two sets and two growth curves a mixed clique growth curve and a root clique growth curve in experiments where random bipartite bns generated using the bpart algorithm are studied we systematically increase the out degree of the root nodes in bipartite bayesian networks by increasing the number of leaf nodes surprisingly root clique growth is well approximated by gompertz growth curves an s shaped family of curves that has previously been used to describe growth processes in biology medicine and neuroscience we believe that this research improves the understanding of the scaling behavior of clique tree clustering for a certain class of bayesian networks presents an aid for trade off studies of clique tree clustering using growth curves and ultimately provides a foundation for benchmarking and developing improved bn inference and machine learning algorithms

leverage and superleverage in nonlinear regression abstract several measures of the leverage of an observation in a nonlinear regression model are defined and developed in contrast to the upper bound on the leverage in a linear model it is found that in a nonlinear model the leverage of an observation may exceed 1 such a case is said to exhibit superleverage relationships between the leverage measures are explored and several examples are developed to illustrate the proposed methodology

assessment of uncertainty in high frequency data the observed asymptotic variance the availability of high frequency financial data has generated a series of estimators based on intra day data improving the quality of large areas of financial econometrics however estimating the standard error of these estimators is often challenging the root of the problem is that traditionally standard errors rely on estimating a theoretically derived asymptotic variance and often this asymptotic variance involves substantially more complex quantities than the original parameter to be estimated standard errors are important they are used to assess the precision of estimators in the form of confidence intervals to create feasible statistics for testing to build forecasting models based on say daily estimates and also to optimize the tuning parameters the contribution of this paper is to provide an alternative and general solution to this problem which we call observed asymptotic variance it is a general nonparametric method for assessing asymptotic variance avar it provides consistent estimators of avar for a broad class of integrated parameters t dt where the spot parameter process can be a general semimartingale with continuous and jump components the observed avar is implemented with the help of a two scales method its construction works well in the presence of microstructure noise and when the observation times are irregular or asynchronous in the multivariate case the methodology is valid for a wide variety of estimators including the standard ones for variance and covariance and also for more complex estimators such as of leverage effects high frequency betas and semivariance

effects of the reference set on frequentist inferences summary we employ second order likelihood asymptotics to investigate how ideal frequentist inferences depend on the probability model for the data through more than the likelihood function referring to this as the effect of the reference set there are two aspects of higher order corrections to first order likelihood methods namely i that involving effects of fitting nuisance parameters and leading to the modified profile likelihood and ii another part pertaining to limitation in adjusted information generally each of these involves a first order adjustment depending on the reference set however we show that for some important settings likelihood irrelevant model specifications have a second order effect on both of these adjustments this result includes specification of the censoring model for survival data on the other hand for sequential experiments the likelihood irrelevant specification of the stopping rule has a second order effect on adjustment i but a first order effect on adjustment ii these matters raise the issue of what are ideal frequentist inferences since consideration of exact frequentist inferences will not suffice we indicate that to second order ideal frequentist inferences may be based on the distribution of the ordinary likelihood ratio statistic without commonly considered adjustments thereto

on the hardness of approximate reasoning abstract many ai problems when formalized reduce to evaluating the probability that a propositional expression is true in this paper we show that this problem is computationally intractable even in surprisingly restricted cases and even if we settle for an approximation to this probability we consider various methods used in approximate reasoning such as computing degree of belief and bayesian belief networks as well as reasoning techniques such as constraint satisfaction and knowledge compilation that use approximation to avoid computational difficulties and reduce them to model counting problems over a propositional domain we prove that counting satisfying assignments of propositional languages is intractable even for horn and monotone formulae and even when the size of clauses and number of occurrences of the variables are extremely limited this should be contrasted with the case of deductive reasoning where horn theories and theories with binary clauses are distinguished by the existence of linear time satisfiability algorithms what is even more surprising is that as we show even approximating the number of satisfying assignments i e approximating approximate reasoning is intractable for most of these restricted theories we also identify some restricted classes of propositional formulae for which efficient algorithms for counting satisfying assignments can be given

an o brien fleming sequential trial for comparing three treatments we consider a sequential procedure for comparing three treatments with the goal of ultimately selecting the best treatment this procedure starts with a sequential test to detect an overall treatment difference and eliminates the apparently inferior treatment if this test rejects the equality of the treatments it then proceeds with a sequential test of the remaining two treatments we base these sequential tests on the stopping boundaries popularized by o brien and fleming our procedure is similar in structure to that used by siegmund in conjunction with modified repeated significance tests we compare the performances of the two procedures via a simulation experiment we derive analytic approximations for an error probability the power and the expected sample size of our procedure which we compare to simulated values furthermore we propose a modification of the procedure for the comparison of a standard treatment with experimental treatments

proportional discounting of future costs and benefits social benefits in the distant future receive very little importance when a public policy analysis uses a constant discounting model with its geometric timing weights to compare future benefits with present costs this paper defines conditions on social preferences that imply a model having arbitrary timing weights we then introduce a condition on preferences that restricts this general model to one in which the timing weights are linear fractional this model is proposed as an alternative to the constant discounting model it accords far more importance to the distant future than does constant discounting and like constant discounting it depends on the assessment of a single parameter

practical small sample asymptotics for regression problems abstract saddlepoint approximations are derived for sums of independent but not necessarily identically distributed random variables along with generalizations to estimating equations and multivariate problems these results are particularly useful for accurately approximating the distribution of regression coefficients general formulas are given for the distribution of the coefficients arising out of a generalized linear model with both canonical and noncanonical link functions we illustrate the case of logistic regression with a real data example and show how the gibbs sampler may be used to obtain confidence sets for each regression parameter based on the saddlepoint approximation

topological parameters for time space tradeoff in this paper we propose a family of algorithms combining tree clustering with conditioning that trade space for time such algorithms are useful for reasoning in probabilistic and deterministic networks as well as for accomplishing optimization tasks by analyzing the problem structure the user can select from a spectrum of algorithms the one that best meets a given time space specification to determine the potential of this approach we analyze the structural properties of problems coming from the circuit diagnosis domain the analysis demonstrates how the tradeoffs associated with various hybrids can be used for each problem instance 2001 elsevier science b v all rights reserved

simple modifications for signed roots of likelihood ratio statistics summary in the context of inference about a scalar parameter in the presence of nuisance parameters some simple modifications for the signed root of the log likelihood ratio statistic r are developed that reduce the order of error in the standard normal approximation to the distribution of r from o n 1 2 to o n 1 barndorff nielsen has introduced a variable u such that the error in the standard normal approximation to the distribution of r r r log u r is of order o n 3 2 but calculation of urequires the specification of an exact or approximate ancillary statistic a this paper proposes an alternative variable to u denoted by t that is available without knowledge of a and satisfies t u op n 1 in general thus the standard normal approximation to the distribution of r r log t r has error of order o n 1 and it can be used to construct approximate confidence limits having coverage error of order o n 1 in certain cases however tand uare identical the derivation of tinvolves the bayesian approach to constructing confidence limits considered by welch and peers and peers similar modifications for the signed root of the conditional likelihood ratio statistic are also developed and these modifications are seen to be useful when a large number of nuisance parameters are present several examples are presented including inference for natural parameters in exponential models and inference about location scale models with type ii censoring in each case the relationship between tand u is discussed numerical examples are also given including inference for regression models inference about the means of log normal distributions and inference for exponential lifetime models with type i censoring where barndorff nielsen s variable u is not available

very weak expansions for sequential confidence levels on obtient des developpements asymptotiques pour une classe de moyennes de probabilites de recouvrement pour des bornes de confiance sequentielles

approximate inference for the generalized gamma distribution methods of approximate conditional inference in location and scale parameter distributions based on normal approximations to the distributions of signed square roots of likelihood ratio statistics are discussed these methods are applied to obtain approximate inference for the quantiles and scale parameter of the log generalized gamma distribution from uncensored samples assuming that the index parameter of the distribution is known as special cases approximate inference in the extreme value and normal distributions from type ii censored samples is considered the accuracy of the approximate methods for small samples is illustrated by comparison with exact results in some examples

a framework for bayesian and likelihood approximations in statistics summary regular bayesian and frequentist approximations in statistics are studied within a unified framework in particular it is shown how some higher order likelihood based approximations arise from their bayesian counterparts via an unsmoothing argument this approach serves to shed new light on these formulae and to clarify relationships between bayesian and frequentist inferences for example bayesian interpretations of higher order approximations can give insights into the acceptability or otherwise of these approximations from the point of view of relevance to the actual data observed furthermore this approach is a very natural one for the study of more general nonregular problems models for dependent data and approximate conditional inference for ease of exposition the development is in terms of a single real parameter the main analytic development proceeds in terms of signed roots of log density ratios

higher order likelihood inference in meta analysis and meta regression this paper investigates the use of likelihood methods for meta analysis within the random effects models framework we show that likelihood inference relying on first order approximations while improving common meta analysis techniques can be prone to misleading results this drawback is very evident in the case of small sample sizes which are typical in meta analysis we alleviate the problem by exploiting the theory of higher order asymptotics in particular we focus on a second order adjustment to the log likelihood ratio statistic simulation studies in meta analysis and meta regression show that higher order likelihood inference provides much more accurate results than its first order counterpart while being of a computationally feasible form we illustrate the application of the proposed approach on a real example

integrable expansions for posterior distributions for a two parameter exponential family asymptotic expansions of posterior distributions are derived for a two dimensional exponential family which includes normal gamma inverse gamma and inverse gaussian distributions reparameterization allows us to use a data dependent transformation convert the likelihood function to the two dimensional standard normal density and apply a version of stein s identity to assess the posterior distributions applications are given to characterize optimal noninformative priors in the sense of stein to suggest the form of a high order correction to the distribution function of a sequential likelihood ratio statistic and to provide confidence intervals for one parameter in the presence of other nuisance parameters

finding maps for belief networks is np hard given a probabilistic world model an important problem is to find the maximum a posteriori probability map instantiation of all the random variables given the evidence numerous researchers using such models employ some graph representation for the distributions such as a bayesian belief network this representation simplifies the complexity of specifying the distributions from exponential in n the number of variables in the model to linear in n in many interesting cases we show however that finding the map is np hard in the general case when these representations are used even if the size of the representation happens to be linear in n furthermore minor modifications to the proof show that the problem remains np hard for various restrictions of the topology of the graphs the same technique can be applied to the results of a related paper by cooper to further restrict belief network topology in the proof that probabilistic inference is np hard

plausibility functions and exact frequentist inference in the frequentist program inferential methods with exact control on error rates are a primary focus the standard approach however is to rely on asymptotic approximations which may not be suitable this article presents a general framework for the construction of exact frequentist procedures based on plausibility functions it is shown that the plausibility function based tests and confidence regions have the desired frequentist properties in finite samples no large sample justification needed an extension of the proposed method is also given for problems involving nuisance parameters examples demonstrate that the plausibility function based method is both exact and efficient in a wide variety of problems

median bias reduction of maximum likelihood estimates for regular parametric problems we show how median centring of the maximum likelihood estimate can be achieved by a simple modification of the score equation for a scalar parameter of interest the estimator is equivariant under interest respecting reparameterizations and is third order median unbiased with a vector parameter of interest componentwise equivariance and third order median centring are obtained like the implicit method of firth 1993 for bias reduction the new method does not require finiteness of the maximum likelihood estimate and is effective in preventing infinite estimates simulation results for continuous and discrete models including binary and beta regression confirm that the method succeeds in achieving componentwise median centring and in solving the boundary estimate problem while keeping comparable dispersion and the same approximate distribution as its main competitors

approximately unbiased tests of regions using multistep multiscale bootstrap resampling approximately unbiased tests based on bootstrap probabilities are considered for the exponential family of distributions with unknown expectation parameter vector where the null hypothesis is represented as an arbitraryshaped region with smooth boundaries this problem has been discussed previously in efron and tibshirani ann statist 26 1998 1687 1718 and a corrected p value with second order asymptotic accuracy is calculated by the two level bootstrap of efron halloran and holmes proc natl acad sci u s a 93 1996 13429 13434 based on the abc bias correction of efron j amer statist assoc 82 1987 171 185 our argument is an extension of their asymptotic theory where the geometry such as the signed distance and the curvature of the boundary plays an important role we give another calculation of the corrected p value without finding the nearest point on the boundary to the observation which is required in the two level bootstrap and is an implementational burden in complicated problems the key idea is to alter the sample size of the replicated dataset from that of the observed dataset the frequency of the replicates falling in the region is counted for several sample sizes and then the p value is calculated by looking at the change in the frequencies along the changing sample sizes this is the multiscale bootstrap of shimodaira systematic biology 51 2002 492 508 which is third order accurate for the multivariate normal model here we introduce a newly devised multistep multiscale bootstrap calculating a third order accurate p value for the exponential family of distributions in fact our p value is asymptotically equivalent to those obtained by the double bootstrap of hall the bootstrap and edgeworth expansion 1992 springer new york and the modified signed likelihood ratio of barndorff nielsen biometrika 73 1986 307 322 ignoring o n 3 2 terms yet the computation is less demanding and free from model specification the algorithm is remarkably simple despite complexity of the theory behind it the differences of the pvalues are illustrated in simple examples and the accuracies of the bootstrap methods are shown in a systematic way

a decomposition for the likelihood ratio statistic and the bartlett correction a bayesian argument let l n 1 log p x be the log likelihood of an n dimensional x under a p dimensional let j be the mle under h j 1 1 0 j j 0 and 0 be the unrestricted mle define t j as 2n l j 1 1 2 sgn j j 1 j 0 let t t 1 t p then under regularity conditions the following theorem is proved under 0 t is asymptotically n n 1 2 a 0 n 1 a j n 1 o n 3 2 where j is the identity matrix the result is proved by first establishing an analogous result when is random and then making the prior converge to a degenerate distribution the existence of the bartlett correction to order n 3 2 follows from the theorem we show that an edgeworth expansion with error o n 2 for t involves only polynomials of degree less than or equal to 3 and hence verify rigorously lawley s 1956 result giving the order of the error in the bartlett correction as o n 2

more accurate confidence intervals in exponential families summary fisher s theory of maximum likelihood estimation routinely provides approximate confidence intervals for a parameter of interest 0 the standard intervals 0 za5 s where 0 is the maximum likelihood estimator 5 is an estimate of standard error based on differentiation of the log likelihood function and za is a normal percentile point recent work has produced systems of better approximate confidence intervals which look more like exact intervals when exact intervals exist and in general have coverage probabilities an order of magnitude more accurate than the standard intervals this paper develops an efficient and dependable algorithm for calculating highly accurate approximate intervals on a routine basis for parameters 0 defined in the framework of a multiparameter exponential family the better intervals require only a few times as much computational effort as the standard intervals a variety of numerical and theoretical arguments are used to show that the algorithm works well and that the improvement over the standard intervals can be striking in realistic situations

bartlett adjustments to the likelihood ratio statistic and the distribution of the maximum likelihood estimator for rather general parametric models a simple connection is established between the bartlett adjustment factor of the log likelihood ratio statistic and the normalizing constant c of the formula c i i 1 2l for the conditional distribution of a maximum likelihood estimator as applied to the full model and the model of the hypothesis tested this leads to a relatively simple demonstration that division of the likelihood ratio statistic by a suitable constant or estimated factor improves the chi squared approximation to its distribution various expressions for these quantities are discussed in particular for the case of a one dimensional parameter an approximation to the constants involved is derived which does not require integration over the sample space

on second order optimality of the observed fisher information the realized error of an estimate is determined not only by the efficiency of the estimator but also by chance for example suppose that we have observed a bivariate normal vector whose expectation is known to be on a circle then intuitively the longer that vector happens to be the more accurately its angle is likely to be estimated yet this chance though its information is contained in the data cannot be accounted for by the variance of the estimate one way to capture it is by the direct estimation of the realized error in this paper we will demonstrate that the squared error of the maximum likelihood estimate to the extent to which it can be estimated can be most accurately estimated by the inverse of the observed fisher information in relation to this optimality we will also study the properties of several other estimators including the inverse of the expected fisher information the sandwich estimators the jackknife and the bootstrap estimators unlike the observed fisher information these estimators are not optimal

biometrika centenary theory and general methodology note reprinted in biometrika one hundred years edited by d m titterington and d r cox oxford university press 11 50 reference stat article 2001 002doi 10 1093 biomet 88 1 13view record in web of science record created on 2006 04 21 modified on 2017 05 12

conditional properties of likelihood based significance tests summary this paper considers the conditional properties of the unconditional significance tests based on standardized versions of the maximum likelihood estimate and the score statistic for the case of a simple null hypothesis and a one parameter model the properties of the tests are assessed by considering the conditional level of the test given the observed value of a statistic that is locally ancillary near the null model it is shown that for the test based on the maximum likelihood estimate the conditional level is equal to the nominal level of the test ignoring terms of order o n 1 and higher provided that the inverse of the observed fisher information is used as the variance estimate if the inverse of the expected fisher information is used the extent to which the conditional level of the test depends on the observed value of the locally ancillary statistic is shown to depend on the statistical curvature of the model as defined by efron 1975 hence a new interpretation of statistical curvature is given in terms of conditional inference similar results are established for the test based on the score statistic

a differential semantics for jointree algorithms a new approach to inference in belief networks has been recently proposed which is based on an algebraic representation of belief networks using multi linear functions according to this approach belief network inference reduces to a simple process of evaluating and differentiating multilinear functions we show here that mainstream inference algorithms based on jointrees are a special case of the approach based on multi linear functions in a very precise sense we use this result to prove new properties of jointree algorithms we also discuss some practical and theoretical implications of this new finding

likelihood based confidence intervals for a log normal mean to construct a confidence interval for the mean of a log normal distribution in small samples we propose likelihood based approaches the signed log likelihood ratio and modified signed log likelihood ratio methods extensive monte carlo simulation results show the advantages of the modified signed log likelihood ratio method over the signed log likelihood ratio method and other methods in particular the modified signed log likelihood ratio method produces a confidence interval with a nearly exact coverage probability and highly accurate and symmetric error probabilities even for extremely small sample sizes we then apply the methods to two sets of real life data

parameter orthogonality and approximate conditional inference on propose une statistique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations etant donne les estimateurs du maximum de vraisemblance pour les parametres de nuisance

a second order investigation of asymptotic ancillarity the paper deals with approximate ancillarity as discussed by efron and hinkley 1978 in the multivariate i i d case we derive the second order edgeworth expansion of the mle given a normalized version of the second derivative of the log likelihood at its maximum the expansion agrees with the one derived by amari 1982a for curved exponential families but holds for any family satisfying the regularity conditions given in the paper it is shown that the fisher information lost by reducing the data to the mle is recovered by the conditioning and it is sketched how the loss of information relates to the deficiency as defined by lecam finally we investigate some properties of three test statistics proving a conjecture by efron and hinkley 1978 concerning the conditional null distribution of the likelihood ratio test statistic and establishing a kind of superiority of the observed fisher information over the expected one as estimate of the inverse variance of the mle

on the construction of bayes confidence regions we obtain approximate bayes confidence intervals for a scalar parameter based on directed likelihood the posterior probabilities of these intervals agree with their unconditional coverage probabilities to fourth order and with their conditional coverage probabilities to third order these intervals are constructed for arbitrary smooth prior distributions a key feature of the construction is that log likelihood derivatives beyond second order are not required unlike the asymptotic expansions of severini

discounted value representations of temporal preferences this paper examines the proposition that temporal preferences may be represented in terms of discount functions an analytical framework is developed within which preferences are defined over a set of programs or prospects over time necessary and sufficient conditions for generalized discounting discounting as well as exponential discounting are established for a class of program sets over discrete and continuous time

tail probabilities from observed likelihoods summary an exponential model not in standard form is fully characterized by an observed likelihood function and its first sample space derivative up to one one transformations of the observable variable this property is used to modify the lugannani rice 1980 tail probability approximation to make it parameterization invariant then for general continuous models a version of tangent exponential model is defined and used to derive a general tail probability approximation that uses only the observed likelihood and its first sample space derivative the analysis extends from density functions to distribution functions the tangent exponential model methods of fraser 1988 a related tail probability approximation has been reported barndorff nielsen 1988b in the discussion to reid 1988

approximations of marginal tail probabilities for a class of smooth functions with applications to bayesian and conditional inference summary this paper presents an asymptotic approximation of marginal tail probabilities for a real valued function of a random vector where the function has continuous gradient that does not vanish at the mode of the joint density of the random vector this approximation has error 0 n 312 and improves upon a related standard normal approximation which has error 0 n 1 derivation involves the application of a tail probability formula given by diciccio field fraser 1990 to an approximation of a marginal density derived by tierney kass kadane 1989 the approximation can be applied for bayesian and conditional inference as well as for approximating sampling distributions and the accuracy of the approximation is illustrated through several numerical examples related to such applications in the context of conditional inference we develop refinements of the standard normal approximation to the distribution of two different signed root likelihood ratio statistics for a component of the natural parameter in exponential families

approximate confidence intervals for the parameters of a stationary binary markov chain the power divergence family of test statistics introduced by cressie and read 1984 is used to obtain approximate confidence intervals for the parameters of a stationary first order binary markov chain intervals based on inverting the likelihood ratio pearson and freeman tukey statistics are included in this family small sample comparisons are used to select the best intervals for the different parameters the results show that the best power divergence intervals for the probability of success and in particular the likelihood ratio interval have error rates closer to nominal levels than the sample proportion based intervals suggested by crow 1979 the properties of power divergence joint confidence regions for the transition probabilities are also explored

likelihood and observed geometries a partir du point de vue de la conditionnalite on etablit un parallele observe a la theorie de chentsov efron amari etc en utilisant l information observee et un tenseur d asymetrie observee

the modified signed likelihood statistic and saddlepoint approximations summary for a number of tests in exponential families we show that the use of a normal approximation to the modified signed likelihood ratio statistic r is equivalent to the use of a saddlepoint approximation this is also true in a large deviation region where the signed likelihood ratio statistic r is of order in

tree based reparameterization framework for analysis of sum product and related algorithms we present a tree based reparameterization trp framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles this class includes the belief propagation bp or sum product algorithm as well as variations and extensions of bp algorithms in this class can be formulated as a sequence of reparameterization updates each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph i e a tree or more generally a hypertree the ultimate goal is to obtain an alternative but equivalent factorization using functions that represent exact or approximate marginal distributions on cliques of the graph our framework highlights an important property of the sum product algorithm and the larger class of reparameterization algorithms the original distribution on the graph with cycles is not changed the perspective of tree based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency we develop interpretations of these results in terms of information geometry the invariance of the distribution in conjunction with the fixed point characterization enables us to derive an exact expression for the difference between the true marginals on an arbitrary graph with cycles and the approximations provided by belief propagation more broadly our analysis applies to any algorithm that minimizes the bethe free energy we also develop bounds on the approximation error which illuminate the conditions that govern their accuracy finally we show how the reparameterization perspective extends naturally to generalizations of bp e g kikuchi 1951 approximations and variants via the notion of hypertree reparameterization

practical small sample asymptotics for distributions used in life data analysis fraser and lawless discussed exact conditional intervals for the parameters and quantiles of the location scale model when complete data are used moreover lawless extended the exact method to failure censored type ii censored data nevertheless the exact intervals are difficult to obtain in practice and are unavailable under time censoring type i censoring as a consequence approximate large sample intervals are widely used in this article a likelihood based third order procedure is developed the method does not require explicit nuisance parameterization and can be easily implemented into algebraic computational packages numerical examples are presented to show the accuracy of the method even when the sample size is small

empirical bayes methods for combining likelihoods abstract suppose that several independent experiments are observed each one yielding a likelihood l k k for a real valued parameter of interest k for example k might be the log odds ratio for a 2 2 table relating to the kth population in a series of medical experiments this article concerns the following empirical bayes question how can we combine all of the likelihoods l k to get an interval estimate for any one of the k s say 1 the results are presented in the form of a realistic computational scheme that allows model building and model checking in the spirit of a regression analysis no special mathematical forms are required for the priors or the likelihoods this scheme is designed to take advantage of recent methods that produce approximate numerical likelihoods l k k even in very complicated situations with all nuisance parameters eliminated the empirical bayes likelihood theory is extended to situations where the k s have a regression structure as well as an empiri

on parameter transformations and interval estimation summary parameterizations which reduce the asymptotic bias and skewness of various pivotal quantities arising in large sample theory are discussed for models depending on an unknown scalar parameter transformation formulae by which such parameterizations can be obtained are derived and these formulae extend those for one dimensional curved exponential families given by hougaard 1982 to assess the accuracy of normal approximations to the distributions of the pivots their second order properties are considered and comparisons with the signed square root of the likelihood ratio statistic are drawn

refining bootstrap simultaneous confidence sets abstract simultaneous confidence sets for a collection of parametric functions may be constructed in several different ways these ways include a the exact pivotal method that underlies tukey s 1953 and scheffe s 1953 simultaneous confidence intervals for linear parametric functions in the normal linear model b the method of asymptotic pivots which is an approximate extension of the pivotal method c the method of bootstrapped roots developed in beran 1988 these three methods share several features each method simultaneously asserts a collection of confidence sets one confidence set for every parametric function of interest each method obtains the uth constituent confidence set by referring a root to a critical value the uth root is a function of the sample and of the uth parametric function each method has the same aim to control the overall level of the simultaneous confidence set and to keep equal the marginal levels of the individual confidence statements that make up the simulta

infereni on full or partial parameters based on the standardized signed log likelihood ratio on etudie des tests et des limites de confiance bases sur la statistique du rapport de log vraisemblance signee en ajustant cette statistique de telle sorte qu a un ordre superieur d approximation la distribution normale standard soit valable

interval estimates for the ratio and difference of two lognormal means health research often gives rise to data that follow lognormal distributions in two sample situations researchers are likely to be interested in estimating the difference or ratio of the population means several methods have been proposed for providing confidence intervals for these parameters however it is not clear which techniques are most appropriate or how their performance might vary additionally methods for the difference of means have not been adequately explored we discuss in the present article five methods of analysis these include two methods based on the log likelihood ratio statistic and a generalized pivotal approach additionally we provide and discuss the results of a series of computer simulations finally the techniques are applied to a real example

on the common mean of several inverse gaussian distributions based on a higher order likelihood method an interval estimation method for the common mean of several heterogeneous inverse gaussian ig populations is discussed the proposed method is based on a higher order likelihood based procedure the merits of the proposed method are numerically compared with the signed log likelihood ratio statistic two generalized pivot quantities and the simple t test method with respect to their expected lengths coverage probabilities and type i errors numerical studies show that the coverage probabilities of the proposed method are very accurate and type i errors are close to the nominal level 05 even for very small samples the methods are also illustrated with two examples

default priors for bayesian and frequentist inference summary we investigate the choice of default priors for use with likelihood for bayesian and frequentist inference such a prior is a density or relative density that weights an observed likelihood function leading to the elimination of parameters that are not of interest and then a density type assessment for a parameter of interest for independent responses from a continuous model we develop a prior for the full parameter that is closely linked to the original bayes approach and provides an extension of the right invariant measure to general contexts we then develop a modified prior that is targeted on a component parameter of interest and by targeting avoids the marginalization paradoxes of dawid and co workers this modifies jeffreys s prior and provides extensions to the development of welch and peers these two approaches are combined to explore priors for a vector parameter of interest in the presence of a vector nuisance parameter examples are given to illustrate the computation of the priors