a stochastic approximation algorithm with markov chain monte carlo method for incomplete data estimation problems we propose a general procedure for solving incomplete data estimation problems the procedure can be used to find the maximum likelihood estimate or to solve estimating equations in difficult cases such as estimation with the censored or truncated regression model the nonlinear structural measurement error model and the random effects model the procedure is based on the general principle of stochastic approximation and the markov chain monte carlo method applying the theory on adaptive algorithms we derive conditions under which the proposed procedure converges simulation studies also indicate that the proposed procedure consistently converges to the maximum likelihood estimate for the structural measurement error logistic regression model

thermodynamical limit for correlated gaussian random energy models let e n n be a family of n 2n centered unit gaussian random variables defined by the covariance matrix cn of elements cn av e n e n and the corresponding random hamiltonian then the quenched thermodynamical limit exists if for every decomposition n n1 n2 and all pairs n n where k k 1 2 are the projections of n into nk the condition is explicitly verified for the sherrington kirkpatrick the even p spin the derrida rem and the derrida gardner grem models

large deviation approach to the generalized random energy model the generalized random energy model is a generalization of the random energy model introduced by derrida to mimic the ultrametric structure of the parisi solution of the sherrington kirkpatrick model of a spin glass it was solved exactly in two special cases by derrida and gardner a complete solution for the thermodynamics in the general case was given by capocaccia et al here we use large deviation theory to analyse the model in a very straightforward way we also show that the variational expression for the free energy can be evaluated easily using the cauchy schwarz inequality

reference bayesian methods for generalized linear mixed models abstract bayesian methods furnish an attractive approach to inference in generalized linear mixed models in the absence of subjective prior information for the random effect variance components these analyses are typically conducted using either the standard invariant prior for normal responses or diffuse conjugate priors previous work has pointed out serious difficulties with both strategies and we show here that as in normal mixed models the standard invariant prior leads to an improper posterior distribution for generalized linear mixed models this article proposes and investigates two alternative reference i e objective or noninformative priors an approximate uniform shrinkage prior and an approximate jeffreys s prior we give conditions for the existence of the posterior distribution under any prior for the variance components in conjunction with a uniform prior for the fixed effects the approximate uniform shrinkage prior is shown to satisfy these conditions for several families of d

gibbs states of the hopfield model with extensively many patterns we consider the hopfield model withm n n patterns wheren is the number of neurons we show that if is sufficiently small and the temperature sufficiently low then there exist disjoint gibbs states for each of the stored patterns almost surely with respect to the distribution of the random patterns this solves a provlem left open in previous work the key new ingredient is a self averaging result on the free energy functional this result has considerable additional interest and some consequences are discussed a similar result for the free energy of the sherrington kirkpatrick model is also given

bayesian latent variable models for clustered mixed outcomes a general framework is proposed for modelling clustered mixed outcomes a mixture of generalized linear models is used to describe the joint distribution of a set of underlying variables and an arbitrary function relates the underlying variables to be observed outcomes the model accommodates multilevel data structures general covariate effects and distinct link functions and error distributions for each underlying variable within the framework proposed novel models are developed for clustered multiple binary unordered categorical and joint discrete and continuous outcomes a markov chain monte carlo sampling algorithm is described for estimating the posterior distributions of the parameters and latent variables because of the flexibility of the modelling framework and estimation procedure extensions to ordered categorical outcomes and more complex data structures are straightforward the methods are illustrated by using data from a reproductive toxicity study

on the existence of thermodynamics for the generalized random energy model derrida s generalized random energy model is considered almost sure andlp convergence of the free energy at any inverse temperature are proven for an arbitrary numbern of hierarchical levels the explicit form of the free energy is given in the most general case and the limitn is discussed

on power and sample size calculations for likelihood ratio tests in generalized linear models summary a direct extension of the approach described in self mauritsen and ohara 1992 biometrics 48 31 39 for power and sample size calculations in generalized linear models is presented the major feature of the proposed approach is that the modification accommodates both a finite and an infinite number of covariate configurations furthermore for the approximation of the noncentrality of the noncentral chi square distribution for the likelihood ratio statistic a simplification is provided that not only reduces substantial computation but also maintains the accuracy simulation studies are conducted to assess the accuracy for various model configurations and covariate distributions

sample size determination for an exponential survival model with an unrestricted covariate we derive formulae for estimating sample size and power for detecting the effect of an unrestricted covariate on survival time these are useful in designing survival studies with different patterns of recruitment and follow up when survival time is exponentially distributed we use the asymptotic covariance matrix conditional expectation and taylor s expansion techniques to develop these formulae computer simulations indicate that the asymptotic approximations used in developing the formulae are good over a range of parameter values and different patterns of recruitment and follow up that are relevant to survival studies

recursive parameter estimation using incomplete data abstract stochastic approximation procedures are considered for the estimation of parameters using incomplete data one procedure is stated and illustrated which often leads to asymptotically efficient estimators others are developed which although possibly not optimal in the above sense will be very much easier to apply this will be particularly advantageous when quick recursive estimates are required examples are given and a link is made between one of the sub optimal methods and the em algorithm author

a note on gauss hermite quadrature summary for gauss hermite quadrature we consider a systematic method for transforming the variable of integration so that the integrand is sampled in an appropriate region the effectiveness of the quadrature then depends on the ratio of the integrand to some gaussian density being a smooth function well approximated by a low order polynomial it is pointed out that in this approach order one gauss hermite quadrature becomes the laplace approximation thus the quadrature as implemented here can be thought of as a higher order laplace approximation

markov chains for exploring posterior distributions several markov chain methods are available for sampling from a posterior distribution two important examples are the gibbs sampler and the metropolis algorithm in addition several strategies are available for constructing hybrid algorithms this paper outlines some of the basic methods and strategies and discusses some related theoretical and practical issues on the theoretical side results from the theory of general state space markov chains can be used to obtain convergence rates laws of large numbers and central limit theorems for estimates obtained from markov chain methods these theoretical results can be used to guide the construction of more efficient algorithms for the practical use of markov chain methods standard simulation methodology provides several variance reduction techniques and also give guidance on the choice of sample size and allocation

a revisit to sample size and power calculations for testing odds ratio in two independent binomials we reexamine the subject of sample size determination ssd when testing logarithm of odds ratio or against zero in two independent binomials four common approaches are considered a closed form ss formula based on the wald test nw closed form formulas that meet ss requirement by score and exact tests respectively ns and ne and a numerical approach to calculating ss based on likelihood ratio lr tests nl several practically useful findings are presented first nw is a strictly convex function of or for or 1 and or 1 respectively implying that ss calculated by nw does not necessarily decrease as or gets further away from 1 however minimum ss often occurs at or values that are deemed relatively extreme and rare in real life ns ne and nl decrease monotonically as or diverges from 1 secondly the optimal sampling ratio osr between two independent binomials that yields maximum power for a given total ss is not always 1 1 but depends on the odds of outcome in each arm nw benefits the most from the application of osr in that total ss can be significantly reduced as compared to the commonly used 1 1 sampling ratio savings in ss by osr in ns nl and ne are relatively immaterial from a practical perspective finally we use simulation studies to examine the power loyalty of each ss approach and explore penalized likelihood as a remedy for undermined power loyalty

high dimensional maximum marginal likelihood item factor analysis by adaptive quadrature although the bock aitkin likelihood based estimation method for factor analysis of dichotomous item response data has important advantages over classical analysis of item tetrachoric correlations a serious limitation of the method is its reliance on fixed point gauss hermite g h quadrature in the solution of the likelihood equations and likelihood ratio tests when the number of latent dimensions is large computational considerations require that the number of quadrature points per dimension be few but with large numbers of items the dispersion of the likelihood given the response pattern becomes so small that the likelihood cannot be accurately evaluated with the sparse fixed points in the latent space in this paper we demonstrate that substantial improvement in accuracy can be obtained by adapting the quadrature points to the location and dispersion of the likelihood surfaces corresponding to each distinct pattern in the data in particular we show that adaptive g h quadrature combined with mean and covariance adjustments at each iteration of an em algorithm produces an accurate fast converging solution with as few as two points per dimension evaluations of this method with simulated data are shown to yield accurate recovery of the generating factor loadings for models of upto eight dimensions unlike an earlier application of adaptive gibbs sampling to this problem by meng and schilling the simulations also confirm the validity of the present method in calculating likelihood ratio chi square statistics for determining the number of factors required in the model finally we apply the method to a sample of real data from a test of teacher qualifications

convergence of a stochastic approximation version of the em algorithm the expectation maximization em algorithm is a powerful computational technique for locating maxima of functions it is widely used in statistics for maximum likelihood or maximum a posteriori estimation in incomplete data models in certain situations however this method is not applicable because the expectation step cannot be performed in closed form to deal with these problems a novel method is introduced the stochastic approximation em saem which replaces the expectation step of the em algorithm by one iteration of a stochastic approximation procedure the convergence of the saem algorithm is established under conditions that are applicable to many practical situations moreover it is proved that under mild additional conditions the attractive stationary points of the saem algorithm correspond to the local maxima of the function presented to support our findings

latent class models for diary method data parameter estimation by local computations the increasing use of diary methods calls for the development of appropriate statistical methods for the resulting panel data latent markov models can be used to model both individual differences and temporal dynamics the computational burden associated with these models can be overcome by exploiting the conditional independence relations implied by the model this is done by associating a probabilistic model with a directed acyclic graph and applying transformations to the graph the structure of the transformed graph provides a factorization of the joint probability function of the manifest and latent variables which is the basis of a modified and more efficient e step of the em algorithm the usefulness of the approach is illustrated by estimating a latent markov model involving a large number of measurement occasions and subsequently a hierarchical extension of the latent markov model that allows for transitions at different levels furthermore logistic regression techniques are used to incorporate restrictions on the conditional probabilities and to account for the effect of covariates throughout models are illustrated with an experience sampling methodology study on the course of emotions among anorectic patients

bayes modal estimation in item response models this article describes a bayesian framework for estimation in item response models with two stage prior distributions on both item and examinee populations strategies for point and interval estimation are discussed and a general procedure based on the em algorithm is presented details are given for implementation under one two and three parameter binary logistic irt models novel features include minimally restrictive assumptions about examinee distributions and the exploitation of dependence among item parameters in a population of interest improved estimation in a moderately small sample is demonstrated with simulated data

sample size determination for logistic regression revisited there is no consensus on the approach to compute the power and sample size with logistic regression some authors use the likelihood ratio test some use the test on proportions some suggest various approximations to handle the multivariate case we advocate the use of the wald test since the z score is routinely used for statistical significance testing of regression coefficients the null variance formula became popular from early studies which contradicts modern software which utilizes the method of maximum likelihood estimation mle when the variance of the mle is estimated at the mle not at the null we derive general wald based power and sample size formulas for logistic regression and then apply them to binary exposure and confounder to obtain a closed form expression these formulas are applied to minimize the total sample size in a case control study to achieve a given power by optimizing the ratio of controls to cases approximately the optimal number of controls to cases is equal to the square root of the alternative odds ratio our sample size and power calculations can be carried out online at www dartmouth edu approximately eugened

a monte carlo implementation of the em algorithm and the poor man s data augmentation algorithms abstract the first part of this article presents the monte carlo implementation of the e step of the em algorithm given the current guess to the maximizer of the posterior distribution latent data patterns are generated from the conditional predictive distribution the expected value of the augmented log posterior is then updated as a mixture of augmented log posteriors mixed over the generated latent data patterns multiple imputations in the m step of the algorithm this mixture is maximized to obtain the update to the maximizer of the observed posterior the gradient and hessian of the observed log posterior are also expressed as mixtures mixed over the multiple imputations the relation between the monte carlo em mcem algorithm and the data augmentation algorithm is noted two modifications to the mcem algorithm the poor man s data augmentation algorithms which allow for the calculation of the entire posterior are then presented these approximations serve as diagnostics for the validity o

the thermodynamic limit in mean field spin glass models we present a simple strategy in order to show the existence and uniqueness of the infinite volume limit of thermodynamic quantities for a large class of mean field disordered models as for example the sherrington kirkpatrick model and the derrida p spin model the main argument is based on a smooth interpolation between a large system made of n spin sites and two similar but independent subsystems made of n 1 and n 2 sites respectively with n 1 n 2 n the quenched average of the free energy turns out to be subadditive with respect to the size of the system this gives immediately convergence of the free energy per site in the infinite volume limit moreover a simple argument based on concentration of measure gives the almost sure convergence with respect to the external noise similar results hold also for the ground state energy per site

power and sample size calculations for longitudinal studies comparing rates of change with a time varying exposure existing study design formulas for longitudinal studies have assumed that the exposure is time invariant we derived sample size formulas for studies comparing rates of change by exposure when the exposure varies with time within a subject focusing on observational studies where this variation is not controlled by the investigator two scenarios are considered one assuming that the effect of exposure on the response is acute and the other assuming that it is cumulative we show that accurate calculations can often be obtained by providing the intraclass correlation of exposure and the exposure prevalence at each time point when comparing rates of change studies with a time varying exposure are in general less efficient than studies with a time invariant one we provide a public access program to perform the calculations described in the paper http www hsph harvard edu faculty spiegelman optitxs html copyright 2009 john wiley sons ltd

lord wingersky algorithm version 2 0 for hierarchical item factor models with applications in test scoring scale alignment and model fit testing lord and wingersky s appl psychol meas 8 453 461 1984 recursive algorithm for creating summed score based likelihoods and posteriors has a proven track record in unidimensional item response theory irt applications extending the recursive algorithm to handle multidimensionality is relatively simple especially with fixed quadrature because the recursions can be defined on a grid formed by direct products of quadrature points however the increase in computational burden remains exponential in the number of dimensions making the implementation of the recursive algorithm cumbersome for truly high dimensional models in this paper a dimension reduction method that is specific to the lord wingersky recursions is developed this method can take advantage of the restrictions implied by hierarchical item factor models e g the bifactor model the testlet model or the two tier model such that a version of the lord wingersky recursive algorithm can operate on a dramatically reduced set of quadrature points for instance in a bifactor model the dimension of integration is always equal to 2 regardless of the number of factors the new algorithm not only provides an effective mechanism to produce summed score to irt scaled score translation tables properly adjusted for residual dependence but leads to new applications in test scoring linking and model fit checking as well simulated and empirical examples are used to illustrate the new applications

limit theorems for sums of random exponentials we study limiting distributions of exponential sums open image in new window as t n where x i are i i d random variables two cases are considered a ess sup x i 0 and b ess sup x i we assume that the function h x log p x i x case b or h x log p x i 1 x case a is regularly varying at with index 1 case b or 0 case a the appropriate growth scale of n relative to t is of the form open image in new window where the rate function h0 t is a certain asymptotic version of the function open image in new window case b or open image in new window case a we have found two critical points 1 2 below which the law of large numbers and the central limit theorem respectively break down for 0 2 under the slightly stronger condition of normalized regular variation of h we prove that the limit laws are stable with characteristic exponent 0 2 and skewness parameter 1

a simple approach to power and sample size calculations in logistic regression and cox regression models for a given regression problem it is possible to identify a suitably defined equivalent two sample problem such that the power or sample size obtained for the two sample problem also applies to the regression problem for a standard linear regression model the equivalent two sample problem is easily identified but for generalized linear models and for cox regression models the situation is more complicated an approximately equivalent two sample problem may however also be identified here in particular we show that for logistic regression and cox regression models the equivalent two sample problem is obtained by selecting two equally sized samples for which the parameters differ by a value equal to the slope times twice the standard deviation of the independent variable and further requiring that the overall expected number of events is unchanged in a simulation study we examine the validity of this approach to power calculations in logistic regression and cox regression models several different covariate distributions are considered for selected values of the overall response probability and a range of alternatives for the cox regression model we consider both constant and non constant hazard rates the results show that in general the approach is remarkably accurate even in relatively small samples some discrepancies are however found in small samples with few events and a highly skewed covariate distribution comparison with results based on alternative methods for logistic regression models with a single continuous covariate indicates that the proposed method is at least as good as its competitors the method is easy to implement and therefore provides a simple way to extend the range of problems that can be covered by the usual formulas for power and sample size determination

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

maximum likelihood estimation of limited and discrete dependent variable models with nested random effects abstract gauss hermite quadrature is often used to evaluate and maximize the likelihood for random component probit models unfortunately the estimates are biased for large cluster sizes and or intraclass correlations we show that adaptive quadrature largely overcomes these problems we then extend the adaptive quadrature approach to general random coefficient models with limited and discrete dependent variables the models can include several nested random effects intercepts and coefficients representing unobserved heterogeneity at different levels of a hierarchical dataset the required multivariate integrals are evaluated efficiently using spherical quadrature rules simulations show that adaptive quadrature performs well in a wide range of situations

generalized multilevel structural equation modeling a unifying framework for generalized multilevel structural equation modeling is introduced the models in the framework called generalized linear latent and mixed models gllamm combine features of generalized linear mixed models glmm and structural equation models sem and consist of a response model and a structural model for the latent variables the response model generalizes glmms to incorporate factor structures in addition to random intercepts and coefficients as in glmms the data can have an arbitrary number of levels and can be highly unbalanced with different numbers of lower level units in the higher level units and missing data a wide range of response processes can be modeled including ordered and unordered categorical responses counts and responses of mixed types the structural model is similar to the structural part of a sem except that it may include latent and observed variables varying at different levels for example unit level latent variables factors or random coefficients can be regressed on cluster level latent variables special cases of this framework are explored and data from the british social attitudes survey are used for illustration maximum likelihood estimation and empirical bayes latent score prediction within the gllamm framework can be performed using adaptive quadrature in gllamm a freely available program running in stata

sample size determination for multiple continuous risk factors in case control studies for desired power of detecting the association between a disease and several potential risk factors in case control studies yang 1978 and halperin 1980 proposed a probit risk model and derived approximate sample size formulas respectively the lack of direct interpretation of the parameters in the probit risk model and the complexity of these approximate sample size formulas however limit the application of yang s and halperin s procedures in practice in this paper we assume an exponential risk model for the incidence rate under our model assumptions we derive an exact sample size calculation procedure that is simpler and easier to use than yang s and halperin s

marginal maximum likelihood estimation of item parameters application of an em algorithm maximum likelihood estimation of item parameters in the marginal distribution integrating over the distribution of ability becomes practical when computing procedures based on an em algorithm are used by characterizing the ability distribution empirically arbitrary assumptions about its form are avoided the em procedure is shown to apply to general item response models lacking simple sufficient statistics for ability this includes models with more than one latent dimension

optimal combination of number of participants and number of repeated measurements in longitudinal studies with time varying exposure in the context of observational longitudinal studies we explored the values of the number of participants and the number of repeated measurements that maximize the power to detect the hypothesized effect given the total cost of the study we considered two different models one that assumes a transient effect of exposure and one that assumes a cumulative effect results were derived for a continuous response variable whose covariance structure was assumed to be damped exponential and a binary time varying exposure under certain assumptions we derived simple formulas for the approximate solution to the problem in the particular case in which the response covariance structure is assumed to be compound symmetry results showed the importance of the exposure intraclass correlation in determining the optimal combination of the number of participants and the number of repeated measurements and therefore the optimized power thus incorrectly assuming a time invariant exposure leads to inefficient designs we also analyzed the sensitivity of results to dropout mis specification of the response correlation structure allowing a time varying exposure prevalence and potential confounding impact we illustrated some of these results in a real study in addition we provide software to perform all the calculations required to explore the combination of the number of participants and the number of repeated measurements

sample size determination for case control studies the influence of the joint distribution of exposure and confounder in case control studies the results about how the exposure distribution affects sample size are well known this paper extends previous results by incorporating the effect of a confounder into the calculation of sample size for a desired size and power of a statistical test the paper also includes a quantitative discussion on the influence of the joint distribution for exposure to a putative cause and a a confounder on required sample sizes the results show that to detect a specified alternative for a given size and power the required sample size decreases as either the variance of exposure or the effect of exposure on disease increases the required sample size however increases as either the variance of the confounder or the effect of the confounder on disease increases generally the higher is the absolute value of the simple correlation between the exposure and the confounder the larger is the required sample size

a general structural equation model with dichotomous ordered categorical and continuous latent variable indicators a structural equation model is proposed with a generalized measurement part allowing for dichotomous and ordered categorical variables indicators in addition to continuous ones a computationally feasible three stage estimator is proposed for any combination of observed variable types this approach provides large sample chi square tests of fit and standard errors of estimates for situations not previously covered two multiple indicator modeling examples are given one is a simultaneous analysis of two groups with a structural equation model underlying skewed likert variables the second is a longitudinal model with a structural model for multivariate probit regressions

estimation of generalized linear latent variable models generalized linear latent variable models gllvms as defined by bartholomew and knott enable modelling of relationships between manifest and latent variables they extend structural equation modelling techniques which are powerful tools in the social sciences however because of the complexity of the log likelihood function of a gllvm an approximation such as numerical integration must be used for inference this can limit drastically the number of variables in the model and can lead to biased estimators we propose a new estimator for the parameters of a gllvm based on a laplace approximation to the likelihood function and which can be computed even for models with a large number of variables the new estimator can be viewed as an m estimator leading to readily available asymptotic properties and correct inference a simulation study shows its excellent finite sample properties in particular when compared with a well established approach such as lisrel a real data example on the measurement of wealth for the computation of multidimensional inequality is analysed to highlight the importance of the methodology

sample size considerations for the evaluation of prognostic factors in survival analysis when the role of a new prognostic factor is investigated careful planning of an appropriate study is required this includes an assessment of the power of the study in terms of sample sizes an adequate analysis of the independent prognostic effect of a new factor has to be adjusted for the existing standard factors with survival time as endpoint this will usually be done with the cox proportional hazards model sample size and power formulae in survival analysis have been developed by schoenfeld for randomized treatment comparisons in the analysis of prognostic factors the covariates included are expected to be correlated with the factor of primary interest in this situation the existing sample size and power formulae are not valid and may not be applied in this paper schoenfeld s formula is first extended to the situation where a correlated factor is included in the analysis the validity of the resulting approximate asymptotic formula is investigated for its asymptotic behaviour by numerical integration and for its finite behaviour by simulation second an approximate formula for sample size and power is provided to detect an interaction between the interesting and a second correlated factor this extends the formula for independent effects finally the approach is illustrated by an example on the prognostic impact of dna ploidy and other factors in advanced ovarian cancer

a simple method of sample size calculation for linear and logistic regression a sample size calculation for logistic regression involves complicated formulae this paper suggests use of sample size formulae for comparing means or for comparing proportions in order to calculate the required sample size for a simple logistic regression model one can then adjust the required sample size for a multiple logistic regression model by a variance inflation factor this method requires no assumption of low response probability in the logistic model as in a previous publication one can similarly calculate the sample size for linear regression models this paper also compares the accuracy of some existing sample size software for logistic regression with computer power simulations an example illustrates the methods

sample size and optimal design for logistic regression with binary interaction there is no consensus on what test to use as the basis for sample size determination and power analysis some authors advocate the wald test and some the likelihood ratio test we argue that the wald test should be used because the z score is commonly applied for regression coefficient significance testing and therefore the same statistic should be used in the power function we correct a widespread mistake on sample size determination when the variance of the maximum likelihood estimate mle is estimated at null value in our previous paper we developed a correct sample size formula for logistic regression with single exposure statist med 2007 26 18 3385 3397 in the present paper closed form formulas are derived for interaction studies with binary exposure and covariate in logistic regression the formula for the optimal control case ratio is derived such that it maximizes the power function given other parameters our sample size and power calculations with interaction can be carried out online at www dartmouth edu approximately eugened

log mean linear models for binary data this paper is devoted to the theory and application of a novel class of models for binary data which we call log mean linear lml models the characterizing feature of these models is that they are specied by linear constraints on the

a correlated probit model for joint modeling of clustered binary and continuous responses a difficulty in joint modeling of continuous and discrete response variables is the lack of a natural multivariate distribution for joint modeling of clustered observations on binary and continuous responses we study a correlated probit model that has an underlying normal latent variable for the binary responses catalano and ryan have factored the model into a marginal and a conditional component and used generalized estimating equations methodology to estimate the effects we propose a monte carlo expectation conditional maximization algorithm for finding maximum likelihood estimates of the mixed model itself extending and accelerating an algorithm for models with binary responses we demonstrate the methodology with a developmental toxicity study measuring fetal weight and a binary malformation status for several litters of mice a simulation study suggests that efficiency gains of joint fittings over separate fittings of the response variables occur mainly for small datasets with strong correlation

power sample size and smallest detectable effect determination for multivariate studies this paper discusses some general methods for determining approximate power sample size and smallest detectable effect for studies of multiple risk factors these methods are based on standard large sample formulae for determining the power of chi square tests and emphasis is given to determinations for pearson chi 2 tests in multiway contingency tables the methods are illustrated in application to the design of a clinical trial of the preventive effect of alpha tocopherol ascorbic acid and beta carotene on colon polyp recurrence and a case control study of the joint effect of smoking and asbestos exposure on lung cancer incidence

sample size tables for logistic regression sample size tables are presented for epidemiologic studies which extend the use of whittemore s formula the tables are easy to use for both simple and multiple logistic regressions monte carlo simulations are performed which show three important results firstly the sample size tables are suitable for studies with either high or low event proportions secondly although the tables can be inaccurate for risk factors having double exponential distributions they are reasonably adequate for normal distributions and exponential distributions finally the power of a study varies both with the number of events and the number of individuals at risk

on the relationship between the higher order factor model and the hierarchical factor model the relationship between the higher order factor model and the hierarchical factor model is explored formally we show that the schmid leiman transformation produces constrained hierarchical factor solutions using a generalized schmid leiman transformation and its inverse we show that for any unconstrained hierarchical factor model there is an equivalent higher order factor model with direct effects loadings on the manifest variables from the higher order factors therefore the class of higher order factor models without direct effects of higher order factors is nested within the class of unconstrained hierarchical factor models in light of these formal results we discuss some implications for testing the higher order factor model and the issue of general factor an interesting aspect concerning the efficient fitting of the higher order factor model with direct effects is noted

a modified approach to estimating sample size for simple logistic regression with one continuous covariate different methods for the calculation of sample size for simple logistic regression lr with one normally distributed continuous covariate give different results sometimes the difference can be large furthermore some methods require the user to specify the prevalence of cases when the covariate equals its population mean rather than the more natural population prevalence we focus on two commonly used methods and show through simulations that the power for a given sample size may differ substantially from the nominal value for one method especially when the covariate effect is large while the other method performs poorly if the user provides the population prevalence instead of the required parameter we propose a modification of the method of hsieh et al that requires specification of the population prevalence and that employs schouten s sample size formula for a t test with unequal variances and group sizes this approach appears to increase the accuracy of the sample size estimates for lr with one continuous covariate

variational approximations for categorical causal modeling with latent variables latent class models in the social and behavioral sciences have remained structurally simple one reason for this is that inference in statistical models can be computationally difficult methods for approximate inference known as variational approximations which have been developed in the machine learning graphical modeling and statistical physics literatures can be used to alleviate the computational difficulties of inference for latent variable models the aim of the present article is to set these methods alongside some social and behavioral science literature to which they are relevant and in particular to consider their potential for categorical causal modeling using latent class analysis we have collated a number of popular categorical data models with latent variables and causal structure typically incorporating a markovian structure the efficacy of the approximation methods has been demonstrated through simulations related to an important behavioral science model

fitting full information item factor models and an empirical investigation of bridge sampling abstract based on item response theory bock and aitken introduced a method of item factor analysis termed full information item factor fiif analysis by bartholomew because it uses all distinct item response vectors as data but a limitation of their fitting algorithm is its reliance on fixed point gauss hermite quadrature which can produce appreciable numerical errors especially in high dimension problems the first purpose of this article is to offer more reliable methods by using recent advances in statistical computation specifically we illustrate two ways of implementing monte carlo expectation maximization em algorithm to fit a fiif model using the gibbs sampler to carry out the computation for the e steps we also show how to use bridge sampling to simulate the likelihood ratios for monitoring the convergence of a monte carlo em a strategy that is useful in general simulations demonstrate substantial improvement over bock and aitken s algorithm in recovering known factor loadings in hig

maximizing generalized linear mixed model likelihoods with an automated monte carlo em algorithm summary two new implementations of the em algorithm are proposed for maximum likelihood fitting of generalized linear mixed models both methods use random independent and identically distributed sampling to construct monte carlo approximations at the e step one approach involves generating random samples from the exact conditional distribution of the random effects given the data by rejection sampling using the marginal distribution as a candidate the second method uses a multivariate t importance sampling approximation in many applications the two methods are complementary rejection sampling is more efficient when sample sizes are small whereas importance sampling is better with larger sample sizes monte carlo approximation using random samples allows the monte carlo error at each iteration to be assessed by using standard central limit theory combined with taylor series methods specifically we construct a sandwich variance estimate for the maximizer at each approximate e step this suggests a rule for automatically increasing the monte carlo sample size after iterations in which the true em step is swamped by monte carlo error in contrast techniques for assessing monte carlo error have not been developed for use with alternative implementations of monte carlo em algorithms utilizing markov chain monte carlo e step approximations three different data sets including the infamous salamander data of mccullagh and nelder are used to illustrate the techniques and to compare them with the alternatives the results show that the methods proposed can be considerably more efficient than those based on markov chain monte carlo algorithms however the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension

maximum likelihood estimation for spatial models by markov chain monte carlo stochastic approximation we propose a two stage algorithm for computing maximum likelihood estimates for a class of spatial models the algorithm combines markov chain monte carlo methods such as the metropolis hastings green algorithm and the gibbs sampler and stochastic approximation methods such as the off line average and adaptive search direction a new criterion is built into the algorithm so stopping is automatic once the desired precision has been set simulation studies and applications to some real data sets have been conducted with three spatial models we compared the algorithm proposed with a direct application of the classical robbins monro algorithm using wiebe s wheat data and found that our procedure is at least 15 times faster

sample size calculations for logistic and poisson regression models summary a method is proposed for improving sample size calculations for logistic and poisson regression models by incorporating the limiting value of the maximum likelihood estimates of nuisance parameters under the composite null hypothesis the method modifies existing approaches of whittemnore 1981 and signorini 1991 and provides explicit formulae for determining the sample size needed to test hypotheses about a single parameter at a specified significance level and power simulation studies assess its accuracy for various model configurations and covariate distributions the results show that the proposed method is more accurate than the previous approaches over the range of conditions considered here

on a third order phase transition the asymptotic behaviour of random variables of the general form ln sum limits i 1 kappa n exp n 1 p beta zeta i with independent identically distributed random variables i is studied this generalizes the random energy model of derrida in the limitn there occurs a particular kind of phase transition which does not incorporate a bifurcation phenomenon or symmetry breaking the hypergeometric character of the problem see definitions of sect 4 its function and its entropy function are discussed

sample size for poisson regression summary for the poisson regression model an exact expression for fisher s information matrix based upon the moment generating function of the distribution of covariates is calculated this parallels a similar approximate calculation by whittemore 1981 for logistic regression the resulting asymptotic variance of the maximum likelihood estimate of the parameters is used to calculate the sample size required to test hypotheses about the parameters at a specified significance and power methods for calculating sample size are derived for various distributions of a single covariate and for a family of multivariate exponential type distributions of multiple covariates the procedures are illustrated with two examples

generalized random energy model the formulae for the free energy when the driving distributions in generalized random energy model grem are of the form ce x gamma for 1 are derived the large deviation technique allows the use of different distributions at different levels of the grem as an illustration we consider in detail a two level grem with exponential and gaussian distributions this simple case itself leads to interesting phenomena

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

accurate approximations for posterior moments and marginal densities abstract this article describes approximations to the posterior means and variances of positive functions of a real or vector valued parameter and to the marginal posterior densities of arbitrary i e not necessarily positive parameters these approximations can also be used to compute approximate predictive densities to apply the proposed method one only needs to be able to maximize slightly modified likelihood functions and to evaluate the observed information at the maxima nevertheless the resulting approximations are generally as accurate and in some cases more accurate than approximations based on third order expansions of the likelihood and requiring the evaluation of third derivatives the approximate marginal posterior densities behave very much like saddle point approximations for sampling distributions the principal regularity condition required is that the likelihood times prior be unimodal

full information item bi factor analysis a plausibles factor solution for many types of psychological and educational tests is one that exhibits a general factor ands 1 group or method related factors the bi factor solution results from the constraint that each item has a nonzero loading on the primary dimension and at most one of thes 1 group factors this paper derives a bi factor item response model for binary response data in marginal maximum likelihood estimation of item parameters the bi factor restriction leads to a major simplification of likelihood equations and a permits analysis of models with large numbers of group factors b permits conditional dependence within identified subsets of items and c provides more parsimonious factor solutions than an unrestricted full information item factor analysis in some cases

high dimensional exploratory item factor analysis by a metropolis hastings robbins monro algorithm a metropolis hastings robbins monro mh rm algorithm for high dimensional maximum marginal likelihood exploratory item factor analysis is proposed the sequence of estimates from the mh rm algorithm converges with probability one to the maximum likelihood solution details on the computer implementation of this algorithm are provided the accuracy of the proposed algorithm is demonstrated with simulations as an illustration the proposed algorithm is applied to explore the factor structure underlying a new quality of life scale for children it is shown that when the dimensionality is high mh rm has advantages over existing methods such as numerical quadrature based em algorithm extensions of the algorithm to other modeling frameworks are discussed

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

sample size determinations using logistic regression with pilot data suppose the goal of a projected study is to estimate accurately the value of a prediction proportion p that is specific to a given set of covariates available pilot data show that 1 the covariates are influential in determining the value of p and 2 their relationship to p can be modelled as a logistic regression a sample size justification for the projected study can be based on the logistic model the resulting sample sizes not only are more reasonable than the usual binomial sample size values from a scientific standpoint since they are based on a model that is more realistic but also give smaller prediction standard errors than the binomial approach with the same sample size in appropriate situations the logistic based sample sizes could make the difference between a feasible proposal and an unfeasible binomial based proposal an example using pilot study data of dental radiographs demonstrates the methods

sample size calculations for evaluating mediation this paper presents sample size calculations for testing the mediation of the effect of a primary predictor by an intermediate variable this problem is related to validating surrogate markers and to testing the effect of a primary predictor in the presence of confounders for those problems proposals for sample size calculation exist in the literature and can be adapted to the problem of mediation methods based on the variance inflation factor in linear regression provide exact sample size calculations for the linear model and approximations for the logistic poisson and cox models we propose another procedure based on simulation of the underlying data structure with applications to the logistic and cox models for the poisson model a new analytic method is also proposed the behavior of the different proposals is investigated by means of simulation studies copyright 2008 john wiley sons ltd

a practical approach to computing power for generalized linear models with nominal count or ordinal responses data analysts facing study design questions on a regular basis could derive substantial benefit from a straightforward and unified approach to power calculations for generalized linear models many current proposals for dealing with binary ordinal or count outcomes are conceptually or computationally demanding limited in terms of accommodating covariates and or have not been extensively assessed for accuracy assuming moderate sample sizes here we present a simple method for estimating conditional power that requires only standard software for fitting the desired generalized linear model for a non continuous outcome the model is fit to an appropriate expanded data set using easily calculated weights that represent response probabilities given the assumed values of the parameters the variance covariance matrix resulting from this fit is then used in conjunction with an established non central chi square approximation to the distribution of the wald statistic alternatively the model can be re fit under the null hypothesis to approximate power based on the likelihood ratio statistic we provide guidelines for constructing a representative expanded data set to allow close approximation of unconditional power based on the assumed joint distribution of the covariates relative to prior proposals the approach proves particularly flexible for handling one or more continuous covariates without any need for discretizing we illustrate the method for a variety of outcome types and covariate patterns using simulations to demonstrate its accuracy for realistic sample sizes

logistic approximation to the normal the kl rationale a rationale is proposed for approximating the normal distribution with a logistic distribution using a scaling constant based on minimizing the kullback leibler kl information that is the expected amount of information available in a sample to distinguish between two competing distributions using a likelihood ratio lr test assuming one of them is true the new constant 1 749 computed assuming the normal distribution is true yields an approximation that is an improvement in fit of the tails of the distribution as compared to the minimax constant of 1 702 widely used in item response theory irt the minimax constant is by definition marginally better in its overall maximal error it is argued that the kl constant is more statistically appropriate for use in irt

sample size for logistic regression with small response probability abstract the fisher information matrix for the estimated parameters in a multiple logistic regression can be approximated by the augmented hessian matrix of the moment generating function for the covariates the approximation is valid when the probability of response is small with its use one can obtain a simple closed form estimate of the asymptotic covariance matrix of the maximum likelihood parameter estimates and thus approximate sample sizes needed to test hypotheses about the parameters the method is developed for selected distributions of a single covariate and for a class of exponential type distributions of several covariates it is illustrated with an example concerning risk factors for coronary heart disease