models for longitudinal data with random effects and ar 1 errors abstract for longitudinal data on several individuals linear models that contain both random effects across individuals and autocorrelation in the within individual errors are studied a score test for autocorrelation in the within individual errors for the conditional independence random effects model is first developed an explicit maximum likelihood estimation procedure using the scoring method for the model with random effects and autoregressive ar 1 errors is then derived empirical bayes estimation of the random effects and prediction of future responses of an individual based on this random effects with ar 1 errors model are also considered a numerical example is presented to illustrate these methods

a comparison of permutation and mixed model regression methods for the analysis of simulated data in the context of a group randomized trial our first purpose was to determine whether in the context of a group randomized trial grt with gaussian errors permutation or mixed model regression methods fare better in the presence of measurable confounding in terms of their monte carlo type i error rates and power our results indicate that given a proper randomization the type i error rate is similar for both methods whether unadjusted or adjusted even in small studies however our results also show that should the investigator face the unfortunate circumstance in which modest confounding exists in the only realization available the unadjusted analysis risks a type i error in this regard there was little to distinguish the two methods finally our results show that power is similar for the two methods and not surprisingly better for the adjusted tests our second purpose was to examine the relative performance of permutation and mixed model regression methods in the context of a grt when the normality assumptions underlying the mixed model are violated published studies have examined the impact of violation of this assumption at the member level only our findings indicate that both methods perform well when the assumption is violated so long as the icc is very small and the design is balanced at the group level however at icc 0 01 the permutation test carries the nominal type i error rate while the model based test is conservative and so less powerful binomial group and member level errors did not otherwise change the relative performance of the two methods with regard to confounding copyright 2005 john wiley sons ltd

parameter expansion to accelerate em the px em algorithm summary the em algorithm and its extensions are popular tools for modal estimation but ar e often criticised for their slow convergence we propose a new method that can often make em much faster the intuitive idea is to use a covariance adjustment to correct the analysis of the m step capitalising on extra information captured in the imputed complete data the way we accomplish this is by parameter expansion we expand the complete data model while preserving the observed data model and use the expanded complete data model to generate em this parameter expanded ei m px em algorithm shares the simplicity and stability of ordinary em but has a faster rate of convergence since its m step performs a more efficient analysis the px em algorithm is illustrated for the multivariate t distribution a random effects model factor analysis probit regression and a poisson imaging model

empirical bayes estimation of rates in longitudinal studies abstract the usually irregular follow up intervals in epidemiologic studies preclude the use of classical growth curve analysis for short follow up intervals it is suggested that individual rates of change are useful for exploratory analysis empirical bayes estimates of these rates of change are recommended and developed an example of bone loss with age in women is also given

estimation and prediction in a multivariate random effects generalized linear model abstract estimation of an individual s regression coefficients in a multivariate random effects linear model is considered the problem of prediction of future responses of an individual based on the random effects model is also considered and the prediction mean squared errors of various predictors are derived and compared for the practical situation where all parameters of the model are unknown and estimates must be used

analysis of mixed finite element methods on locally refined grids we consider the mixed finite element method for locally refined triangulations a local projection operator hat pi h is defined to satisfy the commutation property that is required in the general theory of mixed methods our results can be applied to every known space of arbitrary order over rectangles or triangles optimal order error estimates and superconvergence for the flux along the gauss lines are established

a general framework for constructing and analyzing mixed finite volume methods on quadrilateral grids the overlapping covolume case we present a general framework for constructing and analyzing finite volume methods applied to the mixed formulation of second order elliptic problems on quadrilateral grids the control volumes or covolumes in the grids overlap an overlapping finite volume method of this type was first introduced by russell in t f russell tech report 3 reservoir simulation research corp tulsa ok 1995 and was tested for a variety of problems on rectangular and quadrilateral grids in z cai et al comput geosci 1 1997 pp 289 315 later in s h chou and d y kwak siam j numer anal 37 2000 pp 758 771 chou and kwak reformulated it as their mixed covolume method and proved optimal order error estimates using the covolume methodology from s h chou math comp 66 1997 pp 85 104 and s h chou and d y kwak siam j numer anal 35 1998 pp 494 507 however their treatment was restricted to the case of diagonal coefficient tensor and rectangular grids since a different approach was needed for the quadrilateral distorted rectangular case in this paper we give a new framework which can handle not only the rectangular anisotropic case but also the anisotropic and irregular grid cases in which the locally supported test functions are images of the natural unit coordinate vectors under the piola transformation our theory sheds light on how to create new test functions using quadratures and now covers russell s quadrilateral case

a noniterative sample size procedure for tests based on t distributions a noniterative sample size procedure is proposed for a general hypothesis test based on the t distribution by modifying and extending guenther s approach for the one sample and two sample t tests the generalized procedure is employed to determine the sample size for treatment comparisons using the analysis of covariance ancova and the mixed effects model for repeated measures in randomized clinical trials the sample size is calculated by adding a few simple correction terms to the sample size from the normal approximation to account for the nonnormality of the t statistic and lower order variance terms which are functions of the covariates in the model but it does not require specifying the covariate distribution the noniterative procedure is suitable for superiority tests noninferiority tests and a special case of the tests for equivalence or bioequivalence and generally yields the exact or nearly exact sample size estimate after rounding to an integer the method for calculating the exact power of the two sample t test with unequal variance in superiority trials is extended to equivalence trials we also derive accurate power formulae for ancova and mixed effects model for repeated measures and the formula for ancova is exact for normally distributed covariates numerical examples demonstrate the accuracy of the proposed methods particularly in small samples

an analysis of two period crossover designs with carry over effects the crossover design is a type of longitudinal study with subjects receiving different treatments in different time periods when carry over effects are absent the usual crossover design is structured so that all the information about treatment effects is contained in the within subject contrasts standard analyses are based on these within subject contrasts and ignore any between subject information with carry over effects present these standard analyses can be very inefficient especially for suboptimal designs we describe alternative approaches based on methods for the analysis of longitudinal data

inference of nested variance components in a longitudinal myopia intervention trial this paper was motivated by a double blind randomized clinical trial of myopia intervention in addition to the primary goal of comparing treatment effects we are concerned with the modelling of correlation that may come from two possible sources one among the longitudinal observations and the other between measurements taken from both eyes per subject the data are nested repeated measurements we suggest three models for analysis each one expresses the correlation differently in various covariance structures we articulate their differences and describe the implementations in estimation using commercial statistical software the computer output can be further utilized to perform model selection with schwarz criterion simulation studies are conducted to evaluate the performance under each model data of the myopia intervention trial are reanalysed with these models for illustration the results indicate that atropine is more effective in reducing the progression rate the rates are homogeneous across subjects and among the suggested models the one with independent random effects of two eyes fits best we conclude that model selection is a crucial step before making inference with estimates otherwise the correlation may be attributed incorrectly to a different mechanism the same conclusion applies to other variance components as well copyright 2005 john wiley sons ltd

random effects models with non parametric priors we discuss the performance of non parametric maximum likelihood npml estimators for the distribution of a univariate random effect in the analysis of longitudinal data for continuous data we analyse generated and real data sets and compare the npml method to those that assume a gaussian random effects distribution and to ordinary least squares for binary outcomes we use generated data to study the moderate and large sample performance of the npml compared with a method based on a gaussian random effect distribution in logistic regression we find that estimated fixed effects are compatible for all approaches but that appropriate standard errors for the npml require adjusting the likelihood based standard errors we conclude that the non parametric approach provides an attractive alternative to gaussian based methods though additional evaluations are necessary before it can be recommended for general use

arma covariance structures with time heteroscedasticity for repeated measures experiments abstract rochon and helms 1989 presented a model for analyzing repeated measures experiments the general linear model was used to assess the influence of covariate information and arma time series models were put forward to characterize the covariance matrix among the repeated measures practical experience has suggested however that the arma assumption of constant variances and autocovariances over time is too restrictive for many applications for example observations may be relatively stable toward the beginning of the study but become more variable toward the end this article presents a modification to this structure which provides for heteroscedasticity over time maximum likelihood ml estimation procedures are considered and the estimators are found to enjoy optimal large sample properties a scoring algorithm is described for iterating to a solution of the ml equations the model is illustrated with data from a clinical trial investigating human erythropoietin for treating anemia in end

mixed finite volume methods on nonstaggered quadrilateral grids for elliptic problems we construct and analyze a mixed finite volume method on quadrilateral grids for elliptic problems written as a system of two first order pdes in the state variable e g pressure and its flux e g darcy velocity an important point is that no staggered grids or covolumes are used to stabilize the system only a single primary grid system is adopted and the degrees of freedom are imposed on the interfaces the approximate flux is sought in the lowest order raviart thomas space and the pressure field in the rotated q1 nonconforming space furthermore we demonstrate that the present finite volume method can be interpreted as a rotated q1 nonconforming finite element method for the pressure with a simple local recovery of flux numerical results are presented for a variety of problems which confirm the usefulness and effectiveness of the method

estimation in covariance components models abstract estimation techniques for linear covariance components models are developed and illustrated with special emphasis on explaining computational processes the estimation of fixed and random effects when the variances and covariances are known is presented in bayesian terms point estimates of the unknown variances and covariances are computed using the em algorithm for maximum likelihood estimation from incomplete data the techniques are illustrated with data on law schools field mice and professional football teams

superconvergence for rectangular mixed finite elements in this paper we prove superconvergence error estimates for the vector variable for mixed finite element approximations of second order elliptic problems for the rectangular finite elements of raviart and thomas 19 and for those of brezzi et al 4 we prove that the distance inl 2 between the approximate solution and a projection of the exact one is of higher order than the error itself this result is exploited to obtain superconvergence at gaussian points and to construct higher order approximations by a local postprocessing

likelihood ratio testing of variance components in the linear mixed effects model using restricted maximum likelihood this paper reports the results of an extensive monte carlo study of the distribution of the likelihood ratio test statistic using the value of the restricted likelihood for testing random components in the linear mixed effects model when the number of fixed components remains constant the distribution of this test statistic is considered when one additional random component is added the distribution of the likelihood ratio test statistic computed using restricted maximum likelihood is compared to the likelihood ratio test statistic computed from the usual maximum likelihood the rejection proportion is computed under the null hypothesis using a mixture of chi square distributions the restricted likelihood ratio statistic has a reasonable agreement with the maximum likelihood test statistic for the parameter combinations considered the rejection proportions are in most cases less than the nominal 5 level for both test statistics though on average the rejection proportions for reml are closer to the nominal level than for ml

mean squared error properties of empirical bayes estimators in a multivariate random effects general linear model abstract estimation of an individual s regression coefficients is considered in a multivariate general linear model where it is assumed that the individual s coefficients k are subject to both fixed effects and random effects over different individuals the mean squared error matrix of a natural estimator of k is derived for any individual k in the general situation where all parameters of the model must be estimated and is shown to be smaller than the mean squared error matrix of the individual least squares estimator for every individual extension of this result to more general multiparameter estimation situations is also considered

quadrilateral h div finite elements we consider the approximation properties of quadrilateral finite element spaces of vector fields defined by the piola transform extending results previously obtained for scalar approximation the finite element spaces are constructed starting with a given finite dimensional space of vector fields on a square reference element which is then transformed to a space of vector fields on each convex quadrilateral element via the piola transform associated to a bilinear isomorphism of the square onto the element for affine isomorphisms a necessary and sufficient condition for approximation of order r 1 in l2 is that each component of the given space of functions on the reference element contain all polynomial functions of total degree at most r in the case of bilinear isomorphisms the situation is more complicated and we give a precise characterization of what is needed for optimal order l2 approximation of the function and of its divergence as applications we demonstrate degradation of the convergence order on quadrilateral meshes as compared to rectangular meshes for some standard finite element approximations of h div we also derive new estimates for approximation by quadrilateral raviart thomas elements requiring less regularity and propose a new quadrilateral finite element space which provides optimal order approximation in h div finally we demonstrate the theory with numerical computations of mixed and least squares finite element approximations of the solution of poisson s equation

reml estimation of covariance matrices with restricted parameter spaces abstract restricted parameter spaces for covariance matrices such as 2 i or i j are often used to simplify estimation in addition fixed upper and or lower bounds may be needed to ensure that estimates satisfy a priori hypotheses with multivariate variance components models several covariance matrices need to be simultaneously estimated and even with a reduced parameter space estimation can be difficult in earlier work we have discussed estimation for a widely used class of models where the variance components matrices need only be nonnegative definite in this article we extend these results to handle a wide class of restricted parameter spaces we state the conditions required for a parameterization to be a member of the class discuss the implementation of the results for several different members of the class and discuss estimation with both balanced and unbalanced data we give several examples to demonstrate the results

a random effects model for multiple characteristics with possibly missing data abstract the use of random effects models for the analysis of longitudinal data with missing responses has been discussed by several authors this article extends the random effects model for a single characteristic to the case of multiple characteristics allowing for arbitrary patterns of observed data two different structures for the covariance matrix of measurement error are considered uncorrelated error between responses and correlation of error terms at the same measurement times parameters for this model are estimated via the em algorithm the set of equations for this estimation procedure is derived these equations are appropriately modified to deal with missing data the methodology is illustrated with an example from clinical trials

model selection techniques for the covariance matrix for incomplete longitudinal data in longitudinal studies with incomplete data where the number of time points can become numerous it is often advantageous to model the covariance matrix we describe several covariance models for example mixed models compound symmetry ar 1 type models and combination models that offer parsimonious alternatives to unstructured sigma we evaluate each covariance model with longitudinal data concerning cholesterol as the repeated outcome measure we discuss strategies for deciding the best model and show a graphical technique for judging goodness of fit of covariance models

exact inference for growth curves with intraclass correlation structure summary we consider repeated observations taken over time for each of several subjects for example one might consider the growth curve of a cohort of babies over time we assume a simple linear growth curve model exact results based on sufficient statistics exact tests of the null hypothesis that a coefficient is zero or exact confidence intervals for coefficients are not available to make inference on regression coefficients when an intraclass correlation structure is assumed this paper will demonstrate that such exact inference is possible using generalized inference

tests for gaussian repeated measures with missing data in small samples for small samples of gaussian repeated measures with missing data barton and cramer recommended using the em algorithm for estimation and reducing the degrees of freedom for an analogue of rao s f approximation to wilks test computer simulations led to the conclusion that the modified test was slightly conservative for total sample size of n 40 here we consider additional methods and smaller sample sizes nin 12 24 we describe analogues of the pillai bartlett trace hotelling lawley trace and geisser greenhouse corrected univariate tests which allow for missing data eleven sample size adjustments were examined which replace n by some function of the numbers of non missing pairs of responses in computing error degrees of freedom overall simulation results allowed concluding that an adjusted test can always control test size at or below the nominal rate even with as few as 12 observations and up to 10 per cent missing data the choice of method varies with the test statistic replacing n by the mean number of non missing responses per variable works best for the geisser greenhouse test the pillai bartlett test requires the stronger adjustment of replacing n by the harmonic mean number of non missing pairs of responses for wilks and hotelling lawley an even more aggressive adjustment based on the minimum number of non missing pairs must be used

maximum likelihood estimation for incomplete repeated measures experiments under an arma covariance structure a stochastic model is presented for the analysis of incomplete repeated measures experiments the general linear model is used to relate the response measures to other variables which are thought to account for inherent variation an autoregressive moving average arma time series representation is used to model disturbance terms maximum likelihood estimation procedures are considered and the properties of these estimators are derived it is concluded that while the assumptions underpinning the arma covariance models may be somewhat restrictive they provide a useful inferential vehicle particularly in the presence of missing values

a mixed effects model for the analysis of repeated measures cross over studies a mixed effects model is developed for cross over trials in which the response is measured repeatedly within each time period relative to previous work on repeated measures cross overs the methodology synthesizes two important features first our procedure eliminates preliminary testing for carry over defined loosely as the component of a response that is due to treatment in the preceding period this is achieved by generalizing the methodology to cross over designs in which preliminary testing for carry over is unnecessary we focus largely on simple carry over that is carry over that lasts for exactly one period and is independent of the treatment administered in the period in which the carry over occurs however we also illustrate a modification of the procedure for a repeated measures cross over design which uses a more complicated model of carry over second the model allows both the between and within subject variance to differ among treatments conditions are described wherein closed form cf solutions to the variance components as well as closed form hypothesis tests of the treatment differences exist flexibility in the model is illustrated with an example in which inference based on the cf likelihood based estimates of the variance and estimates formed using an iterative routine proc mixed are compared

contextually specific effects and other generalizations of the hierarchical linear model for comparative analysis abstract contextually specific differences among members of a context pose substantive and methodological problems in multilevel analysis an important example is ethnic identity in comparative studies involving different societies ethnic and religious group membership in many societies is a basis not only for differentiation among members but also for the identification and maintenance of deeply rooted integrative ties so important is ethnicity that the existence of different groups within societies sometimes delays or prohibits the taking of censuses moreover even when ethnic information is available in national statistical data sources researchers are sometimes prohibited from reporting the results of analyses that contain ethnic detail despite these constraints information on ethnicity is often available for pluralistic societies where it is available ethnicity poses a challenging analytic problem for comparative analysis if researchers wish to compare different societies how is ethnicity

maximum likelihood computations with repeated measures application of the em algorithm abstract the purpose of this article is to consider the use of the em algorithm dempster laird and rubin 1977 for both maximum likelihood ml and restricted maximum likelihood reml estimation in a general repeated measures setting using a multivariate normal data model with linear mean and covariance structure anderson 1973 several models and methods of analysis have been proposed in recent years for repeated measures data ware 1985 presented an overview because the em algorithm is a general purpose iterative method for computing ml estimates with incomplete data it has often been used in this particular setting dempster et al 1977 andrade and helms 1984 jennrich and schluchter 1985 there are two apparently different approaches to using the em algorithm in this setting in one application each experimental unit is observed under a standard protocol specifying measurements at each of n occasions or under n conditions and incompleteness implies that the number of measurements actua

joint variable selection for fixed and random effects in linear mixed effects models it is of great practical interest to simultaneously identify the important predictors that correspond to both the fixed and random effects components in a linear mixed effects lme model typical approaches perform selection separately on each of the fixed and random effect components however changing the structure of one set of effects can lead to different choices of variables for the other set of effects we propose simultaneous selection of the fixed and random factors in an lme model using a modified cholesky decomposition our method is based on a penalized joint log likelihood with an adaptive penalty for the selection and estimation of both the fixed and random effects it performs model selection by allowing fixed effects or standard deviations of random effects to be exactly zero a constrained expectation maximization algorithm is then used to obtain the final estimates it is further shown that the proposed penalized estimator enjoys the oracle property in that asymptotically it performs as well as if the true model was known beforehand we demonstrate the performance of our method based on a simulation study and a real data example

global estimates for mixed methods for second order elliptic equations global error estimates in l2 92 l 9 and h s q 2 in r2 or a re derived for a mixed finite element method for the dirichlet problem for the elliptic operator lp div a grad p bp cp based on the raviart thomas nedelec space vh x wh c h div 92 x l2 g2 optimal order estimates are obtained for the approximation of p and the associated velocity field u a grad p bp in l2 g2 and h s 92 0 6 s k 1 and if q c r2 forp in lv2

fast em type implementations for mixed effects models the mixed effects model in its various forms is a common model in applied statistics a useful strategy for fitting this model implements em type algorithms by treating the random effects as missing data such implementations however can be painfully slow when the variances of the random effects are small relative to the residual variance in this paper we apply the working parameter approach to derive alternative em type implementations for fitting mixed effects models which we show empirically can be hundreds of times faster than the common em type implementations in our limited simulations they also compare well with the routines in s plus and stata in terms of both speed and reliability the central idea of the working parameter approach is to search for efficient data augmentation schemes for implementing the em algorithm by minimizing the augmented information over the working parameter and in the mixed effects setting this leads to a transfer of the mixed effects variances into the regression slope parameters we also describe a variation for computing the restricted maximum likelihood estimate and an adaptive algorithm that takes advantage of both the standard and the alternative em type implementations

conditional second order generalized estimating equations for generalized linear and nonlinear mixed effects models generalized linear and nonlinear mixed effects models are used extensively in health care research including applications in pharmacokinetics clinical trials and epidemiology because the underlying model may be nonlinear in the random effects there will generally be no closed form expression for the marginal likelihood or indeed for the marginal moments consequently estimation is often carried out either using numerical integration techniques or by approximating the marginal likelihood and or marginal moments using first order expansion methods an advantage of the first order methods is that they do not necessarily require specification of a conditional like lihood to estimate the regression parameters of interest however in many cases they may not take full advantage of the fact that the conditional variance depends on both fixed and random effects in this article we propose using conditional second order generalized estimating equations cgee2 to estimate both fixed and random effects param

a three step estimation procedure using local polynomial smoothing for inconsistently sampled longitudinal data parametric mixed effects models are useful in longitudinal data analysis when the sampling frequencies of a response variable and the associated covariates are the same we propose a three step estimation procedure using local polynomial smoothing and demonstrate with data where the variables to be assessed are repeatedly sampled with different frequencies within the same time frame we first insert pseudo data for the less frequently sampled variable based on the observed measurements to create a new dataset then standard simple linear regressions are fitted at each time point to obtain raw estimates of the association between dependent and independent variables last local polynomial smoothing is applied to smooth the raw estimates rather than use a kernel function to assign weights only analytical weights that reflect the importance of each raw estimate are used the standard errors of the raw estimates and the distance between the pseudo data and the observed data are considered as the measure of the importance of the raw estimates we applied the proposed method to a weight loss clinical trial and it efficiently estimated the correlation between the inconsistently sampled longitudinal data our approach was also evaluated via simulations the results showed that the proposed method works better when the residual variances of the standard linear regressions are small and the within subjects correlations are high also using analytic weights instead of kernel function during local polynomial smoothing is important when raw estimates have extreme values or the association between the dependent and independent variable is nonlinear copyright 2016 john wiley sons ltd

construction of hearing percentiles in women with non constant variance from the linear mixed effects model current age specific reference standards for adult hearing thresholds are primarily cross sectional in nature and vary in the degree of screening of the reference sample for noise induced hearing loss and other hearing problems we develop methods to construct age specific percentiles for longitudinal data that have been modelled using the linear mixed effects model we apply these methods to construct percentiles of hearing level using data from a carefully screened sample of women from the baltimore longitudinal study of aging however the variation in the residuals and random effects from the linear mixed effects model does not remain constant with age and frequency of the stimulus tone in addition the distribution of the hearing levels is not symmetric about the mean we develop a number of methods to use the output from the linear mixed effects model to construct percentiles that do not have constant variance we use a transformation of the hearing levels to provide for skewness in the final percentile curves the change in the variation of the residuals and random effects is modelled as a function of beginning age and frequency and we use this variance function to construct the hearing percentiles we present a number of approaches first we use the absolute values of the population residuals to model the total deviation about the mean as a function of beginning age and frequency second we model the standard deviation in the person specific cluster residuals as well as the standard deviation in the estimated random effects finally we use weighted least squares with the regressions on the absolute cluster residuals and absolute estimated random effects where the weights are the reciprocal of the standard deviations of their estimates

two families of mixed finite elements for second order elliptic problems two families of mixed finite elements one based on triangles and the other on rectangles are introduced as alternatives to the usual raviart thomas nedelec spaces error estimates inl 2 andh 5 are derived for these elements a hybrid version of the mixed method is also considered and some superconvergence phenomena are discussed

unbalanced repeated measures models with structured covariance matrices the question of how to analyze unbalanced or incomplete repeated measures data is a common problem facing analysts we address this problem through maximum likelihood analysis using a general linear model for expected responses and arbitrary structural models for the within subject covariances models that can be fit include standard univariate and multivariate models with incomplete data random effects models and models with time series and factor analytic error structures we describe newton raphson and fisher scoring algorithms for computing maximum likelihood estimates and generalized em algorithms for computing restricted and unrestricted maximum likelihood estimates an example fitting several models to a set of growth data is included

the application of reml in clinical trials residual maximum likelihood reml is a technique for estimating variance components in multi classified data in contrast to analysis of variance it can be routinely applied to unbalanced data and avoids some of the problems of biased variance estimates found with standard maximum likelihood estimation the full reml method is of particular value for the analysis of unbalanced clinical trials as it allows recovery of all the available information on treatment effects which can lead to significant improvements in their precision the use of reml has until recently been limited by heavy computational requirements and lack of readily available software this is no longer such a restriction however as reml procedures are now available in several widely used statistical packages including bmdp genstat and sas this paper describes the reml technique and discusses its application to three common types of clinical trial crossover repeated measures and multicentre

analysis strategies for serial multivariate ultrasonographic data that are incomplete ultrasonographic measurement of intima media thicknesses in the carotid artery has emerged as an important non invasive means of assessing atherosclerosis and has served to define primary outcome measures related to progression of arterial lesions in several large clinical trials and epidemiologic studies it is characteristic that measurements often cannot be obtained from all sites during repeated examinations this leads to incomplete multivariate serial data for which the set and number of visualized sites may vary across time we have contrasted several conditional and unconditional maximum likelihood analytical approaches and have evaluated these with a simulation experiment based on characteristics of ultrasound measurements collected during the course of the asymptomatic carotid artery plaque study we examined analyses based on unweighted and generalized least squares regression in which we estimated cross sectional summary statistics using raw means unconditional maximum likelihood estimates and full maximum likelihood estimates since the genesis of missing data is not fully clear and since the approaches we examined are based to some degree on the assumption that data are missing at random we also examined the relative impact of deviations from such an assumption on each of the approaches considered we found that maximum likelihood based approaches increased the expected efficiency of the analysis of serial ultrasound data over ignoring missing data by up to 21 per cent

conjugate gradient acceleration of the em algorithm the em algorithm is a very popular and widely applicable algorithm for the computation of maximum likelihood estimates although its implementation is generally simple the em algorithm often exhibits slow convergence and is costly in some areas of application past attempts to accelerate the em algorithm have most commonly been based on some form of aitken acceleration here we propose an alternative method based on conjugate gradients the key as we show is that the em step can be viewed approximately at least as a generalized gradient making it natural to apply generalized conjugate gradient methods in an attempt to accelerate the em algorithm the proposed method is relatively simple to implement and can handle problems with a large number of parameters an important feature of most em algorithms to demonstrate the effectiveness of the proposed acceleration method we consider its application to several problems in each of the following areas estimation of a covariance matrix from incomplete mu

superconvergence of the velocity along the gauss lines in mixed finite element methods superconvergence for the velocity along the gauss lines is derived for the mixed finite element method for the second order elliptic equation in rectangular domains these estimates are based on the rectangular finite elements of raviart thomas type

modelling hearing thresholds in the elderly this paper concerns a linear mixed effects repeated measures model in the analysis of a large data set with over 17 000 observations in a longitudinal study of pure tone hearing perception in the elderly the repeated measurements are described by fixed and random components in the model the fixed effects include the age at entry time of follow up a quadratic component in natural logarithm of frequency a component to allow for participants with hearing impairments as well as interaction terms between age and frequency and between impairment and frequency the random factors include a term for subject a time component and a frequency component the analysis shows that hearing impaired individuals have similar patterns of hearing loss over time but on average have higher hearing thresholds than normal individuals estimation of the random effects in the model by restricted maximum likelihood reml using the newton raphson method made possible the analysis of this large data set with speed and efficiency

acceleration of the em algorithm by using quasi newton methods the em algorithm is a popular method for maximum likelihood estimation its simplicity in many applications and desirable convergence properties make it very attractive its sometimes slow convergence however has prompted researchers to propose methods to accelerate it we review these methods classifying them into three groups pure hybrid and em type accelerators we propose a new pure and a new hybrid accelerator both based on quasi newton methods and numerically compare these and two other quasi newton accelerators for this we use examples in each of three areas poisson mixtures the estimation of covariance from incomplete data and multivariate normal mixtures in these comparisons the new hybrid accelerator was fastest on most of the examples and often dramatically so in some cases it accelerated the em algorithm by factors of over 100 the new pure accelerator is very simple to implement and competed well with the other accelerators it accelerated the em algorithm in some cases by factors of over 50 to obtain standard errors we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator a numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes

a recursive regression for high dimensional models with application to growth curves and repeated measures abstract a fixed effects formulation of repeated measures and growth curve problems usually leads to an unwieldy linear model so mixed models are widely used for inference that the conditional linear error model could otherwise support with weaker distributional assumptions very high dimensional regressions not necessarily arising this way can be fitted by the proposed alternating algorithm which partitions the design matrix into singly manageable strips and recursively calls a regression routine with low dimensional subproblems convergence to the full least squares solution with modest memory and time requirements is a consequence of the behavior of cyclically iterated projections of linear spaces the partitioning can be implicitly and transparently done for a wide class of growth curve problems the method does not hinge on any balance or completeness properties of the design in all cases coefficients and residuals are recoverable from standard regression output after convergence but package su

superconvergence of new mixed finite element spaces in this paper we prove some superconvergence of a new family of mixed finite element spaces of higher order which we introduced in etna vol 37 pp 189 201 2010 among all the mixed finite element spaces having an optimal order of convergence on quadrilateral grids this space has the smallest unknowns however the scalar variable is only suboptimal in general thus we have employed a post processing technique for the scalar variable as a byproduct we have obtained a superconvergence on a rectangular grid the superconvergence of a velocity variable naturally holds and can be shown by a minor modification of existing theory but that of a scalar variable requires a new technique especially for k 1 numerical experiments are provided to support the theory

multivariate repeated measurement or growth curve models with multivariate random effects covariance structure abstract the analysis of multiple response repeated measurement or growth curve models with a multivariate random effects covariance structure for individuals is considered results on estimation hypothesis testing and simultaneous confidence bounds for the parameters of the generalized linear model of potthoff and roy 1964 are derived for the multiple response repeated measurements model under the multivariate random effects covariance structure two numerical examples are presented to illustrate these techniques

random effects models for longitudinal data models for the analysis of longitudinal data must recognize the relationship between serial observations on the same unit multivariate models with general covariance structure are often difficult to apply to highly unbalanced data whereas two stage random effects models can be used easily in two stage models the probability distributions for the response vectors of different individuals belong to a single family but some random effects parameters vary across individuals with a distribution specified at the second stage a general family of models is discussed which includes both growth models and repeated measures models as special cases a unified approach to fitting these models based on a combination of empirical bayes and maximum likelihood estimation of model parameters and using the em algorithm is discussed two examples are taken from a current epidemiological study of the health effects of air pollution

parameter estimation and inference in the linear mixed model the paper reviews the linear mixed model with a focus on parameter estimation and inference parameter estimation for the different components of the model are reviewed with an emphasis on variance parameter estimation inferential procedures for the fixed effects random effects or a combination of both fixed and random effects are also discussed

segmental hidden markov models with random effects for waveform modeling this paper proposes a general probabilistic framework for shape based modeling and classification of waveform data a segmental hidden markov model hmm is used to characterize waveform shape and shape variation is captured by adding random effects to the segmental model the resulting probabilistic framework provides a basis for learning of waveform models from data as well as parsing and recognition of new waveforms expectation maximization em algorithms are derived and investigated for fitting such models to data in particular the expectation conditional maximization either ecme algorithm is shown to provide significantly faster convergence than a standard em procedure experimental results on two real world data sets demonstrate that the proposed approach leads to improved accuracy in classification and segmentation when compared to alternatives such as euclidean distance matching dynamic time warping and segmental hmms without random effects

a finite element method for elliptic equations on surfaces in this paper a new finite element approach for the discretization of elliptic partial differential equations on surfaces is treated the main idea is to use finite element spaces that are induced by triangulations of an outer domain to discretize the partial differential equation on the surface the method is particularly suitable for problems in which there is a coupling with a flow problem in an outer domain that contains the surface we give an analysis that shows that the method has optimal order of convergence both in the h 1 and in the l 2 norm results of numerical experiments are included that confirm this optimality

moment closure approximations of the boltzmann equation based on varphi divergences this paper is concerned with approximations of the boltzmann equation based on the method of moments we propose a generalization of the setting of the moment closure problem from relative entropy to varphi divergences and a corresponding closure procedure based on minimization of varphi divergences the proposed description encapsulates as special cases grad s classical closure based on expansion in hermite polynomials and levermore s entropy based closure we establish that the generalization to divergence based closures enables the construction of extended thermodynamic theories that avoid essential limitations of the standard moment closure formulations such as inadmissibility of the approximate phase space distribution potential loss of hyperbolicity and singularity of flux functions at local equilibrium the divergence based closure leads to a hierarchy of tractable symmetric hyperbolic systems that retain the fundamental structural properties of the boltzmann equation

majorization minimization algorithms in signal processing communications and machine learning this paper gives an overview of the majorization minimization mm algorithmic framework which can provide guidance in deriving problem driven algorithms with low computational cost a general introduction of mm is presented including a description of the basic principle and its convergence results the extensions acceleration schemes and connection to other algorithmic frameworks are also covered to bridge the gap between theory and practice upperbounds for a large number of basic functions derived based on the taylor expansion convexity and special inequalities are provided as ingredients for constructing surrogate functions with the pre requisites established the way of applying mm to solving specific problems is elaborated by a wide range of applications in signal processing communications and machine learning

reml estimation in unbalanced multivariate variance components models using an em algorithm estimation of covariance matrices when data are missing is a difficult problem in the multivariate mixed model setting problems can arise either because the data are unbalanced or because the response vectors are incomplete jennrich and schluchter 1986 biometrics 42 805 820 and laird lange and stram 1987 journal of the american statistical association 82 97 105 have proposed em algorithms for estimating the parameters associated with longitudinal data models in this paper a new em algorithm is proposed for the general multivariate mixed model the algorithm is derivativefree and based on an algorithm for balanced data proposed by calvin and dykstra 1991 annals qf statistics 19 850 869 once the algorithm is described it is used to produce estimates of the variance components matrices associated with a completely random two factor nested model for a bivariate response vector the data for this example come from calvin and sedransk 1991 journal of the american statistical association 86 36 48 who analyze a study of the quality of care received by cancer patients receiving radiation therapy

marginal regression analysis of longitudinal data with time dependent covariates a generalized method of moments approach summary we develop a new approach to using estimating equations to estimate marginal regression models for longitudinal data with time dependent covariates our approach classifies time dependent covariates into three types types i ii and iii the type of covariate determines what estimating equations can be used involving the covariate we use the generalized method of moments to make optimal use of the estimating equations that are made available by the covariates previous literature has suggested the use of generalized estimating equations with the independent working correlation when there are time dependent covariates we conduct a simulation study that shows that our approach can provide substantial gains in efficiency over generalized estimating equations with the independent working correlation when a time dependent covariate is of types i or ii and our approach remains consistent and comparable in efficiency with generalized estimating equations with the independent working correlation when a time dependent covariate is of type iii we apply our approach to analyse the relationship between the body mass index and future morbidity among children in the philippines

using empirical bayes methods in biopharmaceutical research a compound sampling model where a unit specific parameter is sampled from a prior distribution and then observed are generated by a sampling distribution depending on the parameter underlies a wide variety of biopharmaceutical data for example in a multi centre clinical trial the true treatment effect varies from centre to centre observed treatment effects deviate from these true effects through sampling variation knowledge of the prior distribution allows use of bayesian analysis to compute the posterior distribution of clinic specific treatment effects frequently summarized by the posterior mean and variance more commonly with the prior not completely specified observed data can be used to estimate the prior and use it to produce the posterior distribution an empirical bayes or variance component analysis in the empirical bayes model the estimated prior mean gives the typical treatment effect and the estimated prior standard deviation indicates the heterogeneity of treatment effects in both the bayes and empirical bayes approaches estimated clinic effects are shrunken towards a common value from estimates based on single clinics this shrinkage produces more efficient estimates in addition the compound model helps structure approaches to ranking and selection provides adjustments for multiplicity allows estimation of the histogram of clinic specific effects and structures incorporation of external information this paper outlines the empirical bayes approach coverage will include development and comparison of approaches based on parametric priors for example a gaussian prior with unknown mean and variance and non parametric priors discussion of the importance of accounting for uncertainty in the estimated prior comparison of the output and interpretation of fixed and random effects approaches to estimating population values estimating histograms and identification of key considerations in the use and interpretation of empirical bayes methods

a general statistical analysis for fmri data we propose a method for the statistical analysis of fmri data that seeks a compromise between efficiency generality validity simplicity and execution speed the main differences between this analysis and previous ones are a simple bias reduction and regularization for voxel wise autoregressive model parameters the combination of effects and their estimated standard deviations across different runs sessions subjects via a hierarchical random effects analysis using the em algorithm overcoming the problem of a small number of runs session subjects using a regularized variance ratio to increase the degrees of freedom

linear equality constraints in the general linear mixed model scientists may wish to analyze correlated outcome data with constraints among the responses for example piecewise linear regression in a longitudinal data analysis can require use of a general linear mixed model combined with linear parameter constraints although well developed for standard univariate models there are no general results that allow a data analyst to specify a mixed model equation in conjunction with a set of constraints on the parameters we resolve the difficulty by precisely describing conditions that allow specifying linear parameter constraints that insure the validity of estimates and tests in a general linear mixed model the recommended approach requires only straightforward and noniterative calculations to implement we illustrate the convenience and advantages of the methods with a comparison of cognitive developmental patterns in a study of individuals from infancy to early adulthood for children from low income families

a comparison of the generalized estimating equation approach with the maximum likelihood approach for repeated measurements liang and zeger proposed an extension of generalized linear models to the analysis of longitudinal data their approach is closely related to quasi likelihood methods and can handle both normal and non normal outcome variables such as poisson or binary outcomes their approach however has been applied mainly to non normal outcome variables this is probably due to the fact that there is a large class of multivariate linear models available for normal outcomes such as growth models and random effects models furthermore there are many iterative algorithms that yield maximum likelihood estimators mles of the model parameters the multivariate linear model approach based on maximum likelihood ml estimation specifies the joint multivariate normal distribution of outcome variables while the approach of liang and zeger based on the quasi likelihood specifies only the marginal distributions in this paper i compare the approach of liang and zeger and the ml approach for the multivariate normal outcomes i show that the generalized estimating equation gee reduces to the score equation only when the data do not have missing observations and the correlation is unstructured in more general cases however the gee estimation yields consistent estimators that may differ from the mles that is the gee does not always reduce to the score equation even when the outcome variables are multivariate normal i compare the small sample properties of the gee estimators and the mles by means of a monte carlo simulation study

application of nonlinear models with random coefficients to growth data summary the application of nonlinear random coefficient models to the analysis of growth curve data is described the approach is further developed for the estimation of mean growth curves and their variability from mark recapture data when the age of an animal at first capture is unknown but the time between successive captures is known these methods are of wide applicability as shown by the analysis of data on the growth of emus dromaius novaehollandiae noisy scrub birds atrichornis clamosus and whelks dicathais aegaota simulations of a mark recapture experiment on the rock spiny lobster panulirus argus in mexico using a known growth curve showed the effectiveness of the approach in estimating both the population s growth curve and the variability in individual growth

generalized method of moments estimation preface 1 introduction to the generalized method of moments estimation david harris and laszlo matyas 2 gmm estimation techniques masao ogaki 3 covariance matrix estimation matthew j cushing and mary g mcgarvey 4 hypothesis testing in models estimated by gmm alastair r hall 5 finite sample properties of gmm estimators and tests jan m podivinsky 6 gmm estimation of time series models david harris 7 reduced rank regression using gmm frank kleibergen 8 estimation of linear panel data models using gmm seung c ahn and peter schmidt 9 alternative gmm methods for nonlinear panel data models jorg breitung and michael lechner 10 simulation based method of moments roman liesenfeld and jorg breitung 11 logically inconsistent limited dependent variables models j s butler and gabriel picone index

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

empirical bayes analysis for systems of mixed models with linked autocorrelated random effects abstract empirical bayes concepts are implemented in a simultaneous analysis of a system of mixed linear models having linked and serially correlated random effects emphasis is placed on the estimation of the random effects and exploration of the relationships between them application is made to the investigation of several series of laboratory assay data that were observed during overlapping time intervals and were therefore subjected to common systematic errors or daily effects the motivation for this work was the need to investigate methods of adjustment for such daily effects and to estimate the degree to which concurrently run series are impacted in common attention is given to the construction of confidence intervals for daily effects tractable methods are proposed that yield approximately correct coverage for large samples although derived within a bayes empirical bayes framework these intervals are somewhat similar to intervals constructed by the method of kackar and harville implement

relating the classical covariance adjustment techniques of multivariate growth curve models to modern univariate mixed effects models summary the relationship between the modern univariate mixed model for analyzing longitudinal data popularized by laird and ware 1982 biometrics38 963 974 and its predecessor the classical multivariate growth curve model summarized by grizzle and allen 1969 biometrics25 357 381 has never been clearly established here the link between the two methodologies is derived and balanced polynomial and cosinor examples cited in the literature are analyzed with both approaches relating the two models demonstrates that classical covariance adjustment for higher order terms is analogous to including them as random effects in the mixed model the polynomial example clearly illustrates the relationship between the methodologies and shows their equivalence when all matrices are properly defined the cosinor example demonstrates how results from each method may differ when the total variance covariance matrix is positive definite but that the between subjects component of that matrix is not so constrained by the growth curve approach additionally advocates of each approach tend to consider different covariance structures modern mixed model analysts consider only those terms in a model s expectation or linear combinations and preferably the most parsimonious subset as candidates for random effects classical growth curve analysts automatically consider all terms in a model s expectation as random effects and then investigate whether covariance adjusting for higher order terms improves the model we apply mixed model techniques to cosinor analyses of a large unbalanced data set to demonstrate the relevance of classical covariance structures that were previously conceived for use only with completely balanced data

semi parametric and non parametric methods for the analysis of repeated measurements with applications to clinical trials techniques applicable for the analysis of longitudinal data when the response variable is non normal are not nearly as comprehensive as for normally distributed outcomes however there have been several recent developments semi parametric and non parametric methodology for the analysis of repeated measurements is reviewed the commonly encountered design in which for each subject one assesses a univariate response variable at multiple fixed time points is considered the types of outcomes considered include binary ordered categorical and continuous but extremely non normal response variables all of the methods considered allow for incomplete data due to the occurrence of missing observations in addition discrete and or continuous covariates which may be time dependent are accommodated by some of the approaches the methods are demonstrated using data from three clinical trials

estimating statistical properties of mimo fading channels we propose maximum likelihood ml and restricted maximum likelihood reml methods for estimating the mean and covariance parameters of multi input multi output mimo ricean and rayleigh block fading channels using measurements from multiple coherent intervals containing both amplitudes and phases of the received signal correlated and independent fading scenarios with structured and unstructured line of sight los array response models are considered computationally efficient ml and approximate ml aml estimators are proposed for unitary space time modulation schemes and orthogonal designs in correlated fading we also derive crame spl acute r rao bounds crbs for the unknown parameters discuss initialization of the proposed algorithms and evaluate their performance via numerical simulations under the block and continuous fading scenarios