distribution network design new problems and related models we study some complex distribution network design problems which involve facility location warehousing transportation and inventory decisions several realistic scenarios are investigated two kinds of mathematical programming formulations are proposed for all the introduced problems together with a proof of their correctness some formulations extend models proposed by perl and daskin 1985 for some warehouse location routing problems other formulations are based on flow variables and constraints

the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

location routing issues models and methods this paper is a survey of location routing a relatively new branch of locational analysis that takes into account vehicle routing aspects we propose a classification scheme and look at a number of problem variants both exact and heuristic algorithms are investigated finally some suggestions for future research are presented

godement jacquet l functions and full theta lifts we relate poles of local godement jacquet l functions to distributions on matrix spaces with singular supports as an application we show the irreducibility of the full theta lifts to mathrm gl n mathrm f of generic irreducible representations of mathrm gl n mathrm f where mathrm f is an arbitrary local field

on a mixture autoregressive model we generalize the gaussian mixture transition distribution gmtd model introduced by le and co workers to the mixture autoregressive mar model for the modelling of non linear time series the models consist of a mixture of k stationary or non stationary ar components the advantages of the mar model over the gmtd model include a more full range of shape changing predictive distributions and the ability to handle cycles and conditional heteroscedasticity in the time series the stationarity conditions and autocorrelation function are derived the estimation is easily done via a simple em algorithm and the model selection problem is addressed the shape changing feature of the conditional distributions makes these models capable of modelling time series with multimodal conditional distributions and with heteroscedasticity the models are applied to two real data sets and compared with other competing models the mar models appear to capture features of the data better than other competing models do

an akaike information criterion for multiple event mixture cure models abstract we derive the proper form of the akaike information criterion for variable selection for mixture cure models which are often fit via the expectation maximization algorithm separate covariate sets may be used in the mixture components the selection criteria are applicable to survival models for right censored data with multiple competing risks and allow for the presence of a non susceptible group the method is illustrated on credit loan data with pre payment and default as events and maturity as the non susceptible case and is used in a simulation study

on the tempered l functions conjecture we give a general proof of shahidi s tempered l function conjecture which has previously been known in all but one case one of the consequences is the standard module conjecture for p adic groups which means that the langlands quotient of a standard module is generic if and only if the standard module is irreducible and the inducing data generic we have also included the result that every generic tempered representation of a p adic group is a sub representation of a representation parabolically induced from a generic supercuspidal representation with a non negative real central character 1 introduction let f be a non archimedean local field of characteristic 0 let g be the group of points of a quasi split connected reductive f group by a parabolic subgroup borel subgroup levi subgroup torus split torus of g we will mean the group of points of an f parabolic subgroup f borel subgroup f levi subgroup f torus f split torus of the algebraic group under lying g fix a borel subgroup b tu of g a nd lett0 t be the maximal split torus in t i fm is any semi standard levi subgroup of g i e a levi subgroup which contains t0 a standard parabolic subgroup of m will be a parabolic subgroup of m which contains b m denote by w the weyl group of g defined with respect to t0 and by w g the longest element in w by sh3 section 3 we can fix a non degenerate character of u which for every levi subgroup m is compatible with w g 0 w m we will still denote by the restriction of to m u every generic representation of m becomes generic with respect to after changing the splitting in u let p mu be a standard parabolic subgroup of g and tm the maximal split torus in the center of m we will write a for the dual of the real lie algebra am of tm a c for its complexification and a m for the positive weyl chamber in a defined with respect to p following w we define a map hm m am such that m f q hm m for every f rational character a of m i f is a smooth representation of m and a c we denote by the smooth rep resentation of m defined by m q hm m m remark that although the sign in the definition of hm has been changed compared to the one due to harish chandra the meaning of is unchanged the symbol i g will denote the

using mixture models to detect sex bias in health outcomes in bangladesh abstract many interesting economic hypotheses entail differences in behaviors of groups within a population but analyses of pooled samples shed only partial light on underlying segmentations finite mixture models are considered as an alternative to methods based on pooling robustness checks using t regressions and a bayesian analogue to the likelihood ratio test for model evaluation are developed the methodology is used to investigate pro son bias in child health outcomes in bangladesh while regression analysis on the entire sample appears to wash out evidence of bias the mixture models reveal systematic girl boy differences in health outcomes

detecting features in spatial point processes with clutter via model based clustering abstract we consider the problem of detecting features such as minefields or seismic faults in spatial point processes when there is substantial clutter we use model based clustering based on a mixture model for the process in which features are assumed to generate points according to highly linear multivariate normal densities and the clutter arises according to a spatial poisson process nonlinear features are represented by several densities giving a piecewise linear representation hierarchical model based clustering provides a first estimate of the features and this is then refined using the em algorithm the number of features is estimated from an approximation to its posterior distribution the method gives good results for the minefield and seismic fault problems software to implement it is available on the world wide web

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

inference of nested variance components in a longitudinal myopia intervention trial this paper was motivated by a double blind randomized clinical trial of myopia intervention in addition to the primary goal of comparing treatment effects we are concerned with the modelling of correlation that may come from two possible sources one among the longitudinal observations and the other between measurements taken from both eyes per subject the data are nested repeated measurements we suggest three models for analysis each one expresses the correlation differently in various covariance structures we articulate their differences and describe the implementations in estimation using commercial statistical software the computer output can be further utilized to perform model selection with schwarz criterion simulation studies are conducted to evaluate the performance under each model data of the myopia intervention trial are reanalysed with these models for illustration the results indicate that atropine is more effective in reducing the progression rate the rates are homogeneous across subjects and among the suggested models the one with independent random effects of two eyes fits best we conclude that model selection is a crucial step before making inference with estimates otherwise the correlation may be attributed incorrectly to a different mechanism the same conclusion applies to other variance components as well copyright 2005 john wiley sons ltd

on convergence and parameter selection of the em and da em algorithms for gaussian mixtures abstract the expectation maximization em for gaussian mixtures is popular as a clustering algorithm however the em algorithm is sensitive to initial values and so ueda and nakano 4 proposed the deterministic annealing em da em algorithm to improve it in this paper we investigate theoretical behaviors of the em and da em algorithms we first derive a general jacobian matrix of the da em algorithm with respect to posterior probabilities we then propose a theoretical lower bound for initialization of the annealing parameter in the da em algorithm on the other hand some researches mentioned that the em algorithm exhibits a self annealing behavior that is the equal posterior probability with small random perturbations can avoid the em algorithm to output the mass center for gaussian mixtures however there is no theoretical analysis on this self annealing property since the da em will become the em when the annealing parameter is 1 according to the jacobian matrix of the da em we can prove the self annealing property of the em algorithm for gaussian mixtures based on these results we give not only convergence behaviors of the equal posterior probabilities and initialization lower bound of the temperature parameter of the da em but also a theoretical explanation why the em algorithm for gaussian mixtures exhibits a self annealing behavior

likelihood inference for exchangeable binary data with varying cluster sizes ce papier etudie l estimation au maximum de vraisemblance dans des modeles satures et insatures pour des donnees binaires correlees echangeables lorsqu on dispose d un echantillon de clusters independants de tailles variables nous discutons plusieurs parametrisations de ces modeles et nous proposons l utilisation de l algorithme em pour obtenir des estimations du maximum de vraisemblance la methodologie est illustree par des applications a une etude d agregation familiale de maladie et a la construction d un essai randomise par groupe pour la prevention du cancer

regression analysis of interval censored survival data with covariates using log linear models we considered the regression analysis of the event time data with left right or interval censored observations we extended life table techniques for censored survival data using log linear models to incorporate interval censored failures the em algorithm was used to calculate maximum likelihood estimates for the parameters we assumed that the hazard function was a stepwise function over disjoint intervals of time thus the nonparametric model the parametric exponential model and the semiparametric cox proportional hazard model were easily implemented as special cases we adapted the restricted em algorithm to test hypotheses and to construct confidence intervals for the parameters these methods were applied in an analysis of the recurrence time for treated melanoma patients

maximum likelihood variance components estimation for binary data abstract we consider a class of probit normal models for binary data and describe ml and reml estimation of variance components for that class as well as best prediction for the realized values of the random effects ml estimates are calculated using an em algorithm for complicated models em includes a gibbs step the computations are illustrated through two examples

hamiltonian location problems abstract exact algorithms for a series of simultaneous location and routing problems are developed problems are formulated as integer programs which are solved by a constraint relaxation procedure numerical results are reported

standard errors for em estimation log likelihood the well known sem algorithm uses the second approach we consider three additional algorithms one that uses the first approach and two that use the second we evaluate the complexity and precision of these three and the sem algorithm in seven examples the first is a single parameter example used to give insight the others are three examples in each of two areas of em application poisson mixture models and the estimation of covariance from incomplete data the examples show that there are algorithms that are much simpler and more accurate than the sem algorithm hopefully their simplicity will increase the availability of standard error estimates in em applications it is shown that as previously conjectured a symmetry diagnostic can accurately estimate errors arising from numerical differentiation some issues related to the speed of the em algorithm and algorithms that differentiate the em operator are identified

em and beyond the basic theme of the em algorithm to repeatedly use complete data methods to solve incomplete data problems is also a theme of several more recent statistical techniques these techniques multiple imputation data augmentation stochastic relaxation and sampling importance resampling combine simulation techniques with complete data methods to attack problems that are difficult or impossible for em

bayesian inference for causal effects in randomized experiments with noncompliance for most of this century randomization has been a cornerstone of scientific experimentation especially when dealing with humans as experimental units in practice however noncompliance is relatively common with human subjects complicating traditional theories of inference that require adherence to the random treatment assignment in this paper we present bayesian inferential methods for causal estimands in the presence of noncompliance when the binary treatment assignment is random and hence ignorable but the binary treatment received is not ignorable we assume that both the treatment assigned and the treatment received are observed we describe posterior estimation using em and data augmentation algorithms also we investigate the role of two assumptions often made in econometric instrumental variables analyses the exclusion restriction and the monotonicity assumption without which the likelihood functions generally have substantial regions of maxima we apply our procedures to real and artificial data thereby demonstrating the technology and showing that our new methods can yield valid inferences that differ in practically important ways from those based on previous methods for analysis in the presence of noncompliance including intention to treat analyses and analyses based on econometric instrumental variables techniques finally we perform a simulation to investigate the operating characteristics of the competing procedures in a simple setting which indicates relatively dramatic improvements in frequency operating characteristics attainable using our bayesian procedures

bayesian sample size determination for inference on two binomial populations with no gold standard classifier we consider the impact of test properties on the required sample size for the bayesian design problem for comparing two proportions with error prone data specifically we examine four cases a single diagnostic test and two independent diagnostic tests both when the test properties are identical across populations and when they differ interval based and moment based sample size determination criteria are contrasted using monte carlo simulation methods we consider an application in which strongyloides infections are compared in two populations

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

a model selection approach for the identification of quantitative trait loci in experimental crosses summary we consider the problem of identifying the genetic loci called quantitative trait loci qtls contributing to variation in a quantitative trait with data on an experimental cross a large number of different statistical approaches to this problem have been described most make use of multiple tests of hypotheses and many consider models allowing only a single qtl we feel that the problem is best viewed as one of model selection we discuss the use of model selection ideas to identify qtls in experimental crosses we focus on a back cross experiment with strictly additive qtls and concentrate on identifying qtls considering the estimation of their effects and precise locations of secondary importance we present the results of a simulation study to compare the performances of the more prominent methods

extending the state space model to accommodate missing values in responses and covariates this article proposes an extended state space model for accommodating multivariate panel data the novel aspect of this contribution is an adjustment to the classical model for multiple subjects that allows missingness in the covariates in addition to the responses missing covariate data are handled by a second state space model nested inside the first to represent unobserved exogenous information relevant kalman filter equations are derived and explicit expressions are provided for both the e and m steps of an expectation maximization em algorithm to obtain maximum gaussian likelihood estimates of all model parameters in the presence of missing data the resulting em algorithm becomes computationally intractable but a simplification of the m step leads to a new procedure that is shown to be an expectation conditional maximization ecm algorithm under exogeneity of the covariates simulation studies reveal that the approach appears to be relatively robust to moderate percentages of missing dat

an exact algorithm for minimizing routing and operating costs in depot location abstract the problem of locating a single depot among n points is considered the objective is to minimize the sum of depot operating cost and routing cost the best depot location is found by means of an exact algorithm that determines simultaneously both the best depot location and the associated optimal delivery routes a global integer programming formulation of the problem is given the model is solved by relaxing most of its constraints and by introducing them only when they are violated

information matrix computation from conditional information via normal approximation summary this paper provides a method for computing the asymptotic covariance matrix from a likelihood function with known maximum likelihood estimate of the parameters philosophically the basic idea is to assume that the likelihood function should be well approximated by a normal density when asymptotic results about the maximum likelihood estimate are applied for statistical inference technically the method makes use of two facts the information for a one dimensional parameter can be well computed when the loglikelihood is approximately quadratic over the range corresponding to a small positive confidence interval and the covariance matrix of a normal distribution can be obtained from its one dimensional conditional distributions whose sample spaces span the sample space of the joint distribution we illustrate the method with its application to a linear mixed effects model

a general maximum likelihood analysis of variance components in generalized linear models this paper describes an em algorithm for nonparametric maximum likelihood ml estimation in generalized linear models with variance component structure the algorithm provides an alternative analysis to approximate mql and pql analyses mcgilchrist and aisbett 1991 biometrical journal 33 131 141 breslow and clayton 1993 journal of the american statistical association 88 9 25 mcgilchrist 1994 journal of the royal statistical society series b 56 61 69 goldstein 1995 multilevel statistical models and to gee analyses liang and zeger 1986 biometrika 73 13 22 the algorithm first given by hinde and wood 1987 in longitudinal data analysis 110 126 is a generalization of that for random effect models for overdispersion in generalized linear models described in aitkin 1996 statistics and computing 6 251 262 the algorithm is initially derived as a form of gaussian quadrature assuming a normal mixing distribution but with only slight variation it can be used for a completely unknown mixing distribution giving a straightforward method for the fully nonparametric ml estimation of this distribution this is of value because the ml estimates of the glm parameters can be sensitive to the specification of a parametric form for the mixing distribution the nonparametric analysis can be extended straightforwardly to general random parameter models with full npml estimation of the joint distribution of the random parameters this can produce substantial computational saving compared with full numerical integration over a specified parametric distribution for the random parameters a simple method is described for obtaining correct standard errors for parameter estimates when using the em algorithm several examples are discussed involving simple variance component and longitudinal models and small area estimation

the statistical analysis of general processing tree models with the em algorithm multinomial processing tree models assume that an observed behavior category can arise from one or more processing sequences represented as branches in a tree these models form a subclass of parametric multinomial models and they provide a substantively motivated alternative to loglinear models we consider the usual case where branch probabilities are products of nonnegative integer powers in the parameters 0 s 1 and their complements 1 s a version of the em algorithm is constructed that has very strong properties first the e step and the m step are both analytic and computationally easy therefore a fast pc program can be constructed for obtaining mles for large numbers of parameters second a closed form expression for the observed fisher information matrix is obtained for the entire class third it is proved that the algorithm necessarily converges to a local maximum and this is a stronger result than for the exponential family as a whole fourth we show how the algorithm can handle quite general hypothesis tests concerning restrictions on the model parameters fifth we extend the algorithm to handle the read and cressie power divergence family of goodness of fit statistics the paper includes an example to illustrate some of these results

the hamiltonian p median problem abstract this paper is concerned with a new mixed routing location problem embedding the p median and the travelling salesman problems the hamiltonian p median problem hpmp is formulated and several heuristics are proposed computational experience with a set of test problems is reported

on the convergence of monte carlo maximum likelihood calculations summary monte carlo maximum likelihood for normalized families of distributions can be used for an extremely broad class of models given any family he 0 e 0 of non negative integrable functions maximum likelihood estimates in the family obtained by normalizing the functions to integrate to 1 can be approximated by monte carlo simulation the only regularity conditions being a compactification of the parameter space such that the evaluation maps 0 h0 x remain continuous then with probability 1 the monte carlo approximant to the log likelihood hypoconverges to the exact log likelihood its maximizer converges to the exact maximum likelihood estimate approximations to profile likelihoods hypoconverge to the exact profile and level sets of the approximate likelihood support regions converge to the exact sets in painleve kuratowski set convergence the same results hold when there are missing data if a wald type integrability condition is satisfied asymptotic normality of the monte carlo error and convergence of the monte carlo approximation to the observed fisher information are also shown

a generalized speed accuracy response model for dichotomous items we propose a generalization of the speed accuracy response model sarm introduced by maris and van der maas psychometrika 77 615 633 2012 in these models the scores that result from a scoring rule that incorporates both the speed and accuracy of item responses are modeled our generalization is similar to that of the one parameter logistic or rasch model to the two parameter logistic or birnbaum model in item response theory an expectation maximization em algorithm for estimating model parameters and standard errors was developed furthermore methods to assess model fit are provided in the form of generalized residuals for item score functions and saddlepoint approximations to the density of the sum score the presented methods were evaluated in a small simulation study the results of which indicated good parameter recovery and reasonable type i error rates for the residuals finally the methods were applied to two real data sets it was found that the two parameter sarm showed improved fit compared to the one parameter sarm in both data sets

exact and heuristic algorithms for the hamiltonian p median problem this paper presents an exact algorithm a constructive heuristic algorithm and a metaheuristic for the hamiltonian p median problem hpmp the exact algorithm is a branch and cut algorithm based on an enhanced p median based formulation which is proved to dominate an existing p median based formulation the constructive heuristic is a giant tour heuristic based on a dynamic programming formulation to optimally split a given sequence of vertices into cycles the metaheuristic is an iterated local search algorithm using 2 exchange and 1 opt operators computational results show that the branch and cut algorithm outperforms the existing exact solution methods

an overview of representative problems in location research we present a survey of over 50 representative problems in location research our goal is not to review all variants of different location models or to describe solution results but rather to provide a broad overview of major location problems that have been studied indicating briefly how they are formulated and how they relate to one another we review standard problems such as median center and warehouse location problems as well as less traditional location problems which have emerged in recent years our primary focus is on problems for which operations research type models have been developed most of the problems we review have been formulated as optimization problems

constrained maximum likelihood estimation of two level covariance structure model via em type algorithms in this paper the constrained maximum likelihood estimation of a two level covariance structure model with unbalanced designs is considered the two level model is reformulated as a single level model by treating the group level latent random vectors as hypothetical missing data then the popular em algorithm is extended to obtain the constrained maximum likelihood estimates for general nonlinear constraints the multiplier method is used at them step to find the constrained minimum of the conditional expectation an accelerated em gradient procedure is derived to handle linear constraints the empirical performance of the proposed em type algorithms is illustrated by some artifical and real examples

on simulated em algorithms the em algorithm is a popular and useful algorithm for finding the maximum likelihood estimator in incomplete data problems each iteration of the algorithm consists of two simple steps an e step in which a conditional expectation is calculated and an m step where the expectation is maximized in some problems however the em algorithm cannot be applied since the conditional expectation required in the e step cannot be calculated instead the expectation may be estimated by simulation we call this a simulated em algorithm the simulations can at least in principle be done in two ways either new independent random variables are drawn in each iteration or the same uniforms are re used in each iteration in this paper the properties of these two versions of the simulated em algorithm are discussed and compared

the analysis of repeated measures data on schizophrenic reaction times using mixture models reaction times for schizophrenic individuals in a simple visual tracking experiment can be substantially more variable than for non schizophrenic individuals current psychological theory suggests that at least some of this extra variability arises from an attentional lapse that delays some but not all of each schizophrenic s reaction times based on this theory we pursue models in which measurements from non schizophrenics arise from a normal linear model with a separate mean for each individual whereas measurements from schizophrenics arise from a mixture of i a component analogous to the distribution of response times for non schizophrenics and ii a mean shifted component we fit four mixture models within this framework where the distinctions between models arise from assumptions about the variance of the shifted observations and the exchangeability of schizophrenic individuals some of these models can be fit by maximum likelihood using the em algorithm and all can be fit using the ecm algorithm where the covariance matrices associated with the parameters are calculated by the sem and secm algorithms respectively bayesian model monitoring using posterior predictive checks is invoked to discard models that fail to reproduce certain observed features of the data and to stimulate the development of better models

standard errors for em estimates in generalized linear models with random effects a procedure is derived for computing standard errors of em estimates in generalized linear models with random effects quadrature formulas are used to approximate the integrals in the em algorithm where two different approaches are pursued i e gauss hermite quadrature in the case of gaussian random effects and nonparametric maximum likelihood estimation for an unspecified random effect distribution an approximation of the expected fisher information matrix is derived from an expansion of the em estimating equations this allows for inferential arguments based on em estimates as demonstrated by an example and simulations

transcending classical invariant theory let sp2 r sp2 sp be the real symplectic group of rank n let sp denote the two fold cover of sp there is a unitary representation constructed by shale sh and weil wa of sp that is of considerable interest in the theory of automorphic forms we denote this representation by co and call it the oscillator representation the purpose of this paper is to establish about c a fact that should be useful for clarifying the structure of spaces of automorphic forms constructed using co we begin by formulating the result let e be a reductive subgroup of sp and let e denote the inverse image of e in sp denote by r e the set of infinitesimal equivalence classes of continuous irreducible admissible representations of e on locally convex spaces let co be the smooth representation of sp associated to co let co be realized on a hilbert space and let the subspace of smooth vectors on which space co is defined be written denote by r9 e co the set of elements of e which are realized as quotients by co e invariant closed subspaces of t g consider a reductive dual pair g g c sp h2 it is not hard to show that g and g commute with one another hence we may regard co i g g as a representation of gx g it is well known f that r gx g s9 g xr9 g the identification associates to p e r g and p e i g the tensor product p x p we note that p x p is not defined as a topological vector space nevertheless the infinitesimal equivalence class of p op is well defined select p x p e g x g suppose that in fact p p e 9 g g c then clearly p e r g c and p e 9 g c hence g co defines the graph of a correspondence between certain subsets of r9 g cv and j g co in fact the situation is quite precise

acceleration of the em algorithm by using quasi newton methods the em algorithm is a popular method for maximum likelihood estimation its simplicity in many applications and desirable convergence properties make it very attractive its sometimes slow convergence however has prompted researchers to propose methods to accelerate it we review these methods classifying them into three groups pure hybrid and em type accelerators we propose a new pure and a new hybrid accelerator both based on quasi newton methods and numerically compare these and two other quasi newton accelerators for this we use examples in each of three areas poisson mixtures the estimation of covariance from incomplete data and multivariate normal mixtures in these comparisons the new hybrid accelerator was fastest on most of the examples and often dramatically so in some cases it accelerated the em algorithm by factors of over 100 the new pure accelerator is very simple to implement and competed well with the other accelerators it accelerated the em algorithm in some cases by factors of over 50 to obtain standard errors we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator a numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes

using em to obtain asymptotic variance covariance matrices the sem algorithm abstract the expectation maximization em algorithm is a popular and often remarkably simple method for maximum likelihood estimation in incomplete data problems one criticism of em in practice is that asymptotic variance covariance matrices for parameters e g standard errors are not automatic byproducts as they are when using some other methods such as newton raphson in this article we define and illustrate a procedure that obtains numerically stable asymptotic variance covariance matrices using only the code for computing the complete data variance covariance matrix the code for em itself and code for standard matrix operations the basic idea is to use the fact that the rate of convergence of em is governed by the fractions of missing information to find the increased variability due to missing information to add to the complete data variance covariance matrix we call this supplemented em algorithm the sem algorithm theory and particular examples reinforce the conclusion that the sem alg

fitting full information item factor models and an empirical investigation of bridge sampling abstract based on item response theory bock and aitken introduced a method of item factor analysis termed full information item factor fiif analysis by bartholomew because it uses all distinct item response vectors as data but a limitation of their fitting algorithm is its reliance on fixed point gauss hermite quadrature which can produce appreciable numerical errors especially in high dimension problems the first purpose of this article is to offer more reliable methods by using recent advances in statistical computation specifically we illustrate two ways of implementing monte carlo expectation maximization em algorithm to fit a fiif model using the gibbs sampler to carry out the computation for the e steps we also show how to use bridge sampling to simulate the likelihood ratios for monitoring the convergence of a monte carlo em a strategy that is useful in general simulations demonstrate substantial improvement over bock and aitken s algorithm in recovering known factor loadings in hig

discussion on the meeting on statistical modelling and analysis of genetic data david j balding imperial college school of medicine london i extendmy apologies to the authors that an unavoidable commitment arising unexpectedly in the 18 hours before the meeting robbed me of my final preparation so that my comments at the meeting were poorly presented i shall try to do a better job in this written version and to leave enough space i shall omit my introductory comments about the role of statisticians in bioinformatics

improving and evaluating topic models and other models of text abstractan ongoing challenge in the analysis of document collections is how to summarize content in terms of a set of inferred themes that can be interpreted substantively in terms of topics the current practice of parameterizing the themes in terms of most frequent words limits interpretability by ignoring the differential use of words across topics here we show that words that are both frequent and exclusive to a theme are more effective at characterizing topical content and we propose a regularization scheme that leads to better estimates of these quantities we consider a supervised setting where professional editors have annotated documents to topic categories organized into a tree in which leaf nodes correspond to more specific topics each document is annotated to multiple categories at different levels of the tree we introduce a hierarchical poisson convolution model to analyze these annotated documents a parallelized hamiltonian monte carlo sampler allows the inference to scale to millio

survival curve estimation with partial non random exposure information the objective of this paper is to estimate survival curves for two different exposure groups when the exposure group is not known for all observations and the data is subject to left truncation and right censoring the situation we consider is when the probability that the exposure group is missing may depend on whether the observation is censored or uncensored in which case the exposure is not missing at random the problem was motivated by a study of alzheimer s disease to estimate the distribution of ages at diagnosis for individuals with and without an apolipoprotein e4 allele the exposure group genotyping for this risk factor was incomplete and performed more frequently on the cases of alzheimer s disease the uncensored observations than the censored observations the survival curves are estimated in discrete time using an em algorithm a bootstrapping procedure is proposed that guarantees each bootstrap sample has the same proportion of observations with missing exposure a simulation is performed to evaluate the bias of the estimators and to investigate design and efficiency issues the methods are applied to the alzheimer s disease study

parameter estimation and inference in the linear mixed model the paper reviews the linear mixed model with a focus on parameter estimation and inference parameter estimation for the different components of the model are reviewed with an emphasis on variance parameter estimation inferential procedures for the fixed effects random effects or a combination of both fixed and random effects are also discussed

local intrinsic dimension estimation by generalized linear modeling we propose a method for intrinsic dimension estimation by fitting the power of distance from an inspection point and the number of samples included inside a ball with a radius equal to the distance to a regression model we estimate the goodness of fit then by using the maximum likelihood method we estimate the local intrinsic dimension around the inspection point the proposed method is shown to be comparable to conventional methods in global intrinsic dimension estimation experiments furthermore we experimentally show that the proposed method outperforms a conventional local dimension estimation method

a method for calibrating false match rates in record linkage abstract specifying a record linkage procedure requires both 1 a method for measuring closeness of agreement between records typically a scalar weight and 2 a rule for deciding when to classify records as matches or nonmatches based on the weights here we outline a general strategy for the second problem that is for accurately estimating false match rates for each possible cutoff weight the strategy uses a model where the distribution of observed weights are viewed as a mixture of weights for true matches and weights for false matches an em algorithm for fitting mixtures of transformed normal distributions is used to find posterior modes associated posterior variability is due to uncertainty about specific normalizing transformations as well as uncertainty in the parameters of the mixture model the latter being calculated using the sem algorithm this mixture model calibration method is shown to perform well in an applied setting with census data further a simulation experiment reveals that

general formulas for obtaining the mles and the asymptotic variance covariance matrix in mapping quantitative trait loci when using the em algorithm we present in this paper general formulas for deriving the maximum likelihood estimates and the asymptotic variance covariance matrix of the positions and effects of quantitative trait loci qtls in a finite normal mixture model when the em algorithm is used for mapping qtls the general formulas are based on two matrices d and q where d is the genetic design matrix characterizing the genetic effects of the qtls and q is the conditional probability matrix of qtl genotypes given flanking marker genotypes containing the information on qtl positions with the general formulas it is relatively easy to extend qtl mapping analysis to using multiple marker intervals simultaneously for mapping multiple qtls for analyzing qtl epistasis and for estimating the heritability of quantitative traits simulations were performed to evaluate the performance of the estimates of the asymptotic variances of qtl positions and effects

maximum likelihood estimation for probit linear mixed models with correlated random effects the probit normal model for binary data mcculloch 1994 journal of the american statistical association 89 330 335 is extended to allow correlated random effects to obtain maximum likelihood estimates we use the em algorithm with its m step greatly simplified under the assumption of a probit link and its e step made feasible by gibbs sampling standard errors are calculated by inverting a monte carlo approximation of the information matrix rather than via the sem algorithm a method is also suggested that accounts for the monte carlo variation explicitly as an illustration we present a new analysis of the famous salamander mating data unlike previous analyses we find it necessary to introduce different variance components for different species of animals finally we consider models with correlated errors as well as correlated random effects

an em algorithm fitting first order conditional autoregressive models to longitudinal data abstract an em algorithm fits a state space formulation of the longitudinal regression model in which a continuous response depends on the lagged response and both time dependent and time independent covariates the baseline response depends only on covariates the model handles both missing data and gaussian measurement error on both response and continuous covariates the e step uses the kalman filter and associated filtering algorithms to update the unknown true response and predictor series for the observed data the m step uses standard closed form gaussian results standard errors come from the supplemented em sem algorithm the model accurately fits 6 years of pulmonary function measurements on 158 children with many missing observations

on the estimation of nonrandom signal coefficients from jittered samples this paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter timing noise and additive independent identically distributed i i d gaussian noise where the signal lies in the span of a finite basis for the presented classical estimation problem the cramer rao lower bound crb is computed and an expectation maximization em algorithm approximating the maximum likelihood ml estimator is developed simulations are performed to study the convergence properties of the em algorithm and compare the performance both against the crb and a basic linear estimator these simulations demonstrate that by postprocessing the jittered samples with the proposed em algorithm greater jitter can be tolerated potentially reducing on chip adc power consumption substantially

a fast improvement to the em algorithm on its own terms the em algorithm is a numerical technique for the evaluation of maximum likelihood estimates for parameters describing incomplete data models it is easy to apply in many problems and is stable but slow the algorithm fails to provide a consistent estimator of the standard errors of the maximum likelihood estimates unless the additional analysis required by the louis method is performed newton type or other gradient methods are faster and provide error estimates but tend to be unstable and require the analytical evaluation of likelihoods to derive expressions for the score function and at least approximations to the fisher information matrix the purpose of this paper is to expand on a result by fisher that permits a unification of em methodology and newton methods the evaluation of the individual observation by observation score functions of the incomplete data is a byproduct of the application of the e step of the em algorithm once these become available the fisher information matrix may be consistently estimated and the m step may be replaced by a fast newton type step

the restricted em algorithm for maximum likelihood estimation under linear restrictions on the parameters abstract the em algorithm is one of the most powerful algorithms for obtaining maximum likelihood estimates for many incomplete data problems but when the parameters must satisfy a set of linear restrictions the em algorithm may be too complicated to apply directly in this article we propose maximum likelihood estimation procedures under a set of linear restrictions for situations in which the em algorithm could be used if there were no such restrictions on the parameters we develop a modification to the em algorithm which we call the restricted em algorithm incorporating the linear restrictions on the parameters this algorithm is easily updated by using the code for the complete data information matrix and the code for the usual em algorithm major applications of the restricted em algorithm are to construct likelihood ratio tests and profile likelihood confidence intervals we illustrate the procedure with two models a variance component model and a bivariate normal model

likelihood analysis for errors in variables regression with replicate measurements summary this paper advocates likelihood analysis for regression models with measurement errors in explanatory variables for data problems in which the relevant distributions can be adequately modelled although computationally difficult maximum likelihood estimates are more efficient than those based on first and second moment assumptions and likelihood ratio inferences can be substantially better than those based on asymptotic normality of estimates the em algorithm is presented as a straightforward approach for likelihood analysis of normal linear regression with normal explanatory variables and normal replicate measurements likelihood methods for regression analysis with explanatory variable measurement error have received little attention relative to methods based on first and second moment assumptions largely because of computational difficulties uncertainty about the robustness of likelihood methods and the belief that the simpler methods perform just as well in practice with improved computational tools for likelihood analysis as discussed for example by tanner 1993 it is worthwhile to reinvestigate these issues in particular the common belief that the simpler methods perform just as well as likelihood methods may be naive first the comparison is usually cast in terms of efficiency without regard to test and confidence interval validity likelihood ratio tests and confidence intervals can be substantially better than tests and confidence intervals based on estimates and standard errors of the commonly used method of moments approaches the sampling distributions of the estimators are very often skewed especially if the measurement errors are large furthermore knowledge about the relative efficiency of the moment methods comes from a small number of efficiency comparisons from simulations of isolated situations in papers promoting these simpler methods carroll rupert stefanski 1995 7 1 an important and often overlooked issue is that the moment methods may be especially inappropriate when the distribution of the true covariates is skewed pierce et al 1992 section 2 discusses computational approaches and difficulties associated with likelihood analysis in the measurement error problem generally section 3 illustrates the em algorithm dempster laird rubin 1977 for likelihood analysis of normal linear regression with a normally distributed explanatory variable measured with normally distributed error when there are replicate measurements for at least some of the cases sections 5 and 6

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

fast bayesian factor analysis via automatic rotations to sparsity abstractrotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis regularization methods also serve to achieve this goal by prioritizing sparse loading matrices in this work we bridge these two paradigms with a unifying bayesian framework our approach deploys intermediate factor rotations throughout the learning process greatly enhancing the effectiveness of sparsity inducing priors these automatic rotations to sparsity are embedded within a pxl em algorithm a bayesian variant of parameter expanded em for posterior mode detection by iterating between soft thresholding of small factor loadings and transformations of the factor basis we obtain a dramatic accelerations b robustness against poor initializations and c better oriented sparse solutions to avoid the prespecification of the factor cardinality we extend the loading matrix to have infinitely many columns with the indian buffet process ibp prior the factor dimen

markov normal analysis of iterative simulations before their convergence abstract iterative simulation techniques are becoming standard tools in bayesian statistics a notable example being the gibbs sampler whose draws form a markov chain standard practice is to run the simulation until convergence is approached in the sense of the draws appearing to be stationary at this point the set of stationary draws can be used to provide an estimate of the target distribution however when the distributions involved are normal and the draws form a markov chain the target distribution can be reliably estimated by maximum likelihood ml using draws before their convergence to the target distribution this fact suggests that the normal based ml estimates can be exploited to estimate the mean and covariance matrix of an approximately normal target distribution before convergence is reached and that these estimates can be used to define a restarting distribution for the simulation here we describe the needed technology and explore its relevance to practice the tentative conclusion is that the markov normal restarting procedure can be computationally advantageous when the target distribution is nearly normal especially in massively parallel or distributed computing environments where many sequences can be run for the same effective cost as one sequence

variable selection with incomplete covariate data summary application of classical model selection methods such as akaike s information criterion aic becomes problematic when observations are missing in this article we propose some variations on the aic which are applicable to missing covariate problems the method is directly based on the expectation maximization em algorithm and is readily available for em based estimation methods without much additional computational efforts the missing data aic criteria are formally derived and shown to work in a simulation study and by application to data on diabetic retinopathy

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea