a primal dual method for large scale image reconstruction in emission tomography in emission tomography images can be reconstructed from a set of measured projections using a maximum likelihood ml criterion in this paper we present a primal dual algorithm for large scale three dimensional image reconstruction the primal dual method is specialized to the ml reconstruction problem the reconstruction problem is extremely large in several of our data sets the hessian of the objective function is the product of a 1 4 million by 63 million matrix and its scaled transpose as such we consider only approaches that are suitable for large scale parallel computation we apply a stabilization technique to the system of equations for computing the primal direction and demonstrate the need for stabilization when approximately solving the system using an early terminated conjugate gradient iteration we demonstrate that the primal dual method for this problem converges faster than the logarithmic barrier method and considerably faster than the expectation maximization algorithm the use of extrapolation in conjunction with the primal dual method further reduces the overall computation required to achieve convergence

maximum entropy and the nearly black object summary maximum entropy me inversion is a non linear inversion technique for inverse problems where the object to be recovered is known to be positive it has been applied in areas ranging from radio astronomy to various forms of spectroscopy sometimes with dramatic success in some cases me has attained an order of magnitude finer resolution and or an order of magnitude smaller noise level than that obtainable by standard linear methods the dramatic successes all seem to occur in cases where the object to be recovered is nearly black essentially zero in the vast majority of samples we show that near blackness is required both for signal to noise enhancements and for superresolution however other methods in particular minimum 1 norm reconstruction may exploit near blackness to an even greater extent

bayesian reconstructions from emission tomography data using a modified em algorithm a novel method of reconstruction from single photon emission computerized tomography data is proposed this method builds on the expectation maximization em approach to maximum likelihood reconstruction from emission tomography data but aims instead at maximum posterior probability estimation which takes account of prior belief about smoothness in the isotope concentration a novel modification to the em algorithm yields a practical method the method is illustrated by an application to data from brain scans

a penalized em algorithm incorporating missing data mechanism for gaussian parameter estimation missing data rates could depend on the targeted values in many settings including mass spectrometry based proteomic profiling studies here we consider mean and covariance estimation under a multivariate gaussian distribution with non ignorable missingness including scenarios in which the dimension p of the response vector is equal to or greater than the number n of independent observations a parameter estimation procedure is developed by maximizing a class of penalized likelihood functions that entails explicit modeling of missing data probabilities the performance of the resulting penalized em algorithm incorporating missing data mechanism pemm estimation procedure is evaluated in simulation studies and in a proteomic data illustration

network tomography estimating source destination traffic intensities from link data abstract the problem of estimating the node to node traffic intensity from repeated measurements of traffic on the links of a network is formulated and discussed under poisson assumptions and two types of traffic routing regimens deterministic a fixed known path between each directed pair of nodes and markovian a random path between each directed pair of nodes determined according to a known markov chain fixed for that pair maximum likelihood estimation and related approximations are discussed and computational difficulties are pointed out a detailed methodology is presented for estimates based on the method of moments the estimates are derived algorithmically taking advantage of the fact that the first and second moment equations give rise to a linear inverse problem with positivity restrictions that can be approached by an em algorithm resulting in a particularly simple solution to a hard problem a small simulation study is carried out

nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci estimation of the covariance structure of longitudinal processes is a fundamental prerequisite for the practical deployment of functional mapping designed to study the genetic regulation and network of quantitative variation in dynamic complex traits we present a nonparametric approach for estimating the covariance structure of a quantitative trait measured repeatedly at a series of time points specifically we adopt huang et al s 2006 biometrika 93 85 98 approach of invoking the modified cholesky decomposition and converting the problem into modeling a sequence of regressions of responses a regularized covariance estimator is obtained using a normal penalized likelihood with an l 2 penalty this approach embedded within a mixture likelihood framework leads to enhanced accuracy precision and flexibility of functional mapping while preserving its biological relevance simulation studies are performed to reveal the statistical properties and advantages of the proposed method a real example from a mouse genome project is analyzed to illustrate the utilization of the methodology the new method will provide a useful tool for genome wide scanning for the existence and distribution of quantitative trait loci underlying a dynamic trait important to agriculture biology and health sciences

semiparametric bayesian analysis of survival data abstract this review article investigates the potential of bayes methods for the analysis of survival data using semiparametric models based on either the hazard or the intensity function the nonparametric part of every model is assumed to be a realization of a stochastic process the parametric part which may include a regression parameter or a parameter quantifying the heterogeneity of a population is assumed to have a prior distribution with possibly unknown hyperparameters careful applications of some recently popular computational tools including sampling based algorithms are used to find posterior estimates of several quantities of interest even when dealing with complex models and unusual data structures the methodologies developed herein are motivated and aimed at analyzing some common types of survival data from different medical studies here we focus on univariate survival data in the presence of fixed and time dependent covariates multiple event time data for repeated nonfatal events

multivariate discriminant analysis and maximum penalized likelihood density estimation summary a new theoretical point of view is discussed in the framework of density estimation the multivariate true density viewed as a prior or penalizing factor in a bayesian framework is modelled by a gibbs potential estimating the density consists in maximizing the posterior for efficiency of time we are interested in an approximate estimator f bwr of the true density f where b is a stochastic operator and 7r is the raw histogram then we investigate the discrimination problem introducing an adaptive bandwidth depending on the k nearest neighbours and chosen to optimize the cross validation criterion our final classification algorithm referred to as apml for approximate penalized maximum likelihood compares favourably in terms of error rate and time efficiency with other algorithms tested including multinormal nearest neighbour and convex hull classifiers

averaging maximum penalized likelihood and bayesian estimation for improving gaussian mixture probability density estimates we apply the idea of averaging ensembles of estimators to probability density estimation in particular we use gaussian mixture models which are important components in many neural network applications we investigate the performance of averaging using three data sets for comparison we employ two traditional regularization approaches i e a maximum penalized likelihood approach and a bayesian approach in the maximum penalized likelihood approach we use penalty functions derived from conjugate bayesian priors such that an expectation maximization em algorithm can be used for training in all experiments the maximum penalized likelihood approach and averaging improved performance considerably if compared to a maximum likelihood approach in two of the experiments the maximum penalized likelihood approach outperformed averaging in one experiment averaging was clearly superior our conclusion is that maximum penalized likelihood gives good results if the penalty term in the cost function is appropriate for the particular problem if this is not the case averaging is superior since it shows greater robustness by not relying on any particular prior assumption the bayesian approach worked very well on a low dimensional toy problem but failed to give good performance in higher dimensional problems

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

tomographic image reconstruction based on a content adaptive mesh model in this paper we explore the use of a content adaptive mesh model camm for tomographic image reconstruction in the proposed framework the image to be reconstructed is first represented by a mesh model an efficient image description based on nonuniform sampling in the camm image samples represented as mesh nodes are placed most densely in image regions having fine detail tomographic image reconstruction in the mesh domain is performed by maximum likelihood ml or maximum a posteriori map estimation of the nodal values from the measured data a camm greatly reduces the number of unknown parameters to be determined leading to improved image quality and reduced computation time we demonstrated the method in our experiments using simulated gated single photon emission computed tomography spect cardiac perfusion images a channelized hotelling observer cho was used to evaluate the detectability of perfusion defects in the reconstructed images a task based measure of image quality a minimum description length mdl criterion was also used to evaluate the effect of the representation size in our application both mdl and cho suggested that the optimal number of mesh nodes is roughly five to seven times smaller than the number of projection bins when compared to several commonly used methods for image reconstruction the proposed approach achieved the best performance in terms of defect detection and computation time the research described in this paper establishes a foundation for future development of a four dimensional space time reconstruction framework for image sequences in which a built in deformable mesh model is used to track the image motion

a unified method for measurement and tracking of contacts from an array of sensors a multiple target track estimation method that operates directly from array data is presented the maximum a posteriori map estimator for contact states is derived for temporally uncorrelated signals and uncorrelated contact tracks where the number of contacts is assumed known this estimator is an iterative algorithm employing a nonlinear programming nlp penalty method in conjunction with an expectation maximization em algorithm the nlp technique is used to find the map track estimate based on the synthetic signal estimates produced by the em algorithm this method eliminates the data association step of traditional multitarget tracking approaches by conditioning the measurement process on individual target state distributions it results in a process similar to the em algorithm for direction finding with an additional penalty term imposed by the track distributions the algorithm is derived as a batch method an extension to support sequential tracking is also developed simulation results for a relevant submarine towed array scenario are presented and discussed

accelerated image reconstruction using ordered subsets of projection data the authors define ordered subset processing for standard algorithms such as expectation maximization em for image restoration from projections ordered subsets methods group projection data into an ordered sequence of subsets or blocks an iteration of ordered subsets em is defined as a single pass through all the subsets in each subset using the current estimate to initialize application of em with that data subset this approach is similar in concept to block kaczmarz methods introduced by eggermont et al 1981 for iterative reconstruction simultaneous iterative reconstruction sirt and multiplicative algebraic reconstruction mart techniques are well known special cases ordered subsets em os em provides a restoration imposing a natural positivity condition and with close links to the em algorithm os em is applicable in both single photon spect and positron emission tomography pet in simulation studies in spect the os em algorithm provides an order of magnitude acceleration over em with restoration quality maintained

nash equilibrium strategy for fuzzy non cooperative games in this article two person zero sum games are investigated in the fuzzy environment several models constructed by maeda in the symmetrical fuzzy environment are extended to the models in the asymmetric fuzzy environment the existence of equilibrium strategies for these extended models is proposed in the asymmetric fuzzy environment however in some cases nash equilibrium strategies may not exist therefore two special cases are presented for which nash equilibrium strategies do exist in order to investigate the existence of weak pareto nash equilibrium strategies for fuzzy matrix games we introduce the concept of crisp bi matrix games with parameters by solving the parametric bi matrix games we obtain the weak pareto nash equilibrium strategies for the fuzzy matrix games

shared kernel models for class conditional density estimation we present probabilistic models which are suitable for class conditional density estimation and can be regarded as shared kernel models where sharing means that each kernel may contribute to the estimation of the conditional densities of an classes we first propose a model that constitutes an adaptation of the classical radial basis function rbf network with full sharing of kernels among classes where the outputs represent class conditional densities in the opposite direction is the approach of separate mixtures model where the density of each class is estimated using a separate mixture density no sharing of kernels among classes we present a general model that allows for the expression of intermediate cases where the degree of kernel sharing can be specified through an extra model parameter this general model encompasses both the above mentioned models as special cases in all proposed models the training process is treated as a maximum likelihood problem and expectation maximization algorithms have been derived for adjusting the model parameters

shrinkage estimation for multivariate hidden markov models abstractmotivated from a changing market environment over time we consider high dimensional data such as financial returns generated by a hidden markov model that allows for switching between different regimes or states to get more stable estimates of the covariance matrices of the different states potentially driven by a number of observations that are small compared to the dimension we modify the expectation maximization em algorithm so that it yields the shrinkage estimators for the covariance matrices the final algorithm turns out to reproduce better estimates not only for the covariance matrices but also for the transition matrix it results into a more stable and reliable filter that allows for reconstructing the values of the hidden markov chain in addition to a simulation study performed in this article we also present a series of theoretical results that include dimensionality asymptotics and provide the motivation for certain techniques used in the algorithm supplementary materials fo

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

variances for maximum penalized likelihood estimates obtained via the em algorithm we address the problem of providing variances for parameter estimates obtained under a penalized likelihood formulation through use of the em algorithm the solution proposed represents a synthesis of two existent techniques firstly we exploit the supplemented em algorithm developed by meng and rubin that provides variance estimates for maximum likelihood estimates obtained via the em algorithm their procedure relies on evaluating the jacobian of the mapping induced by the em algorithm secondly we utilize results from green that provide expressions for jacobians of mappings induced by em algorithms applied to a penalized likelihood the resultant procedure requires no more code than that needed for the penalized em algorithm itself

fast bayesian factor analysis via automatic rotations to sparsity abstractrotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis regularization methods also serve to achieve this goal by prioritizing sparse loading matrices in this work we bridge these two paradigms with a unifying bayesian framework our approach deploys intermediate factor rotations throughout the learning process greatly enhancing the effectiveness of sparsity inducing priors these automatic rotations to sparsity are embedded within a pxl em algorithm a bayesian variant of parameter expanded em for posterior mode detection by iterating between soft thresholding of small factor loadings and transformations of the factor basis we obtain a dramatic accelerations b robustness against poor initializations and c better oriented sparse solutions to avoid the prespecification of the factor cardinality we extend the loading matrix to have infinitely many columns with the indian buffet process ibp prior the factor dimen

on use of the em algorithm for penalized likelihood estimation summary the em algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation this paper discusses properties of the em algorithm in such contexts concentrating on rates of conver gence and presents an alternative that is usually more practical and converges at least as quickly the em algorithm is a general approach to maximum likelihood estimation rather than a specific algorithm dempster et al 1977 discussed the method and derived basic properties demonstrating that a variety of procedures previously developed rather informally could be unified the common strand to problems where the approach is applicable is a notion of incomplete data this includes the conventional sense of missing data but is much broader than that the em algorithm demon strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical the incomplete data actually at hand are regarded as observable functions of these complete data the resulting algorithms while usually slow to converge are often extremely simple and remain practical in large problems where no other approaches may be feasible dempster et al 1977 briefly refer to the use of the same approach to the problem of finding the posterior mode maximum a posteriori estimate in a bayesian estima

discussion latent variable graphical model selection via convex optimization we want to congratulate the authors for a thought provoking and very interesting paper sparse modeling of the concentration matrix has enjoyed popularity in recent years it has been framed as a computationally convenient convex 1constrained estimation problem in yuan and lin 2007 and can be applied readily to higher dimensional problems the authors argue we think correctly that the sparsity of the concentration matrix is for many applications more plausible after the effects of a few latent variables have been removed the most attractive point about their method is surely that it is formulated as a convex optimization problem latent variable fitting and sparse graphical modeling of the conditional distribution of the observed variables can then be obtained through a single fitting procedure practical aspects the method deserves wide adoption but this will only be realistic if software is made available for example as an r package not many users will go to the trouble of implementing the method on their own so we will strongly urge the authors to do so an imputation method in the absence of readily available software it is worth thinking whether the proposed fitting procedure can be approximated by methods involving known and well tested computational techniques the concentration matrix of observed and hidden variables is k ko koh k ho kh where we have deviated from the notation in the paper by omitting the asterisk the proposed estimator sn ko of ko was defined as

finding curvilinear features in spatial point patterns principal curve clustering with noise clustering about principal curves combines parametric modeling of noise with nonparametric modeling of feature shape this is useful for detecting curvilinear features in spatial point patterns with or without background noise applications include the detection of curvilinear minefields from reconnaissance images some of the points in which represent false detections and the detection of seismic faults from earthquake catalogs our algorithm for principal curve clustering is in two steps the first is hierarchical and agglomerative hpcc and the second consists of iterative relocation based on the classification em algorithm cem pcc hpcc is used to combine potential feature clusters while cem pcc refines the results and deals with background noise it is important to have a good starting point for the algorithm this can be found manually or automatically using for example nearest neighbor clutter removal or model based clustering we choose the number of features and the amount of smoothing simultaneously using approximate bayes factors

a method for assessing age time disease incidence using serial prevalence data summary this paper considers nonparametric estimation of age and time specific trends in disease incidence using serial prevalence data collected from multiple cross sectional samples of a population over time the methodology accounts for differential selection of diseased and undiseased individuals resulting for example from differences in mortality it is shown that when a log linear incidence odds model is adopted an em algorithm provides a convenient method for carrying out maximum likelihood estimation primarily using existing generalized linear models software the procedure is quite general allowing a range of age time incidence models to be fitted under the same framework furthermore by making use of existing software for fitting generalized additive models the procedure can be generalized with virtually no extra complexity to allow maximization of a penalized likelihood for smooth nonparametric estimation automatic choice of smoothing level for the penalized likelihood estimates is discussed using generalized cross validation the method is applied to a data set on serial toxoplasmosis prevalence which has previously been analyzed under the assumption of nondifferential selection a variety of age time incidence models are fitted and the sensitivity to plausible differential selection patterns is considered it is found that nonmultiplicative models are unnecessary and that qualitative incidence trends are fairly robust to differential selection

a theoretical study of some maximum likelihood algorithms for emission and transmission tomography this paper has the dual purpose of introducing some new algorithms for emission and transmission tomography and proving mathematically that these algorithms and related antecedent algorithms converge like the em algorithms for positron single photon and transmission tomography the algorithms provide maximum likelihood estimates of pixel concentration or linear attenuation parameters one particular innovation we discuss is a computationally practical scheme for modifying the em algorithms to include a bayesian prior the bayesian versions of the em algorithms are shown to have superior convergence properties in a vicinity of the maximum we anticipate that some of the other algorithms will also converge faster than the em algorithms

penalized model based clustering with application to variable selection variable selection in clustering analysis is both challenging and important in the context of model based clustering analysis with a common diagonal covariance matrix which is especially suitable for high dimension low sample size settings we propose a penalized likelihood approach with an l1 penalty function automatically realizing variable selection via thresholding and delivering a sparse solution we derive an em algorithm to fit our proposed model and propose a modified bic as a model selection criterion to choose the number of components and the penalization parameter a simulation study and an application to gene function prediction with gene expression profiles demonstrate the utility of our method

backfitting in smoothing spline anova a computational scheme for fitting smoothing spline anova models to large data sets with a near tensor product design is proposed such data sets are common in spatial temporal analyses the proposed scheme uses the backfitting algorithm to take advantage of the tensor product design to save both computational memory and time several ways to further speed up the backfitting algorithm such as collapsing component functions and successive over relaxation are discussed an iterative imputation procedure is used to handle the cases of near tensor product designs an application to a global historical surface air temperature data set which motivated this work is used to illustrate the scheme proposed

mixed h2 h multi channel linear parameter varying control in discrete time this paper develops a new method for the synthesis of linear parameter varying lpv controllers in discrete time lpv plants under consideration have a linear fractional transformation lft representation in contrast to earlier results which are restricted to single objective lpv problems the proposed method can handle a set of h2 h specifications that can be defined channel wise this practically attractive extension is derived by using specific transformations of both the lyapunov and scaling multiplier variables in tandem with appropriate linearizing transformations of the controller data and of the controller scheduling function it is shown that the controller gain scheduling function can be constructed as an affine matrix valued function in the polytopic coordinates of the scheduled parameter hence is easily implemented on line finally these manipulations give rise to a tractable and practical lmi formulation of the multi objective lpv control problem

shape based regularization of electron tomographic reconstruction we introduce a tomographic reconstruction method implemented using a shape based regularization technique spatial models of known features in the structure being reconstructed are integrated into the reconstruction process as regularizers our regularization scheme is driven locally through shape information obtained from segmentation and compared with a known spatial model we demonstrated our method on tomography data from digital phantoms simulated data and experimental electron tomography et data of virus complexes our reconstruction showed reduced blurring and an improvement in the resolution of the reconstructed volume was also measured this method also produced improved demarcation of spike boundaries in viral membranes when compared with popular techniques like weighted back projection and the algebraic reconstruction technique improved et reconstructions will provide better structure elucidation and improved feature visualization which can aid in solving key biological issues our method can also be generalized to other tomographic modalities

orthogonalizing em a design based least squares algorithm we introduce an efficient iterative algorithm intended for various least squares problems based on a design of experiments perspective the algorithm called orthogonalizing em oem works for ordinary least squares ols and can be easily extended to penalized least squares the main idea of the procedure is to orthogonalize a design matrix by adding new rows and then solve the original problem by embedding the augmented design in a missing data framework we establish several attractive theoretical properties concerning oem for the ols with a singular regression matrix an oem sequence converges to the moore penrose generalized inverse based least squares estimator for ordinary and penalized least squares with various penalties it converges to a point having grouping coherence for fully aliased regression matrices convergence and the convergence rate of the algorithm are examined finally we demonstrate that oem is highly efficient for large scale least squares and penalized least squares proble

parameter dependent lyapunov functions and the discrete time popov criterion for robust analysis abstract parameter dependent lyapunov functions are developed for discrete time systems with constant real parameter uncertainty this construction of a family of lyapunov functions is then used to analyze robust stability with h 2 performance bounds for state space systems a special case of these results generalizes the classical discrete time popov criterion

h sub spl infin design with pole placement constraints an lmi approach this paper addresses the design of state or output feedback h sub spl infin controllers that satisfy additional constraints on the closed loop pole location sufficient conditions for feasibility are derived for a general class of convex regions of the complex plane these conditions are expressed in terms of linear matrix inequalities lmis and the authors formulation is therefore numerically tractable via lmi optimization in the state feedback case mixed h sub 2 h sub spl infin synthesis with regional pole placement is also discussed finally the validity and applicability of this approach are illustrated by a benchmark example

a smoothed em approach to indirect estimation problems with particular reference to stereology and emission tomography there are many practical problems where the observed data are not drawn directly from the density g of real interest but rather from another distribution derived from g by the application of an integral operator the estimation of g then entails both statistical and numerical difficulties a natural statistical approach is by maximum likelihood conveniently implemented using the em algorithm but this provides unsatisfactory reconstructions of g in this paper we modify the maximum likelihood em approach by introducing a simple smoothing step at each em iteration in our experience this algorithm converges in relatively few iterations to good estimates of g that do not depend on the choice of starting configuration some theoretical background is given that relates this smoothed em algorithm to a maximum penalized likelihood approach two applications are considered in detail the first is the classical stereology problem of determining particle size distributions from data collected on a plane section through a composite medium the second concerns the recovery of the structure of a section of the human body from external observations obtained by positron emission tomography for this problem we also suggest several technical improvements on existing methodology

space alternating generalized expectation maximization algorithm the expectation maximization em method can facilitate maximizing likelihood functions that arise in statistical estimation problems in the classical em paradigm one iteratively maximizes the conditional log likelihood of a single unobservable complete data space rather than maximizing the intractable likelihood function for the measured or incomplete data em algorithms update all parameters simultaneously which has two drawbacks 1 slow convergence and 2 difficult maximization steps due to coupling when smoothness penalties are used the paper describes the space alternating generalized em sage method which updates the parameters sequentially by alternating between several small hidden data spaces defined by the algorithm designer the authors prove that the sequence of estimates monotonically increases the penalized likelihood objective derive asymptotic convergence rates and provide sufficient conditions for monotone convergence in norm two signal processing applications illustrate the method estimation of superimposed signals in gaussian noise and image reconstruction from poisson measurements in both applications the sage algorithms easily accommodate smoothness penalties and converge faster than the em algorithms

from image deblurring to optimal investments maximum likelihood solutions for positive linear inverse problems the problem of recovering an input signal from a blurred output in an input output system with linear distortion is ubiquitous in science and technology when the blurred output is not degraded by statistical noise the problem is entirely deterministic and amounts to a mathematical inversion of a linear system with positive parameters subject to positivity constraints on the solution we show that all such linear inverse problems with positivity restrictions lininpos problems for short can be interpreted as statistical estimation problems from incomplete data based on infinitely large samples and that maximum likelihood ml estimation and the em algorithm provide a straightforward method of solution for such problems

interior point methodology for 3 d pet reconstruction interior point methods have been successfully applied to a wide variety of linear and nonlinear programming applications this paper presents a class of algorithms based on path following interior point methodology for performing regularized maximum likelihood ml reconstructions on three dimensional 3 d emission tomography data the algorithms solve a sequence of subproblems that converge to the regularized maximum likelihood solution from the interior of the feasible region the nonnegative orthant the authors propose two methods a primal method which updates only the primal image variables and a primal dual method which simultaneously updates the primal variables and the lagrange multipliers a parallel implementation permits the interior point methods to scale to very large reconstruction problems termination is based on well defined convergence measures namely the karush kuhn tucker first order necessary conditions for optimality the authors demonstrate the rapid convergence of the path following interior point methods using both data from a small animal scanner and monte carlo simulated data the proposed methods can readily be applied to solve the regularized weighted least squares reconstruction problem

estimation for general birth death processes birth death processes bdps are continuous time markov chains that track the number of particles in a system over time while widely used in population biology genetics and ecology statistical inference of the instantaneous particle birth and death rates remains largely limited to restrictive linear bdps in which per particle birth and death rates are constant researchers often observe the number of particles at discrete times necessitating data augmentation procedures such as expectation maximization em to find maximum likelihood estimates mles for bdps on finite state spaces there are powerful matrix methods for computing the conditional expectations needed for the e step of the em algorithm for bdps on infinite state spaces closed form solutions for the e step are available for some linear models but most previous work has resorted to time consuming simulation remarkably we show that the e step conditional expectations can be expressed as convolutions of computable transition probabi

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

two component mixture cure rate model with spline estimated nonparametric components in some survival analysis of medical studies there are often long term survivors who can be considered as permanently cured the goals in these studies are to estimate the noncured probability of the whole population and the hazard rate of the susceptible subpopulation when covariates are present as often happens in practice to understand covariate effects on the noncured probability and hazard rate is of equal importance the existing methods are limited to parametric and semiparametric models we propose a two component mixture cure rate model with nonparametric forms for both the cure probability and the hazard rate function identifiability of the model is guaranteed by an additive assumption that allows no time covariate interactions in the logarithm of hazard rate estimation is carried out by an expectation maximization algorithm on maximizing a penalized likelihood for inferential purpose we apply the louis formula to obtain point wise confidence intervals for noncured probability and hazard rate asymptotic convergence rates of our function estimates are established we then evaluate the proposed method by extensive simulations we analyze the survival data from a melanoma study and find interesting patterns for this study

regularized interpolation for noisy images interpolation is the means by which a continuously defined model is fit to discrete data samples when the data samples are exempt of noise it seems desirable to build the model by fitting them exactly in medical imaging where quality is of paramount importance this ideal situation unfortunately does not occur in this paper we propose a scheme that improves on the quality by specifying a tradeoff between fidelity to the data and robustness to the noise we resort to variational principles which allow us to impose smoothness constraints on the model for tackling noisy data based on shift rotation and scale invariant requirements on the model we show that the lp norm of an appropriate vector derivative is the most suitable choice of regularization for this purpose in addition to tikhonov like quadratic regularization this includes edge preserving total variation like tv regularization we give algorithms to recover the continuously defined model from noisy samples and also provide a data driven scheme to determine the optimal amount of regularization we validate our method with numerical examples where we demonstrate its superiority over an exact fit as well as the benefit of tv like nonquadratic regularization over tikhonov like quadratic regularization

assessing quantification for the ems algorithm abstract a framework is developed for assessing the potential of the ems estimate maximize and smooth algorithm to correctly quantify the emission it aims to reconstruct maximal eigenvector localization bounds of perron frobenius theory are used to show that the ability of the ems algorithm to quantify differences in emission is fundamentally limited by the smoothing imposed and the properties of the underlying model matrix in the algorithm in particular it is established that for nonnegative irreducible smoothing the ems algorithm will always fail to correctly reconstruct zero emission intensity

maximum likelihood estimation in the joint analysis of time to event and multiple longitudinal variables joint modelling of longitudinal and survival data has received much attention in recent years most have concentrated on a single longitudinal variable this paper considers joint modelling in the presence of multiple longitudinal variables we explore direct association of time to event and multiple longitudinal processes through a frailty model and use a mixed effects model for each of the longitudinal variables correlations among the longitudinal variables are induced through correlated random effects we allow effects of categorical and continuous covariates on both longitudinal and time to event responses and explore interactions between the longtudinal variables and other covariates on time to event estimates of the parameters are obtained by maximizing the joint likelihood for the longitudinal variable processes and the event process we use a one step late em algorithm to handle the direct dependence of the event process on the modelled longitudinal variables along with the presence of other fixed covariates in both processes we argue that such a joint analysis with multiple longitudinal variables is advantageous to one with only a single longitudinal variable in revealing interplay among multiple longitudinal variables and the time to event copyright 2002 john wiley sons ltd

cross reference weighted least square estimates for positron emission tomography an efficient new method termed as the cross reference weighted least square estimate wlse crwlse is proposed to integrate the incomplete local smoothness information to improve the reconstruction of positron emission tomography pet images in the presence of accidental coincidence events and attenuation the algebraic reconstruction technique art is applied to this new estimate and the convergence is proved this numerical technique is based on row operations the computational complexity is only linear in the sizes of pixels and detector tubes hence it is efficient in storage and computation for a large and sparse system moreover the easy incorporation of range limits and spatially variant penalty will not deprive the efficiency all this makes the new method practically applicable an automatically data driven selection method for this new estimate based on the generalized cross validation is also studied the monte carlo studies demonstrate the advantages of this new method

a convex characterization of gain scheduled h sub spl infin controllers an important class of linear time varying systems consists of plants where the state space matrices are fixed functions of some time varying physical parameters spl theta small gain techniques can be applied to such systems to derive robust time invariant controllers yet this approach is often overly conservative when the parameters spl theta undergo large variations during system operation in general higher performance can be achieved by control laws that incorporate available measurements of spl theta and therefore adjust to the current plant dynamics this paper discusses extensions of h sub spl infin synthesis techniques to allow for controller dependence on time varying but measured parameters when this dependence is linear fractional the existence of such gain scheduled h sub spl infin controllers is fully characterized in terms of linear matrix inequalities the underlying synthesis problem is therefore a convex program for which efficient optimization techniques are available the formalism and derivation techniques developed here apply to both the continuous and discrete time problems existence conditions for robust time invariant controllers are recovered as a special case and extensions to gain scheduling in the face of parametric uncertainty are discussed in particular simple heuristics are proposed to compute such controllers

a quasi empirical bayes method for small area estimation abstract this article develops an empirical bayes type approach for small area estimation based only on the specification of a set of conditionally independent hierarchical mean and variance functions describing the first two moments of the process generating the data the objective of the analysis is to draw inference about the intermediate or the random means we combine the quasi likelihood functions to construct the quasi posterior density of the random means conditional on the data and the marginal parameters we describe a method for estimating the marginal parameters that are then substituted in the quasi posterior density of the random means the empirical quasi posterior density so derived is used to obtain the quasi empirical bayes estimates of the random parameters we apply the methodology to estimate the utilization rates of cancer chemotherapy for the selected counties in the state of washington

multivariate regression analyses for categorical data summary it is common to observe a vector of discrete and or continuous responses in scientific problems where the objective is to characterize the dependence of each response on explanatory variables and to account for the association between the outcomes the response vector can comprise repeated observations on one variable as in longitudinal studies or genetic studies of families or can include observations for different variables this paper discusses a class of models for the marginal expectations of each response and for pairwise associations the marginal models are contrasted with log linear models two generalized estimating equation approaches are compared for parameter estimation the first focuses on the regression parameters the second simultaneously estimates the regression and association parameters the robustness and efficiency of each is discussed the methods are illustrated with analyses of two data sets from public health research

kullback proximal algorithms for maximum likelihood estimation accelerated algorithms for maximum likelihood image reconstruction are essential for emerging applications such as three dimensional 3 d tomography dynamic tomographic imaging and other high dimensional inverse problems in this paper we introduce and analyze a class of fast and stable sequential optimization methods for computing maximum likelihood estimates and study its convergence properties these methods are based on a proximal point algorithm implemented with the kullback liebler kl divergence between posterior densities of the complete data as a proximal penalty function when the proximal relaxation parameter is set to unity one obtains the classical expectation maximization em algorithm for a decreasing sequence of relaxation parameters relaxed versions of em are obtained which can have much faster asymptotic convergence without sacrifice of monotonicity we present an implementation of the algorithm using more s 1983 trust region update strategy for illustration the method is applied to a nonquadratic inverse problem with poisson distributed data