a primal dual method for large scale image reconstruction in emission tomography in emission tomography images can be reconstructed from a set of measured projections using a maximum likelihood ml criterion in this paper we present a primal dual algorithm for large scale three dimensional image reconstruction the primal dual method is specialized to the ml reconstruction problem the reconstruction problem is extremely large in several of our data sets the hessian of the objective function is the product of a 1 4 million by 63 million matrix and its scaled transpose as such we consider only approaches that are suitable for large scale parallel computation we apply a stabilization technique to the system of equations for computing the primal direction and demonstrate the need for stabilization when approximately solving the system using an early terminated conjugate gradient iteration we demonstrate that the primal dual method for this problem converges faster than the logarithmic barrier method and considerably faster than the expectation maximization algorithm the use of extrapolation in conjunction with the primal dual method further reduces the overall computation required to achieve convergence

block truncated newton methods for parallel optimization truncated newton methods are a class of optimization methods suitable for large scale problems at each iteration a search direction is obtained by approximately solving the newton equations using an iterative method in this way matrix costs and second derivative calculations are avoided hence removing the major drawbacks of newton s method in this form the algorithms are well suited for vectorization further improvements in performance are sought by using block iterative methods for computing the search direction in particular conjugate gradient type methods are considered computational experience on a hypercube computer is reported indicating that on some problems the improvements in performance can be better than that attributable to parallelism alone

long term prediction of non linear time series using multiresolution wavelet models the long term prediction of non linear dynamical time series based on identified multiresolution wavelet models from historically observed data sets is investigated and a new direct prediction approach is introduced prediction results based on the new direct scheme are compared with those from iterative methods and it is shown that improved predictions can be obtained using the new approach

maximum entropy and the nearly black object summary maximum entropy me inversion is a non linear inversion technique for inverse problems where the object to be recovered is known to be positive it has been applied in areas ranging from radio astronomy to various forms of spectroscopy sometimes with dramatic success in some cases me has attained an order of magnitude finer resolution and or an order of magnitude smaller noise level than that obtainable by standard linear methods the dramatic successes all seem to occur in cases where the object to be recovered is nearly black essentially zero in the vast majority of samples we show that near blackness is required both for signal to noise enhancements and for superresolution however other methods in particular minimum 1 norm reconstruction may exploit near blackness to an even greater extent

bayesian reconstructions from emission tomography data using a modified em algorithm a novel method of reconstruction from single photon emission computerized tomography data is proposed this method builds on the expectation maximization em approach to maximum likelihood reconstruction from emission tomography data but aims instead at maximum posterior probability estimation which takes account of prior belief about smoothness in the isotope concentration a novel modification to the em algorithm yields a practical method the method is illustrated by an application to data from brain scans

network tomography estimating source destination traffic intensities from link data abstract the problem of estimating the node to node traffic intensity from repeated measurements of traffic on the links of a network is formulated and discussed under poisson assumptions and two types of traffic routing regimens deterministic a fixed known path between each directed pair of nodes and markovian a random path between each directed pair of nodes determined according to a known markov chain fixed for that pair maximum likelihood estimation and related approximations are discussed and computational difficulties are pointed out a detailed methodology is presented for estimates based on the method of moments the estimates are derived algorithmically taking advantage of the fact that the first and second moment equations give rise to a linear inverse problem with positivity restrictions that can be approached by an em algorithm resulting in a particularly simple solution to a hard problem a small simulation study is carried out

applying the method of surrogate data to cyclic time series the surrogate data methodology is used to test a given time series for membership of specific classes of dynamical systems currently there are three algorithms that are widely applied in the literature the most general of these tests the hypothesis of nonlinearly scaled linearly filtered noise however these tests and the many extensions of them that have been suggested are inappropriate for data exhibiting strong cyclic components for such data it is more natural to ask if there exist any long term of period longer than the data cycle length determinism in this paper we discuss existing techniques that attempt to address this hypothesis and introduce a new approach this new approach generates surrogates that are constrained i e they look like the data and for cyclic time series tests the null hypothesis of a periodic orbit with uncorrelated noise we examine various alternative implementations of this algorithm applying it to a variety of known test systems and experimental time series with unknown dynamics 2002 elsevier science b v all rights reserved

convergence of em image reconstruction algorithms with gibbs smoothing p j green has defined an osl one step late algorithm that retains the e step of the em algorithm for image reconstruction in emission tomography but provides an approximate solution to the m step further modifications of the osl algorithm guarantee convergence to the unique maximum of the log posterior function convergence is proved under a specific set of sufficient conditions several of these conditions concern the potential function of the gibb s prior and a number of candidate potential functions are identified generalization of the osl algorithm to transmission tomography is also considered

a penalized em algorithm incorporating missing data mechanism for gaussian parameter estimation missing data rates could depend on the targeted values in many settings including mass spectrometry based proteomic profiling studies here we consider mean and covariance estimation under a multivariate gaussian distribution with non ignorable missingness including scenarios in which the dimension p of the response vector is equal to or greater than the number n of independent observations a parameter estimation procedure is developed by maximizing a class of penalized likelihood functions that entails explicit modeling of missing data probabilities the performance of the resulting penalized em algorithm incorporating missing data mechanism pemm estimation procedure is evaluated in simulation studies and in a proteomic data illustration

nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci estimation of the covariance structure of longitudinal processes is a fundamental prerequisite for the practical deployment of functional mapping designed to study the genetic regulation and network of quantitative variation in dynamic complex traits we present a nonparametric approach for estimating the covariance structure of a quantitative trait measured repeatedly at a series of time points specifically we adopt huang et al s 2006 biometrika 93 85 98 approach of invoking the modified cholesky decomposition and converting the problem into modeling a sequence of regressions of responses a regularized covariance estimator is obtained using a normal penalized likelihood with an l 2 penalty this approach embedded within a mixture likelihood framework leads to enhanced accuracy precision and flexibility of functional mapping while preserving its biological relevance simulation studies are performed to reveal the statistical properties and advantages of the proposed method a real example from a mouse genome project is analyzed to illustrate the utilization of the methodology the new method will provide a useful tool for genome wide scanning for the existence and distribution of quantitative trait loci underlying a dynamic trait important to agriculture biology and health sciences

numerical stability and efficiency of penalty algorithms penalty algorithms have been somewhat forgotten due to numerical instabilities once believed to be inherent to those methods one usually has to solve a sequence of such problems and when the penalty factor decreases too fast the subproblems may become intractable moreover as the penalty factor decreases the unconstrained subproblem becomes ill conditioned and thus difficult to solve also in several intermediate computations numerical instability may show up the author proposes remedies to such problems and presents a wide class of numerically stable penalty algorithms the work is done in the more general context of variational inequality problems which encompasses optimization problems the author s results yield a family of globally convergent two step superlinearly convergent numerically stable algorithms for variational inequality problems finally issues in the numerically stable implementation of intermediate computations within those algorithms are discussed

a generalized em algorithm for 3 d bayesian reconstruction from poisson data using gibbs priors a generalized expectation maximization gem algorithm is developed for bayesian reconstruction based on locally correlated markov random field priors in the form of gibbs functions and on the poisson data model for the m step of the algorithm a form of coordinate gradient ascent is derived the algorithm reduces to the em maximum likelihood algorithm as the markov random field prior tends towards a uniform distribution three different gibbs function priors are examined reconstructions of 3 d images obtained from the poisson model of single photon emission computed tomography are presented

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

tomographic image reconstruction based on a content adaptive mesh model in this paper we explore the use of a content adaptive mesh model camm for tomographic image reconstruction in the proposed framework the image to be reconstructed is first represented by a mesh model an efficient image description based on nonuniform sampling in the camm image samples represented as mesh nodes are placed most densely in image regions having fine detail tomographic image reconstruction in the mesh domain is performed by maximum likelihood ml or maximum a posteriori map estimation of the nodal values from the measured data a camm greatly reduces the number of unknown parameters to be determined leading to improved image quality and reduced computation time we demonstrated the method in our experiments using simulated gated single photon emission computed tomography spect cardiac perfusion images a channelized hotelling observer cho was used to evaluate the detectability of perfusion defects in the reconstructed images a task based measure of image quality a minimum description length mdl criterion was also used to evaluate the effect of the representation size in our application both mdl and cho suggested that the optimal number of mesh nodes is roughly five to seven times smaller than the number of projection bins when compared to several commonly used methods for image reconstruction the proposed approach achieved the best performance in terms of defect detection and computation time the research described in this paper establishes a foundation for future development of a four dimensional space time reconstruction framework for image sequences in which a built in deformable mesh model is used to track the image motion

a unified method for measurement and tracking of contacts from an array of sensors a multiple target track estimation method that operates directly from array data is presented the maximum a posteriori map estimator for contact states is derived for temporally uncorrelated signals and uncorrelated contact tracks where the number of contacts is assumed known this estimator is an iterative algorithm employing a nonlinear programming nlp penalty method in conjunction with an expectation maximization em algorithm the nlp technique is used to find the map track estimate based on the synthetic signal estimates produced by the em algorithm this method eliminates the data association step of traditional multitarget tracking approaches by conditioning the measurement process on individual target state distributions it results in a process similar to the em algorithm for direction finding with an additional penalty term imposed by the track distributions the algorithm is derived as a batch method an extension to support sequential tracking is also developed simulation results for a relevant submarine towed array scenario are presented and discussed

accelerated image reconstruction using ordered subsets of projection data the authors define ordered subset processing for standard algorithms such as expectation maximization em for image restoration from projections ordered subsets methods group projection data into an ordered sequence of subsets or blocks an iteration of ordered subsets em is defined as a single pass through all the subsets in each subset using the current estimate to initialize application of em with that data subset this approach is similar in concept to block kaczmarz methods introduced by eggermont et al 1981 for iterative reconstruction simultaneous iterative reconstruction sirt and multiplicative algebraic reconstruction mart techniques are well known special cases ordered subsets em os em provides a restoration imposing a natural positivity condition and with close links to the em algorithm os em is applicable in both single photon spect and positron emission tomography pet in simulation studies in spect the os em algorithm provides an order of magnitude acceleration over em with restoration quality maintained

shrinkage estimation for multivariate hidden markov models abstractmotivated from a changing market environment over time we consider high dimensional data such as financial returns generated by a hidden markov model that allows for switching between different regimes or states to get more stable estimates of the covariance matrices of the different states potentially driven by a number of observations that are small compared to the dimension we modify the expectation maximization em algorithm so that it yields the shrinkage estimators for the covariance matrices the final algorithm turns out to reproduce better estimates not only for the covariance matrices but also for the transition matrix it results into a more stable and reliable filter that allows for reconstructing the values of the hidden markov chain in addition to a simulation study performed in this article we also present a series of theoretical results that include dimensionality asymptotics and provide the motivation for certain techniques used in the algorithm supplementary materials fo

on selecting models for nonlinear time series abstract constructing models from time series with nontrivial dynamics involves the problem of how to choose the best model from within a class of models or to choose between competing classes this paper discusses a method of building nonlinear models of possibly chaotic systems from data while maintaining good robustness against noise the models that are built are close to the simplest possible according to a description length criterion the method will deliver a linear model if that has shorter description length than a nonlinear model we show how our models can be used for prediction smoothing and interpolation in the usual way we also show how to apply the results to identification of chaos by detecting the presence of homoclinic orbits directly from time series

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

a general purpose parallel algorithm for unconstrained optimization this paper describes a general purpose algorithm for unconstrained optimization that is suitable for a parallel computer it is designed to be as easy to use as traditional algorithms for this problem requiring only that a scalar subroutine be provided to evaluate the objective function and its gradient vector of first derivatives the algorithm used is a block truncated newton method truncated newton methods are a class of methods that compromise on newton s method so that large problems can be solved enhancements to the basic method suitable for a parallel computer are described these include a revised data storage scheme new preconditioning and initialization strategies to accelerate the method a parallel line search revised stopping rules for the inner algorithm and a new nonlinearity test to determine the adequacy of the quadratic model numerical results are presented to illustrate the performance of the method and comparisons are made with other scalar and parallel algorithms

preconditioning methods for improved convergence rates in iterative reconstructions because of the characteristics of the tomographic inversion problem iterative reconstruction techniques often suffer from poor convergence rates especially at high spatial frequencies by using preconditioning methods the convergence properties of most iterative methods can be greatly enhanced without changing their ultimate solution to increase reconstruction speed spatially invariant preconditioning filters that can be designed using the tomographic system response and implemented using 2 d frequency domain filtering techniques have been applied in a sample application reconstructions from noiseless simulated projection data were performed using preconditioned and conventional steepest descent algorithms the preconditioned methods demonstrated residuals that were up to a factor of 30 lower than the assisted algorithms at the same iteration applications of these methods to regularized reconstructions from projection data containing poisson noise showed similar although not as dramatic behavior

multiscale support vector learning with projection operator wavelet kernel for nonlinear dynamical system identification a giant leap has been made in the past couple of decades with the introduction of kernel based learning as a mainstay for designing effective nonlinear computational learning algorithms in view of the geometric interpretation of conditional expectation and the ubiquity of multiscale characteristics in highly complex nonlinear dynamic systems 1 3 this paper presents a new orthogonal projection operator wavelet kernel aiming at developing an efficient computational learning approach for nonlinear dynamical system identification in the framework of multiresolution analysis the proposed projection operator wavelet kernel can fulfill the multiscale multidimensional learning to estimate complex dependencies the special advantage of the projection operator wavelet kernel developed in this paper lies in the fact that it has a closed form expression which greatly facilitates its application in kernel learning to the best of our knowledge it is the first closed form orthogonal projection wavelet kernel reported in the literature it provides a link between grid based wavelets and mesh free kernel based methods simulation studies for identifying the parallel models of two benchmark nonlinear dynamical systems confirm its superiority in model accuracy and sparsity

discussion latent variable graphical model selection via convex optimization we want to congratulate the authors for a thought provoking and very interesting paper sparse modeling of the concentration matrix has enjoyed popularity in recent years it has been framed as a computationally convenient convex 1constrained estimation problem in yuan and lin 2007 and can be applied readily to higher dimensional problems the authors argue we think correctly that the sparsity of the concentration matrix is for many applications more plausible after the effects of a few latent variables have been removed the most attractive point about their method is surely that it is formulated as a convex optimization problem latent variable fitting and sparse graphical modeling of the conditional distribution of the observed variables can then be obtained through a single fitting procedure practical aspects the method deserves wide adoption but this will only be realistic if software is made available for example as an r package not many users will go to the trouble of implementing the method on their own so we will strongly urge the authors to do so an imputation method in the absence of readily available software it is worth thinking whether the proposed fitting procedure can be approximated by methods involving known and well tested computational techniques the concentration matrix of observed and hidden variables is k ko koh k ho kh where we have deviated from the notation in the paper by omitting the asterisk the proposed estimator sn ko of ko was defined as

on use of the em algorithm for penalized likelihood estimation summary the em algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation this paper discusses properties of the em algorithm in such contexts concentrating on rates of conver gence and presents an alternative that is usually more practical and converges at least as quickly the em algorithm is a general approach to maximum likelihood estimation rather than a specific algorithm dempster et al 1977 discussed the method and derived basic properties demonstrating that a variety of procedures previously developed rather informally could be unified the common strand to problems where the approach is applicable is a notion of incomplete data this includes the conventional sense of missing data but is much broader than that the em algorithm demon strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical the incomplete data actually at hand are regarded as observable functions of these complete data the resulting algorithms while usually slow to converge are often extremely simple and remain practical in large problems where no other approaches may be feasible dempster et al 1977 briefly refer to the use of the same approach to the problem of finding the posterior mode maximum a posteriori estimate in a bayesian estima

finding curvilinear features in spatial point patterns principal curve clustering with noise clustering about principal curves combines parametric modeling of noise with nonparametric modeling of feature shape this is useful for detecting curvilinear features in spatial point patterns with or without background noise applications include the detection of curvilinear minefields from reconnaissance images some of the points in which represent false detections and the detection of seismic faults from earthquake catalogs our algorithm for principal curve clustering is in two steps the first is hierarchical and agglomerative hpcc and the second consists of iterative relocation based on the classification em algorithm cem pcc hpcc is used to combine potential feature clusters while cem pcc refines the results and deals with background noise it is important to have a good starting point for the algorithm this can be found manually or automatically using for example nearest neighbor clutter removal or model based clustering we choose the number of features and the amount of smoothing simultaneously using approximate bayes factors

pet regularization by envelope guided conjugate gradients the authors propose a new way to iteratively solve large scale ill posed problems and in particular the image reconstruction problem in positron emission tomography by exploiting the relation between tikhonov regularization and multiobjective optimization to obtain iteratively approximations to the tikhonov l curve and its corner monitoring the change of the approximate l curves allows the authors to adjust the regularization parameter adaptively during a preconditioned conjugate gradient iteration so that the desired solution can be reconstructed with a small number of iterations

an infeasible interior point method for linear complementarity problems in this work we present an infeasible interior point algorithm which is based on a method for the general nonlinear programming problem to solve linear complementarity problems for this algorithm we prove global convergence from any strictly positive starting point under minor assumptions numerical results are reported which demonstrate very good computational performance on large scale linear complementarity problems

a method for assessing age time disease incidence using serial prevalence data summary this paper considers nonparametric estimation of age and time specific trends in disease incidence using serial prevalence data collected from multiple cross sectional samples of a population over time the methodology accounts for differential selection of diseased and undiseased individuals resulting for example from differences in mortality it is shown that when a log linear incidence odds model is adopted an em algorithm provides a convenient method for carrying out maximum likelihood estimation primarily using existing generalized linear models software the procedure is quite general allowing a range of age time incidence models to be fitted under the same framework furthermore by making use of existing software for fitting generalized additive models the procedure can be generalized with virtually no extra complexity to allow maximization of a penalized likelihood for smooth nonparametric estimation automatic choice of smoothing level for the penalized likelihood estimates is discussed using generalized cross validation the method is applied to a data set on serial toxoplasmosis prevalence which has previously been analyzed under the assumption of nondifferential selection a variety of age time incidence models are fitted and the sensitivity to plausible differential selection patterns is considered it is found that nonmultiplicative models are unnecessary and that qualitative incidence trends are fairly robust to differential selection

a theoretical study of some maximum likelihood algorithms for emission and transmission tomography this paper has the dual purpose of introducing some new algorithms for emission and transmission tomography and proving mathematically that these algorithms and related antecedent algorithms converge like the em algorithms for positron single photon and transmission tomography the algorithms provide maximum likelihood estimates of pixel concentration or linear attenuation parameters one particular innovation we discuss is a computationally practical scheme for modifying the em algorithms to include a bayesian prior the bayesian versions of the em algorithms are shown to have superior convergence properties in a vicinity of the maximum we anticipate that some of the other algorithms will also converge faster than the em algorithms

on the convergence of an em type algorithm for penalized likelihood estimation in emission tomography recently we proposed an extension of the expectation maximization em algorithm that was able to handle regularization terms in a natural way although very general convergence proofs were not valid for many possibly useful regularizations we present here a simple convergence result that is valid assuming only continuous differentiability of the penalty term and can be also extended to other methods for penalized likelihood estimation in tomography

backfitting in smoothing spline anova a computational scheme for fitting smoothing spline anova models to large data sets with a near tensor product design is proposed such data sets are common in spatial temporal analyses the proposed scheme uses the backfitting algorithm to take advantage of the tensor product design to save both computational memory and time several ways to further speed up the backfitting algorithm such as collapsing component functions and successive over relaxation are discussed an iterative imputation procedure is used to handle the cases of near tensor product designs an application to a global historical surface air temperature data set which motivated this work is used to illustrate the scheme proposed

shape based regularization of electron tomographic reconstruction we introduce a tomographic reconstruction method implemented using a shape based regularization technique spatial models of known features in the structure being reconstructed are integrated into the reconstruction process as regularizers our regularization scheme is driven locally through shape information obtained from segmentation and compared with a known spatial model we demonstrated our method on tomography data from digital phantoms simulated data and experimental electron tomography et data of virus complexes our reconstruction showed reduced blurring and an improvement in the resolution of the reconstructed volume was also measured this method also produced improved demarcation of spike boundaries in viral membranes when compared with popular techniques like weighted back projection and the algebraic reconstruction technique improved et reconstructions will provide better structure elucidation and improved feature visualization which can aid in solving key biological issues our method can also be generalized to other tomographic modalities

estimation of lyapunov exponents of dynamical systems using a spatial average we present an unconventional method of estimating all of the lyapunov exponents of a dynamical system from either a known map or a set of experimental data rather than averaging exponents along a single trajectory we instead represent each exponent as an integral over all of phase space the contribution to each exponent calculated at each point in phase space is averaged spatially by weighting areas of high density more heavily than areas of low density according to the invariant measure of the system explicit formulas for approximating both the contributions to the exponents and the invariant measure are given and convergence results stated the techniques are illustrated in detail for the henon system

image reconstruction and restoration overview of common estimation structures and problems developments in the theory of image reconstruction and restoration over the past 20 or 30 years are outlined particular attention is paid to common estimation structures and to practical problems not properly solved yet the problem of image reconstruction and restoration is first formulated some of the current regularization approaches used to solve the problem are then described the concepts of a priori information and compound criterion are introduced a bayesian interpretation of the regularization techniques is given which clarifies the role of the tuning parameters and indicates how they could be estimated the practical aspects of computing the solution first when the hyperparameters are known and second when they must be estimated are then considered conclusions are drawn and points that still need to be investigated are outlined

orthogonalizing em a design based least squares algorithm we introduce an efficient iterative algorithm intended for various least squares problems based on a design of experiments perspective the algorithm called orthogonalizing em oem works for ordinary least squares ols and can be easily extended to penalized least squares the main idea of the procedure is to orthogonalize a design matrix by adding new rows and then solve the original problem by embedding the augmented design in a missing data framework we establish several attractive theoretical properties concerning oem for the ols with a singular regression matrix an oem sequence converges to the moore penrose generalized inverse based least squares estimator for ordinary and penalized least squares with various penalties it converges to a point having grouping coherence for fully aliased regression matrices convergence and the convergence rate of the algorithm are examined finally we demonstrate that oem is highly efficient for large scale least squares and penalized least squares proble

maximum likelihood least squares and penalized least squares for pet the em algorithm is the basic approach used to maximize the log likelihood objective function for the reconstruction problem in positron emission tomography pet the em algorithm is a scaled steepest ascent algorithm that elegantly handles the nonnegativity constraints of the problem it is shown that the same scaled steepest descent algorithm can be applied to the least squares merit function and that it can be accelerated using the conjugate gradient approach the experiments suggest that one can cut the computation by about a factor of 3 by using this technique the results are applied to various penalized least squares functions which might be used to produce a smoother image

a smoothed em approach to indirect estimation problems with particular reference to stereology and emission tomography there are many practical problems where the observed data are not drawn directly from the density g of real interest but rather from another distribution derived from g by the application of an integral operator the estimation of g then entails both statistical and numerical difficulties a natural statistical approach is by maximum likelihood conveniently implemented using the em algorithm but this provides unsatisfactory reconstructions of g in this paper we modify the maximum likelihood em approach by introducing a simple smoothing step at each em iteration in our experience this algorithm converges in relatively few iterations to good estimates of g that do not depend on the choice of starting configuration some theoretical background is given that relates this smoothed em algorithm to a maximum penalized likelihood approach two applications are considered in detail the first is the classical stereology problem of determining particle size distributions from data collected on a plane section through a composite medium the second concerns the recovery of the structure of a section of the human body from external observations obtained by positron emission tomography for this problem we also suggest several technical improvements on existing methodology

random matrix elements and eigenfunctions in chaotic systems the expected root mean square value of a matrix element a ensuremath alpha ensuremath beta in a classically chaotic system where a is a smooth ensuremath elzxh independent function of the coordinates and momenta and ensuremath alpha and ensuremath beta label different energy eigenstates has been evaluated in the literature in two different ways by treating the energy eigenfunctions as gaussian random variables and averaging a ensuremath alpha ensuremath beta 2 over them and by relating a ensuremath alpha ensuremath beta 2 to the classical time correlation function of a we show that these two methods give the same answer only if berry s formula for the spatial correlations in the energy eigenfunctions which is based on a microcanonical density in phase space is modified at large separations in a manner that we previously proposed

from image deblurring to optimal investments maximum likelihood solutions for positive linear inverse problems the problem of recovering an input signal from a blurred output in an input output system with linear distortion is ubiquitous in science and technology when the blurred output is not degraded by statistical noise the problem is entirely deterministic and amounts to a mathematical inversion of a linear system with positive parameters subject to positivity constraints on the solution we show that all such linear inverse problems with positivity restrictions lininpos problems for short can be interpreted as statistical estimation problems from incomplete data based on infinitely large samples and that maximum likelihood ml estimation and the em algorithm provide a straightforward method of solution for such problems

space alternating generalized expectation maximization algorithm the expectation maximization em method can facilitate maximizing likelihood functions that arise in statistical estimation problems in the classical em paradigm one iteratively maximizes the conditional log likelihood of a single unobservable complete data space rather than maximizing the intractable likelihood function for the measured or incomplete data em algorithms update all parameters simultaneously which has two drawbacks 1 slow convergence and 2 difficult maximization steps due to coupling when smoothness penalties are used the paper describes the space alternating generalized em sage method which updates the parameters sequentially by alternating between several small hidden data spaces defined by the algorithm designer the authors prove that the sequence of estimates monotonically increases the penalized likelihood objective derive asymptotic convergence rates and provide sufficient conditions for monotone convergence in norm two signal processing applications illustrate the method estimation of superimposed signals in gaussian noise and image reconstruction from poisson measurements in both applications the sage algorithms easily accommodate smoothness penalties and converge faster than the em algorithms

a practical method for calculating largest lyapunov exponents from small data sets detecting the presence of chaos in a dynamical system is an important problem that is solved by measuring the largest lyapunov exponent lyapunov exponents quantify the exponential divergence of initially close state space trajectories and estimate the amount of chaos in a system we present a new method for calculating the largest lyapunov exponent from an experimental time series the method follows directly from the definition of the largest lyapunov exponent and is accurate because it takes advantage of all the available data we show that the algorithm is fast easy to implement and robust to changes in the following quantities embedding dimension size of data set reconstruction delay and noise level furthermore one may use the algorithm to calculate simultaneously the correlation dimension thus one sequence of computations will yield an estimate of both the level of chaos and the system complexity

a maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography the expectation maximization method for maximum likelihood image reconstruction in emission tomography based on the poisson distribution of the statistically independent components of the image and measurement vectors is extended to a maximum aposteriori image reconstruction using a multivariate gaussian a priori probability distribution of the image vector the approach is equivalent to a penalized maximum likelihood estimation with a special choice of the penalty function the expectation maximization method is applied to find the a posteriori probability maximizer a simple iterative formula is derived for a penalty function that is a weighted sum of the squared deviations of image vector components from their a priori mean values the method is demonstrated to be superior to pure likelihood maximization in that the penalty function prevents the occurrence of irregular high amplitude patterns in the image with a large number of iterations the so called checkerboard effect or noise artifact

interior point methodology for 3 d pet reconstruction interior point methods have been successfully applied to a wide variety of linear and nonlinear programming applications this paper presents a class of algorithms based on path following interior point methodology for performing regularized maximum likelihood ml reconstructions on three dimensional 3 d emission tomography data the algorithms solve a sequence of subproblems that converge to the regularized maximum likelihood solution from the interior of the feasible region the nonnegative orthant the authors propose two methods a primal method which updates only the primal image variables and a primal dual method which simultaneously updates the primal variables and the lagrange multipliers a parallel implementation permits the interior point methods to scale to very large reconstruction problems termination is based on well defined convergence measures namely the karush kuhn tucker first order necessary conditions for optimality the authors demonstrate the rapid convergence of the path following interior point methods using both data from a small animal scanner and monte carlo simulated data the proposed methods can readily be applied to solve the regularized weighted least squares reconstruction problem

asymptotic rate of quantum ergodicity in chaotic euclidean billiards the quantum unique ergodicity que conjecture of rudnick and sarnak is that every eigenfunction n of the laplacian on a manifold with uniformly hyperbolic geodesic flow becomes equidistributed in the semiclassical limit eigenvalue en that is strong scars are absent we study numerically the rate of equidistribution for a uniformly hyperbolic sinai type planar euclidean billiard with dirichlet boundary condition the drum problem at unprecedented high e and statistical accuracy via the matrix elements n m of a piecewise constant test function a by collecting 30 000 diagonal elements up to level n 7 105 we find that their variance decays with eigenvalue as a power 0 48 0 01 close to the semiclassical estimate of feingold and peres this contrasts with the results of existing studies which have been limited to en a factor 102 smaller we find strong evidence for que in this system we also compare off diagonal variance as a function of distance from the diagonal against feingold peres or spectral measure at the highest accuracy 0 7 thus far in any chaotic system we outline the efficient scaling method and boundary integral formulae used to calculate eigenfunctions 2006 wiley periodicals inc

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

two component mixture cure rate model with spline estimated nonparametric components in some survival analysis of medical studies there are often long term survivors who can be considered as permanently cured the goals in these studies are to estimate the noncured probability of the whole population and the hazard rate of the susceptible subpopulation when covariates are present as often happens in practice to understand covariate effects on the noncured probability and hazard rate is of equal importance the existing methods are limited to parametric and semiparametric models we propose a two component mixture cure rate model with nonparametric forms for both the cure probability and the hazard rate function identifiability of the model is guaranteed by an additive assumption that allows no time covariate interactions in the logarithm of hazard rate estimation is carried out by an expectation maximization algorithm on maximizing a penalized likelihood for inferential purpose we apply the louis formula to obtain point wise confidence intervals for noncured probability and hazard rate asymptotic convergence rates of our function estimates are established we then evaluate the proposed method by extensive simulations we analyze the survival data from a melanoma study and find interesting patterns for this study

autocorrelation function of eigenstates in chaotic and mixed systems we study the autocorrelation function of different types of eigenfunctions in quantum mechanical systems with either chaotic or mixed classical limits we obtain an expansion of the autocorrelation function in terms of the correlation distance for localized states in billiards like bouncing ball modes or states living on tori a simple model using only classical input gives good agreement with the exact result in particular a prediction for irregular eigenfunctions in mixed systems is derived and tested for chaotic systems the expansion of the autocorrelation function can be used to test quantum ergodicity on different length scales

assessing quantification for the ems algorithm abstract a framework is developed for assessing the potential of the ems estimate maximize and smooth algorithm to correctly quantify the emission it aims to reconstruct maximal eigenvector localization bounds of perron frobenius theory are used to show that the ability of the ems algorithm to quantify differences in emission is fundamentally limited by the smoothing imposed and the properties of the underlying model matrix in the algorithm in particular it is established that for nonnegative irreducible smoothing the ems algorithm will always fail to correctly reconstruct zero emission intensity

indistinguishable states ii the imperfect model scenario given a perfect model of a chaotic system and a set of noisy observations of arbitrary duration it is not possible to determine the state of this system precisely rather one must consider a set of states which are indistinguishable from one another given the observations k judd l a smith indistinguishable states i physica d 151 2001 yet the perfect model scenario is a fiction in practice all models are imperfect how do the results from the perfect model scenario change under imperfect models it is shown to be essential to take even small model imperfections into account failure to do so can systematically degrade state estimation or prediction of nonlinear systems with an imperfect model the system state space and model state space are rarely if ever equivalent and so one must consider a projection of the system state into the model state space furthermore it is almost certain that no trajectory of the model is consistent with an infinite series of observations thus there is no consistent way to estimate the projection of system state using trajectories there are pseudo orbits however that are consistent with observations and these can be used to estimate the projection of the system state using pseudo orbits one finds that as in the perfect model scenario there is a set of states that are indistinguishable from the projection of the system state estimation of this set of indistinguishable states and the probability density on these states is discussed the main conclusions are i that there is no state of the model that can be identified with the state of the system and ii that great care must be taken when using an imperfect model to forecast the system because the initialization of the model state from observations can provide a poor analogue for the system the forecast may not shadow the future behaviour of the system for very long even if one were able to obtain a noise free projection of the system state the ultimate aims of probability forecasts should be re examined in light of these results

fast gradient based methods for bayesian reconstruction of transmission and emission pet images the authors describe conjugate gradient algorithms for reconstruction of transmission and emission pet images the reconstructions are based on a bayesian formulation where the data are modeled as a collection of independent poisson random variables and the image is modeled using a markov random field a conjugate gradient algorithm is used to compute a maximum a posteriori map estimate of the image by maximizing over the posterior density to ensure nonnegativity of the solution a penalty function is used to convert the problem to one of unconstrained optimization preconditioners are used to enhance convergence rates these methods generally achieve effective convergence in 15 25 iterations reconstructions are presented of an sup 18 fdg whole body scan from data collected using a siemens cti ecat931 whole body system these results indicate significant improvements in emission image quality using the bayesian approach in comparison to filtered backprojection particularly when reprojections of the map transmission image are used in place of the standard attenuation correction factors

identification and prediction of low dimensional dynamics this contribution focuses upon extracting information from dynamic reconstructions of experimental time series data in addition to the problem of distinguishing between deterministic dynamics and stochastic dynamics applied questions such as the detection of parametric drift are addressed nonlinear prediction and dimension algorithms are applied to geophysical laboratory data and the significance of these results is established by comparison with results from similar surrogate series generated so as not to contain the property of interest a global nonlinear predictor is introduced which attempts to correct systematic bias due to the inhomogeneous distribution of data common in strange attractors variations in the quality of predictions with location in phase space are examined in order to estimate the uncertainty in a forecast at the time it is made finally the application of these methods to truly stochastic systems is discussed and the distinction between deterministic stochastic and low dimensional dynamics is considered

constrained iterative reconstruction by the conjugate gradient method the conjugate gradient method incorporating the object extent constraint is applied to image reconstruction of a three dimensional object using an incomplete projection data set the missing information is recovered by constraining the solution with the knowledge of the outer boundary of the object extent which may be a priori measured or known the algorithm is derived from the least squares criterion as an advanced version of conventional iterative reconstruction algorithms such as sirt simultaneous iterative reconstruction technique and ilst iterative least squares technique in the case of reconstruction from noisy projection data a method based on the minimum mean square error criterion is also proposed computer simulated reconstruction images of a phantom using limited angle and number of views are presented the result shows that the conjugate gradient method incorporating the object extent constraining provides the fastest convergence and the least error

a statistical model for positron emission tomography abstract positron emission tomography pet still in its research stages is a technique that promises to open new medical frontiers by enabling physicians to study the metabolic activity of the body in a pictorial manner much as in x ray transmission tomography and other modes of computerized tomography the quality of the reconstructed image in pet is very sensitive to the mathematical algorithm to be used for reconstruction in this article we tailor a mathematical model to the physics of positron emissions and we use the model to describe the basic image reconstruction problem of pet as a standard problem in statistical estimation from incomplete data we describe various estimation procedures such as the maximum likelihood ml method using the em algorithm the method of moments and the least squares method a computer simulation of a pet experiment is then used to demonstrate the ml and the least squares reconstructions the main purposes of this article are to report on what we believe is an

ill conditioning and computational error in interior methods for nonlinear programming ill conditioning has long been regarded as a plague on interior methods but its damaging effects have rarely been documented in fact implementors of interior methods who ignore warnings about the dire consequences of ill conditioning usually manage to compute accurate solutions we offer some insight into this seeming contradiction by analyzing ill conditioning within a primal dual method in which the full usually well conditioned primal dual matrix is transformed to a condensed inherently ill conditioned matrix mpd we show that ill conditioning in the exact condensed matrix closely resembles that known for the primal barrier hessian and then examine the influence of cancellation in the computed constraints using the structure of mpd various bounds are obtained on the absolute accuracy of the computed primal dual steps without cancellation the portion of the computed x step in the small space of mpd a subspace close to the null space of the jacobian of the active constraints has an absolute error bound comparable to machine precision and its large space component has a much smaller error bound with cancellation the usual case the absolute error bounds for both the small and large space components of the computed x step are comparable to machine precision in either case the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision the computed multiplier steps for inactive constraints although converging to zero retain approximately full relative precision because of errors in forming the right hand side the absolute error in the computed solution of the full well conditioned primal dual system is shown to be comparable to machine precision thus under quite general conditions ill conditioning in mpd does not noticeably impair the accuracy of the computed primal dual steps a similar analysis applies to search directions obtained by direct solution of the primal newton equations

a modified expectation maximization algorithm for penalized likelihood estimation in emission tomography the maximum likelihood ml expectation maximization em approach in emission tomography has been very popular in medical imaging for several years in spite of this no satisfactory convergent modifications have been proposed for the regularized approach here a modification of the em algorithm is presented the new method is a natural extension of the em for maximizing likelihood with concave priors convergence proofs are given

a practical interior point method for convex programming the authors present a primal interior point algorithm for solving convex programs with nonlinear constraints the algorithm uses a predictor corrector strategy to follow a smooth path that leads from a given starting point to an optimal solution a convergence analysis is given showing that under mild assumptions the algorithm simultaneously iterates towards feasibility and optimality the matrices involved can be kept sparse if the nonlinear functions are separable or depend on only a few variables a preliminary implementation has been developed some promising numerical results indicate that the algorithm may be efficient in practice and that it can deal in a single phase with infeasible starting points without relying on some big m parameter

towards long term prediction abstract this paper describes a simple method of obtaining longer term predictions from a nonlinear time series assuming one already has a reasonably good short term predictor the usefulness of the technique is that it eliminates to some extent the systematic errors of the iterated short term predictor the technique we describe also provides an indication of the prediction horizon we consider systems with both observational and dynamic noise and analyse a number of artificial and experimental systems obtaining consistent results we also compare this method of longer term prediction with ensemble prediction

fast bayesian factor analysis via automatic rotations to sparsity abstractrotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis regularization methods also serve to achieve this goal by prioritizing sparse loading matrices in this work we bridge these two paradigms with a unifying bayesian framework our approach deploys intermediate factor rotations throughout the learning process greatly enhancing the effectiveness of sparsity inducing priors these automatic rotations to sparsity are embedded within a pxl em algorithm a bayesian variant of parameter expanded em for posterior mode detection by iterating between soft thresholding of small factor loadings and transformations of the factor basis we obtain a dramatic accelerations b robustness against poor initializations and c better oriented sparse solutions to avoid the prespecification of the factor cardinality we extend the loading matrix to have infinitely many columns with the indian buffet process ibp prior the factor dimen

cross reference weighted least square estimates for positron emission tomography an efficient new method termed as the cross reference weighted least square estimate wlse crwlse is proposed to integrate the incomplete local smoothness information to improve the reconstruction of positron emission tomography pet images in the presence of accidental coincidence events and attenuation the algebraic reconstruction technique art is applied to this new estimate and the convergence is proved this numerical technique is based on row operations the computational complexity is only linear in the sizes of pixels and detector tubes hence it is efficient in storage and computation for a large and sparse system moreover the easy incorporation of range limits and spatially variant penalty will not deprive the efficiency all this makes the new method practically applicable an automatically data driven selection method for this new estimate based on the generalized cross validation is also studied the monte carlo studies demonstrate the advantages of this new method

kullback proximal algorithms for maximum likelihood estimation accelerated algorithms for maximum likelihood image reconstruction are essential for emerging applications such as three dimensional 3 d tomography dynamic tomographic imaging and other high dimensional inverse problems in this paper we introduce and analyze a class of fast and stable sequential optimization methods for computing maximum likelihood estimates and study its convergence properties these methods are based on a proximal point algorithm implemented with the kullback liebler kl divergence between posterior densities of the complete data as a proximal penalty function when the proximal relaxation parameter is set to unity one obtains the classical expectation maximization em algorithm for a decreasing sequence of relaxation parameters relaxed versions of em are obtained which can have much faster asymptotic convergence without sacrifice of monotonicity we present an implementation of the algorithm using more s 1983 trust region update strategy for illustration the method is applied to a nonquadratic inverse problem with poisson distributed data

some properties of the hessian of the logarithmic barrier function more than twenty years ago murray and lootsma showed that hessian matrices of the logarithmic barrier function become increasingly ill conditioned at points on the barrier trajectory as the solution is approached this paper explores some further characteristics of the barrier hessian we first show that except in two special cases the barrier hessian is ill conditioned in an entire region near the solution at points in a more restricted region including the barrier trajectory itself this ill conditioning displays a special structure connected with subspaces defined by the jacobian of the active constraints we then indicate how a cholesky factorization with diagonal pivoting can be used to detect numerical rank deficiency in the barrier hessian and to provide information about the underlying subspaces without making an explicit prediction of the active constraints using this subspace information a close approximation to the newton direction can be calculated by solving linear systems whose condition reflects that of the original problem

a general class of preconditioners for statistical iterative reconstruction of emission computed tomography a major drawback of statistical iterative image reconstruction for emission computed tomography is its high computational cost the ill posed nature of tomography leads to slow convergence for standard gradient based iterative approaches such as the steepest descent or the conjugate gradient algorithm here new theory and methods for a class of preconditioners are developed for accelerating the convergence rate of iterative reconstruction to demonstrate the potential of this class of preconditioners a preconditioned conjugate gradient pcg iterative algorithm for weighted least squares reconstruction wls was formulated for emission tomography using simulated positron emission tomography pet data of the hoffman brain phantom it was shown that the convergence rate of the pcg can reduce the number of iterations of the standard conjugate gradient algorithm by a factor of 2 8 times depending on the convergence criterion