on some convergence properties of the subspace constrained mean shift abstract subspace constrained mean shift scms is a non parametric iterative algorithm that has recently been proposed to find principal curves and surfaces based on a new definition involving the gradient and hessian of a kernel probability density estimate although simulation results using synthetic and real data have demonstrated the usefulness of the scms algorithm a rigorous study of its convergence is still missing this paper aims to take initial steps in this direction by showing that the scms algorithm inherits some important convergence properties of the mean shift ms algorithm in particular the monotonicity and convergence of the density estimate values along the sequence of output values of the algorithm is shown also it is shown that the distance between consecutive points of the output sequence converges to zero as does the projection of the gradient vector onto the subspace spanned by the d d eigenvectors corresponding to the d d largest eigenvalues of the local inverse covariance matrix these last two properties provide theoretical guarantees for stopping criteria by modifying the projection step three variations of the scms algorithm are proposed and the running times and performance of the resulting algorithms are compared

relations between 2 d and multichannel linear prediction a procedure for solving 2 d normal equations is derived by relating the 2 d linear prediction problem to a multichannel linear prediction problem and applying the multichannel levinson recursion

identification of arx models subject to missing data parameter estimation when the measurement information may be incomplete is discussed an arx model is used as a basic system representation the presentation covers both missing output and missing input first reconstruction of the missing values is discussed the reconstruction is based on a state space formulation of the system and is performed using kalman filtering or fixed interval smoothing formulas several approaches to the identification problem are presented including a new method based on the em expectation maximization algorithm the different approaches are tested and compared using monte carlo simulations the choice of method is always a tradeoff between estimation accuracy and computational complexity according to the simulations the gain in accuracy using the em method can be considerable if many data are missing

frequency estimation for closely spaced sinsoids simple approximations to the cramer rao lower bound simple approximate formulas for the evaluation of the cramer ran lower bound crlb of unbiased frequency estimates for the deterministic and stochastic cases of multiple snapshots of two closely spaced cisoids in complex noise are introduced these are based on general expressions recently published by stoica and nehorai also explored along the same lines is the case of a single snapshot com prising either one real sinusoid in real noise or two cisoids in complex noise these formulas though simple are accurate to within a few percent where the number of data points per snapshot is greater than about 6 it is felt that they add insight into the behavior of the crlb and cover many of the situations commonly cited in the literature

wideband maximum likelihood direction finding and signal parameter estimation by using the tree structured em algorithm the expectation maximization em algorithm is presented for the case of estimating direction of arrivals of unknown deterministic wideband signals alternative regularized least squares estimation techniques for the required signal estimation and a tree structure for the data mapping in the em algorithm are proposed extensive simulation results are presented for comparison of the proposed algorithms with the conventional em approach and the current high resolution methods of wideband direction finding

on the resolvability of normally distributed vector parameter estimates this paper presents a metric useful in analyzing the resolution of multidimensional vector parameter estimators that produce normal estimates this metric is based on the distance between ellipsoidal confidence regions about each of the parameters when the ellipsoids are disjoint the parameters are deemed resolvable this paper presents a method for finding the noise level at which all of the ellipsoids are disjoint but at least two are tangent this is deemed the resolution threshold noise level the underlying mathematical result treats the problem of finding a dilation noise level at which two coupled quadratic equations have a unique solution

a disaggregate negative binomial regression procedure for count data analysis various research areas face the methodological problems presented by nonnegative integer count data drawn from heterogeneous populations we present a disaggregate negative binomial regression procedure for analysis of count data observed for a heterogeneous sample of cross sections possibly over some fixed time periods this procedure simultaneously pools or groups cross sections while estimating a separate negative binomial regression model for each group an e m algorithm is described within a maximum likelihood framework to estimate the group proportions the group specific regression coefficients and the degree of overdispersion in event rates within each derived group the proposed procedure is illustrated with count data entailing nonnegative integer counts of purchases events for a frequently bought consumer good

on convergence and parameter selection of the em and da em algorithms for gaussian mixtures abstract the expectation maximization em for gaussian mixtures is popular as a clustering algorithm however the em algorithm is sensitive to initial values and so ueda and nakano 4 proposed the deterministic annealing em da em algorithm to improve it in this paper we investigate theoretical behaviors of the em and da em algorithms we first derive a general jacobian matrix of the da em algorithm with respect to posterior probabilities we then propose a theoretical lower bound for initialization of the annealing parameter in the da em algorithm on the other hand some researches mentioned that the em algorithm exhibits a self annealing behavior that is the equal posterior probability with small random perturbations can avoid the em algorithm to output the mass center for gaussian mixtures however there is no theoretical analysis on this self annealing property since the da em will become the em when the annealing parameter is 1 according to the jacobian matrix of the da em we can prove the self annealing property of the em algorithm for gaussian mixtures based on these results we give not only convergence behaviors of the equal posterior probabilities and initialization lower bound of the temperature parameter of the da em but also a theoretical explanation why the em algorithm for gaussian mixtures exhibits a self annealing behavior

conjugate mixture models for clustering multimodal data the problem of multimodal clustering arises whenever the data are gathered with several physically different sensors observations from different modalities are not necessarily aligned in the sense there there is no obvious way to associate or compare them in some common space a solution may consist in considering multiple clustering tasks independently for each modality the main difficulty with such an approach is to guarantee that the unimodal clusterings are mutually consistent in this letter we show that multimodal clustering can be addressed within a novel framework conjugate mixture models these models exploit the explicit transformations that are often available between an unobserved parameter space objects and each of the observation spaces sensors we formulate the problem as a likelihood maximization task and derive the associated conjugate expectation maximization algorithm the convergence properties of the proposed algorithm are thoroughly investigated several local and global optimization techniques are proposed in order to increase its convergence speed two initialization strategies are proposed and compared a consistent model selection criterion is proposed the algorithm and its variants are tested and evaluated within the task of 3d localization of several speakers using both auditory and visual data

mixtures of empirical distributions an iterative application of the age length key it is well known that an age length key is capable of providing biased estimates of age distributions when length distributions are applied to age length keys constructed from sampling in a different year from that in which the length data were collected in separate unpublished papers fukuda and chikuni far seas fisheries research laboratory fisheries agency of japan 1974 proposed a method of iterating an age length key that corrects this problem for a length distribution collected in any year their method provides an estimate of the age distribution corresponding to this length distribution which is consistent with an age length key constructed from the estimated age distribution and previously estimated length at age distributions we show that this method is an application of the em algorithm to mixtures of empirical distributions also we give conditions for which these estimates of age distribution are unique maximum likelihood estimates the method is used to estimate age distributions of the catch of greenland turbot reinhardtius hippoglossoides by small japanese freezer trawlers in the eastern bering sea

a unified framework and sparse bayesian perspective for direction of arrival estimation in the presence of array imperfections self calibration methods play an important role in reducing the negative effects of array imperfections during direction of arrival doa estimation however the dependence of most such methods on the eigenstructure techniques greatly degrades their adaptation to demanding scenarios such as low signal to noise ratio snr and limited snapshots this paper aims at formulating a unified framework and sparse bayesian perspective for array calibration and doa estimation a comprehensive model of the array output is presented first when a single type of array imperfection is considered with mutual coupling gain phase uncertainty and sensor location error treated as typical examples the spatial sparsity of the incident signals is then exploited and a bayesian method is proposed to realize array calibration and source doa estimation the array perturbation magnitudes are assumed to be small according to most application scenarios and the geometries of mutually coupled arrays are assumed to be uniform linear and those of arrays with sensor location errors are assumed to be linear cramer rao lower bounds crlbs for the array calibration and doa estimation precisions are also obtained the sparse bayesian method is finally extended to deal with the doa estimation problem when more than one type of array perturbation coexists

array shape calibration using sources in unknown locations a maximum likelihood approach sensor location uncertainty can severely degrade the performance of direction finding systems an iterative maximum likelihood method for simultaneously estimating directions of arrival doa and sensor locations is developed to alleviate this problem the case of nondisjoint sources i e sources observed in the same frequency cell and at the same time is emphasized the algorithm converges to the global maximum of the likelihood function if the initial conditions are sufficiently good numerical examples are presented illustrating the performance of the proposed technique a distinctive feature of the algorithm is its ability to locate the sensors accurately without deploying calibration sources at known locations

theoretical analyses of gain and phase error calibration with optimal implementation for linear equispaced array in array signal processing high resolution direction of arrival doa estimation algorithms are known to be sensitive to system errors in practice the system should be properly calibrated before doa estimation in this paper a calibration method of gain and phase errors of linear equispaced arrays leas is considered a class of simplified calibration algorithms based on different diagonal lines of the covariance matrix is proposed the statistical performance analyses of the calibration algorithms due to finite data perturbations are presented expressions for average bias abias and square root of average variance srav of the calibration algorithms are derived using first order approximation these statistical and computer simulation results reveal and explain why the more diagonal lines that are used for gain and phase error estimations the more inferior performance that may be obtained based on this conclusion the simple and optimal gain and phase error calibration algorithms for such model are obtained

image enhancement using the hypothesis selection filter theory and application to jpeg decoding we introduce the hypothesis selection filter hsf as a new approach for image quality enhancement we assume that a set of filters has been selected a priori to improve the quality of a distorted image containing regions with different characteristics at each pixel hsf uses a locally computed feature vector to predict the relative performance of the filters in estimating the corresponding pixel intensity in the original undistorted image the prediction result then determines the proportion of each filter used to obtain the final processed output in this way the hsf serves as a framework for combining the outputs of a number of different user selected filters each best suited for a different region of an image we formulate our scheme in a probabilistic framework where the hsf output is obtained as the bayesian minimum mean square error estimate of the original image maximum likelihood estimates of the model parameters are determined from an offline fully unsupervised training procedure that is derived from the expectation maximization algorithm to illustrate how to apply the hsf and to demonstrate its potential we apply our scheme as a post processing step to improve the decoding quality of jpeg encoded document images the scheme consistently improves the quality of the decoded image over a variety of image content with different characteristics we show that our scheme results in quantitative improvements over several other state of the art jpeg decoding methods

a monte carlo implementation of the em algorithm and the poor man s data augmentation algorithms abstract the first part of this article presents the monte carlo implementation of the e step of the em algorithm given the current guess to the maximizer of the posterior distribution latent data patterns are generated from the conditional predictive distribution the expected value of the augmented log posterior is then updated as a mixture of augmented log posteriors mixed over the generated latent data patterns multiple imputations in the m step of the algorithm this mixture is maximized to obtain the update to the maximizer of the observed posterior the gradient and hessian of the observed log posterior are also expressed as mixtures mixed over the multiple imputations the relation between the monte carlo em mcem algorithm and the data augmentation algorithm is noted two modifications to the mcem algorithm the poor man s data augmentation algorithms which allow for the calculation of the entire posterior are then presented these approximations serve as diagnostics for the validity o

bearing estimation without calibration for randomly perturbed arrays the bearing estimation problem of linearly periodic arrays with the presence of sensor position errors is discussed conventional approaches for solving this problem generally consist of two steps calibrating sensor locations and then performing bearing estimation based on the calibrated sensor locations approaches to carrying out bearing estimation based on the toeplitz and eigenstructure reconstruction of the covariance matrix without the need for calibration are proposed the toeplitz approximation method tam and a modification of it mtam are used to reconstruct a matrix with toeplitz structure to further enhance the capabilities of the tam and mtam an iterative algorithm incorporating with the tam itam and the mtam imtam is proposed to iteratively reconstruct both the toeplitz and the desired eigenstructure from the observed covariance matrix computer simulations show that the mtam is more effective than the tam moreover the iterative methods are superior to the noniterative ones at the price of computations

using the em algorithm to increase the number of signals estimable by 2 d parametric techniques this paper provides an easy method for increasing the number of modes estimable by 2 d parameter estimation schemes which use a single snapshot of the data for data of size m sub 1 spl times m sub 2 few of these techniques allow the number of modes to exceed m sub 1 and or m sub 2 as these restrictions are not inherent to the model but the algorithms themselves this problem is circumvented by treating the observed data as an incomplete version of a larger data set an existing 2 d parameter estimation scheme is then used for the maximization m step of the expectation maximization em algorithm in this way one can increase the number of modes the scheme can estimate without changing the scheme itself

sparse power angle spectrum estimation a novel method for estimating the power angle spectrum pas is presented that decomposes the true pas into a small set of basis functions the basis coefficients for this sparse representation are found by enforcing equality to the covariance or bartlett pas subject to a minimum lscr1 norm constraint the method referred to as sparse pas estimation spase can be implemented conveniently using existing linear programming lp solvers further because only a few clusters are required in the representation the method enables reduced complexity stochastic models for the channel and possibly allows reduced overhead in channel feedback schemes application of the method to simulated channels and multiple input multiple output mimo propagation data demonstrates the utility of the method

proper complex random processes with applications to information theory the covariance of complex random variables and processes when defined consistently with the corresponding notion for real random variables is shown to be determined by the usual complex covariance together with a quantity called the pseudo covariance a characterization of uncorrelatedness and wide sense stationarity in terms of covariance and pseudo covariance is given complex random variables and processes with a vanishing pseudo covariance are called proper it is shown that properness is preserved under affine transformations and that the complex multivariate gaussian density assumes a natural form only for proper random variables the maximum entropy theorem is generalized to the complex multivariate case the differential entropy of a complex random vector with a fixed correlation matrix is shown to be maximum if and only if the random vector is proper gaussian and zero mean the notion of circular stationarity is introduced for the class of proper complex random processes a discrete fourier transform correspondence is derived relating circular stationarity in the time domain to uncorrelatedness in the frequency domain an application of the theory is presented

statistical resolution limits and the complexified crame spl acute r rao bound array resolution limits and accuracy bounds on the multitude of signal parameters e g azimuth elevation doppler range cross range depth frequency chirp polarization amplitude phase etc estimated by array processing algorithms are essential tools in the evaluation of system performance the case in which the complex amplitudes of the signals are unknown is of particular practical interest a computationally efficient formulation of these bounds from the perspective of derivations and analysis is presented for the case of deterministic and unknown signal amplitudes a new derivation is given using the unknown complex signal parameters and their complex conjugates the new formula is readily applicable to obtaining either symbolic or numerical solutions to estimation bounds for a very wide class of problems encountered in adaptive sensor array processing this formula is shown to yield several of the standard crame spl acute r rao results for array processing along with new results of fundamental interest specifically a new closed form expression for the statistical resolution limit of an aperture for any asymptotically unbiased superresolution algorithm e g music esprit is provided the statistical resolution limit is defined as the source separation that equals its own crame spl acute r rao bound providing an algorithm independent bound on the resolution of any high resolution method it is shown that the statistical resolution limit of an array or coherent integration window is about 1 2 spl middot snr sup 1 4 relative to the fourier resolution limit of 2 spl pi n radians large number n of array elements that is the highest achievable resolution is proportional to the reciprocal of the fourth root of the signal to noise ratio snr in contrast to the square root snr sup 1 2 dependence of standard accuracy bounds these theoretical results are consistent with previously published bounds for specific superresolution algorithms derived by other methods it is also shown that the potential resolution improvement obtained by separating two collinear arrays synthetic ultra wideband each with a fixed aperture b wavelengths by m wavelengths assumed large is approximately m b sup 1 2 in contrast to the resolution improvement of m b for a full aperture exact closed form results for these problems with their asymptotic approximations are presented

estimation of structured gaussian mixtures the inverse em algorithm this contribution is devoted to the estimation of the parameters of multivariate gaussian mixture where the covariance matrices are constrained to have a linear structure such as toeplitz hankel or circular constraints we propose a simple modification of the expectation maximization em algorithm to take into account the structure constraints the basic modification consists of virtually updating the observed covariance matrices in a first stage then in a second stage the estimated covariances undergo the reversed updating the proposed algorithm is called the inverse em algorithm the increasing property of the likelihood through the algorithm iterations is proved the strict increasing for nonstationary points is proved as well numerical results are shown to corroborate the effectiveness of the proposed algorithm for the joint unsupervised classification and spectral estimation of stationary autoregressive time series

a novel online mutual coupling compensation algorithm for uniform and linear arrays in this paper a novel online mutual coupling compensation algorithm especially tailored to uniform and linear arrays is presented it is conceived to simultaneously compensate for mutual coupling and estimate the direction of arrivals doas of signals impinging on the array since the estimated calibration matrix can be embedded within any classical super resolution direction finding method an alternating minimization procedure based on closed form solutions is performed to estimate the mutual coupling matrix in the field of complex symmetric toeplitz matrices unlike many existing array calibration methods it requires neither the presence of calibration sources nor previous calibration information as initialization computer simulations show the effectiveness of the proposed technique and prove that the nice statistical properties of classical super resolution doa estimation algorithms can be restored despite the presence of mutual coupling

on the global and componentwise rates of convergence of the em algorithm abstract the em algorithm is a very general and popular iterative algorithm in statistics for finding maximum likelihood estimates in the presence of incomplete data in the paper that defined and popularized em dempster laird and rubin 1977 showed that its global rate of convergence is governed by the largest eigenvalue of the matrix of fractions of missing information due to incomplete data it was also mentioned that componentwise rates of convergence can differ from each other when the fractions of information loss vary across different components of a parameter vector in this article using the well known diagonability theorem we present a general description on how and when the componentwise rates differ as well as their relationships with the global rate we also provide an example a standard contaminated normal model to show that such phenomena are not necessarily pathological but can occur in useful statistical models

the statistical analysis of general processing tree models with the em algorithm multinomial processing tree models assume that an observed behavior category can arise from one or more processing sequences represented as branches in a tree these models form a subclass of parametric multinomial models and they provide a substantively motivated alternative to loglinear models we consider the usual case where branch probabilities are products of nonnegative integer powers in the parameters 0 s 1 and their complements 1 s a version of the em algorithm is constructed that has very strong properties first the e step and the m step are both analytic and computationally easy therefore a fast pc program can be constructed for obtaining mles for large numbers of parameters second a closed form expression for the observed fisher information matrix is obtained for the entire class third it is proved that the algorithm necessarily converges to a local maximum and this is a stronger result than for the exponential family as a whole fourth we show how the algorithm can handle quite general hypothesis tests concerning restrictions on the model parameters fifth we extend the algorithm to handle the read and cressie power divergence family of goodness of fit statistics the paper includes an example to illustrate some of these results

bayesian and regularization methods for hyperparameter estimation in image restoration in this paper we propose the application of the hierarchical bayesian paradigm to the image restoration problem we derive expressions for the iterative evaluation of the two hyperparameters applying the evidence and maximum a posteriori map analysis within the hierarchical bayesian paradigm we show analytically that the analysis provided by the evidence approach is more realistic and appropriate than the map approach for the image restoration problem we furthermore study the relationship between the evidence and an iterative approach resulting from the set theoretic regularization approach for estimating the two hyperparameters or their ratio defined as the regularization parameter finally the proposed algorithms are tested experimentally

further simple approximations to the cramer rao lower bound on frequency estimates for closely spaced sinusoids it is demonstrated that the cramer rao lower bound on frequency estimates for a data record containing two closely spaced cisoids in complex white gaussian noise can be approximated by an extremely simple nonmatrix expression it extends earlier work by explicitly retaining the difference in initial phases as a parameter of interest the approximation to the bound is shown to have a root mean square error of about 10 with occasional peak errors of about spl plusmn 25 over a wide range of data lengths and for frequency separations down to about one tenth of the rayleigh resolution limit further it is demonstrated that the same basic form of the approximation handles the related cases of a frequency estimation of a single real sinusoid in real noise and b frequency estimation for a closely spaced pair of real sinusoids in real noise

an exact recursion for the composite nearest neighbor degeneracy for a 2 n lattice space a set theoretic argument is utilized to develop a recursion relation that yields exactly the composite nearest neighbor degeneracy for simple indistinguishable particles distributed on a 2 n lattice space the associated generating functions as well as the expectation of the resulting statistics are also treated

robust maximum likelihood estimation of multivariable dynamic systems this paper examines the problem of estimating linear time invariant state space system models in particular it addresses the parametrization and numerical robustness concerns that arise in the multivariable case these difficulties are well recognised in the literature resulting for example in extensive study of subspace based techniques as well as recent interest in data driven local co ordinate approaches to gradient search solutions the paper here proposes a different strategy that employs the expectation maximisation em technique the consequence is an algorithm that is iterative with associated likelihood values that are locally convergent to stationary points of the gaussian likelihood function furthermore theoretical and empirical evidence presented here establishes additional attractive properties such as numerical robustness avoidance of difficult parametrization choices the ability to naturally and easily estimate non zero initial conditions and moderate computational cost moreover since the methods here are maximum likelihood based they have associated known and asymptotically optimal statistical properties

an eigenstructure method for estimating doa and sensor gain phase errors in this paper we consider the problem of direction of arrival doa estimation in the presence of sensor gain phase errors under some mild assumptions we propose a new doa estimation method based on the eigendecomposition of a covariance matrix which is constructed by the dot product of the array output vector and its conjugate by combining the new doa estimation with the conventional gain phase error estimation a method is proposed to simultaneously estimate the doa and gain phase errors without joint iteration theoretical analysis shows that the proposed method performs independently of phase errors and thus behaves well regardless of phase errors however the resolution capability of the proposed method is lower than that of the method in a j weiss and b friedlander eigenstructure methods for direction finding with sensor gain and phase uncertainties circuits systems signal process vol 9 no 3 pp 271 300 1990 named as the wf method in order to improve the resolution capability and maintain phase error independence a combined strategy is developed using the proposed and wf methods the advantage of the proposed methods is that they are independent of phase errors leading to the cancellation of phase error calibration during the operation life of an array moreover the proposed methods avoid the problem of suboptimal convergence which occurs in the wf method the drawbacks of the proposed methods are their high computational complexity and their requirement for the condition that at least two signals are spatially far from each other and they are not applicable to a linear array simulation results verify the effectiveness of the proposed methods

on convergence properties of the em algorithm for gaussian mixtures we build up the mathematical connection between the expectation maximization em algorithm and gradient based approaches for maximum likelihood learning of finite gaussian mixtures we show that the em step in parameter space is obtained from the gradient via a projection matrix p and we provide an explicit expression for the matrix we then analyze the convergence of em in terms of special properties of p and provide new results analyzing the effect that p has on the likelihood surface based on these mathematical results we present a comparative discussion of the advantages and disadvantages of em and other algorithms for the learning of gaussian mixture models

probits of mixtures the tolerances of individuals insects parasites in a population have a frequency or probability distribution called a tolerance distribution many tolerance distributions in bioassay studies can be the result of a rather heterogeneous population of individuals and can often be modelled as a mixture of a number of standard unimodal distributions a probit analysis can be generalized to the case where the tolerance distribution is a mixture of location and scale parameter distributions in this article the existence and determination of the maximum likelihood estimates are investigated an expectation maximization em algorithm for probits of mixtures is developed and it is shown that by application of the em algorithm the problem of probits of mixtures can be separated into a series of probits of individual component tolerance distributions

on the recursive computation of interpolators with nonrectangular masks an algorithm is presented for the recursive computation of finite order interpolators and predictors for scalar random processes on multidimensional parameter sets the algorithm is able to achieve computational savings even for interpolation filters with nonrectangularly shaped support because it avoids direct exploitation of toeplitz structure in the normal equations by using the displacement invariance structure of the interpolation filter and the low displacement rank properties of the correlation matrix the paper presents the method for step by step growth of the interpolation support and shows that an interpolation filter can be constructed from the interpolator of the previous step along with certain interpolators corresponding to the boundary points of the filter support in the previous step when restricted to rectangularly shaped masks the algorithm has the same order of complexity as previous algorithms for solving toeplitz block toeplitz systems

deterministic stochastic wavelet decomposition for recovery of signal from noisy data in a series of recent articles on nonparametric regression donoho and johnstone developed waveletshrinkage methods for recovering unknown piecewise smooth deterministic signals from noisy data wavelet shrinkage based on the bayesian approach involves specifying a prior distribution on the wavelet coefficients which is usually assumed to have a distribution with zero mean there is no a priori reason why all prior means should be 0 indeed one can imagine certain types of signals in which this is not a good choice of model in this article we take an empirical bayes approach in which we propose an estimator for the prior mean that is plugged into the bayesian shrinkage formulas another way we are more general than previous work is that we assume that the underlying signal is composed of a piecewise smooth deterministic part plus a zero mean stochastic part that is the signal may contain a reasonably large number of nonzero wavelet coefficients our goal is to predict this signal from noisy data

roughness penalties on finite domains a class of penalty functions for use in estimation and image regularization is proposed these penalty functions are defined for vectors whose indexes are locations in a finite lattice as the discrepancy between the vector and a shifted version of itself after motivating this class of penalty functions their relationship to markov random field priors is explored one of the penalty functions proposed a divergence roughness penalty is shown to be a discretization of a penalty proposed by good and gaskins 1971 for use in density estimation one potential use in estimation problems is explored an iterative algorithm that takes advantage of induced neighborhood structures is proposed and convergence of the algorithm is proven under specified conditions examples in emission tomographic imaging and radar imaging are given

sensor array calibration using a maximum likelihood approach high resolution array processing algorithms for source localization are known to be sensitive to errors in the model for the sensor array spatial response in particular unknown gain phase and mutual coupling as well as errors in the sensor positions can seriously degrade the performances of array processing algorithms this paper describes a calibration algorithm that estimates the calibration matrix consisting of the unknown gain phase and mutual coupling coefficients as well as the sensor positions using a set of calibration sources in known locations the estimation of the various parameters is based on a maximum likelihood approach cramer rao lower bound crb expressions for the sensor positions and the calibration matrix parameters are also derived numerical results are shown to illustrate the potential usefulness of the proposed calibration algorithm toward better accuracy and resolution in parametric array processing algorithms

segmentation of brain parenchyma and cerebrospinal fluid in multispectral magnetic resonance images presents a new method to segment brain parenchyma and cerebrospinal fluid spaces automatically in routine axial spin echo multispectral mr images the algorithm simultaneously incorporates information about anatomical boundaries shape and tissue signature grey scale using a priori knowledge the head and brain are divided into four regions and seven different tissue types each tissue type c is modeled by a multivariate gaussian distribution n spl mu sub c spl sigma sub c each region is associated with a finite mixture density corresponding to its constituent tissue types initial estimates of tissue parameters spl mu sub c spl sigma sub c sub c 1 7 are obtained from k means clustering of a single slice used for training the first algorithmic step uses the em algorithm for adjusting the initial tissue parameter estimates to the mr data of new patients the second step uses a recently developed model of dynamic contours to detect three simply closed nonintersecting curves in the plane constituting the arachnoid dura mater boundary of the brain the border between the subarachnoid space and brain parenchyma and the inner border of the parenchyma toward the lateral ventricles the model which is formulated by energy functions in a bayesian framework incorporates a priori knowledge smoothness constraints and updated tissue type parameters satisfactory maximum a posteriori probability estimates of the closed contour curves defined by the model were found using simulated annealing

nonparametric em algorithms for estimating prior distributions consider a sequence of repetitive experiments e 1 e 2 e n each e i is represented by a pair of random vectors y i q i such that y i is observed q i is not observed and the conditional density of y i given q i is known the q i s are independent and identically distributed with common but unknown distribution function f q the problem is to estimate f given the data y 1 y 2 y n two nonparametric maximum likelihood methods based on the em algorithm are presented motivation for this problem comes from the analysis of the distribution of parameters in pharmacokinetic models based on population data

modeling 2 d ar processes with various regions of support we show that there exists a causal 2 d linear process in the nonsymmetric half plane having the same autocorrelations as a noncausal 2 d linear process in the whole plane this property is called the autocorrelation equivalence relation and can be used for practical fitting and modeling of 2 d processes some causal 2 d autoregressive ar models with various regions of support are considered such as half cross half diamond quarter plane square half square half hexagon half octagon and half circle considerations of parsimony in 2 d model fitting are then focused not only on the number of parameters in our model but also most importantly on the optimal shape of the region of support their 2 d yule walker equations are derived and a computationally efficient order recursive algorithm is proposed to solve them the autocorrelation equivalence relation and the order recursive algorithm are utilized to specify a noncausal 2 d ar process as well as its spectrum from a given realization of a random field

space alternating generalized expectation maximization algorithm the expectation maximization em method can facilitate maximizing likelihood functions that arise in statistical estimation problems in the classical em paradigm one iteratively maximizes the conditional log likelihood of a single unobservable complete data space rather than maximizing the intractable likelihood function for the measured or incomplete data em algorithms update all parameters simultaneously which has two drawbacks 1 slow convergence and 2 difficult maximization steps due to coupling when smoothness penalties are used the paper describes the space alternating generalized em sage method which updates the parameters sequentially by alternating between several small hidden data spaces defined by the algorithm designer the authors prove that the sequence of estimates monotonically increases the penalized likelihood objective derive asymptotic convergence rates and provide sufficient conditions for monotone convergence in norm two signal processing applications illustrate the method estimation of superimposed signals in gaussian noise and image reconstruction from poisson measurements in both applications the sage algorithms easily accommodate smoothness penalties and converge faster than the em algorithms

nonparametric estimation of the distribution of time to onset for specific diseases in survival sacrifice experiments this paper concerns the analysis of an animal survival sacrifice experiment designed to investigate the incidence of a particular disease of interest the disease is assumed to be irreversible and detectable only at death for example by a necropsy each observation can be of one of three types i death caused by the disease ii death from a competing cause such as sacrifice with the disease present or iii death with the disease absent a two dimensional em algorithm is proposed for the nonparametric maximum likelihood estimation of the distributions of the time to onset and of the time to death from the disease these can be compared with nonparametric estimators recently proposed by kodell shaw and johnson 1982 biometrics 38 43 58 and by dinse and lagakos 1982 biometrics 38 921 932 a slight modification of the algorithm permits the construction of likelihood based interval estimates of quantiles of the distributions some extensions and generalizations are indicated

direction of arrival estimation using a sparse representation of array covariance vectors a new direction of arrival doa estimation method is proposed based on a novel data model using the concept of a sparse representation of array covariance vectors sracv in which doa estimation is achieved by jointly finding the sparsest coefficients of the array covariance vectors in an overcomplete basis the proposed method not only has high resolution and the capability of estimating coherent signals based on an arbitrary array but also gives an explicit error suppression criterion that makes it statistically robust even in low signal to noise ratio snr cases simulation experiments are conducted to validate the effectiveness of the proposed method the performance is compared with several existing doa estimation methods and the cramer rao lower bound crlb

maximization of the marginal likelihood of grouped survival data summary grouped failure time data occur in studies where subjects are monitored periodically to determine whether failure has occurred in the intervening interval here the model under consideration is cox s 1972 1975 proportional hazards model but the commonly used method of partial likelihood needs modification with grouped data due to a potentially large number of ties this paper demonstrates how the monte carlo em algorithm wei tanner 1990 can be used on grouped data to find the maximum likelihood estimate from the marginal likelihood based on the incomplete ranks of the event times the methodology is exemplified with a data set with precise failure times after artificially introducing grouping

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

efficient inversion of toeplitz block toeplitz matrix an iterative algorithm for the inversion of a toeplitz block toeplitz matrix consisting of m m blocks of size p p is described the algorithm presented exploits the structure of the toeplitz block toeplitz matrix and outperforms akaike s algorithm by a factor of max 2 p m 2 the use of this algorithm for an iterative solution of a toeplitz block toeplitz set of linear equations is also presented

asymptotically optimal blind calibration of uniform linear sensor arrays for narrowband gaussian signals an asymptotically optimal blind calibration scheme of uniform linear arrays for narrowband gaussian signals is proposed rather than taking the direct maximum likelihood ml approach for joint estimation of all the unknown model parameters which leads to a multi dimensional optimization problem with no closed form solution we revisit paulraj and kailath s p k s classical approach in exploiting the special toeplitz structure of the observations covariance however we offer a substantial improvement over p k s ordinary least squares ls estimates by using asymptotic approximations in order to obtain simple non iterative quasi linear optimally weighted ls owls estimates of the sensors gains and phases offsets with asymptotically optimal weighting based only on the empirical covariance matrix of the measurements moreover we prove that our resulting estimates are also asymptotically optimal w r t the raw data and can therefore be deemed equivalent to the ml estimates mle which are otherwise obtained by joint ml estimation of all the unknown model parameters after deriving computationally convenient expressions of the respective cramer rao lower bounds we also show that our estimates offer improved performance when applied to non gaussian signals and or noise as quasi mle in a similar setting the optimal performance of our estimates is demonstrated in simulation experiments with a considerable improvement reaching an order of magnitude and more in the resulting mean squared errors w r t p k s ordinary ls estimates we also demonstrate the improved accuracy in a multiple sources directions of arrivals estimation task

an em algorithm for fitting two level structural equation models maximum likelihood is an important approach to analysis of two level structural equation models different algorithms for this purpose have been available in the literature in this paper we present a new formulation of two level structural equation models and develop an em algorithm for fitting this formulation this new formulation covers a variety of two level structural equation models as a result the proposed em algorithm is widely applicable in practice a practical example illustrates the performance of the em algorithm and the maximum likelihood statistic

on the analysis of accumulation curves summary identifying and counting the total number of biological species observed when plotted against a measure of the effort used to record them gives rise to a species accumulation curve we investigate estimation of the total number of species and other relevant properties of accumulation by elaborating on the multinomial model nakamura and peraza 1998 journal of agricultural biological and environmental statistics 3 17 36 that includes specification of a beta density for recording probabilities we consider a unified description including complete and incomplete aggregated curves a more general scheme for recording and a bayesian framework to allow for inclusion of the biologist s knowledge with regard to typical recording probabilities and the total number of species the beta distribution is used as a prior but recording probabilities are not restrained to be beta distributed the methods yield either closed analytical expressions for inference or relatively simple numerical procedures predictive distributions of future recordings that would eventually lead to an optimal decision theoretical rule for stopping collection effort may be easily calculated a case study regarding species of bats is considered including some guidelines for elicitation of an informative prior

extensions of estimation methods using the em algorithm abstract the em algorithm described by dempster laid and rubin 1977 is reviewed with the purpose of clarifying several misconceptions in the statistical and econometric literature the clarifications lead to several applications of the algorithm to models that have appeared to be less tractable the relationship between the em algorithm and the method of scoring is also explained providing estimators of the score and the information from the em algorithm the em algorithm is extended to missing data problems and an estimation method based on simulations

network inference from co occurrences the discovery of networks is a fundamental problem arising in numerous fields of science and technology including communication systems biology sociology and neuroscience unfortunately it is often difficult or impossible to obtain data that directly reveal network structure and so one must infer a network from incomplete data this paper considers inferring network structure from co occurrence data observations that identify which network components e g switches routers genes carry each transmission but do not indicate the order in which they handle the transmission without order information the number of networks that are consistent with the data grows exponentially with the size of the network i e the number of nodes yet the basic engineering evolutionary principles underlying most networks strongly suggest that not all data consistent networks are equally likely in particular nodes that co occur in many observations are probably closely connected with this in mind we model the co occurrence observations as independent realizations of a random walk on the network subjected to a random permutation to account for the lack of order information treating permutations as missing data we derive an expectation maximization em algorithm for estimating the random walk parameters the model and em algorithm significantly simplify the problem but the computational complexity of the reconstruction process does grow exponentially in the length of each transmission path for networks with long paths the exact e step may be computationally intractable we propose a polynomial time monte carlo em algorithm based on importance sampling and derive conditions that ensure convergence of the algorithm with high probability simulations and experiments with internet measurements demonstrate the promise of this approach

minimum mean square distance estimation of a subspace we consider the problem of subspace estimation in a bayesian setting since we are operating in the grassmann manifold the usual approach which consists of minimizing the mean square error mse between the true subspace u and its estimate u may not be adequate as the mse is not the natural metric in the grassmann manifold gn p i e the set of p dimensional subspaces in rn as an alternative we propose to carry out subspace estimation by minimizing the mean square distance between u and its estimate where the considered distance is a natural metric in the grassmann manifold viz the distance between the projection matrices we show that the resulting estimator is no longer the posterior mean of u but entails computing the principal eigenvectors of the posterior mean of uut derivation of the minimum mean square distance mmsd estimator is carried out in a few illustrative examples including a linear gaussian model for the data and bingham or von mises fisher prior distributions for u in all scenarios posterior distributions are derived and the mmsd estimator is obtained either analytically or implemented via a markov chain monte carlo simulation method the method is shown to provide accurate estimates even when the number of samples is lower than the dimension of u an application to hyperspectral imagery is anally investigated

adaptive interpolation of discrete time signals that can be modeled as autoregressive processes this paper presents an adaptive algorithm for the restoration of lost sample values in discrete time signals that can locally be described by means of autoregressive processes the only restrictions are that the positions of the unknown samples should be known and that they should be embedded in a sufficiently large neighborhood of known samples the estimates of the unknown samples are obtained by minimizing the sum of squares of the residual errors that involve estimates of the autoregressive parameters a statistical analysis shows that for a burst of lost samples the expected quadratic interpolation error per sample converges to the signal variance when the burst length tends to infinity the method is in fact the first step of an iterative algorithm in which in each iteration step the current estimates of the missing samples are used to compute the new estimates furthermore the feasibility of implementation in hardware for real time use is established the method has been tested on artificially generated auto regressive processes as well as on digitized music and speech signals

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

inference based on the em algorithm for the competing risks model with masked causes of failure summary in this paper we propose inference methods based on the em algorithm for estimating the parameters of a weakly parameterised competing risks model with masked causes of failure and second stage data with a carefully chosen definition of complete data the maximum likelihood estimation of the cause specific hazard functions and of the masking probabilities is performed via an em algorithm both the e and m steps can be solved in closed form under the full model and under some restricted models of interest we illustrate the flexibility of the method by showing how grouped data and tests of common hypotheses in the literature on missing cause of death can be handled the method is applied to a real dataset and the asymptotic and robustness properties of the estimators are investigated through simulation

properties of mean shift we study properties of the mean shift ms type algorithms for estimating modes of probability density functions pdfs via regarding these algorithms as gradient ascent on estimated pdfs with adaptive step sizes we rigorously prove convergence of mode estimate sequences generated by the ms type algorithms under the assumption that an analytic kernel function is used moreover our analysis on the ms function finds several new properties of mode estimate sequences and corresponding density estimate sequences including the result that in the ms type algorithm using a gaussian kernel the density estimate monotonically increases between two consecutive mode estimates this implies that in the one dimensional case the mode estimate sequence monotonically converges to the stationary point nearest to an initial point without jumping over any stationary point

performance study of conditional and unconditional direction of arrival estimation a numerical and analytical study of conditional and unconditional direction of arrival doa estimation is presented explicit expressions for the unconditional cramer rao bounds on the doa estimation accuracy and the covariance matrix of the conditional maximum likelihood method are given it is shown that many doa estimation methods have the same asymptotic statistical properties under conditional and unconditional models the situation of two narrowband plane signals impinging on a uniformly spaced linear array is discussed

estimating the number of species a review how many kinds are there suppose that a population is partitioned into c classes in many situations interest focuses not on estimation of the relative sizes of the classes but on estimation of c itself for example biologists and ecologists may be interested in estimating the number of species in a population of plants or animals numismatists may be concemed with estimating the number of dies used to produce an ancient coin issue and linguists may be interested in estimating the size of an author s vocabulary in this article we review the problem of statistical estimation of c many approaches have been proposed some purely data analytic and others based in sampling theory in the latter case numerous variations have been considered the population may be finite or infinite if finite samples may be taken with replacement multinomial sampling or without replacement hypergeometric sampling or by bernoulli sampling if infinite sampling may be multinomial or bernoulli or the sample may be th

cs decomposition based bayesian subspace estimation in numerous applications it is required to estimate the principal subspace of the data possibly from a very limited number of samples additionally it often occurs that some rough knowledge about this subspace is available and could be used to improve subspace estimation accuracy in this case this is the problem we address herein and in order to solve it a bayesian approach is proposed the main idea consists of using the cs decomposition of the semi orthogonal matrix whose columns span the subspace of interest this parametrization is intuitively appealing and allows for non informative prior distributions of the matrices involved in the cs decomposition and very mild assumptions about the angles between the actual subspace and the prior subspace the posterior distributions are derived and a gibbs sampling scheme is presented to obtain the minimum mean square distance estimator of the subspace of interest numerical simulations and an application to real hyperspectral data assess the validity and the performances of the estimator

hierarchical color correction for camera cell phone images in this paper we propose a hierarchical color correction algorithm for enhancing the color of digital images obtained from low quality digital image capture devices such as cell phone cameras the proposed method is based on a multilayer hierarchical stochastic framework whose parameters are learned in an offline training procedure using the well known expectation maximization em algorithm this hierarchical framework functions by first making soft assignments of images into defect classes and then processing the images in each defect class with an optimized algorithm the hierarchical color correction is performed in three stages in the first stage global color attributes of the low quality input image are used in a gaussian mixture model gmm framework to perform a soft classification of the image into m predefined global image classes in the second stage the input image is processed with a nonlinear color correction algorithm that is designed for each of the m global classes this color correction algorithm which we refer to as resolution synthesis color correction rscc applies a spatially varying color correction determined by the local color attributes of the input image in the third stage the outputs of the rscc predictors are combined using the global classification weights to yield the color corrected output image we compare the performance of the proposed method to other commercial color correction algorithms on cell phone camera images obtained from different sources both subjective and objective measures of quality indicate that the new color correction algorithm improves quality over the existing methods

maximum likelihood estimation of delta domain model parameters from noisy output signals fast sampling is desirable to describe signal transmission through wide bandwidth systems the delta operator provides an ideal discrete time modeling description for such fast sampled systems however the estimation of delta domain model parameters is usually biased by directly applying the delta transformations to a sampled signal corrupted by additive measurement noise this problem is solved here by expectation maximization where the delta transformations of the true signal are estimated and then used to obtain the model parameters the method is demonstrated on a numerical example to improve on the accuracy of using a shift operator approach when the sample rate is fast

accelerated iterative reconstruction for positron emission tomography based on the em algorithm for maximum likelihood estimation the em method that was originally developed for maximum likelihood estimation in the context of mathematical statistics may be applied to a stochastic model of positron emission tomography pet the result is an iterative algorithm for image reconstruction that is finding increasing use in pet due to its attractive theoretical and practical properties its major disadvantage is the large amount of computation that is often required due to the algorithm s slow rate of convergence this paper presents an accelerated form of the em algorithm for pet in which the changes to the image as calculated by the standard algorithm are multiplied at each iteration by an overrelaxation parameter the accelerated algorithm retains two of the important practical properties of the standard algorithm namely the selfnormalization and nonnegativity of the reconstructed images experimental results are presented using measured data obtained from a hexagonal detector system for pet the likelihood function and the norm of the data residual were monitored during the iterative process according to both of these measures the images reconstructed at iterations 7 and 11 of the accelerated algorithm are similar to those at iterations 15 and 30 of the standard algorithm for two different sets of data important theoretical properties remain to be investigated namely the convergence of the accelerated algorithm and its performance as a maximum likelihood estimator

convergence theorems for generalized alternating minimization procedures the em algorithm is widely used to develop iterative parameter estimation procedures for statistical models in cases where these procedures strictly follow the em formulation the convergence properties of the estimation procedures are well understood in some instances there are practical reasons to develop procedures that do not strictly fall within the em framework we study em variants in which the e step is not performed exactly either to obtain improved rates of convergence or due to approximations needed to compute statistics under a model family over which e steps cannot be realized since these variants are not em procedures the standard g em convergence results do not apply to them we present an information geometric framework for describing such algorithms and analyzing their convergence properties we apply this framework to analyze the convergence properties of incremental em and variational em for incremental em we discuss conditions under these algorithms converge in likelihood for variational em we show how the e step approximation prevents convergence to local maxima in likelihood

nonparametric hierarchical bayes via sequential imputations we consider the empirical bayes estimation of a distribution using binary data via the dirichlet process let d denote a dirichlet process with being a finite measure on 0 1 instead of having direct samples from an unknown random distribution f from d we assume that only indirect binomial data are observable this paper presents a new interpretation of lo s formula and thereby relates the predictive density of the observations based on a dirichlet process model to likelihoods of much simpler models as a consequence the log likelihood surface as well as the maximum likelihood estimate of c 0 1 is found when the shape of is assumed known together with a formula for the fisher information evaluated at the estimate the sequential imputation method of kong liu and wong is recommended for overcoming computational difficulties commonly encountered in this area the related approximation formulas are provided an analysis of the tack data of beckett and diaconis which motivated this study is supplemented to illustrate our methods

carrier phase and frequency estimation for pilot symbol assisted transmission bounds and algorithms in this paper we consider the cramer rao lower bound crb for the joint estimation of the carrier phase and the frequency offset from a noisy linearly modulated burst signal containing random data symbols dss as well as known pilot symbols pss we point out that the crb depends on the location of the pss in the burst the number of pss the number of dss the signal to noise ratio snr and the data modulation scheme distributing the pss symmetrically about the center of the burst and estimating the carrier phase in the center of the burst interval decouples the frequency and phase estimation making the crb for phase estimation independent of the specific location of the pss at low and moderate snr the crbs for both phase and frequency estimation decrease as the fraction of the pss in the burst increases in addition the crb for frequency estimation decreases as the pss are separated with more dss numerical evaluation of the crb indicates that the carrier phase and frequency of a hybrid burst i e containing pss and dss can be estimated more accurately when exploiting both the presence of the dss and the a priori knowledge about the pss instead of using only the knowledge about the pss and ignoring the dss or considering all the received symbols pss and dss as unknown and ignoring the knowledge about the pss comparison of the crb with the performance of existing carrier synchronizers shows that the iterative soft decision directed sdd estimator with data aided da initialization performs very closely to the crb and provides a large improvement over the classical non data aided nda estimator at lower snr