the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

bayesian approach with hidden markov modeling and mean field approximation for hyperspectral data analysis the main problems in hyperspectral image analysis are spectral classification segmentation and data reduction in this paper we propose a bayesian estimation approach which gives a joint solution for these problems the problem is modeled as a blind sources separation bss the data are m hyperspectral images and the sources are k m images which are composed of compact homogeneous regions and have mutually disjoint supports the set of all these regions cover the total surface of the observed scene to insure these properties we propose a hierarchical markov model for the sources with a common hidden classification field which is modeled via a potts markov field the joint bayesian estimation of the hidden variable sources and the mixing matrix of the bss gives a solution for all three problems spectra classification segmentation and data reduction of hyperspectral images the mean field approximation mfa algorithm for the posterior laws is proposed for the effective bayesian computation finally some results of the application of the proposed methods on simulated and real data are given to illustrate the performance of the proposed method compared to other classical methods such as pca and ica

global hopf bifurcation of two parameter flows the behavior of center indices as introduced by j mallet paret j yorke is analyzed for two parameter flows the integer sum of center indices along a one dimensional curve in parameter space is called the h index a nonzero h index implies global hopf bifurcation the index h is not a homotopy invariant this fact is due to the occurrence of stationary points with an algebraically double eigenvalue zero which we call b points to each b point we assign an integer b index such that the h index relates to the b indices by a formula such as occurs in the calculus of residues

homoclinic doubling cascades cascades of period doubling bifurcations have attracted much interest from researchers of dynamical systems in the past two decades as they are one of the routes to onset of chaos in this paper we consider routes to onset of chaos involving homoclinic doubling bifurcations we show the existence of cascades of homoclinic doubling bifurcations which occur persistently in two parameter families of vector fields on 3 the cascades are found in an unfolding of a codimension three homoclinic bifurcation which occur an orbit flip at resonant eigenvalues we develop a continuation theory for homoclinic orbits in order to follow homoclinic orbits through infinitely many homoclinic doubling bifurcations

semi supervised linear spectral unmixing using a hierarchical bayesian model for hyperspectral imagery this paper proposes a hierarchical bayesian model that can be used for semi supervised hyperspectral image unmixing the model assumes that the pixel reflectances result from linear combinations of pure component spectra contaminated by an additive gaussian noise the abundance parameters appearing in this model satisfy positivity and additivity constraints these constraints are naturally expressed in a bayesian context by using appropriate abundance prior distributions the posterior distributions of the unknown model parameters are then derived a gibbs sampler allows one to draw samples distributed according to the posteriors of interest and to estimate the unknown abundances an extension of the algorithm is finally studied for mixtures with unknown numbers of spectral components belonging to a know library the performance of the different unmixing strategies is evaluated via simulations conducted on synthetic and real data

toward a sparse bayesian markov random field approach to hyperspectral unmixing and classification recent work has shown that existing powerful bayesian hyperspectral unmixing algorithms can be significantly improved by incorporating the inherent local spatial correlations between pixel class labels via the use of markov random fields we here propose a new bayesian approach to joint hyperspectral unmixing and image classification such that the previous assumption of stochastic abundance vectors is relaxed to a formulation whereby a common abundance vector is assumed for pixels in each class this allows us to avoid stochastic reparameterizations and instead we propose a symmetric dirichlet distributionmodel with adjustable parameters for the common abundance vector of each class inference over the proposed model is achieved via a hybrid gibbs sampler and in particular simulated annealing is introduced for the label estimation in order to avoid the local trap problem experiments on a synthetic image and a popular publicly available real data set indicate the proposed model is faster than and outperforms the existing approach quantitatively and qualitatively moreover for appropriate choices of the dirichlet parameter it is shown that the proposed approach has the capability to induce sparsity in the inferred abundance vectors it is demonstrated that this offers increased robustness in cases where the preprocessing endmember extraction algorithms overestimate the number of active endmembers present in a given scene

brief papers independent tracking and regulation adaptive control with forgetting factor this paper considers a discrete time adaptive control algorithm with a forgetting factor applicable to minimum phase plants the tracking and regulation objectives are independently specified the relevance of the eigenvalues of the gain matrix f k used in the updating equation for the adaptive parameters gq k is shown it is proved that if the maximum eigenvalue of the inverse of the gain matrix f k has an upper bound and a non zero lower bound then the global convergence of the control algorithm is insured the result of the design is a simple control scheme using a linear constant feedforward controller and a nonlinear feedback controller the performance of the control structure in tracking and regulation are evaluated by simulations

hyperspectral image unmixing using a multiresolution sticky hdp this paper is concerned with joint bayesian endmember extraction and linear unmixing of hyperspectral images using a spatial prior on the abundance vectors we propose a generative model for hyperspectral images in which the abundances are sampled from a dirichlet distribution dd mixture model whose parameters depend on a latent label process the label process is then used to enforces a spatial prior which encourages adjacent pixels to have the same label a gibbs sampling framework is used to generate samples from the posterior distributions of the abundances and the parameters of the dd mixture model the spatial prior that is used is a tree structured sticky hierarchical dirichlet process shdp and when used to determine the posterior endmember and abundance distributions results in a new unmixing algorithm called spatially constrained unmixing scu the directed markov model facilitates the use of scale recursive estimation algorithms and is therefore more computationally efficient as compared to standard markov random field mrf models furthermore the proposed scu algorithm estimates the number of regions in the image in an unsupervised fashion the effectiveness of the proposed scu algorithm is illustrated using synthetic and real data

least squares parameter estimation of continuous time arx models from discrete time data when modeling a system from discrete time data a continuous time parameterization is desirable in some situations in a direct estimation approach the derivatives are approximated by appropriate differences for an arx model this lead to a linear regression the well known least squares method would then be very desirable since it can have good numerical properties and low computational burden in particular for fast or nonuniform sampling it is examined under what conditions a least squares fit for this linear regression will give adequate results for an arx model the choice of derivative approximation is crucial for this approach to be useful standard approximations like euler backward or euler forward cannot be used directly the precise conditions on the derivative approximation are derived and analyzed it is shown that if the highest order derivative is selected with care a least squares estimate will be accurate the theoretical analysis is complemented by some numerical examples which provide further insight into the choice of derivative approximation

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

a convex analysis based minimum volume enclosing simplex algorithm for hyperspectral unmixing hyperspectral unmixing aims at identifying the hidden spectral signatures or endmembers and their corresponding proportions or abundances from an observed hyperspectral scene many existing hyperspectral unmixing algorithms were developed under a commonly used assumption that pure pixels exist however the pure pixel assumption may be seriously violated for highly mixed data based on intuitive grounds craig reported an unmixing criterion without requiring the pure pixel assumption which estimates the endmembers by vertices of a minimum volume simplex enclosing all the observed pixels in this paper we incorporate convex analysis and craig s criterion to develop a minimum volume enclosing simplex mves formulation for hyperspectral unmixing a cyclic minimization algorithm for approximating the mves problem is developed using linear programs lps which can be practically implemented by readily available lp solvers we also provide a non heuristic guarantee of our mves problem formulation where the existence of pure pixels is proved to be a sufficient condition for mves to perfectly identify the true endmembers some monte carlo simulations and real data experiments are presented to demonstrate the efficacy of the proposed mves algorithm over several existing hyperspectral unmixing methods

reconstruction of linear and non linear continuous time system models from input output data using the kernel invariance algorithm abstract a new kernel invariance algorithm kia is introduced to determine both the significant model terms and estimate the unknown parameters in non linear continuous time differential equation models of unknown systems

unsupervised post nonlinear unmixing of hyperspectral images using a hamiltonian monte carlo algorithm this paper presents a nonlinear mixing model for hyperspectral image unmixing the proposed model assumes that the pixel reflectances are post nonlinear functions of unknown pure spectral components contaminated by an additive white gaussian noise these nonlinear functions are approximated using second order polynomials leading to a polynomial post nonlinear mixing model a bayesian algorithm is proposed to estimate the parameters involved in the model yielding an unsupervised nonlinear unmixing algorithm due to the large number of parameters to be estimated an efficient hamiltonian monte carlo algorithm is investigated the classical leapfrog steps of this algorithm are modified to handle the parameter constraints the performance of the unmixing strategy including convergence and parameter tuning is first evaluated on synthetic data simulations conducted with real data finally show the accuracy of the proposed unmixing strategy for the analysis of hyperspectral images

period doubling cascades of attractors a prerequisite for horseshoes this paper shows that if a horseshoe is created in a natural manner as a parameter is varied then the process of creation involves the appearance of attracting periodic orbits of all periods furthermore each of these orbits will period double repeatedly with those periods going to infinity

joint bayesian endmember extraction and linear unmixing for hyperspectral imagery this paper studies a fully bayesian algorithm for endmember extraction and abundance estimation for hyperspectral imagery each pixel of the hyperspectral image is decomposed as a linear combination of pure endmember spectra following the linear mixing model the estimation of the unknown endmember spectra is conducted in a unified manner by generating the posterior distribution of abundances and endmember parameters under a hierarchical bayesian model this model assumes conjugate prior distributions for these parameters accounts for nonnegativity and full additivity constraints and exploits the fact that the endmember proportions lie on a lower dimensional simplex a gibbs sampler is proposed to overcome the complexity of evaluating the resulting posterior distribution this sampler generates samples distributed according to the posterior distribution and estimates the unknown parameters using these generated samples the accuracy of the joint bayesian estimator is illustrated by simulations conducted on synthetic and real aviris images

alternating subspace spanning resampling to accelerate markov chain monte carlo simulation this article provides a simple method to accelerate markov chain monte carlo sampling algorithms such as the data augmentation algorithm and the gibbs sampler via alternating subspace spanning resampling assr the assr algorithm often shares the simplicity of its parent sampler but has dramatically improved efficiency the methodology is illustrated with bayesian estimation for analysis of censored data from fractionated experiments the relationships between assr and existing methods are also discussed

seeking efficient data augmentation schemes via conditional and marginal augmentation data augmentation sometimes known as the method of auxiliary variables is a powerful tool for constructing optimisation and simulation algorithms in the context of optimisation meng van dyk 1997 1998 reported several successes of the working parameter approach for constructing efficient data augmentation schemes for fast and simple em type algorithms this paper investigates the use of working parameters in the context of markov chain monte carlo in particular in the context of tanner wong s 1987 data augmentation algorithm via a theoretical study of two working parameter approaches the conditional augmentation approach and the marginal augmentation approach posterior sampling under the univariate t model is used as a running example which particularly illustrates how the marginal augmentation approach obtains a fast mixing positive recurrent markov chain by first constructing a nonpositive recurrent markov chain in a larger space

global families of limit cycles of planar analytic systems the global behavior of any one parameter family of limit cycles of a planar analytic system x f x a depending on a parameter a e r is determined it is shown that any one parameter family of limit cycles belongs to a maximal one parameter family which is either open or cyclic if the family is open then it terminates as the parameter or the orbits become unbounded or it terminates at a critical point or on a compound separatrix cycle of the system this implies that the periods in a one parameter family of limit cycles can become unbounded only if the orbits become unbounded or if they approach a degenerate critical point or compound separatrix cycle of the system this is a more specific result for planar analytic systems than wintner s principle of natural termination for n dimensional systems where the periods can become unbounded in strange ways this work generalizes duff s results for one parameter families of limit cycles generated by a one parameter family of rotated vector fields in particular it is shown that the behavior at a nonsingular multiple limit cycle of any one parameter family of limit cycles is exactly the same as the behavior at a multiple limit cycle of a one parameter family of liniit cycles generated by a one parameter family of rotated vector fields

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

bias compensating least squares method for identification of continuous time systems from sampled data abstract least squares ls techniques are applied to the identification problem of continuous time systems from sampled data of input output measurements the linear integral filter which solves the intial condition problem is employed for handling time derivatives the asymptotic bias in the ls estimator is derived and is compensated in a modified ls algorithm called a bias compensating ls method to obtain consistent estimates of the unknown parameters when it is unknown the variance of the noise can be estimated recursively together with the system parameters the bias compensating ls method not only has all merits of the ls method but also possesses good consistency properties it is applicable to both time invariant and time varying continuous time systems in the time varying case an adaptive ls algorithm with a variable forgetting factor is chosen to make the estimates track the changing parameters quickly and the bias is compensated as is done in the time invariant case representative nu

a new bias compensating ls method for continuous system identification in the presence of coloured noise a new bias compensating least squares ls method is presented for the parameter estimation of linear single input single output siso continuous time systems a discrete time model obtained by using the linear integral filter is augmented by introducing a pre filter on the input and then the parameters of the augmented model are estimated by the conventional ls method the distinct characteristic roots of the pre filter are used to estimate the bias in the ls estimate the pre filter should be chosen so that its frequency bandwidth is wider than those of the system and the input signals since the new method requires minimal information on the noise characteristics it is easily applicable to the case of coloured noise

the collapsed gibbs sampler in bayesian computations with applications to a gene regulation problem abstract this article describes a method of grouping and collapsing in using the gibbs sampler and proves from an operator theory viewpoint that the method is in general beneficial the norms of the forward operators associated with the corresponding nonreversible markov chains are used to discriminate among different simulation schemes when applied to bayesian missing data problems the idea of collapsing suggests skipping the steps of sampling parameter s values in standard data augmentation by doing this we obtain a predictive update version of the gibbs sampler a procedure of calculating the posterior odds ratio via the collapsed gibbs sampler when incomplete observations are involved is presented as an illustration of possible applications three examples along with a bayesian treatment for identifying common protein binding sites in unaligned dna sequences are provided

estimating the granularity coefficient of a potts markov random field within a markov chain monte carlo algorithm this paper addresses the problem of estimating the potts parameter jointly with the unknown parameters of a bayesian model within a markov chain monte carlo mcmc algorithm standard mcmc methods cannot be applied to this problem because performing inference on requires computing the intractable normalizing constant of the potts model in the proposed mcmc method the estimation of is conducted using a likelihood free metropolis hastings algorithm experimental results obtained for synthetic data show that estimating jointly with the other unknown parameters leads to estimation results that are as good as those obtained with the actual value of on the other hand choosing an incorrect value of can degrade estimation performance significantly to illustrate the interest of this method the proposed algorithm is successfully applied to real bidimensional sar and tridimensional ultrasound images

snakes oriented families of periodic orbits their sources sinks and continuation abstract poincare observed that for a differential equation x x depending on a parameter each periodic orbit generally lies in a connected family of orbits in x space in order to investigate certain large connected sets denoted q of orbits containing a given orbit we introduce two indices an orbit index and a center index defined at certain stationary points we show that genetically there are two types of hopf bifurcation those we call sources 1 and sinks 1 generically if the set q is bounded in x space and if there is an upper bound for periods of the orbits in q then q must have as many source hopf bifurcations as sink hopf bifurcations and each source is connected to a sink by an oriented one parameter snake of orbits a snake is a maximal path of orbits that contains no orbits whose orbit index is 0 see fig 1 1

families of periodic orbits virtual periods and global continuability abstract for a differential equation depending on a parameter there have been numerous investigations of the continuation of periodic orbits as the parameter is varied mallet paret and yorke investigated in generic situations how connected components of orbits must terminate here we extend the theory to the general case dropping genericity assumptions

extended self tuning algorithm this article concerns the practical implications of an extended self tuning property the extension represents a refinement of existing asymptotic self tuning properties in the sense that it allows the designer to make arbitrary assumptions concerning the disturbance dynamics the important practical consequence of this is that a simple pole assignment self tuner may now be designed which combines servo tracking and regulatory closed loop criteria in the pure regulation case a priori parametrization of the noise dynamics can be used to aid convergence of the self tuning algorithm in addition an implicit pole assignment regulator is easily formulated from the extended self tuner

estimation of continuous time ar process parameters from discrete time data the problem of estimating continuous time autoregressive process parameters from discrete time data is considered the basic approach used here is based on replacing the derivatives in the model by discrete time differences forming a linear regression and using the least squares method such a procedure is simple to apply computationally flexible and efficient and may have good numerical properties it is known however that all standard approximations of the highest order derivative such as repeated use of the delta operator gives a biased least squares estimate even as the sampling interval tends to zero some of our previous approaches to overcome this problem are reviewed then two new methods which avoid the shift in our previous results are presented one of them which is termed bias compensation is computationally very efficient finally the relationship of the above least squares approaches with an instrumental variable method is investigated comparative simulation results are also presented

on a least squares based algorithm for identification of stochastic linear systems a new form of bias eliminated least squares bels algorithm is developed to identify transfer function parameters of a linear time invariant system irrespective of noise dynamics unlike the bels estimator previously presented the main feature with the developed algorithm is that the transfer function parameters are consistently estimated in such a direct way that there is no need to prefilter observed data or to deal with a high order augmented system this greatly simplifies implementation of the bels based algorithms and reduces numerical efforts whereas a desirable estimation accuracy can still be achieved two simulation examples are presented that clearly illustrate the good performances of the developed algorithm including its superiority over one type of simple instrumental variable method

on the continuability of periodic orbits of parametrized three dimensional differential equations on considere un systeme autonome a n dimensions d equations differentielles ordinaires a 1 parametre x f x on suppose que f est definie et continue sur un sous ensemble ouvert o de r r n et que f x 0 sur o on etudie les orbites periodiques du systeme contenues dans o

spatial statistics and bayesian computation on wednesday may 6th 1992 professor b w silverman in the chair summary markov chain monte carlo mcmc algorithms such as the gibbs sampler have provided a bayesian inference machine in image analysis and in other areas of spatial statistics for several years founded on the pioneering ideas of ulf grenander more recently the observation that hyperparameters can be included as part of the updating schedule and the fact that almost any multivariate distribution is equivalently a markov random field has opened the way to the use of mcmc in general bayesian computation in this paper we trace the early development of mcmc in bayesian inference review some recent computational progress in statistical physics based on the introduction of auxiliary variables and discuss its current and future relevance in bayesian applications we briefly describe a simple mcmc implementation for the bayesian analysis of agricultural field experiments with which we have some practical experience

partially collapsed gibbs samplers theory and methods ever increasing computational power along with ever more sophisticated statistical computing techniques is making it possible to fit ever more complex statistical models among the more computationally intensive methods the gibbs sampler is popular because of its simplicity and power to effectively generate samples from a high dimensional probability distribution despite its simple implementation and description however the gibbs sampler is criticized for its sometimes slow convergence especially when it is used to fit highly structured complex models here we present partially collapsed gibbs sampling strategies that improve the convergence by capitalizing on a set of functionally incompatible conditional distributions such incompatibility generally is avoided in the construction of a gibbs sampler because the resulting convergence properties are not well understood we introduce three basic tools marginalization permutation and trimming that allow us to transform a gibbs sampler into a part