multiple window parallel adaptive boundary finding in computer vision the problem considered in this paper is the estimation of highly variable object boundaries in noisy images boundaries may be those of a tank in an ir image a spinal canal in a cat scan a cloud in a visible light image etc or they may be internal to an object such as the boundary between a spherical surface and a cylindrical surface in a manufactured object the focus of the paper is on parallel multiple window boundary estimation algorithms here the image field is parti tioned into an array of rectangular windows and boundary finders are run simultaneously within the windows the boundary segments found within the windows are then seamed together to obtain meaningful global boundaries the entire procedure is treated within a maximum likelihood estimation framework that we have developed for boundary finding although our multiple window estimation approach can be used with a number of local boundary finding algorithms we concen trate on one which is based on dynamic programming and will produce the true maximum likelihood boundary some theoretical considera tions for boundary model design and boundary finding runtime are covered included is the use of a low computational cost f test for test ing whether a window contains a boundary and an analytical treatment which shows that use of coarse pixels with a chi square test or an f test improves the probability of correctly recognizing whether a boundary is present in a window

central limit theorem for triangular arrays of non homogeneous markov chains in this paper we obtain the central limit theorem for triangular arrays of non homogeneous markov chains under a condition imposed to the maximal coefficient of correlation the proofs are based on martingale techniques and a sharp lower bound estimate for the variance of partial sums the results complement an important central limit theorem of dobrushin based on the contraction coefficient

on a positive semidefinite relaxation of the cut polytope abstract we study the convex set l n defined by l n z x x xij a positive semidefinite n n matrix xii 1 for all i we describe several geometric properties of l n in particular we show that l n has 2n 1 vertices which are its rank one matrices corresponding to all bipartitions of the set 1 2 n our main motivation for investigating the convex set l n comes from combinatorial optimization namely from approximating the max cut problem an important property of l n is that due to the positive semidefinite constraints one can optimize over it in polynomial time on the other hand l n still inherits the difficult structure of the underlying combinatorial problem in particular it is np hard to decide whether the optimum of the problem min tr cx x ln is reached at a vertex this result follows from the complete characterization of the matrices c of the form c bbt for some vector b for which the optimum of the above program is reached at a vertex

optimal monte carlo updating based on peskun s theorem it is shown that optimal transition matrices in markov chain monte carlo should have zero diagonal elements except for the diagonal element corresponding to the largest weight we will compare the statistical efficiency of this sampler to existing algorithms such as heat bath updating and the metropolis algorithm we provide numerical results for the potts model as an application in classical physics as an application in quantum physics we consider the spin 3 2xy model and the bose hubbard model which have been simulated by the directed loop algorithm in the stochastic series expansion framework

on random and systematic scan samplers we introduce a simple time homogeneous markov embedding of a class of time inhomogeneous markov chains widely used in the context of monte carlo sampling algorithms such as systematic scan metropolis within gibbs samplers this allows us to establish that systematic scan samplers involving two factors are always better than their random scan counterparts when asymptotic variance is the criterion of interest we also show that this embedding sheds some light on the result of maire et al 2014 and discuss the scenario involving more than two factors

nonlinear modeling and prediction by successive approximation using radial basis functions abstract in this paper a multilayer radial basis network is proposed for nonlinear system modeling and prediction it uses successive approximations first obtaining a number of coarse approximations and then optimally linearly combining the coarsely defined functions to achieve an accurate end result the more conventional approach of strict interpolation using radial basis functions does not necessarily yield high predictive accuracy is very time consuming for large data sets and can also give suboptimal accuracy due to numerical problems with nearly singular systems of equations the present approach considerably reduces computation time and can improve the predictive ability of radial basis networks while retaining good training accuracy numerical examples for chaotic time series and some classical real world time series are presented it is shown that the successive approximation radial basis network presented here is a useful tool for nonlinear modeling and prediction

conditional simulation of max stable processes summary since many environmental processes such as heat waves or precipitation are spatial in extent it is likely that a single extreme event affects several loca tions and the areal modelling of extremes is therefore essential if the spatial dependence of e xtremes has to be appropriately taken into account this paper proposes a framework for conditional simulations of max stable processes and give closed forms for brown resnick and schlather processes we test the method on simulated data and give an application to extreme rainfall around zurich and extreme temperature in switzerland results show that the proposed framework provides accurate conditional simulations and can handle real sized problems

the real positive definite completion problem for a simple cycle abstract we consider the question of whether a real partial positive definite matrix in which the specified off diagonal entries consist of a full n cycle has a positive definite completion this lies in contrast to the previously studied chordal case we give two solutions in one we describe about n 2 independent conditions on angles associated with a normalization of the data that are necessary and sufficient the second is more computational and allows presentation of all positive definite completions as well as answering the existence question

bayesian models for multiple local sequence alignment and gibbs sampling strategies abstract a wealth of data concerning life s basic molecules proteins and nucleic acids has emerged from the biotechnology revolution the human genome project has accelerated the growth of these data multiple observations of homologous protein or nucleic acid sequences from different organisms are often available but because mutations and sequence errors misalign these data multiple sequence alignment has become an essential and valuable tool for understanding structures and functions of these molecules a recently developed gibbs sampling algorithm has been applied with substantial advantage in this setting in this article we develop a full bayesian foundation for this algorithm and present extensions that permit relaxation of two important restrictions we also present a rank test for the assessment of the significance of multiple sequence alignment as an example we study the set of dinucleotide binding proteins and predict binding segments for dozens of its members

markov chain monte carlo methods for lattice gaussian sampling convergence analysis and enhancement sampling from lattice gaussian distribution has emerged as an important problem in coding decoding and cryptography in this paper the classic gibbs algorithm from markov chain monte carlo mcmc methods is demonstrated to be geometrically ergodic for lattice gaussian sampling which means that the markov chain arising from it converges exponentially fast to the stationary distribution meanwhile the exponential convergence rate of the markov chain is also derived through the spectral radius of the forward operator then a comprehensive analysis of the convergence rate is carried out and two sampling schemes are proposed to further enhance the convergence performance the first one referred to as a metropolis within gibbs mwg algorithm improves the convergence by refining the state space of the univariate sampling the second is a blocked strategy of the gibbs algorithm which performs sampling over multivariates at each markov move and is shown to yield a better convergence rate than the traditional univariate sampling in order to perform blocked sampling efficiently the gibbs klein gk algorithm is proposed which samples block by block using the kleins algorithm furthermore the validity of the gk algorithm is demonstrated by showing its ergodicity simulation results based on mimo detections are presented to confirm the convergence gain brought by the proposed gibbs sampling schemes

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

digital image restoration using spatial interaction models this paper is concerned with developing fast nonrecursive algorithms for the minimum mean squared error restoration of degraded images the degradation is assumed to be due to a space invariant periodic nonseparable known point spread function and additive white noise our basic approach is to represent the images by a class of spatial interaction models namely the simultaneous autoregressve models and the conditional markov models defined on toroidal lattices and develop minimum mean squared error restoration algorithms using these models the restoration algorithms are optimal if the parameters characterizing the interaction models are exactly known however in practice the parameters are estimated from the images by using spatial interaction models we develop restoration algorithms that do not require the availability of the original image or its prototype the specific structure of the underlying lattice enables the implementation of the filters using fast fourier transform fft computations several restoration examples are given

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

properties of a simple bilinear stochastic model estimation and predictability we analyze the properties of arguably the simplest bilinear stochastic multiplicative process proposed as a model of financial returns and of other complex systems combining both nonlinearity and multiplicative noise by construction it has no linear predictability zero two point correlation but a certain nonlinear predictability non zero three point correlation it can thus be considered as a paradigm for testing the existence of a possible nonlinear predictability in a given time series we present a rather exhaustive study of the process including its ability to produce fat tailed distributions from gaussian innovations the unstable characteristics of the inversion of the key nonlinear parameters and of the two initial conditions necessary for the implementation of a prediction scheme and an analysis of the associated super exponential sensitivity of the inversion of the innovations in the presence of a large impulse our study emphasizes the conditions under which a degree of predictability can be achieved and describe a number of different attempts which overall illuminates the properties of the process in conclusion notwithstanding its remarkable simplicity the bilinear stochastic process exhibits remarkably rich and complex behavior which makes it a serious candidate for the modeling of financial time series among others

bayesian estimation of switching arma models abstract switching arma processes have recently appeared as an efficient modelling to nonlinear time series models because they can represent multiple or heterogeneous dynamics through simple components the levels of dependence between the observations are double at a first level the parameters of the model are selected by a markovian procedure at a second level the next observation is generated according to a standard time series model when the model involves a moving average structure the complexity of the resulting likelihood function is such that simulation techniques like those proposed by shephard 1994 biometrika 81 115 131 and billio and monfort 1998 journal of statistical planning and inference 68 65 103 are necessary to derive an inference on the parameters of the model we propose in this paper a bayesian approach with a non informative prior distribution developed in mengersen and robert 1996 bayesian statistics 5 oxford university press oxford pp 255 276 and robert and titterington 1998 statistics and computing 8 2 145 158 in the setup of mixtures of distributions and hidden markov models respectively the computation of the bayes estimates relies on mcmc techniques which iteratively simulate missing states innovations and parameters until convergence the performances of the method are illustrated on several simulated examples this work also extends the papers by chib and greenberg 1994 journal of econometrics 64 183 206 and chib 1996 journal of econometrics 75 1 79 97 which deal with arma and hidden markov models respectively

brief paper parameter identification for nonlinear systems guaranteed confidence regions through lscr in this paper we consider the problem of constructing confidence regions for the parameters of nonlinear dynamical systems the proposed method uses higher order statistics and extends the lscr leave out sign dominant correlation regions algorithm for linear systems introduced in campi and weyer 2005 guaranteed non asymptotic confidence regions in system identification automatica 41 10 1751 1764 extended version available at the confidence regions contain the true parameter value with a guaranteed probability for any finite number of data points moreover the confidence regions shrink around the true parameter value as the number of data points increases the usefulness of the proposed approach is illustrated on some simple examples

current developments in time series modelling abstract in this paper a survey is given of some current developments in time series analysis after a brief discussion of linear models arma models and state space reprentations the discussion concentrates on a fairly general class of non linear models called state dependent models and the applications of these to the problem of identifying general forms of non linear structure the effectiveness of the estimation technique is studied by means of numerical examples on simulated data towards the end of the paper it is indicated how these ideas may be applied to the study of systems governed by non linear differential equations

lattice reduction aided gibbs algorithm for lattice gaussian sampling convergence enhancement and decoding optimization sampling from the lattice gaussian distribution has emerged as an important problem in coding decoding and cryptography in this paper lattice reduction technique is adopted to gibbs sampler for lattice gaussian sampling first with respect to lattice gaussian distribution we show the convergence rate of systematic scan gibbs sampling is characterized by the hirschfeld gebelein renyi maximal correlation among the multivariate of being sampled then lattice reduction aided gibbs algorithm is proposed to sample from an equivalent lattice gaussian distribution but with less correlated multivariate thus leading to a better markov mixing after that we extend the proposed lattice reduction aided gibbs sampling to lattice decoding where the choice of the standard deviation for the sampling is fully investigated a customized solution that suits for each specific decoding case by euclidean distance is given which results in a better tradeoff between markov mixing and sampler decoding based on it a startup mechanism is also proposed for gibbs sampler decoding where decoding complexity can be reduced without performance loss moreover the recycling gibbs sampling that exploits the potential of samples is also considered to improve the decoding performance in lattice decoding simulation results based on large scale uncoded multiple input multiple output detection are presented to confirm the performance gain and complexity reduction

on the foundations of relaxation labeling processes a large class of problems can be formulated in terms of the assignment of labels to objects frequently processes are needed which reduce ambiguity and noise and select the best label among several possible choices relaxation labeling processes are just such a class of algorithms they are based on the parallel use of local constraints between labels this paper develops a theory to characterize the goal of relaxation labeling the theory is founded on a definition of con sistency in labelings extending the notion of constraint satisfaction in certain restricted circumstances an explicit functional exists that can be maximized to guide the search for consistent labelings this functional is used to derive a new relaxation labeling operator when the restrictions are not satisfied the theory relies on variational cal culus it is shown that the problem of finding consistent labelings is equivalent to solving a variational inequality a procedure nearly identical to the relaxation operator derived under restricted circum stances serves in the more general setting further a local convergence result is established for this operator the standard relaxation labeling formulas are shown to approximate our new operator which leads us to conjecture that successful applications of the standard methods are explainable by the theory developed here observations about con vergence and generalizations to higher order compatibility relations are described

markov random field texture models we consider a texture to be a stochastic possibly periodic two dimensional image field a texture model is a mathematical procedure capable of producing and describing a textured image we explore the use of markov random fields as texture models the binomial model where each point in the texture has a binomial distribution with parameter controlled by its neighbors and number of tries equal to the number of gray levels was taken to be the basic model for the analysis a method of generating samples from the binomial model is given followed by a theoretical and practical analysis of the method s convergence examples show how the parameters of the markov random field control the strength and direction of the clustering in the image the power of the binomial model to produce blurry sharp line like and blob like textures is demonstrated natural texture samples were digitized and their parameters were estimated under the markov random field model a hypothesis test was used for an objective assessment of goodness of fit under the markov random field model overall microtextures fit the model well the estimated parameters of the natural textures were used as input to the generation procedure the synthetic microtextures closely resembled their real counterparts while the regular and inhomogeneous textures did not

convergence rate of markov chain methods for genomic motif discovery we analyze the convergence rate of a popular gibbs sampling method used for statistical discovery of gene regulatory binding motifs in dna sequences this sampler satisfies a very strong form of ergodicity uniform however we show that due to multimodality of the posterior distribution the rate of convergence often decreases exponentially as a function of the length of the dna sequence specifically we show that this occurs whenever there is more than one true repeating pattern in the data in practice there are typically multiple even numerous such patterns in biological data the goal being to detect the most well conserved and frequently occurring of these our findings match empirical results in which the motif discovery gibbs sampler has exhibited such poor convergence that it is used only for finding modes of the posterior distribution candidate motifs rather than for obtaining samples from that distribution ours appear to be the first meaningful bounds on the convergence rate of a markov chain method for sampling from a multimodal posterior distribution as a function of statistical quantities like the number of observations

the collapsed gibbs sampler in bayesian computations with applications to a gene regulation problem abstract this article describes a method of grouping and collapsing in using the gibbs sampler and proves from an operator theory viewpoint that the method is in general beneficial the norms of the forward operators associated with the corresponding nonreversible markov chains are used to discriminate among different simulation schemes when applied to bayesian missing data problems the idea of collapsing suggests skipping the steps of sampling parameter s values in standard data augmentation by doing this we obtain a predictive update version of the gibbs sampler a procedure of calculating the posterior odds ratio via the collapsed gibbs sampler when incomplete observations are involved is presented as an illustration of possible applications three examples along with a bayesian treatment for identifying common protein binding sites in unaligned dna sequences are provided

antithetic coupling of two gibbs sampler chains two coupled gibbs sampler chains both with invariant probability density are run in parallel so that the chains are negatively correlated we define an asymptotically unbiased estimator of the expectation e f x which achieves significant variance reduction with respect to the usual gibbs sampler at comparable computational cost the variance of the estimator based on the new algorithm is always smaller than the variance of a single gibbs sampler chain if is attractive and f is monotone nondecreasing in all components of x for nonattractive targets our results are not complete the new antithetic algorithm outperforms the standard gibbs sampler when is a multivariate normal density or the ising model more generally nonrigorous arguments and numerical experiments support the usefulness of the antithetically coupled gibbs samplers also for other nonattractive models in our experiments the variance is reduced to at least a third and the efficiency also improves significantly

null spaces of correlation matrices abstract let r be the real numbers and r n the vector space of all column vectors of length n let c n be the convex set of all real correlation matrices of size n if v is a subspace of r n of dimension k we consider the face f v of c n consisting of all a c n such that v n a i e av 0 if f v is nonempty we say that v is realizable we give complete geometric descriptions of f v in the cases k 1 n 4 and k 2 n 5 for k 2 n 5 we provide a simple algebraic method for describing f v

blind identification of bilinear systems this paper is concerned with the blind identification of a class of bilinear systems excited by non gaussian higher order white noise the matrix of coefficients of mixed input output terms of the bilinear system model is assumed to be triangular in this work under the additional assumption that the system output is corrupted by gaussian measurement noise we derive an exact parameter estimation procedure based on the output cumulants of orders up to four results of the simulation experiments presented in the paper demonstrate the validity and usefulness of our approach

extremal correlation matrices abstract let r n denote the convex compact set of all real n by n positive semidefinite matrices with main diagonal entries equal to 1 we examine the extreme points of r n focusing mainly on their rank the principal result is that r n contains extreme points of rank k if and only if k k 1 2 n

covariance structure and convergence rate of the gibbs sampler with various scans this paper presents results on covariance structure and convergence for the gibbs sampler with both systematic and random scans it is shown that under conditions that guarantee the compactness of the markov forward operator and irreducibility of the corresponding chain the gibbs sampling scheme converges geometrically in terms of pearson 2 distance in particular for the random scan the autocovariance can be expressed as variances of iterative conditional expectations as a consequence the autocorrelations are all positive and decrease monotonically

rao blackwellisation of sampling schemes summary this paper proposes a post simulation improvement for two common monte carlo methods the accept reject and metropolis algorithms the improvement is based on a rao blackwellisation method that integrates over the uniform random variables involved in the algorithms and thus post processes the standard estimators we show how the rao blackwellised versions of these algorithms can be implemented and through examples illustrate the improvement in variance brought by these new procedures we also compare the improved version of the metropolis algorithm with ordinary and rao blackwellised importance sampling procedures for independent and general metropolis set ups

on the theory of bilinear time series models summary the theory of bilinear time series models is considered in this paper the sufficient conditions for asymptotic stationarity of the bilinear time series models are derived and the expressions for the variance and covariance are obtained the conditions for the invertibility of the model are also included the estimation of the parameters of the scalar bilinear time series model is considered the bilinear models are fitted to sunspot numbers and also to a p wave of a nuclear explosion the forecasting of sunspot numbers is also considered

covariance structure of the gibbs sampler with applications to the comparisons of estimators and augmentation schemes summary we study the covariance structure of a markov chain generated by the gibbs sampler with emphasis on data augmentation when applied to a bayesian missing data problem the gibbs sampler produces two natural approximations for the posterior distribution of the parameter vector the empirical distribution based on the sampled values of the parameter vector and a mixture of complete data posteriors we prove that rao blackwellization causes a one lag delay for the autocovariances among dependent samples obtained from data augmentation and consequently the mixture approximation produces estimates with smaller variances than the empirical approximation the covariance structure results are used to compare different augmentation schemes it is shown that collapsing and grouping random components in a gibbs sampler with two or three components usually result in more efficient sampling schemes

bayesian methods for hidden markov models markov chain monte carlo mcmc sampling strategies can be used to simulate hidden markov model hmm parameters from their posterior distribution given observed data some mcmc methods used in practice for computing likelihood conditional probabilities of hidden states and the most likely sequence of states can be improved by incorporating established recursive algorithms the most important of these is a set of forward backward recursions calculating conditional distributions of the hidden states given observed data and model parameters i show how to use the recursive algorithms in an mcmc context and demonstrate mathematical and empirical results showing a gibbs sampler using the forward backward recursions mixes more rapidly than another sampler often used for hmms iintroduce an augmented variables technique for obtaining unique state labels in hmms and finite mixture models i show how recursive computing allows the statistically efficient use of mcmc output when estimating the hidden states i

rank inequalities for positive semidefinite matrices abstract several inequalities relating the rank of a positive semidefinite matrix with the ranks of various principal submatrices are presented these inequalities are analogous to known determinantal inequalities for positive definite matrices such as fischer s inequality koteljanskii s inequality and extensions of these associated with chordal graphs

estimation and choice of neighbors in spatial interaction models of images some aspects of statistical inference for a class of spatial interaction models for finite images are presented primarily the simultaneous autoregressive sar models and conditional markov cm models each of these models is characterized by a set of neighbors a set of coefficients and a noise sequence of specified characteristics we are concerned with two problems the estimation of the unknown parameters in both sar and cm models and the choice of an appropriate model from a class of such competing models assuming gaussian distributed variables we discuss maximum likelihood ml estimation methods in general the ml scheme leads to nonlinear optimization problems to avoid excessive computation an iterative scheme is given for sar models which gives approximate ml estimates in the gaussian case and reasonably good estimates in some non gaussian situations as well likewise for cm models an easily computable consistent estimate is given the asymptotic mean squared error mse of this estimate for a four neighbor cm model is shown tn be substantially less than the mse of the popular coding estimate asymptotically consistent decision rules are given for choosing an appropriate sar or cm model the usefulness of the estimation scheme and the decision rule for the choice of neighbors is illustrated by using synthetic patterns synthetic patterns obeying known sar and cm models are generated and the models corresponding to true and several competing neighbor sets are fitted the estimation scheme yields estimates close to the parameters of the true models and the decision rule for the choice of neighbors picks up the true model from the class of competing models

peskun s theorem and a modified discrete state gibbs sampler attention is drawn to the use of peskun s theorem in improving statistical efficiency of discretestate gibbs sampling

higher order spectral densities of fractional random fields this paper presents the second and higher order spectral densities of stationary in space random fields arising as approximations of rescaled solutions of the heat and fractional heat equations with singular initial conditions the development is based on the diagram formalism and the riesz composition formula our results are the first step to full parametrization of higher order spectra of some classes of fractional random fields

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea