lagrangian multiforms and multidimensional consistency we show that well chosen lagrangians for a class of two dimensional integrable lattice equations obey a closure relation when embedded in a higher dimensional lattice on the basis of this property we formulate a lagrangian description for such systems in terms of lagrangian multiforms we discuss the connection of this formalism with the notion of multidimensional consistency and the role of the lattice from the point of view of the relevant variational principle

estimating normal means with a dirichlet process prior abstract in this article the dirichlet process prior is used to provide a nonparametric bayesian estimate of a vector of normal means in the past there have been computational difficulties with this model this article solves the computational difficulties by developing a gibbs sampler algorithm the estimator developed in this article is then compared to parametric empirical bayes estimators peb and nonparametric empirical bayes estimators npeb in a monte carlo study the monte carlo study demonstrates that in some conditions the peb is better than the npeb and in other conditions the npeb is better than the peb the monte carlo study also shows that the estimator developed in this article produces estimates that are about as good as the peb when the peb is better and produces estimates that are as good as the npeb estimator when that method is better

a comparison of sampford s sampling procedure versus unequal probability sampling with replacement summary the classical result that simple random sampling without replacement is better than simple random sampling with replacement holds in unequal probability sampling for the sampling procedure of sampford

bayes procedures for adaptive inference in inverse problems for the white noise model we study empirical and hierarchical bayes approaches to the problem of estimating an infinite dimensional parameter in mildly ill posed inverse problems we consider a class of prior distributions indexed by a hyperparameter that quantifies regularity we prove that both methods we consider succeed in automatically selecting this parameter optimally resulting in optimal convergence rates for truths with sobolev or analytic smoothness without using knowledge about this regularity both methods are illustrated by simulation examples

unsupervised texture segmentation in a deterministic annealing framework we present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity texture segmentation is formulated as a data clustering problem based on sparse proximity data dissimilarities of pairs of textured regions are computed from a multiscale gabor filter image representation we discuss and compare a class of clustering objective functions which is systematically derived from invariance principles as a general optimization framework we propose deterministic annealing based on a mean field approximation the canonical way to derive clustering algorithms within this framework as well as an efficient implementation of mean field annealing and the closely related gibbs sampler are presented we apply both annealing variants to brodatz like microtexture mixtures and real word images

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

the construction of pi ps sampling designs through a method of emptying boxes we present a simple but universal technique for the construction of iips sampling designs a tool that is used in the construction consists of playing a game in which objects are removed from n boxes n at a time and at most one from each box at a time it is shown that every iips design can be derived from such a game sampling designs with additional properties are obtained through additional restrictions on emptying the boxes the emphasis is on controlling sample selection probabilities and inequalities for the first and second order inclusion probabilities

spatially balanced sampling through the pivotal method a simple method to select a spatially balanced sample using equal or unequal inclusion probabilities is presented for populations with spatial trends in the variables of interest the estimation c

bagging for path based clustering a resampling scheme for clustering with similarity to bootstrap aggregation bagging is presented bagging is used to improve the quality of path based clustering a data clustering method that can extract elongated structures from data in a noise robust way the results of an agglomerative optimization method are influenced by small fluctuations of the input data to increase the reliability of clustering solutions a stochastic resampling method is developed to infer consensus clusters a related reliability measure allows us to estimate the number of clusters based on the stability of an optimized cluster solution under resampling the quality of path based clustering with resampling is evaluated on a large image data set of human segmentations

a variational model for image classification and restoration we present a variational model devoted to image classification coupled with an edge preserving regularization process the discrete nature of classification i e to attribute a label to each pixel has led to the development of many probabilistic image classification models but rarely to variational ones in the last decade the variational approach has proven its efficiency in the field of edge preserving restoration we add a classification capability which contributes to provide images composed of homogeneous regions with regularized boundaries a region being defined as a set of pixels belonging to the same class the soundness of our model is based on the works developed on the phase transition theory in mechanics the proposed algorithm is fast easy to implement and efficient we compare our results on both synthetic and satellite images with the ones obtained by a stochastic model using a potts regularization

boundary detection by constrained optimization a statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions the model is a joint probability distribution for the array of pixel gray levels and an array of labels in boundary finding the labels are binary zero or one representing the absence or presence of boundary elements in partitioning the label values are generic two labels are the same when the corresponding scene locations are considered to belong to the same region the distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels using the kolmogorov smirnov nonparametric measures of difference between the distributions of these features the number of model parameters is minimized by forbidding label configurations which are assigned probability zero the maximum a posteriori estimator of boundary placements and partitionings is examined the forbidden states introduce constraints into the calculation of these configurations stochastic relaxation methods are extended to accommodate constrained optimization

stability based validation of clustering solutions data clustering describes a set of frequently employed techniques in exploratory data analysis to extract natural group structure in data such groupings need to be validated to separate the signal in the data from spurious structure in this context finding an appropriate number of clusters is a particularly important model selection question we introduce a measure of cluster stability to assess the validity of a cluster model this stability measure quantifies the reproducibility of clustering solutions on a second sample and it can be interpreted as a classification risk with regard to class labels produced by a clustering algorithm the preferred number of clusters is determined by minimizing this classification risk as a function of the number of clusters convincing results are achieved on simulated as well as gene expression data sets comparisons to other methods demonstrate the competitive performance of our method and its suitability as a general validation tool for clustering solutions in real world problems

semi supervised dimensionality reduction of hyperspectral imagery using pseudo labels abstract dimensionality reduction has been proven to be efficient in preparing high dimensional data for various tasks in machine learning as supervised dimensionality reduction methods such as fisher discriminant analysis fda and local fisher discriminant analysis lfda tend to suffer from overfitting when only a small number of labeled samples are available the abundant unlabeled samples could be helpful in finding a better embedding space however applying discriminant analysis on unlabeled data is challenging since we do not have labels for unlabeled data in this paper we propose a semi supervised semi supervised local fisher discriminant analysis sslfda using pseudo labels aiming to perform discriminant analysis on both labeled and unlabeled samples sslfda makes use of pseudo labels learned from the dirichlet process mixture model dpmm based clustering algorithm to enable local fisher discriminant analysis on unlabeled data in addition a kernel extension of sslfda is derived for non linear dimensionality reduction we present experimental results with real hyperspectral data to show that our method provides better classification performance compared to other existing dimensionality reduction methods

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

efficient balanced sampling the cube method a balanced sampling design is defined by the property that the horvitz thompson estimators of the population totals of a set of auxiliary variables equal the known totals of these variables therefore the variances of estimators of totals of all the variables of interest are reduced depending on the correlations of these variables with the controlled variables in this paper we develop a general method called the cube method for selecting approximately balanced samples with equal or unequal inclusion probabilities and any number of auxiliary variables

image segmentation by map ml estimations image segmentation plays an important role in computer vision and image analysis in this paper image segmentation is formulated as a labeling problem under a probability maximization framework to estimate the label configuration an iterative optimization scheme is proposed to alternately carry out the maximum a posteriori map estimation and the maximum likelihood ml estimation the map estimation problem is modeled with markov random fields mrfs and a graph cut algorithm is used to find the solution to the map estimation the ml estimation is achieved by computing the means of region features in a gaussian model our algorithm can automatically segment an image into regions with relevant textures or colors without the need to know the number of regions in advance its results match image edges very well and are consistent with human perception comparing to six state of the art algorithms extensive experiments have shown that our algorithm performs the best

a markov random field regulated pitman yor process prior for spatially constrained data clustering in this work we propose a markov random field regulated pitman yor process mrf pyp prior for nonparametric clustering of data with spatial interdependencies the mrf pyp is constructed by imposing a pitman yor process over the distribution of the latent variables that allocate data points to clusters model states the discount hyperparameter of which is regulated by an additionally postulated simplified pointwise markov random field gibbsian distribution with a countably infinite number of states further based on the stick breaking construction of the pitman yor process we derive an efficient truncated variational bayesian algorithm for model inference we examine the efficacy of our approach by considering an unsupervised image segmentation application using a real world dataset we show that our approach completely outperforms related methods from the field of bayesian nonparametrics including the recently proposed infinite hidden markov random field model and the dirichlet process prior

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

biometrika centenary sample surveys summary despite karl pearson s stated aim that the evolutionist has to become in the widest sense of the words a registrar general for all forms of life the papers published in biometrika on sample surveys have been mainly theoretical and do not start to appear until the 1940s there has been little work on sampling in the life sciences with the exception of capture recapture methods the major areas have been in variable probability designs the foundations of inference for finite populations analytic surveys and the role of selection mechanisms in inference in recent years the work on survey analysis has been the outcome of an interaction between sample selection mechanisms and mainstream theory with interesting consequences for both areas

optimal controlled sampling designs optimal controlled sampling designs that minimize the probability of selecting a nonpreferred sample are obtained these plans satisfy either of the following two requirements i match specified joint inclusion probabilities ij associated with an uncontrolled sampling plan having certain desirable properties ii match specified first order inclusion probabilities and provide stable nonnegative unbiased variance estimators limitations of other approaches using methods of experimental designs are demonstrated

the infinite hidden markov random field model hidden markov random field hmrf models are widely used for image segmentation as they appear naturally in problems where a spatially constrained clustering scheme is asked for a major limitation of hmrf models concerns the automatic selection of the proper number of their states i e the number of region clusters derived by the image segmentation procedure existing methods including likelihood or entropy based criteria and reversible markov chain monte carlo methods usually tend to yield noisy model size estimates while imposing heavy computational requirements recently dirichlet process dp infinite mixture models have emerged in the cornerstone of nonparametric bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori infinite mixture models based on the original dp or spatially constrained variants of it have been applied in unsupervised image segmentation applications showing promising results under this motivation to resolve the aforementioned issues of hmrf models in this paper we introduce a nonparametric bayesian formulation for the hmrf model the infinite hmrf model formulated on the basis of a joint dirichlet process mixture dpm and markov random field mrf construction we derive an efficient variational bayesian inference algorithm for the proposed model and we experimentally demonstrate its advantages over competing methodologies

continuous symmetric reductions of the adler bobenko suris equations continuously symmetric solutions of the adler bobenko suris class of discrete integrable equations are presented initially defined by their invariance under the action of both of the extended three point generalized symmetries admitted by the corresponding equations these solutions are shown to be determined by an integrable system of partial differential equations the connection of this system to the nijhoff hone joshi generating partial differential equations is established and an auto backlund transformation and a lax pair for it are constructed applied to the h1 and q1 0 members of the adler bobenko suris family the method of continuously symmetric reductions yields explicit solutions determined by the painleve trancendents

bayesian nonparametric inference for random distributions and related functions in recent years bayesian nonparametric inference both theoretical and computational has witnessed considerable advances however these advances have not received a full critical and comparative analysis of their scope impact and limitations in statistical modelling many aspects of the theory and methods remain a mystery to practitioners and many open guestions remain in this paper we discuss and illustrate the rich modelling and analytic possibilities that are available to the statistician within the bayesian nonparametric and or semiparametric framework

optimal cluster preserving embedding of nonmetric proximity data for several major applications of data analysis objects are often not represented as feature vectors in a vector space but rather by a matrix gathering pairwise proximities such pairwise data often violates metricity and therefore cannot be naturally embedded in a vector space concerning the problem of unsupervised structure detection or clustering in this paper a new embedding method for pairwise data into euclidean vector spaces is introduced we show that all clustering methods which are invariant under additive shifts of the pairwise proximities can be reformulated as grouping problems in euclidian spaces the most prominent property of this constant shift embedding framework is the complete preservation of the cluster structure in the embedding space restating pairwise clustering problems in vector spaces has several important consequences such as the statistical description of the clusters by way of cluster prototypes the generic extension of the grouping procedure to a discriminative prediction rule and the applicability of standard preprocessing methods like denoising or dimensionality reduction

estimating the granularity coefficient of a potts markov random field within a markov chain monte carlo algorithm this paper addresses the problem of estimating the potts parameter jointly with the unknown parameters of a bayesian model within a markov chain monte carlo mcmc algorithm standard mcmc methods cannot be applied to this problem because performing inference on requires computing the intractable normalizing constant of the potts model in the proposed mcmc method the estimation of is conducted using a likelihood free metropolis hastings algorithm experimental results obtained for synthetic data show that estimating jointly with the other unknown parameters leads to estimation results that are as good as those obtained with the actual value of on the other hand choosing an incorrect value of can degrade estimation performance significantly to illustrate the interest of this method the proposed algorithm is successfully applied to real bidimensional sar and tridimensional ultrasound images

yang baxter maps and symmetries of integrable equations on quad graphs a connection between the yang baxter relation for maps and the multidimensional consistency property of integrable equations on quad graphs is investigated the approach is based on the symmetry analysis of the corresponding equations it is shown that the yang baxter variables can be chosen as invariants of the multiparameter symmetry groups of the equations we use the classification results by adler bobenko and suris to demonstrate this method some new examples of yang baxter maps are derived in this way from multifield integrable equations

generalized hyperbolic ernst equations for an einstein maxwell weyl field a new family of nonlinear partial differential equations is presented they represent a generalization of the hyperbolic ernst equations for an einstein mawxell weyl field in general relativity a backlund transformation for the system of equations under consideration is given and their direct relation to the complete boussinesq hierarchy of soliton equations is illustrated

on chao s unequal probability sampling plan summary we give necessary and sufficient conditions for the second order inclusion probabilities to be strictly positive under chao s 1982 sampling plan for some order of arrangement of the population units we also prove that the variance of the horvitz thompson estimator of the population total based on this sampling plan is smaller than the variance of the hansen hurwitz estimator based on a probability proportional to size with replacement sample involving the same number of draws

applying the horvitz thompson criterion in complex designs a computer intensive perspective for estimating inclusion probabilities a modification of the horvitz thompson estimator is proposed for complex sampling designs the inclusion probabilities are estimated by means of independent replications of the sampling scheme the properties of the resulting estimator are derived guidelines for choosing the appropriate number of replications are given and some applications are considered

variational infinite hidden conditional random fields hidden conditional random fields hcrfs are discriminative latent variable models which have been shown to successfully learn the hidden structure of a given classification problem an infinite hidden conditional random field is a hidden conditional random field with a countably infinite number of hidden states which rids us not only of the necessity to specify a priori a fixed number of hidden states available but also of the problem of overfitting markov chain monte carlo mcmc sampling algorithms are often employed for inference in such models however convergence of such algorithms is rather difficult to verify and as the complexity of the task at hand increases the computational cost of such algorithms often becomes prohibitive these limitations can be overcome by variational techniques in this paper we present a generalized framework for infinite hcrf models and a novel variational inference approach on a model based on coupled dirichlet process mixtures the hcrf dpm we show that the variational hcrf dpm is able to converge to a correct number of represented hidden states and performs as well as the best parametric hcrfs chosen via cross validation for the difficult tasks of recognizing instances of agreement disagreement and pain in audiovisual sequences

on the lagrangian formulation of multidimensionally consistent systems multidimensional consistency has emerged as a key integrability property for partial difference equations p es defined on the space time lattice it has led among other major insights to a classification of scalar affine linear quadrilateral p es possessing this property leading to the so called adler bobenko suris abs list recently a new variational principle has been proposed that describes the multidimensional consistency in terms of discrete lagrangian multi forms this description is based on a fundamental and highly non trivial property of lagrangians for those integrable lattice equations namely the fact that on the solutions of the corresponding p e the lagrange forms are closed i e they obey a closure relation here we extend those results to the continuous case it is known that associated with the integrable p es there exist systems of partial differential equations pdes in fact differential equations with regard to the parameters of the lattice as independent variables which equally possess the property of multidimensional consistency in this paper we establish a universal lagrange structure for affine linear quad lattices alongside a universal lagrange multi form structure for the corresponding continuous pdes and we show that the lagrange forms possess the closure property

a general purpose unequal probability sampling plan summary we present a general purpose unequal probability without replacement sampling plan with fixed sample size in contrast to existing such plans our scheme keeps the sample size fixed and lets the population units enter the sample one at a time through a carefully designed random mechanism consequently all high order inclusion probabilities can be easily computed

nonparametric bayesian image segmentation image segmentation algorithms partition the set of pixels of an image into a specific number of different spatially homogeneous groups we propose a nonparametric bayesian model for histogram clustering which automatically determines the number of segments when spatial smoothness constraints on the class assignments are enforced by a markov random field a dirichlet process prior controls the level of resolution which corresponds to the number of clusters in data with a unique cluster structure the resulting posterior is efficiently sampled by a variant of a conjugate case sampling algorithm for dirichlet process mixture models experimental results are provided for real world gray value images synthetic aperture radar images and magnetic resonance imaging data

second order inclusion probabilities in sequential sampling without replacement with unequal probabilities chao 1982 presents a sequential procedure for sampling without replacement with unequal probabilities his sampling plan combines the simplicity of sequential processing with the possibility of computation of second order inclusion probabilities the second order inclusion probabilities produced by this method depend on the order in which the elements are processed and are not always strictly positive this paper presents necessary and sufficient conditions for the order of the elements in the population giving positive second order probabilities thus enabling variance estimation

unequal probability sampling without replacement through a splitting method summary a very general class of sampling methods without replacement and with unequal probabilities is proposed it consists of splitting the inclusion probability vector into several new inclusion probability vectors one of these vectors is chosen randomly thus the initial problem is reduced to another sampling problem with unequal probabilities this splitting is then repeated on these new vectors of inclusion probabilities at each step the sampling problem is reduced to a simpler problem the simplicity of this technique allows one to generate easily new sampling procedures with unequal probabilities the splitting method also generalises well known methods such as the midzuno method the elimination procedure and the chao procedure next a sufficient condition is given in order that a splitting method satisfies the sen yates grundy condition finally it is shown that the elimination procedure satisfies the gabler sufficient condition

on a 2 1 dimensional integrable ernst type equation a strong 2 1 dimensional integrable extension of ernst s equation of general relativity is proposed its richness is demonstrated by means of various canonical dimensional reductions and specializations which lead to formal analogues of well known 1 1 and 2 1 dimensional integrable systems such as the self induced transparency equations the konopelchenko rogers equations a 2 1 dimensional darboux system descriptive of conjugate coordinate systems a single 2 1 dimensional sine gordon equation and the equations representing its backlund transformation a darboux levi type transformation is given and its compatibility with the above mentioned reductions is shown

an elimination procedure for unequal probability sampling without replacement summary an exact without replacement sampling procedure with fixed sample size and unequal inclusion probabilities is presented the proposed method eliminates a unit of the population at each of the n n steps of the algorithm in order finally to retain n units a simple expression is given for the joint inclusion probabilities

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

a family of integrable nonlinear equations of hyperbolic type a new system of integrable nonlinear equations of hyperbolic type obtained by a two dimensional reduction of the anti self dual yang mills equations is presented it represents a generalization of the ernst weyl equation of general relativity related to colliding neutrino and gravitational waves as well as of the fourth order equation of schwarzian type related to the kdv hierarchies which was introduced by nijhoff hone and joshi recently an auto backlund transformation of the new system is constructed leading to a superposition principle remarkably similar to the one connecting four solutions of the kdv equation at the level of the ernst weyl equation this backlund transformation and the associated superposition principle yield directly a generalization of the single and double harrison transformations of the ernst equation respectively the very method of construction also allows for revealing in an essentially algorithmic fashion other integrability features of the main subsystems such as their reduction to the painleve transcendents

on unequal probability sampling sufficient conditions for the superiority of sampling without replacement summary a challenging theoretical problem in unequal probability sampling is the question which design sampling with or without replacement should be implemented in this context we compare the horvitz thompson estimator with the hansen hurwitz estimator and derive conditions under which the variance of the horvitz thompson estimator is smaller than the variance of the hansen hurwitz estimator examples are given showing the validity of these conditions

method for searching higher symmetries for quad graph equations a generalized symmetry integrability test for discrete equations on the square lattice is studied integrability conditions are discussed a method for searching higher symmetries including non autonomous ones for quad graph equations is suggested based on characteristic vector fields