cooling schedules for optimal annealing a monte carlo optimization technique called simulated annealing is a descent algorithm modified by random ascent moves in order to escape local minima which are not global minima the level of randomization is determined by a control parameter t called temperature which tends to zero according to a deterministic cooling schedule we give a simple necessary and sufficient condition on the cooling schedule for the algorithm state to converge in probability to the set of globally minimum cost states in the special case that the cooling schedule has parametric form tt c log1 t the condition for convergence is that c be greater than or equal to the depth suitably defined of the deepest local minimum which is not a global minimum state

pseudopotentials and lie symmetries for the generalized nonlinear schrodinger equation the generalized nonlinear schrodinger equation zxx izt f z z is analyzed from the point of view of the existence of pseudopotentials backlund transformations lie symmetries and conservation laws applying the wahlquist estabrook method of closed differential ideals we show that eight classes of nontrivial interaction terms f z z exist for which the equation allows the existence of pseudopotentials five of them simply lead to conservation laws the remaining three to backlund transformations the usual cubic nonlinear schrodinger equation with f z z ez z 2 is obtained as a special case it is also the only case for which the backlund transformation contains a free parameter we show that the real and complex parts of this parameter are generated by the dilation and galilei invariance of the equation

topographic mapping of large dissimilarity data sets topographic maps such as the self organizing map som or neural gas ng constitute powerful data mining techniques that allow simultaneously clustering data and inferring their topological structure such that additional features for example browsing become available both methods have been introduced for vectorial data sets they require a classical feature encoding of information often data are available in the form of pairwise distances only such as arise from a kernel matrix a graph or some general dissimilarity measure in such cases ng and som cannot be applied directly in this article we introduce relational topographic maps as an extension of relational clustering algorithms which offer prototype based representations of dissimilarity data to incorporate neighborhood structure these methods are equivalent to the standard vectorial techniques if a euclidean embedding exists while preventing the need to explicitly compute such an embedding extending these techniques for the general case of non euclidean dissimilarities makes possible an interpretation of relational clustering as clustering in pseudo euclidean space we compare the methods to well known clustering methods for proximity data based on deterministic annealing and discuss how far convergence can be guaranteed in the general case relational clustering is quadratic in the number of data points which makes the algorithms infeasible for huge data sets we propose an approximate patch version of relational clustering that runs in linear time the effectiveness of the methods is demonstrated in a number of examples

unsupervised texture segmentation in a deterministic annealing framework we present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity texture segmentation is formulated as a data clustering problem based on sparse proximity data dissimilarities of pairs of textured regions are computed from a multiscale gabor filter image representation we discuss and compare a class of clustering objective functions which is systematically derived from invariance principles as a general optimization framework we propose deterministic annealing based on a mean field approximation the canonical way to derive clustering algorithms within this framework as well as an efficient implementation of mean field annealing and the closely related gibbs sampler are presented we apply both annealing variants to brodatz like microtexture mixtures and real word images

universal formats for nonlinear ordinary differential systems it is shown that very general nonlinear ordinary differential systems embracing all that arise in practice may first be brought down to polynomial systems where the nonlinearities occur only as polynomials in the dependent variables by introducing suitable new variables into the original system second that polynomial systems are reducible to riccati systems where the nonlinearities are quadratic at most third that riccati systems may be brought to elemental universal formats containing purely quadratic terms with simple arrays of coefficients that are all zero or unity the elemental systems have representations as novel types of matrix riccati equations different starting systems and their associated riccati systems differ from one another at the final elemental level in order and in initial data but not in format

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

prandtl reuss elastoplasticity on off switch and superposition formulae abstract constitutive postulates for prandtl reuss elastoplasticity are selected based on them sufficient and necessary conditions for plastic irreversibility are found to be the yield condition and the straining condition this is then analyzed and it is pointed out that prandtl reuss elastoplasticity is nothing but a two phase linear system with an on off switch which is operated in the pace of an intrinsic measure of plastic irreversibility then the temporally global concept of the switch on time t on and the switch off time t off and their determination and bound estimation is developed owing to the implicit linearity superposition formulae for the stress response are discovered as an example the exact constitutive solutions for circular paths based on the superposition formulae are obtained and t on and t off are determined

boundary detection by constrained optimization a statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions the model is a joint probability distribution for the array of pixel gray levels and an array of labels in boundary finding the labels are binary zero or one representing the absence or presence of boundary elements in partitioning the label values are generic two labels are the same when the corresponding scene locations are considered to belong to the same region the distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels using the kolmogorov smirnov nonparametric measures of difference between the distributions of these features the number of model parameters is minimized by forbidding label configurations which are assigned probability zero the maximum a posteriori estimator of boundary placements and partitionings is examined the forbidden states introduce constraints into the calculation of these configurations stochastic relaxation methods are extended to accommodate constrained optimization

lie algebras with finite dimensional polynomial centralizer we give criteria for finite or infinite dimensionality of the polynomial centralizer of the lie algebra of a linear lie group in terms of invariants and relative invariants of the group in the finite dimensional scenario some applications to normal forms and to certain equations with fundamental solutions are presented

how to avoid spurious cluster validation a methodological investigation on simulated and fmri data abstract this paper presents an evaluation of a common approach that has been considered as a promising option for exploratory fmri data analyses the approach includes two stages creating from the data a sequence of partitions with increasing number of subsets clustering and selecting the one partition in this sequence that exhibits the clearest indications of an existing structure cluster validation in order to achieve that the selected partition is actually the best characterization of the data structure previous studies were directed to find the most appropriate validity function s in our analysis protocol we first optimize the sequence of partitions according to the given objective function our study showed that an insufficient optimization of the partition for one or more numbers of clusters can easily yield a spurious validation result which in turn may lead the analyst to a misleading interpretation of the fmri experiment however a sufficient optimization for each included number of clusters provided the basis for a reliable adequate characterization of the data furthermore it enabled an adequate evaluation of the validity functions these findings were obtained independently for three clustering algorithms representing the hard and fuzzy clustering variant and three up to date cluster validity functions the findings were derived from analyses of gaussian clusters simulated data sets that mimic typical fmri response signals and real fmri data based on our results we propose a number of options of how to configure improved clustering tools

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

a characterization of linkage based hierarchical clustering the class of linkage based algorithms is perhaps the most popular class of hierarchical algorithms we identify two properties of hierarchical algorithms and prove that linkage based algorithms are the only ones that satisfy both of these properties our characterization clearly delineates the difference between linkage based algorithms and other hierarchical methods we formulate an intuitive notion of locality of a hierarchical algorithm that distinguishes between linkage based and global hierarchical algorithms like bisecting k means and prove that popular divisive hierarchical algorithms produce clusterings that cannot be produced by any linkage based algorithm

vector quantization by deterministic annealing a deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data the problem is reformulated within a probabilistic framework no prior knowledge is assumed on the source density and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion the corresponding lagrange multiplier is inversely related to the temperature and is used to control the annealing process in this process as the temperature is lowered the system undergoes a sequence of phase transitions when existing clusters split naturally without use of heuristics the resulting codebook is independent of the codebook used to initialize the iterations

riccati equations and lie series lie series and a special matrix notation for first order differential operators are used to show that the lie group properties of matrix riccati equations arise in a natural way the lie series notation makes it evident that the solutions of a matrix riccati equation are curves in a group of nonlinear transformations that is a generalization of the linear fractional transformations familiar from the classical complex analysis it is easy to obtain a linear representation of the lie algebra of the nonlinear group of transformations and then this linearization leads directly to the standard linearization of the matrix riccati equations we note that the matrix riccati equations considered here are of the general rectangular type

simple subgroups of simple lie groups and nonlinear differential equations with superposition principles nonlinear ordinary differential equations admitting a superposition principle based on the action of the group sl n c on the homogeneous spaces sl n c o n c and sl n c sp n c are derived the superposition formulas are presented explicitly in the o n c case the general solution is expressed in terms of three particular solutions for any n for the sp n c case three solutions are needed for n 2k 8 four solutions for n 6 and five solutions for n 4

unsupervised texture segmentation using gabor filters abstract this paper presents a texture segmentation algorithm inspired by the multi channel filtering theory for visual information processing in the early stages of human visual system the channels are characterized by a bank of gabor filters that nearly uniformly covers the spatial frequency domain and a systematic filter selection scheme is proposed which is based on reconstruction of the input image from the filtered images texture features are obtained by subjecting each selected filtered image to a nonlinear transformation and computing a measure of energy in a window around each pixel a square error clustering algorithm is then used to integrate the feature images and produce a segmentation a simple procedure to incorporate spatial information in the clustering process is proposed a relative index is used to estimate the true number of texture categories

normal forms of elements of classical real and complex lie and jordan algebras elements of the classical complex and real lie and jordan algebras with involutions are classified into conjugacy classes under the action of the corresponding classical lie group normal forms of representatives of each conjugacy class are chosen so as to resemble the jordan normal forms of n n complex matrices for completeness similar results are given for gl n c gl n r and gl n h

a sequence based extension of mean field annealing using the forward backward algorithm application to image segmentation optimization based on mean field theory including mean field annealing mfa is widely used for discrete optimization label assignment problems defined on the pixel sites of an image one formulation of mfa is via maximum entropy where one seeks the joint distribution over the random assignments subject to an average level of cost mfa is obtained by assuming the assignments at each pixel are statistically independent given the observed image alternatively we make the less restrictive assumption of independent row labelings the independence assumption means that at each step mfa optimizes over only one pixel whereas our method jointly optimizes over an entire row i e our method is less greedy in principle an mfa extension could be developed that explicitly re estimates the row labeling distributions but such an approach is in practice infeasible even so we can indirectly implement this re estimation by re estimating quantities that determine the row labeling distributions these quantities are the a posteriori site probabilities re estimable via the well known forward backward f b algorithm thus our algorithm which descends in the me lagrangian free energy consists of iterative application of f b to the image rows columns at convergence maximum a posteriori site labeling is performed our method was applied to segmentation of both real and synthetic noise corrupted images it achieved lower markov random field model potentials and better segmentations compared with other methods and in high noise standard mfa

completely integrable relativistic hamiltonian systems and separation of variables in hermitian hyperbolic spaces the hamilton jacobi and laplace beltrami equations on the hermitian hyperbolic space hh 2 are shown to allow the separation of variables in precisely 12 classes of coordinate systems the isometry group of this two complex dimensional riemannian space su 2 1 has four mutually nonconjugate maximal abelian subgroups these subgroups are used to construct the separable coordinates explicitly all of these subgroups are two dimensional and this leads to the fact that in each separable coordinate system two of the four variables are ignorable ones the symmetry reduction of the free hh 2 hamiltonian by a maximal abelian subgroup of su 2 1 reduces this hamiltonian to one defined on an o 2 1 hyperboloid and involving a nontrivial singular potential separation of variables on hh 2 and more generally on hh n thus provides a new method of generating nontrivial completely integrable relativistic hamiltonian systems

a comparison of texture feature extraction using adaptive gabor filtering pyramidal and tree structured wavelet transforms in recent studies on image analysis an increasing effort has been carried out in the area of wavelet transform techniques for the discrimination and classification of textural images these methods compete with multichannel filtering techniques especially with the nonorthogonal and incomplete gabor filtering in this paper we introduce two feature extraction algorithms based on pyramidal and tree structured wavelet transforms and compare their performance with the feature extraction which employs adaptive gabor filtering this comparison is based on the segmentation results of several texture image examples using the identical segmentation algorithm for all three feature extraction methods the visible differences of the segmentation results are discussed and their algorithmic causes are analysed

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

generalizing swendsen wang to sampling arbitrary posterior probabilities many vision tasks can be formulated as graph partition problems that minimize energy functions for such problems the gibbs sampler provides a general solution but is very slow while other methods such as ncut and graph cuts are computationally effective but only work for specific energy forms and are not generally applicable in this paper we present a new inference algorithm that generalizes the swendsen wang method to arbitrary probabilities defined on graph partitions we begin by computing graph edge weights based on local image features then the algorithm iterates two steps 1 graph clustering it forms connected components by cutting the edges probabilistically based on their weights 2 graph relabeling it selects one connected component and flips probabilistically the coloring of all vertices in the component simultaneously thus it realizes the split merge and regrouping of a chunk of the graph in contrast to gibbs sampler that flips a single vertex we prove that this algorithm simulates ergodic and reversible markov chain jumps in the space of graph partitions and is applicable to arbitrary posterior probabilities or energy functions defined on graphs we demonstrate the algorithm on two typical problems in computer vision image segmentation and stereo vision experimentally we show that it is 100 400 times faster in cpu time than the classical gibbs sampler and 20 40 times faster then the ddmcmc segmentation algorithm for stereo we compare performance with graph cuts and belief propagation we also show that our algorithm can automatically infer generative models and obtain satisfactory results better than the graphic cuts or belief propagation in the same amount of time

entropic repulsion for the high dimensional gaussian lattice field between two walls the main aim of this paper is to discuss the entropic repulsion of random interfaces between two hard walls we consider the d 3 dimensional gaussian lattice field on n n n n d d and identify the repulsion of the field as n under the condition that the field lies between two hard walls at the height level 0 and l in n where l is large enough but finite we also study the same problem for two layered interfaces case

entropic repulsion and the maximum of the two dimensional harmonic crystal we consider the lattice version of the free field in two dimensions also called harmonic crystal the main aim of the paper is to discuss quantitatively the entropic repulsion of the random surface in the presence of a hard wall the basic ingredient of the proof is the analysis of the maximum of the field which requires a multiscale analysis reducing the problem essentially to a problem on a field with a tree structure

optimal cluster preserving embedding of nonmetric proximity data for several major applications of data analysis objects are often not represented as feature vectors in a vector space but rather by a matrix gathering pairwise proximities such pairwise data often violates metricity and therefore cannot be naturally embedded in a vector space concerning the problem of unsupervised structure detection or clustering in this paper a new embedding method for pairwise data into euclidean vector spaces is introduced we show that all clustering methods which are invariant under additive shifts of the pairwise proximities can be reformulated as grouping problems in euclidian spaces the most prominent property of this constant shift embedding framework is the complete preservation of the cluster structure in the embedding space restating pairwise clustering problems in vector spaces has several important consequences such as the statistical description of the clusters by way of cluster prototypes the generic extension of the grouping procedure to a discriminative prediction rule and the applicability of standard preprocessing methods like denoising or dimensionality reduction

axioms for graph clustering quality functions we investigate properties that intuitively ought to be satisfied by graph clustering quality functions that is functions that assign a score to a clustering of a graph graph clustering also known as network community detection is often performed by optimizing such a function two axioms tailored for graph clustering quality functions are introduced and the four axioms introduced in previous work on distance based clustering are reformulated and generalized for the graph setting we show that modularity a standard quality function for graph clustering does not satisfy all of these six properties this motivates the derivation of a new family of quality functions adaptive scale modularity which does satisfy the proposed axioms adaptive scale modularity has two parameters which give greater flexibility in the kinds of clusterings that can be found standard graph clustering quality functions such as normalized cut and unnormalized cut are obtained as special cases of adaptive scale modularity in general the results of our investigation indicate that the considered axiomatic framework covers existing good quality functions for graph clustering and can be used to derive an interesting new family of quality functions

on the information and representation of non euclidean pairwise data two common data representations are mostly used in intelligent data analysis namely the vectorial and the pairwise representation pairwise data which satisfy the restrictive conditions of euclidean spaces can be faithfully translated into a euclidean vectorial representation by embedding non metric pairwise data with violations of symmetry reflexivity or triangle inequality pose a substantial conceptual problem for pattern recognition since the amount of predictive structural information beyond what can be measured by embeddings is unclear we show by systematic modeling of non euclidean pairwise data that there exists metric violations which can carry valuable problem specific information furthermore euclidean and non metric data can be unified on the level of structural information contained in the data stable component analysis selects linear subspaces which are particularly insensitive to data fluctuations experimental results from different domains support our pattern recognition strategy

complexity optimized data clustering by competitive neural networks data clustering is a complex optimization problem with applications ranging from vision and speech processing to data transmission and data storage in technical as well as in biological systems we discuss a clustering strategy that explicitly reflects the tradeoff between simplicity and precision of a data representation the resulting clustering algorithm jointly optimizes distortion errors and complexity costs a maximum entropy estimation of the clustering cost function yields an optimal number of clusters their positions and their cluster probabilities our approach establishes a unifying framework for different clustering methods like k means clustering fuzzy clustering entropy constrained vector quantization or topological feature maps and competitive neural networks

a theory of proximity based clustering structure detection by optimization abstract in this paper a systematic optimization approach for clustering proximity or similarity data is developed starting from fundamental invariance and robustness properties a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria the approach covers the case of sparse proximity matrices and is extended to nested partitionings for hierarchical data clustering to solve the associated optimization problems a rigorous mathematical framework for deterministic annealing and mean field approximation is presented efficient optimization heuristics are derived in a canonical way which also clarifies the relation to stochastic optimization by gibbs sampling similarity based clustering techniques have a broad range of possible applications in computer vision pattern recognition and data analysis as a major practical application we present a novel approach to the problem of unsupervised texture segmentation which relies on statistical tests as a measure of homogeneity the quality of the algorithms is empirically evaluated on a large collection of brodatz like micro texture mondrians and on a set of real word images to demonstrate the broad usefulness of the theory of proximity based clustering the performances of different criteria and algorithms are compared on an information retrieval task for a document database the superiority of optimization algorithms for clustering is supported by extensive experiments

a cutting plane algorithm for a clustering problem in this paper we consider a clustering problem that arises in qualitative data analysis this problem can be transformed to a combinatorial optimization problem the clique partitioning problem we have studied the latter problem from a polyhedral point of view and determined large classes of facets of the associated polytope these theoretical results are utilized in this paper we describe a cutting plane algorithm that is based on the simplex method and uses exact and heuristic separation routines for some of the classes of facets mentioned before we discuss some details of the implementation of our code and present our computational results we mention applications from e g zoology economics and the political sciences

subgroups of lie groups and separation of variables separable systems of coordinates for the helmholtz equation d e in pseudo riemannian spaces of dimension d have previously been characterized algebraically in terms of sets of commuting second order symmetry operators for the operator d they have also been characterized geometrically by the form that the metric ds2 gik x dxidxk can take we complement these characterizations by a group theoretical one in which the second order operators are related to continuous and discrete subgroups of g the symmetry group of d for d 3 we study all separable coordinates that can be characterized in terms of the lie algebra l of g and show that they are of eight types seven of which are related to the subgroup structure of g our method clearly generalizes to the case d 3 although each separable system corresponds to a pair of commuting symmetry operators there do exist pairs of commuting symmetries s1 s2 that are not associated with separable coordinates for subgroup related operators we show in detail just

hierarchical unsupervised learning with growing via phase transitions we address unsupervised learning subject to structural constraints with particular emphasis placed on clustering with an imposed decision tree structure most known methods are greedy optimizing one node of the tree at a time to minimize a local cost by constrast we develop a joint optimization method derived based on information theoretic principles and closely related to known methods in statistical physics the approach is inspired by the deterministic annealing algorithm for unstructured data clustering which was based on maximum entropy inference the new approach is founded on the principle of minimum cross entropy using informative priors to approximate the unstructured clustering solution while imposing the structural constraint the resulting method incorporates supervised learning principles applied in an unsupervised problem setting in our approach the tree grows by a sequence of bifurcations that occur while optimizing an effective free energy cost at decreasing temperature scales thus estimates of the tree size and structure are naturally obtained at each temperature in the process examples demonstrate considerable improvement over known methods

information based clustering in an age of increasingly large data sets investigators in many different disciplines have turned to clustering as a tool for data analysis and exploration existing clustering methods however typically depend on several nontrivial assumptions about the structure of data here we reformulate the clustering problem from an information theoretic perspective that avoids many of these assumptions in particular our formulation obviates the need for defining a cluster prototype does not require an a priori similarity metric is invariant to changes in the representation of the data and naturally captures nonlinear relations we apply this approach to different domains and find that it consistently produces clusters that are more coherent than those extracted by existing algorithms finally our approach provides a way of clustering based on collective notions of similarity rather than the traditional pairwise measures

nonlinear action of lie groups and superposition principles for nonlinear differential equations abstract recently obtained results on superposition principles for systems of ordinary nonlinear equations are reviewed in particular the general solution of the matrix riccati equation is expressed in terms of five particular solutions the study of superposition laws is related to the problem of backlund transformations for nonlinear partial differential equations to the problem of transforming nonlinear equations into linear ones and to the classification of primitive filtered lie algebras