strong universal source coding subject to a rate distortion constraint the following problem in universal source coding is explored some members of a class of sources have various constraints on the maximum rate at which they may be encoded and thc remainder have various constraints on the maximum distortion that may result from encoding it is desired to find a universal code that will adapt its performance so that whatever source in the class is encoded the resulting performance meets the constraint and is optimal in the rate distortion function sense for that particular source it is shown that such codes exist when the class is totally bounded and the constraints are uniformly continuous in an appropriate sense a key result shows that the value of any uniformly continuous function on a totally bounded class can be uniformly well estimated from observations of the output of any source in

natural type selection in adaptive lossy compression consider approximate lossy matching of a source string spl sim p with a random codebook generated from reproduction distribution q at a specified distortion d previous work determined the minimum coding rate r sub 1 r p q d for this setting we observe that for a large word length and with high probability the matching codeword is typical with a distribution q sub 1 which is different from q if a new random codebook is generated spl sim q sub 1 then the source string will favor codewords which are typical with a new distribution q sub 2 resulting in a minimum coding rate r sub 2 r p q sub 1 d and so on we show that the sequences of distributions q sub 1 q sub 2 and rates r sub 1 r sub 2 generated by this procedure converge to an optimum reproduction distribution q and the rate distortion function r p d respectively we also derive a fixed rate distortion slope version of this natural type selection process in the latter case an iteration of the process stochastically simulates an iteration of the blahut arimoto 1972 algorithm for rate distortion function computation without recourse to prior knowledge of the underlying source distribution to strengthen these limit statements we also characterize the steady state error of these procedures when iterating at a finite string length implications of the main results provide fresh insights into the workings of lossy variants of the lempel ziv algorithm for adaptive compression

multiple window parallel adaptive boundary finding in computer vision the problem considered in this paper is the estimation of highly variable object boundaries in noisy images boundaries may be those of a tank in an ir image a spinal canal in a cat scan a cloud in a visible light image etc or they may be internal to an object such as the boundary between a spherical surface and a cylindrical surface in a manufactured object the focus of the paper is on parallel multiple window boundary estimation algorithms here the image field is parti tioned into an array of rectangular windows and boundary finders are run simultaneously within the windows the boundary segments found within the windows are then seamed together to obtain meaningful global boundaries the entire procedure is treated within a maximum likelihood estimation framework that we have developed for boundary finding although our multiple window estimation approach can be used with a number of local boundary finding algorithms we concen trate on one which is based on dynamic programming and will produce the true maximum likelihood boundary some theoretical considera tions for boundary model design and boundary finding runtime are covered included is the use of a low computational cost f test for test ing whether a window contains a boundary and an analytical treatment which shows that use of coarse pixels with a chi square test or an f test improves the probability of correctly recognizing whether a boundary is present in a window

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

optimal phase transitions in compressed sensing compressed sensing deals with efficient recovery of analog signals from linear encodings this paper presents a statistical study of compressed sensing by modeling the input signal as an i i d process with known distribution three classes of encoders are considered namely optimal nonlinear optimal linear and random linear encoders focusing on optimal decoders we investigate the fundamental tradeoff between measurement rate and reconstruction fidelity gauged by error probability and noise sensitivity in the absence and presence of measurement noise respectively the optimal phase transition threshold is determined as a functional of the input distribution and compared to suboptimal thresholds achieved by popular reconstruction algorithms in particular we show that gaussian sensing matrices incur no penalty on the phase transition threshold with respect to optimal nonlinear encoding our results also provide a rigorous justification of previous results based on replica heuristics in the weak noise regime

intraband discrete breathers in disordered nonlinear systems ii localization abstract we find spatially localized time periodic solutions discrete breathers or dbs in disordered nonlinear systems with frequency inside the linear phonon spectrum under conditions that strictly prohibit their existence in their periodic counterparts for that purpose we develop a new in situ method for the accurate calculation of these solutions which does not make use of any continuation from an anticontinuous limit using this method we demonstrate that intraband localized modes intraband discrete breathers or idbs at a given site with frequencies inside the discrete linear spectrum do exist provided these frequencies do not belong to forbidden resonance gaps since there is a dense set of resonant frequencies we illustrate numerically in agreement with a theorem by albanese and frohlich that the localized dbs exist provided that their frequencies belong to fat cantor sets i e with finite measure such a set contains as accumulation points the linear frequency of the normal mode at the occupied site we check that many of these solutions are linearly stable and conjecture that their frequency belongs to another smaller fat cantor set our numerical methods provide a much wider set of exact solutions which are multisite breathers and suggest conjectures extending the existing theorems the physical implications of the existence of idbs and possible applications for glasses and the persistent spectral hole burning are discussed

recognition using region correspondences recognition systems attempt to recover information about the identity of observed objects and their location in the environment a fundamental problem in recognition is pose estimation this is the problem of using a correspondence between some portions of an object model and some portions of an image to determine whether the image contains an instance of the object and in case it does to determine the transformation that relates the model to the image the current approaches to this problem are divided into methods that use global properties of the object e g centroid and moments of inertia and methods that use local properties of the object e g corners and line segments global properties are sensitive to occlusion and specifically to self occlusion local properties are difficult to locate reliably and their matching involves intensive computation we present a novel method for recognition that uses region information in our approach the model and the image are divided into regions given a match between subsets of regions without any explicit correspondence between different pieces of the regions the alignment transformation is computed the method applies to planar objects under similarity affine and projective transformations and to projections of 3 d objects undergoing affine and projective transformations the new approach combines many of the advantages of the previous two approaches while avoiding some of their pitfalls like the global methods our approach makes use of region information that reflects the true shape of the object but like local methods our approach can handle occlusion

wavelet based image estimation an empirical bayes approach using jeffrey s noninformative prior the sparseness and decorrelation properties of the discrete wavelet transform have been exploited to develop powerful denoising methods however most of these methods have free parameters which have to be adjusted or estimated in this paper we propose a wavelet based denoising technique without any free parameters it is in this sense a universal method our approach uses empirical bayes estimation based on a jeffreys noninformative prior it is a step toward objective bayesian wavelet based denoising the result is a remarkably simple fixed nonlinear shrinkage thresholding rule which performs better than other more computationally demanding methods

an mcmc approach to universal lossy compression of analog sources motivated by the markov chain monte carlo mcmc approach to the compression of discrete sources developed by jalali and weissman we propose a lossy compression algorithm for analog sources that relies on a finite reproduction alphabet which grows with the input length the algorithm achieves in an appropriate asymptotic sense the optimum shannon theoretic tradeoff between rate and distortion universally for stationary ergodic continuous amplitude sources we further propose an mcmc based algorithm that resorts to a reduced reproduction alphabet when such reduction does not prevent achieving the shannon limit the latter algorithm is advantageous due to its reduced complexity and improved rates of convergence when employed on sources with a finite and small optimum reproduction alphabet

analysis of multiresolution image denoising schemes using generalized gaussian and complexity priors research on universal and minimax wavelet shrinkage and thresholding methods has demonstrated near ideal estimation performance in various asymptotic frameworks however image processing practice has shown that universal thresholding methods are outperformed by simple bayesian estimators assuming independent wavelet coefficients and heavy tailed priors such as generalized gaussian distributions ggds in this paper we investigate various connections between shrinkage methods and maximum a posteriori map estimation using such priors in particular we state a simple condition under which map estimates are sparse we also introduce a new family of complexity priors based upon rissanen s universal prior on integers one particular estimator in this class outperforms conventional estimators based on earlier applications of the minimum description length mdl principle we develop analytical expressions for the shrinkage rules implied by ggd and complexity priors this allows us to show the equivalence between universal hard thresholding map estimation using a very heavy tailed ggd and mdl estimation using one of the new complexity priors theoretical analysis supported by numerous practical experiments shows the robustness of some of these estimates against mis specifications of the prior a basic concern in image processing applications

universal compressed sensing for almost lossless recovery in this paper the problem of developing universal algorithms for noiseless compressed sensing of stochastic processes is studied first renyi s notion of information dimension id is generalized to continuous valued discrete time stationary processes this provides a measure of complexity for such processes and is connected to the rate of measurement sampling rate required for their accurate recovery then based on occam s razor a minimum entropy pursuit mep optimization approach for universal compressed sensing is proposed it is proven that for any stationary process satisfying certain mixing conditions if the sampling rate is larger than the id of the source process mep optimization can reliably recover the source vector almost losslessly without having any prior information about its distribution then a lagrangian type relaxation of mep optimization problem referred to as lagrangian mep is studied it is shown that lagrangian mep is identical to an implementable algorithm proposed by baron and coauthors and for the right choice of parameters has the same asymptotic performance as mep optimization finally it is proven that lagrangian mep is robust to small measurement noise

achieving the rate distortion bound with low density generator matrix codes it is shown that binary low density generator matrix codes can achieve the rate distortion bound of discrete memoryless sources with general distortion measure via multilevel quantization a practical encoding scheme based on the survey propagation algorithm is proposed the effectiveness of the proposed scheme is verified through simulation

approximate message passing algorithm with universal denoising and gaussian mixture learning we study compressed sensing cs signal reconstruction problems where an input signal is measured via matrix multiplication under additive white gaussian noise our signals are assumed to be stationary and ergodic but the input statistics are unknown the goal is to provide reconstruction algorithms that are universal to the input statistics we present a novel algorithmic framework that combines 1 the approximate message passing cs reconstruction framework which solves the matrix channel recovery problem by iterative scalar channel denoising 2 a universal denoising scheme based on context quantization which partitions the stationary ergodic signal denoising into independent and identically distributed i i d subsequence denoising and 3 a density estimation approach that approximates the probability distribution of an i i d sequence by fitting a gaussian mixture gm model in addition to the algorithmic framework we provide three contributions 1 numerical results showing that state evolution holds for nonseparable bayesian sliding window denoisers 2 an i i d denoiser based on a modified gm learning algorithm and 3 a universal denoiser that does not need information about the range where the input takes values from or require the input signal to be bounded we provide two implementations of our universal cs recovery algorithm with one being faster and the other being more accurate the two implementations compare favorably with existing universal reconstruction algorithms in terms of both reconstruction quality and runtime

triplet markov fields for the classification of complex structure data we address the issue of classifying complex data we focus on three main sources of complexity namely the high dimensionality of the observed data the dependencies between these observations and the general nature of the noise model underlying their distribution we investigate the recent triplet markov fields and propose new models in this class designed for such data and in particular allowing very general noise models in addition our models can handle the inclusion of a learning step in a consistent way so that they can be used in a supervised framework one advantage of our models is that whatever the initial complexity of the noise model parameter estimation can be carried out using state of the art bayesian clustering techniques under the usual simplifying assumptions as generative models they can be seen as an alternative in the supervised case to discriminative conditional random fields identifiability issues underlying the models in the nonsupervised case are discussed while the models performance is illustrated on simulated and real data exhibiting the mentioned various sources of complexity

two dimensional discrete breathers construction stability and bifurcations we develop a methodology for the construction of two dimensional discrete breather excitations application to the discrete nonlinear schroedinger equation on a square lattice reveals three different types of breathers considering an elementary plaquette the most unstable mode is centered on the plaquette the most stable mode is centered on its vertices while the intermediate but also unstable mode is centered at the middle of one of the edges below the turning points of each branch in a frequency power phase diagram the construction methodology fails and a continuation method is used to obtain the unstable branches of the solutions until a triple point is reached at this triple point the branches meet and subsequently bifurcate into the final state of an extended phonon mode c 2000 the american physical society

an mdl framework for sparse coding and dictionary learning the power of sparse signal modeling with learned overcomplete dictionaries has been demonstrated in a variety of applications and fields from signal processing to statistical inference and machine learning however the statistical properties of these models such as underfitting or overfitting given sets of data are still not well characterized in the literature as a result the success of sparse modeling depends on hand tuning critical parameters for each data and application this work aims at addressing this by providing a practical and objective characterization of sparse models by means of the minimum description length mdl principle a well established information theoretic approach to model selection in statistical inference the resulting framework derives a family of efficient sparse coding and dictionary learning algorithms which by virtue of the mdl principle are completely parameter free furthermore such framework allows to incorporate additional prior information to existing models such as markovian dependencies or to define completely new problem formulations including in the matrix analysis area in a natural way these virtues will be demonstrated with parameter free algorithms for the classic image denoising and classification problems and for low rank matrix recovery in video applications however the framework is not limited to this imaging data and can be applied to a wide range of signal and data types and tasks

em and beyond the basic theme of the em algorithm to repeatedly use complete data methods to solve incomplete data problems is also a theme of several more recent statistical techniques these techniques multiple imputation data augmentation stochastic relaxation and sampling importance resampling combine simulation techniques with complete data methods to attack problems that are difficult or impossible for em

compressive imaging via approximate message passing with image denoising we consider compressive imaging problems where images are reconstructed from a reduced number of linear measurements our objective is to improve over existing compressive imaging algorithms in terms of both reconstruction error and runtime to pursue our objective we propose compressive imaging algorithms that employ the approximate message passing amp framework amp is an iterative signal reconstruction algorithm that performs scalar denoising at each iteration in order for amp to reconstruct the original input signal well a good denoiser must be used we apply two wavelet based image denoisers within amp the first denoiser is the amplitude scale invariant bayes estimator abe and the second is an adaptive wiener filter we call our amp based algorithms for compressive imaging amp abe and amp wiener numerical results show that both amp abe and amp wiener significantly improve over the state of the art in terms of runtime in terms of reconstruction quality amp wiener offers lower mean square error mse than existing compressive imaging algorithms in contrast amp abe has higher mse because abe does not denoise as well as the adaptive wiener filter

digital image restoration using spatial interaction models this paper is concerned with developing fast nonrecursive algorithms for the minimum mean squared error restoration of degraded images the degradation is assumed to be due to a space invariant periodic nonseparable known point spread function and additive white noise our basic approach is to represent the images by a class of spatial interaction models namely the simultaneous autoregressve models and the conditional markov models defined on toroidal lattices and develop minimum mean squared error restoration algorithms using these models the restoration algorithms are optimal if the parameters characterizing the interaction models are exactly known however in practice the parameters are estimated from the images by using spatial interaction models we develop restoration algorithms that do not require the availability of the original image or its prototype the specific structure of the underlying lattice enables the implementation of the filters using fast fourier transform fft computations several restoration examples are given

recovery from linear measurements with complexity matching universal signal estimation we study the compressed sensing cs signal estimation problem where an input signal is measured via a linear matrix multiplication under additive noise while this setup usually assumes sparsity or compressibility in the input signal during recovery the signal structure that can be leveraged is often not known a priori in this paper we consider universal cs recovery where the statistics of a stationary ergodic signal source are estimated simultaneously with the signal itself inspired by kolmogorov complexity and minimum description length we focus on a maximum a posteriori map estimation framework that leverages universal priors to match the complexity of the source our framework can also be applied to general linear inverse problems where more measurements than in cs might be needed we provide theoretical results that support the algorithmic feasibility of universal map estimation using a markov chain monte carlo implementation which is computationally challenging we incorporate some techniques to accelerate the algorithm while providing comparable and in many cases better reconstruction quality than existing algorithms experimental results show the promise of universality in cs particularly for low complexity sources that do not exhibit standard sparsity or compressibility

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

discrete breathers in a two dimensional spring mass lattice we consider a two dimensional spring mass lattice with square symmetry in which each particle experiences a nonlinear onsite potential and nonlinear nearest neighbour interactions at equilibrium the particles are equally spaced in both the horizontal and vertical directions and all springs are unextended motivated by the work of marin et al 1998 phys lett a 248 225 9 2001 phys lett a 281 21 5 we seek a solution in which most of the breather s energy is focused along three chains we construct an asymptotic approximation to the breather using the method of multiple scales to describe the coherent oscillations in the three main chains that constitute the discrete breather we reduce the equation of motion to a nonlinear schrodinger equation for the leading order term and find a family of solutions which encompasses both stationary and moving bright soliton solutions we use numerical simulations of the lattice to verify the shape and velocity of breathers and find that while stationary breathers are found to persist for long times moving breathers decay by radiating energy in the direction perpendicular to their motion

geometric invariants and object recognition we discuss the role of the general invariance concept in object recognition and review the classical and recent literature on projective invariance invariants help solve major problems of object recognition for instance different images of the same object often differ from each other because of the different viewpoint from which they were obtained to match the two images common methods thus need to find the correct viewpoint a difficult problem that can involve search in a high dimensional space of all possible points of view and or finding point correspondences geometric invariants are shape descriptors computed from the geometry of the shape that remain unchanged under geometric transformations such as changing the viewpoint thus they can be matched without search deformations of objects are another important class of geometric changes for which invariance is useful

breathers in nonlinear lattices existence linear stability and quantization abstract it has been proved by the principle of anti integrable limit renamed here anti continuous limit and the implicit function theorem that under rather weak hypotheses there exist spatially localized and time periodic solutions breathers in arrays of nonlinear coupled classical oscillators provided the coupling is not too strong the models can be translationally invariant or not and be of any dimension there are also time periodic solutions corresponding to arbitrary distributions on the lattice of such solutions multibreathers the whole set of these solutions can be labeled by a discrete coding sequence the condition that the frequency of the breather or multibreather solution has no harmonics in the phonon spectrum can be discarded for a large subset of spatially extended time periodic solutions called phonobreather which correspond to special codes which can be also spatially chaotic and where no oscillator is at rest these results extend almost without change to models with coupled rotors the existence of rotobreathers which are exact solutions consisting of a single rotor rotating while the other rotors oscillate is proved there are also multirotobreather solutions and phono rotobreathers etc it is also shown that the phase of the single breathers constituting a multibreather or multirotobreather solution can be submitted to a phase torsion these new dynamical structures carry a nonzero energy flow in models with two dimensions and more the phase torsion of multibreather solutions can generate new dynamical solutions with vortices in the energy flow the dynamics of a quantum electron coupled to a classical lattice is also studied within the same approach from the anti continuous limit the existence of two kinds of polarobreathers which are spatially exponentially localized dynamical solutions is proved for a small enough electronic transfer integral the polarobreathers of the first kind include the standard static polaron the lattice configuration is static time independent when the electron is in an excited state i e the electronic oscillator oscillates periodically they can be spatially chaotic a polarobreather of the second kind corresponds to a localized periodic lattice oscillation associated with a localized quasiperiodic electronic oscillation the most simple of these solutions are those for which both the lattice oscillation and the electron wave function are mostly concentrated close to the same single site the standard floquet analysis of the linear stability of the breather and multibreather solutions is shown to be related with the band spectrum property of the newton operator involved in the implicit function theorem the krein theory of bifurcations is reinterpreted from this point of view the linear stability of the single breather and rotobreather solutions is then proved at least at small enough coupling finally it is shown that in the quantum version of translationally invariant classical models where there cannot exist in principle any strictly localized excitation there are narrow bands of excitations consisting of bound states of many bosons which are the quantum counterpart of the classical breathers the bandwidth of these excitations goes exponentially to zero with the number of bound phonons this theory provides simultaneously existence theorems and a new powerful and accurate numerical method for calculating practically any of the solutions which has been formally predicted to exist these numerical applications are currently written elsewhere

numerical study of two dimensional disordered klein gordon lattices with cubic soft anharmonicity localized oscillations appear both in ordered nonlinear lattices breathers and in disordered linear lattices anderson modes numerical studies on a class of two dimensional systems of the klein gordon type show that there exist two different types of bifurcation in the path from nonlinearity order to linearity disorder inverse pitchforks with or without period doubling and saddle nodes this was discovered for a one dimensional system in a previous work of archilla mackay and marin the appearance of a saddle node bifurcation indicates that nonlinearity and disorder begin to interfere destructively and localization is not possible in contrast the appearance of a pitchfork bifurcation indicates that localization persists

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

segmentation given partial grouping constraints we consider data clustering problems where partial grouping is known a priori we formulate such biased grouping problems as a constrained optimization problem where structural properties of the data define the goodness of a grouping and partial grouping cues define the feasibility of a grouping we enforce grouping smoothness and fairness on labeled data points so that sparse partial grouping information can be effectively propagated to the unlabeled data considering the normalized cuts criterion in particular our formulation leads to a constrained eigenvalue problem by generalizing the rayleigh ritz theorem to projected matrices we find the global optimum in the relaxed continuous domain by eigendecomposition from which a near global optimum to the discrete labeling problem can be obtained effectively we apply our method to real image segmentation problems where partial grouping priors can often be derived based on a crude spatial attentional map that binds places with common salient features or focuses on expected object locations we demonstrate not only that it is possible to integrate both image structures and priors in a single grouping process but also that objects can be segregated from the background without specific object knowledge

on the foundations of relaxation labeling processes a large class of problems can be formulated in terms of the assignment of labels to objects frequently processes are needed which reduce ambiguity and noise and select the best label among several possible choices relaxation labeling processes are just such a class of algorithms they are based on the parallel use of local constraints between labels this paper develops a theory to characterize the goal of relaxation labeling the theory is founded on a definition of con sistency in labelings extending the notion of constraint satisfaction in certain restricted circumstances an explicit functional exists that can be maximized to guide the search for consistent labelings this functional is used to derive a new relaxation labeling operator when the restrictions are not satisfied the theory relies on variational cal culus it is shown that the problem of finding consistent labelings is equivalent to solving a variational inequality a procedure nearly identical to the relaxation operator derived under restricted circum stances serves in the more general setting further a local convergence result is established for this operator the standard relaxation labeling formulas are shown to approximate our new operator which leads us to conjecture that successful applications of the standard methods are explainable by the theory developed here observations about con vergence and generalizations to higher order compatibility relations are described

markov random field texture models we consider a texture to be a stochastic possibly periodic two dimensional image field a texture model is a mathematical procedure capable of producing and describing a textured image we explore the use of markov random fields as texture models the binomial model where each point in the texture has a binomial distribution with parameter controlled by its neighbors and number of tries equal to the number of gray levels was taken to be the basic model for the analysis a method of generating samples from the binomial model is given followed by a theoretical and practical analysis of the method s convergence examples show how the parameters of the markov random field control the strength and direction of the clustering in the image the power of the binomial model to produce blurry sharp line like and blob like textures is demonstrated natural texture samples were digitized and their parameters were estimated under the markov random field model a hypothesis test was used for an objective assessment of goodness of fit under the markov random field model overall microtextures fit the model well the estimated parameters of the natural textures were used as input to the generation procedure the synthetic microtextures closely resembled their real counterparts while the regular and inhomogeneous textures did not

a mapping approach to rate distortion computation and analysis in rate distortion theory results are often derived and stated in terms of the optimizing density over the reproduction space in this paper the problem is reformulated in terms of the optimal mapping from the unit interval with lebesgue measure that would induce the desired reproduction probability density this results in optimality conditions that are random relatives of the known lloyd 1982 optimality conditions for deterministic quantizers the validity of the mapping approach is assured by fundamental isomorphism theorems for measure spaces we show that for the squared error distortion the optimal reproduction random variable is purely discrete at supercritical distortion where the shannon 1948 lower bound is not tight the gaussian source is thus the only source that produces continuous reproduction variables for the entire range of positive rate to analyze the evolution of the optimal reproduction distribution we use the mapping formulation and establish an analogy to statistical mechanics the solutions are given by the distribution at isothermal statistical equilibrium and are parameterized by the temperature in direct correspondence to the parametric solution of the variational equations in rate distortion theory the analysis of an annealing process shows how the number of symbols grows as the system undergoes phase transitions thus an algorithm based on the mapping approach often needs but a few variables to find the exact solution while the blahut 1972 algorithm would only approach it at the limit of infinite resolution finally a quick deterministic annealing algorithm to generate the rate distortion curve is suggested the resulting curve is exact as long as continuous phase transitions in the process are accurately followed

genetic algorithms for ambiguous labelling problems abstract consistent labelling problems frequently have more than one solution most work in the field has aimed at disambiguating early in the interpretation process using only local evidence this paper starts with a review of the literature on labelling problems and ambiguity based on this review we propose a strategy for simultaneously extracting multiple related solutions to the consistent labelling problem in a preliminary experimental study we show that an appropriately modified genetic algorithm is a robust tool for finding multiple solutions to the consistent labelling problem these solutions are related by common labellings of the most strongly constrained junctions we have proposed three run time measures of algorithm performance the maximum fitness of the genetic algorithm s population its shannon entropy and the total hamming distance between its distinct members the results to date indicate that when the shannon entropy falls below a certain threshold new solutions are unlikely to emerge and that most of the diversity in the population disappears within the first few generations

on the performance of data compression algorithms based upon string matching lossless and lossy data compression algorithms based on string matching are considered in the lossless case a result of wyner and ziv 1989 is extended in the lossy case a data compression algorithm based on approximate string matching is analyzed in the following two frameworks 1 the database and the source together form a markov chain of finite order 2 the database and the source are independent with the database coming from a markov model and the source from a general stationary ergodic model in either framework it is shown that the resulting compression rate converges with probability one to a quantity computable as the infimum of an information theoretic functional over a set of auxiliary random variables the quantity is strictly greater than the rate distortion function of the source except in some symmetric cases in particular this result implies that the lossy algorithm proposed by steinberg and gutman 1993 is not optimal even for memoryless or markov sources

nonparametric bayesian dictionary learning for analysis of noisy and incomplete images nonparametric bayesian methods are considered for recovery of imagery based upon compressive incomplete and or noisy measurements a truncated beta bernoulli process is employed to infer an appropriate dictionary for the data under test and also for image recovery in the context of compressive sensing significant improvements in image recovery are manifested using learned dictionaries relative to using standard orthonormal image expansions the compressive measurement projections are also optimized for the learned dictionary additionally we consider simpler incomplete measurements defined by measuring a subset of image pixels uniformly selected at random spatial interrelationships within imagery are exploited through use of the dirichlet and probit stick breaking processes several example results are presented with comparisons to other methods in the literature

invariant descriptors for 3d object recognition and pose invariant descriptors are shape descriptors that are unaffected by object pose by perspective projection or by the intrinsic parameters of the camera these descriptors can be constructed using the methods of invariant theory which are briefly surveyed a range of applications of invariant descriptors in 3d model based vision is demonstrated first a model based vision system that recognizes curved plane objects irrespective of their pose is demonstrated curves are not reduced to polyhedral approximations but are handled as objects in their own right models are generated directly from image data once objects have been recognized their pose can be computed invariant descriptors for 3d objects with plane faces are described all these ideas are demonstrated using images of real scenes the stability of a range of invariant descriptors to measurement error is treated in detail

an algorithm for source coding subject to a fidelity criterion based on string matching a practical suboptimal universal block source coding scheme subject to a fidelity criterion is proposed the algorithm is an extension of the lempel ziv algorithm and is based on string matching with distortion it is shown that given average distortion d 0 the algorithm achieves a rate of exceeding r d 2 for a large class of sources and distortion measures tighter bounds on the rate are derived for discrete memoryless sources and for memoryless gaussian sources

fixed slope universal lossy data compression corresponding to any lossless codeword length function l three universal lossy data compression schemes are presented one is with a fixed rate another is with a fixed distortion and a third is with a fixed slope the former two universal lossy data compression schemes are the generalization of yang kieffer s see ibid vol 42 no 1 p 239 45 1995 results to the general case of any lossless codeword length function l whereas the third is new in the case of fixed slope spl lambda 0 our universal lossy data compression scheme works as follows for any source sequence x sup n of length n the encoder first searches for a reproduction sequence y sup n of length n which minimizes a cost function n sup 1 l y sup n spl lambda spl rho sub n x sup n y sup n over all reproduction sequences of length n and then encodes x sup n into the binary codeword of length l y sup n associated with y sup n via the lossless codeword length function l where spl rho sub n x sup n y sup n is the distortion per sample between x sup n and y sup n under some mild assumptions on the lossless codeword length function l it is shown that when this fixed slope data compression scheme is applied to encode a stationary ergodic source the resulting encoding rate per sample and the distortion per sample converge with probability one to r sub spl lambda and d sub spl lambda respectively where d sub spl lambda r sub spl lambda is the point on the rate distortion curve at which the slope of the rate distortion function is spl lambda this result holds particularly for the arithmetic codeword length function and lempel ziv codeword length function the main advantage of this fixed slope universal lossy data compression scheme over the fixed rate fixed distortion universal lossy data compression scheme lies in the fact that it converts the encoding problem to a search problem through a trellis and then permits one to use some sequential search algorithms to implement it simulation results show that this fixed slope universal lossy data compression scheme combined with a suitable search algorithm is promising

3 d to 2 d pose determination with regions this paper presents a novel approach to parts based object recognition in the presence of occlusion we focus on the problem of determining the pose of a 3 d object from a single 2 d image when convex parts of the object have been matched to corresponding regions in the image we consider three types of occlusions self occlusion occlusions whose locus is identified in the image and completely arbitrary occlusions we show that in the first two cases this is a convex optimization problem derive efficient algorithms and characterize their performance for the last case we prove that the problem of finding valid poses is computationally hard but provide an efficient approximate algorithm this work generalizes our previous work on region based object recognition which focused on the case of planar models

adaptive wavelet thresholding for image denoising and compression the first part of this paper proposes an adaptive data driven threshold for image denoising via wavelet soft thresholding the threshold is derived in a bayesian framework and the prior used on the wavelet coefficients is the generalized gaussian distribution ggd widely used in image processing applications the proposed threshold is simple and closed form and it is adaptive to each subband because it depends on data driven estimates of the parameters experimental results show that the proposed method called bayesshrink is typically within 5 of the mse of the best soft thresholding benchmark with the image assumed known it also outperforms sureshrink donoho and johnstone 1994 1995 donoho 1995 most of the time the second part of the paper attempts to further validate claims that lossy compression can be used for denoising the bayesshrink threshold can aid in the parameter selection of a coder designed with the intention of denoising and thus achieving simultaneous denoising and compression specifically the zero zone in the quantization step of compression is analogous to the threshold value in the thresholding function the remaining coder design parameters are chosen based on a criterion derived from rissanen s minimum description length mdl principle experiments show that this compression method does indeed remove noise significantly especially for large noise power however it introduces quantization noise and should be used only if bitrate were an additional concern to denoising

image denoising by sparse 3 d transform domain collaborative filtering we propose a novel image denoising strategy based on an enhanced sparse representation in transform domain the enhancement of the sparsity is achieved by grouping similar 2d image fragments e g blocks into 3d data arrays which we call groups collaborative altering is a special procedure developed to deal with these 3d groups we realize it using the three successive steps 3d transformation of a group shrinkage of the transform spectrum and inverse 3d transformation the result is a 3d estimate that consists of the jointly filtered grouped image blocks by attenuating the noise the collaborative filtering reveals even the finest details shared by grouped blocks and at the same time it preserves the essential unique features of each individual block the filtered blocks are then returned to their original positions because these blocks are overlapping for each pixel we obtain many different estimates which need to be combined aggregation is a particular averaging procedure which is exploited to take advantage of this redundancy a significant improvement is obtained by a specially developed collaborative wiener filtering an algorithm based on this novel denoising strategy and its efficient implementation are presented in full detail an extension to color image denoising is also developed the experimental results demonstrate that this computationally scalable algorithm achieves state of the art denoising performance in terms of both peak signal to noise ratio and subjective visual quality

an on line universal lossy data compression algorithm via continuous codebook refinement i basic results a new on line universal lossy data compression algorithm is presented for finite memoryless sources with unknown statistics its performance asymptotically approaches the fundamental rate distortion limit the codebook is generated on the fly and continuously adapted by simple rules there is no separate codebook training or codebook transmission candidate codewords are randomly generated according to an arbitrary and possibly suboptimal distribution through a carefully designed gold washing or information theoretic sieve mechanism good codewords and only good codewords are promoted to permanent status with high probability we also determine the rate at which our algorithm approaches the fundamental limit

a common set of perceptual observables for grouping figure ground discrimination and texture classification we present a complete set of perceptual observables that provides a unified image description for grouping figure ground separation and texture analysis although much progress has been made recently in treating contours and texture simultaneously for image segmentation and grouping current approaches rely on different models for contours regions and texture such as one dimensional intensity discontinuities for contours and filter bank responses for texture this results in expensive computation that arbitrates between these disparate representations at each pixel in our approach salient image content such as contours regions and texture are represented in a common low level framework of image observables we model the image as a partition of surfaces bounded by intensity discontinuities and derive perceptual measures as relations between neighboring surfaces this enables us to extend the traditional gestalt measures based on local edge geometry and contrast to region based measures that jointly exploit large scale image topology photometry and geometry these measures provide a natural basis for grouping on multidimensional similarity criteria and texture is directly derived as relational properties on local region neighborhoods the viability of our model is demonstrated by applying the common observables to texture recognition figure ground separation and generic image segmentation

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

segmentation based on fusion of range and intensity images using robust trimmed methods abstract this paper proposes a segmentation algorithm based on fusion of range and intensity images using robust trimmed methods based on the bayesian theory a priori knowledge is represented using the markov random field mrf a maximum a posteriori map estimator is constructed using the edge features extracted from both range and intensity images objects are represented by a number of local planar surfaces in range images and the parametric space for surface representation is constructed with the surface parameters estimated pixel by pixel based on the least trimmed squares lts method whereas in intensity images the trimmed variance is adopted as the feature for edge extraction a final edge map is obtained by the map estimator that is constructed using the likelihood functions based on the edge information obtained from range and intensity images finally an image is segmented using the fused edge map computer simulation results show that our new segmentation algorithm effectively segments test images independent of shadow noise and lighting environment

block and sliding block lossy compression via mcmc we propose an approach to lossy compression of finite alphabet sources that utilizes markov chain monte carlo mcmc and simulated annealing methods the idea is to define an energy function over the space of reconstruction sequences the energy of a candidate reconstruction sequence is defined such that it incorporates its distortion relative to the source sequence its compressibility and the point sought on the rate distortion curve the proposed algorithm samples from the boltzmann distribution associated with this energy function using the heat bath algorithm the complexity of each iteration is independent of the sequence length and is only linearly dependent on a certain context parameter which grows sub logarithmically with the sequence length we show that the proposed algorithm achieves optimum rate distortion performance in the limits of large number of iterations and sequence length when employed on any stationary ergodic source inspired by the proposed block coding algorithm we also propose an algorithm for constructing sliding block sb codes using similar ideas

a universal data compression system a universal data compression algorithm is described which is capable of compressing long strings generated by a finitely generated source with a near optimum per symbol length without prior knowledge of the source this class of sources may be viewed as a generalization of markov sources to random fields moreover the algorithm does not require a working storage much larger than that needed to describe the source generating parameters

minimum complexity pursuit for universal compressed sensing the nascent field of compressed sensing is founded on the fact that high dimensional signals with simple structure can be recovered accurately from just a small number of randomized samples several specific kinds of structures have been explored in the literature from sparsity and group sparsity to low rankness however two fundamental questions have been left unanswered what are the general abstract meanings of structure and simplicity do there exist universal algorithms for recovering such simple structured objects from fewer samples than their ambient dimension in this paper we address these two questions using algorithmic information theory tools such as the kolmogorov complexity we provide a unified definition of structure and simplicity leveraging this new definition we develop and analyze an abstract algorithm for signal recovery motivated by occam s razor minimum complexity pursuit mcp requires approximately 2 randomized samples to recover a signal of complexity and ambient dimension n we also discuss the performance of the mcp in the presence of measurement noise and with approximately simple signals

fast acquisition and reconstruction of optical coherence tomography images via sparse representation in this paper we present a novel technique based on compressive sensing principles for reconstruction and enhancement of multi dimensional image data our method is a major improvement and generalization of the multi scale sparsity based tomographic denoising msbtd algorithm we recently introduced for reducing speckle noise our new technique exhibits several advantages over msbtd including its capability to simultaneously reduce noise and interpolate missing data unlike msbtd our new method does not require an a priori high quality image from the target imaging subject and thus offers the potential to shorten clinical imaging sessions this novel image restoration method which we termed sparsity based simultaneous denoising and interpolation sbsdi utilizes sparse representation dictionaries constructed from previously collected datasets we tested the sbsdi algorithm on retinal spectral domain optical coherence tomography images captured in the clinic experiments showed that the sbsdi algorithm qualitatively and quantitatively outperforms other state of the art methods

a suboptimal lossy data compression based on approximate pattern matching a practical suboptimal variable source coding algorithm for lossy data compression is presented this scheme is based on approximate string matching and it naturally extends the lossless lempel ziv 1977 data compression scheme among others we consider the typical length of an approximately repeated pattern within the first n positions of a stationary mixing sequence where d percent of mismatches is allowed we prove that there exists a constant r sub 0 d such that the length of such an approximately repeated pattern converges in probability to 1 r sub 0 d log n pr but it almost surely oscillates between 1 r sub spl infin d log n and 2 r sub 1 d log n where r sub spl infin d r sub 0 d r sub 1 d 2 are some constants these constants are natural generalizations of renyi entropies to the lossy environment more importantly we show that the compression ratio of a lossy data compression scheme based on such an approximate pattern matching is asymptotically equal to r sub 0 d we also establish the asymptotic behavior of the so called approximate waiting time n sub l which is defined as the time until a pattern of length c repeats approximately for the first time we prove that log n sub l l spl rarr r sub 0 d pr as l spl rarr spl infin in general r sub 0 d r d where r d is the rate distortion function thus for stationary mixing sequences we settle in the negative the problem investigated by steinberg and gutman by showing that a lossy extension of the wyner ziv 1989 scheme cannot be optimal

an implementable lossy version of the lempel ziv algorithm i optimality for memoryless sources a new lossy variant of the fixed database lempel ziv coding algorithm for encoding at a fixed distortion level is proposed and its asymptotic optimality and universality for memoryless sources with respect to bounded single letter distortion measures is demonstrated as the database size m increases to infinity the expected compression ratio approaches the rate distortion function the complexity and redundancy characteristics of the algorithm are comparable to those of its lossless counterpart a heuristic argument suggests that the redundancy is of order log log m log m and this is also confirmed experimentally simulation results are presented that agree well with this rate also the complexity of the algorithm is seen to be comparable to that of the corresponding lossless scheme we show that there is a tradeoff between compression performance and encoding complexity and we discuss how the relevant parameters can be chosen to balance this tradeoff in practice we also discuss the performance of the algorithm when applied to sources with memory and extensions to the cases of unbounded distortion measures and infinite reproduction alphabets

estimation and choice of neighbors in spatial interaction models of images some aspects of statistical inference for a class of spatial interaction models for finite images are presented primarily the simultaneous autoregressive sar models and conditional markov cm models each of these models is characterized by a set of neighbors a set of coefficients and a noise sequence of specified characteristics we are concerned with two problems the estimation of the unknown parameters in both sar and cm models and the choice of an appropriate model from a class of such competing models assuming gaussian distributed variables we discuss maximum likelihood ml estimation methods in general the ml scheme leads to nonlinear optimization problems to avoid excessive computation an iterative scheme is given for sar models which gives approximate ml estimates in the gaussian case and reasonably good estimates in some non gaussian situations as well likewise for cm models an easily computable consistent estimate is given the asymptotic mean squared error mse of this estimate for a four neighbor cm model is shown tn be substantially less than the mse of the popular coding estimate asymptotically consistent decision rules are given for choosing an appropriate sar or cm model the usefulness of the estimation scheme and the decision rule for the choice of neighbors is illustrated by using synthetic patterns synthetic patterns obeying known sar and cm models are generated and the models corresponding to true and several competing neighbor sets are fitted the estimation scheme yields estimates close to the parameters of the true models and the decision rule for the choice of neighbors picks up the true model from the class of competing models

rigid body segmentation and shape description from dense optical flow under weak perspective we present an algorithm for identifying and tracking independently moving rigid objects from optical flow some previous attempts at segmentation via optical flow have focused on finding discontinuities in the flow field while discontinuities do indicate a change in scene depth they do not in general signal a boundary between two separate objects the proposed method uses the fact that each independently moving object has a unique epipolar constraint associated with its motion thus motion discontinuities based on self occlusion can be distinguished from those due to separate objects the use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object the algorithm assumes an affine camera where perspective effects are limited to changes in overall scale no camera calibration parameters are required a kalman filter based approach is used for tracking motion parameters with time

projective alignment with regions we have previously proposed basri and jacobs 1999 and jacobs and basri 1999 an approach to recognition that uses regions to determine the pose of objects while allowing for partial occlusion of the regions regions introduce an attractive alternative to existing global and local approaches since unlike global features they can handle occlusion and segmentation errors and unlike local features they are not as sensitive to sensor errors and they are easier to match the region based approach also uses image information directly without the construction of intermediate representations such as algebraic descriptions which may be difficult to reliably compute we further analyze properties of the method for planar objects undergoing projective transformations in particular we prove that three visible regions are sufficient to determine the transformation uniquely and that for a large class of objects two regions are insufficient for this purpose however we show that when several regions are available the pose of the object can generally be recovered even when some or all regions are significantly occluded our analysis is based on investigating the flow patterns of points under projective transformations in the presence of fixed points