multimodality image registration by maximization of mutual information a new approach to the problem of multimodality medical image registration is proposed using a basic concept from information theory mutual information mi or relative entropy as a new matching criterion the method presented in this paper applies mi to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images which is assumed to be maximal if the images are geometrically aligned maximization of mi is a very general and powerful criterion because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved the accuracy of the mi criterion is validated for rigid body registration of computed tomography ct magnetic resonance mr and photon emission tomography pet images by comparison with the stereotactic registration solution while robustness is evaluated with respect to implementation issues such as interpolation and optimization and image content including partial overlap and image degradation our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation feature extraction or other preprocessing steps which makes this method very well suited for clinical applications

a progressive transmission capable diagnostically lossless compression scheme for 3d medical image sets this paper presents a novel and efficient diagnostically lossless compression for 3d medical image sets this compression scheme provides the 3d medical image sets with a progressive transmission capability an automated filter and threshold based preprocessing technique is used to remove noise outside the diagnostic region then a wavelet decomposition feature vector based approach is applied to determine the reference image for the entire 3d medical image set the selected reference image contains the most discernible anatomical structures within a relative large diagnostic region it is progressively encoded by a lossless embedded zerotree wavelet method so the validity of an entire set can be determined early this preprocessing technique is followed by an optimal predictor plus a 1st level integer wavelet transform to de correlate the 3d medical image set run length and arithmetic coding are used to further remove coding redundancy this diagnostically lossless compression method achieves an average compression of 2 1038 2 4292 and 1 6826 bits per pixel for three types of 3d magnetic resonance image sets the integrated progressive transmission capability degrades the compression performance by an average of 7 25 6 60 and 4 49 for the above three types moreover our compression without and with progressive transmission achieves better compression than the state of the art

multidimensional incremental parsing for universal source coding a multidimensional incremental parsing algorithm mdip for multidimensional discrete sources as a generalization of the lempel ziv coding algorithm is investigated it consists of three essential component schemes maximum decimation matching hierarchical structure of multidimensional source coding and dictionary augmentation as a counterpart of the longest match search in the lempel ziv algorithm two classes of maximum decimation matching are studied also an underlying behavior of the dictionary augmentation scheme for estimating the source statistics is examined for an m dimensional source m augmentative patches are appended into the dictionary at each coding epoch thus requiring the transmission of a substantial amount of information to the decoder the property of the hierarchical structure of the source coding algorithm resolves this issue by successively incorporating lower dimensional coding procedures in the scheme in regard to universal lossy source coders we propose two distortion functions the local average distortion and the local minimax distortion with a set of threshold levels for each source symbol for performance evaluation we implemented three image compression algorithms based upon the mdip one is lossless and the others are lossy the lossless image compression algorithm does not perform better than the lempel ziv welch coding but experimentally shows efficiency in capturing the source structure the two lossy image compression algorithms are implemented using the two distortion functions respectively the algorithm based on the local average distortion is efficient at minimizing the signal distortion but the images by the one with the local minimax distortion have a good perceptual fidelity among other compression algorithms our insights inspire future research on feature extraction of multidimensional discrete sources

one class classifiers based on entropic spanning graphs one class classifiers offer valuable tools to assess the presence of outliers in data in this paper we propose a design methodology for one class classifiers based on entropic spanning graphs our approach also takes into account the possibility to process nonnumeric data by means of an embedding procedure the spanning graph is learned on the embedded input data and the outcoming partition of vertices defines the classifier the final partition is derived by exploiting a criterion based on mutual information minimization here we compute the mutual information by using a convenient formulation provided in terms of the alpha jensen difference once training is completed in order to associate a confidence level with the classifier decision a graph based fuzzy model is constructed the fuzzification process is based only on topological information of the vertices of the entropic spanning graph as such the proposed one class classifier is suitable also for data characterized by complex geometric structures we provide experiments on well known benchmarks containing both feature vectors and labeled graphs in addition we apply the method to the protein solubility recognition problem by considering several representations for the input samples experimental results demonstrate the effectiveness and versatility of the proposed method with respect to other state of the art approaches

using spanning graphs for efficient image registration we provide a detailed analysis of the use of minimal spanning graphs as an alignment method for registering multimodal images this yields an efficient graph theoretic algorithm that for the first time jointly estimates both an alignment measure and a viable descent direction with respect to a parameterized class of spatial transformations we also show how prior information about the interimage modality relationship from prealigned image pairs can be incorporated into the graph based algorithm a comparison of the graph theoretic alignment measure is provided with more traditional measures based on plug in entropy estimators this highlights previously unrecognized similarities between these two registration methods our analysis gives additional insight into the tradeoffs the graph based algorithm is making and how these will manifest themselves in the registration algorithm s performance

overlap and channel errors in adaptive vector quantization for image coding adaptive vector quantization avq is a flexible adaptive single pass image compression method with fast decompression time for which practical compression achieved compares favorably with existing methods such as the jpeg standard and traditional trained vq in this paper we analyze its asymptotic performance for the lossless case and present variations to limit the effect of transmission errors on the decoding image

on the information rates of the plenoptic function the plenoptic function describes the visual information available to an observer at any point in space and time samples of the plenoptic function pof are seen in video and in general visual content images mosaics panoramic scenes etc and represent large amounts of information in this paper we propose a stochastic model to study the compression limits of a simplified version of the plenoptic function in the proposed framework we isolate the two fundamental sources of information in the pof the one representing the camera motion and the other representing the information complexity of the reality being acquired and transmitted the sources of information are combined generating a stochastic process that we study in detail we first propose a model for ensembles of realities that do not change over time the proposed model is simple in that it enables us to derive precise coding bounds in the information theoretic sense that are sharp in a number of cases of practical interest for this simple case of static realities and camera motion our results indicate that coding practice is in accordance with optimal coding from an information theoretic standpoint the model is further extended to account for visual realities that change over time we derive bounds on the lossless and lossy information rates for this dynamic reality model stating conditions under which the bounds are tight examples with synthetic sources suggest that within our proposed model common hybrid coding using motion displacement estimation with dpcm performs considerably suboptimally relative to the true rate distortion bound

universal fir mmse filtering we consider the problem of causal estimation i e filtering of a real valued signal corrupted by zero mean time independent real valued additive noise under the mean squared error mse criterion we build a universal filter whose per symbol squared error for every bounded underlying signal is essentially as small as that of the best finite duration impulse response fir filter of a given order we do not assume a stochastic mechanism generating the underlying signal and assume only that the variance of the noise is known to the filter the regret of the expected mse of our scheme is shown to decay as o logn n where n is the length of the signal moreover we present a stronger concentration result which guarantees the performance of our scheme not only in expectation but also with high probability our result implies a conventional stochastic setting result i e when the underlying signal is a stationary process our filter achieves the performance of the optimal fir filter we back our theoretical findings with several experiments showcasing the potential merits of our universal filter in practice our analysis combines tools from the problems of universal filtering and competitive online regression

mutually compatible gibbs random fields a unified theory for the mathematical description of gibbs random fields that answers some important theoretical and practical questions about their statistical behavior is presented the local transfer function is introduced and the joint probability measure of the general gibbs random field is derived in terms of this function the resulting probability structure is required to satisfy the property of mutual compatibility a necessary and sufficient condition for a gibbs random field to be mutually compatible is developed and used to prove that a mutually compatible gibbs random field is a unilateral markov random field the existence of some special nontrivial cases of gibbs random fields that are mutually compatible is demonstrated conditions on the translation invariance and isotropy of the general gibbs random field with a free boundary are studied the class of gibbs random fields with a homogeneous local transfer function and the class of horizontally and vertically translation invariant gibbs random fields are introduced and treated the concept of a translation invariant gibbs random field is also explored the problem of the statistical inference of mutually compatible gibbs random fields is discussed

information distance while kolmogorov 1965 complexity is the accepted absolute measure of information content in an individual finite object a similarly absolute notion is needed for the information distance between two individual objects for example two pictures we give several natural definitions of a universal information metric based on length of shortest programs for either ordinary computations or reversible dissipationless computations it turns out that these definitions are equivalent up to an additive logarithmic term we show that the information distance is a universal cognitive similarity distance we investigate the maximal correlation of the shortest programs involved the maximal uncorrelation of programs a generalization of the slepian wolf theorem of classical information theory and the density properties of the discrete metric spaces induced by the information distances a related distance measures the amount of nonreversibility of a computation using the physical theory of reversible computation we give an appropriate universal antisymmetric and transitive measure of the thermodynamic work required to transform one object in another object by the most efficient process information distance between individual objects is needed in pattern recognition where one wants to express effective notions of pattern similarity or cognitive similarity between individual objects and in thermodynamics of computation where one wants to analyze the energy dissipation of a computation from a particular input to a particular output

a coupling regeneration scheme for diagnosing convergence in markov chain monte carlo algorithms abstract here i propose a convergence diagnostic for markov chain monte carlo mcmc algorithms based on couplings of a markov chain with an auxiliary chain that is periodically restarted from a fixed parameter value the diagnostic provides a mechanism for estimating the specific constants governing the rate of convergence of geometrically and uniformly ergodic chains and provides a lower bound on the effective sample size of a mcmc run it also provides a simple procedure for obtaining what is with high probability an independent sample from the stationary distribution

fused trees and some new approaches to source coding the concept of fusion partition for the node set of a tree and the concept of a fused tree corresponding to it are discussed a fused tree requires in principle significantly less memory for storage than the original tree yet allows the recovery of the original hence it makes sense to store a tree in a fused form furthermore if labels from an ordered set c are assigned to the branches of the original tree so that a lexographic alphabetic order on the set of terminal nodes is induced a fused tree still enables the execution of direct and inverse enumeration algorithms a fusion algorithm to produce the completely fused tree for a given tree is presented some approaches to data compression problems related to the algorithms given are discussed

a combined alignment and registration scheme of lesions with psoriasis a scheme that registers and aligns digital image lesions of psoriasis within and between sessions is proposed lesions to be tracked are found under the assumption of being the object of largest size in thematic maps produced by a two step hierarchical classification scheme that uses the output of an expectation maximization algorithm to obtain a classification window of optimal size advantage is taken of the fact that the shape and the size of lesions with psoriasis do not change very much along the time a first alignment of the lesions is done assuming that the correspondence between points is given by equivalence of positions of pixels after translations and rotations finally a combined contextual registration and alignment scheme is applied the alignment and registration schemes both use an extreme value detection algorithm based on a retinal mapping model the output of the scheme is satisfactory not only in terms of visual appreciation of the aligned lesions but also because the variation within sessions in the aligned lesions resembles randomly distributed gaussian noise

iterative simulation methods abstract the standard methods of generating sample from univariate distributions often become hopelessly inefficient when applied to realizations of stochastic processes iterative methods are not widely known amongst statisticians but some are standard practice in statistical physics and chemistry the methods are surveyed and compared with particular reference to their convergence properties

statistical mechanics of image restoration we develop the statistical mechanics formulation of the image restoration problem pioneered by geman and geman 1984 using bayesian methods we establish the posterior probability distribution for restored images for given data corrupted image and prior assumptions about source and corruption process in the simplest cases studied here the posterior is controlled by a cost function analagous to the configurational energy of an ising model with local fields whose sense is defined by the data through a combination of monte carlo simulation and mean field theory we address three key issues first we explore the sensitivity of the posterior distribution to the choice of prior parameters we find phase transitions separating regions in which the distribution is effective data dominated from regions in which it is ineffective prior dominated second we examine the question of how best to use the posterior distribution to prescribe a single optimal restored image we argue that the mean of the posterior is in general to be preferred over the mode both in principle and in practice finally borrowing from monte carlo techniques for free energy calculations we address the question of prior parameter estimation within the evidence framework of gull 1989 and mackay 1992 our results suggest that parameters identified by this framework provide effective priors leading to optimal restoration only to the extent that the forms of the priors are well matched to the processes they claim to represent

discrete denoising with shifts we introduce s dude a new algorithm for denoising discrete memoryless channel dmc corrupted data the algorithm which generalizes the recently introduced dude discrete universal denoiser aims to compete with a genie that has access in addition to the noisy data also to the underlying clean data and that can choose to switch up to m times between sliding window denoisers in a way that minimizes the overall loss when the underlying data form an individual sequence we show that the s dude performs essentially as well as this genie provided that m is sublinear in the size of the data when the clean data are emitted by a piecewise stationary process we show that the s dude achieves the optimum distribution dependent performance provided that the same sublinearity condition is imposed on the number of switches to further substantiate the universal optimality of the s dude we show that when the number of switches is allowed to grow linearly with the size of the data any sequence of scheme s fails to compete in the above sense using dynamic programming we derive an efficient implementation of the s dude which has complexity time and memory growing linearly with the data size and the number of switches m preliminary experimental results are presented suggesting that s dude has the capacity to improve on the performance attained by the original dude in applications where the nature of the data abruptly changes in time or space as is often the case in practice

arakelov self intersection numbers of minimal regular models of modular curves x_0 p 2 x0 p2 we compute an asymptotic expression for the arakelov self intersection number of the relative dualizing sheaf of edixhoven s minimal regular model for the modular curve x 0 p 2 over mathbb q the computation of the self intersection numbers are used to prove an effective version of the bogolomov conjecture for the semi stable models of modular curves x 0 p 2 and obtain a bound on the stable faltings height for those curves in a companion article banerjee and chaudhuri in isr j math 2020

the cost of achieving the best portfolio in hindsight for a market with m assets consider the minimum over all possible sequences of asset prices through time n of the ratio of the final wealth of a nonanticipating investment strategy to the wealth obtained by the best constant rebalanced portfolio computed in hindsight for that price sequence we show that the maximum value of this ratio over all nonanticipating investment strategies is vn 2 nh n1 n nm n n1 nm 1 where h is the shannon entropy and we specify a strategy achieving it the optimal ratio vn is shown to decrease only polynomially in n indicating that the rate of return of the optimal strategy converges uniformly to that of the best constant rebalanced portfolio determined with full hindsight we also relate this result to the pricing of a new derivative security which might be called the hindsight allocation option

universal zero delay joint source channel coding we consider zero delay joint source channel coding of individual source sequences for a general known channel given an arbitrary finite set of schemes with finite memory not necessarily time invariant decoders a scheme is devised that does essentially as well as the best in the set on all individual source sequences using this scheme we construct a universal zero delay joint source channel coding scheme that is guaranteed to achieve asymptotically the performance of the best zero delay encoding decoding scheme with a finite state encoder and a markov decoder on all individual sequences for the case where the channel is a discrete memoryless channel dmc we construct an implementable zero delay joint source channel coding scheme that is based on the follow the perturbed leader scheme of gyoumlrgy for lossy source coding of individual sequences our scheme is guaranteed to attain asymptotically the performance of the best in the set of all encoding decoding schemes with a symbol by symbol decoder and arbitrary encoder on all individual sequences

on the sup norm of maass cusp forms of large level iii let f be a hecke maass cuspidal newform of square free level n and laplacian eigenvalue lambda it is shown that left f right infty ll lambda epsilon n frac 1 6 epsilon left f right 2 for any epsilon 0

universal switching linear least squares prediction in this paper we consider sequential regression of individual sequences under the square error loss we focus on the class of switching linear predictors that can segment a given individual sequence into an arbitrary number of blocks within each of which a fixed linear regressor is applied using a competitive algorithm framework we construct sequential algorithms that are competitive with the best linear regression algorithms for any segmenting of the data as well as the best partitioning of the data into any fixed number of segments where both the segmenting of the data and the linear predictors within each segment can be tuned to the underlying individual sequence the algorithms do not require knowledge of the data length or the number of piecewise linear segments used by the members of the competing class yet can achieve the performance of the best member that can choose both the partitioning of the sequence as well as the best regressor within each segment we use a transition diagram f m j willems 1996 to compete with an exponential number of algorithms in the class using complexity that is linear in the data length the regret with respect to the best member is o ln n per transition for not knowing the best transition times and o ln n for not knowing the best regressor within each segment where n is the data length we construct lower bounds on the performance of any sequential algorithm demonstrating a form of min max optimality under certain settings we also consider the case where the members are restricted to choose the best algorithm in each segment from a finite collection of candidate algorithms performance on synthetic and real data are given along with a matlab implementation of the universal switching linear predictor

universal prediction this paper consists of an overview on universal prediction from an information theoretic perspective special attention is given to the notion of probability assignment under the self information loss function which is directly related to the theory of universal data compression both the probabilistic setting and the deterministic setting of the universal prediction problem are described with emphasis on the analogy and the differences between results in the two settings

schemes for bidirectional modeling of discrete stationary sources we develop adaptive schemes for bidirectional modeling of unknown discrete stationary sources these algorithms can be applied to statistical inference problems such as noncausal universal discrete denoising that exploit bidirectional dependencies efficient algorithms for constructing those models are developed and we compare their performance to that of the dude algorithm for universal discrete denoising

a combined transform coding ctc scheme for medical images a combined transform coding ctc scheme is proposed to reduce the blocking artifact of conventional block transform coding and hence to improve the subjective performance the proposed ctc scheme is described and its information theoretic properties are investigated computer simulation results for a class of chest x ray images are presented a comparison between the ctc scheme and the conventional discrete cosine transform dct and discrete walsh hadamard transform dwht demonstrates the performance improvement of the proposed scheme in addition combined coding can also be used in noiseless coding yielding a slight improvement in the compression performance if it is used properly

points de petite hauteur sur les courbes modulaires x0 n let n be a square free integer prime to 6 let the imbeding of x 0 n in its jacobian relative to the point we show that the set is finite and that is infinite this explicit form of the bogomolov conjecture is obtained by an estimation of the self intersection of the dualizing sheaf in the sense of arakelov theory of modular curves this result is obtained by estimating several quantities attached to the arakelov metric on x 0 n starting with petersson s trace formula

efficient adaptive algorithms and minimax bounds for zero delay lossy source coding zero delay lossy source coding schemes are considered for both individual sequences and random sources performance is measured by the distortion redundancy which is defined as the difference between the normalized cumulative mean squared distortion of the scheme and the normalized cumulative distortion of the best scalar quantizer of the same rate that is matched to the entire sequence to be encoded by improving and generalizing a scheme of linder and lugosi weissman and merhav showed the existence of a randomized scheme that for any bounded individual sequence of length n achieves a distortion redundancy o n sup 1 3 logn however both schemes have prohibitive complexity both space and time which makes practical implementation infeasible in this paper we present an algorithm that computes weissman and merhav s scheme efficiently in particular we introduce an algorithm with encoding complexity o n sup 4 3 and distortion redundancy o n sup 1 3 logn the complexity can be made linear in the sequence length n at the price of increasing the distortion redundancy to o n sup 1 4 spl radic logn we also consider the problem of minimax distortion redundancy in zero delay lossy coding of random sources by introducing a simplistic scheme and proving a lower bound we show that for the class of bounded memoryless sources the minimax expected distortion redundancy is upper and lower bounded by constant multiples of n sup 1 2

a methodology for modeling the distributions of medical images and their stochastic properties the probabilistic distribution properties of a set of medical images are studied it is shown that the generalized gaussian function provides a good approximation to the distribution of ap chest radiographs based on this result and a goodness of fit test a generalized gaussian autoregressive model ggar is proposed its properties and limitations are also discussed it is expected that the ggar model will be useful in describing the stochastic characteristics of some classes of medical images and in image data compression and other applications

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

modified tests of independence in 2 x 2 tables with spatial data in this paper we suggest two modified chi squared tests of independence in 2 x 2 tables when the data are spatially autocorrelated for this purpose we first derive the asymptotic distribution of standard test statistics based on the sample cross product ratio under mild conditions on the nature of spatial dependence the performance of the tests is assessed by means of a monte carlo experiment in a lattice case from simulation results the need to adjust standard test statistics is apparent when spatial autocorrelation is present in both variables an illustrative application to ecological data is also given

universal filtering via prediction we consider the filtering problem where a finite alphabet individual sequence is corrupted by a discrete memoryless channel and the goal is to causally estimate each sequence component based on the past and present noisy observations we establish a correspondence between the filtering problem and the problem of prediction of individual sequences which leads to the following result given an arbitrary finite set of filters there exists a filter which performs with high probability essentially as well as the best in the set regardless of the underlying noiseless individual sequence we use this relationship between the problems to derive a filter guaranteed of attaining the finite state filterability of any individual sequence by leveraging results from the prediction problem

image registration by compression abstract image registration consists in finding the transformation that brings one image into the best possible spatial correspondence with another image in this paper we present a new framework for image registration based on compression the basic idea underlying our approach is the conjecture that two images are correctly registered when we can maximally compress one image given the information in the other the contribution of this paper is twofold first we show that image registration can be formulated as a compression problem second we demonstrate the good performance of the similarity metric introduced by li et al in image registration two different approaches for the computation of this similarity metric are described the kolmogorov version computed using standard real world compressors and the shannon version calculated from an estimation of the entropy rate of the images

hauteur de faltings de quotients de j 0 n discriminants d algebres de hecke et congruences entre formes modulaires in this paper we compute the faltings height of the jacobian of the modular curve j 0 n we explain how it is related to the discriminant of the hecke algebra this discriminant measures congruences between modular forms as a corollary we obtain a lower bound for this discriminant the faltings height of a quotient a of j 0 n is also computed

tracking the best quantizer an algorithm is presented for online prediction that allows to track the best expert efficiently even when the number of experts is exponentially large provided that the set of experts has a certain additive structure as an example we work out the case where each expert is represented by a path in a directed graph and the loss of each expert is the sum of the weights over the edges in the path these results are then used to construct universal limited delay schemes for lossy coding of individual sequences in particular we consider the problem of tracking the best scalar quantizer that is adaptively matched to the source sequence with piecewise different behavior a randomized algorithm is presented which can perform on any source sequence asymptotically as well as the best scalar quantization algorithm that is matched to the sequence and is allowed to change the employed quantizer for a given number of times the complexity of the algorithm is quadratic in the sequence length but at the price of some deterioration in performance the complexity can be made linear analogous results are obtained for sequential multiresolution and multiple description scalar quantization of individual sequences

two dimensional encoding by finite state encoders distortion free compressibility of individual pictures by finite state encoders is investigated in a recent paper see ieee trans inform theory vol 32 no 1 p 1 8 1986 the compressibility of a given picture i was defined and shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for i by any finite state encoder here a different and more direct approach is taken to prove similar results which are summarized in a converse to coding theorem and a constructive coding theorem that leads to a universal asymptotically optimal compression algorithm

hidden markov processes an overview of statistical and information theoretic aspects of hidden markov processes hmps is presented an hmp is a discrete time finite state homogeneous markov chain observed through a discrete time memoryless invariant channel in recent years the work of baum and petrie 1966 on finite state finite alphabet hmps was expanded to hmps with finite as well as continuous state spaces and a general alphabet in particular statistical properties and ergodic theorems for relative entropy densities of hmps were developed consistency and asymptotic normality of the maximum likelihood ml parameter estimator were proved under some mild conditions similar results were established for switching autoregressive processes these processes generalize hmps new algorithms were developed for estimating the state parameter and order of an hmp for universal coding and classification of hmps and for universal decoding of hidden markov channels these and other related topics are reviewed

ml parameter estimation for markov random fields with applications to bayesian tomography markov random fields mrfs have been widely used to model images in bayesian frameworks for image reconstruction and restoration typically these mrf models have parameters that allow the prior model to be adjusted for best performance however optimal estimation of these parameters sometimes referred to as hyperparameters is difficult in practice for two reasons i direct parameter estimation for mrfs is known to be mathematically and numerically challenging ii parameters can not be directly estimated because the true image cross section is unavailable we propose a computationally efficient scheme to address both these difficulties for a general class of mrf models and we derive specific methods of parameter estimation for the mrf model known as generalized gaussian mrf ggmrf we derive methods of direct estimation of scale and shape parameters for a general continuously valued mrf for the ggmrf case we show that the ml estimate of the scale parameter spl sigma has a simple closed form solution and we present an efficient scheme for computing the ml estimate of the shape parameter p by an off line numerical computation of the dependence of the partition function on p we present a fast algorithm for computing ml parameter estimates when the true image is unavailable to do this we use the expectation maximization em algorithm we develop a fast simulation method to replace the e step and a method to improve the parameter estimates when the simulations are terminated prior to convergence experimental results indicate that our fast algorithms substantially reduce the computation and result in good scale estimates for real tomographic data sets

universal denoising of discrete time continuous amplitude signals we consider the problem of reconstructing a discrete time signal sequence with continuous valued components corrupted by a known memoryless channel when performance is measured using a per symbol loss function satisfying mild regularity conditions we develop a sequence of denoisers that although independent of the distribution of the underlying ldquocleanrdquo sequence is universally optimal in the limit of large sequence length this sequence of denoisers is universal in the sense of performing as well as any sliding window denoising scheme which may be optimized for the underlying clean signal our results are initially developed in a ldquosemi stochasticrdquo setting where the noiseless signal is an unknown individual sequence and the only source of randomness is due to the channel noise it is subsequently shown that in the fully stochastic setting where the noiseless sequence is a stationary stochastic process our schemes universally attain optimum performance the proposed schemes draw from nonparametric density estimation techniques and are practically implementable we demonstrate efficacy of the proposed schemes in denoising gray scale images in the conventional additive white gaussian noise awgn setting with additional promising results for less conventional noise distributions

the weighted majority algorithm we study the construction of prediction algorithms in a situation in which a learner faces a sequence of trials with a prediction to be made in each and the goal of the learner is to make few mistakes we are interested in the case where the learner has reason to believe that one of some pool of known algorithms will perform well but the learner does not know which one a simple and effective method based on weighted voting is introduced for constructing a compound algorithm in such a circumstance we call this method the weighted majority algorithm we show that this algorithm is robust in the presence of errors in the data we discuss various versions of the weighted majority algorithm and prove mistake bounds for them that are closely related to the mistake bounds of the best algorithms of the pool for example given a sequence of trials if there is an algorithm in the pool a that makes at most m mistakes then the weighted majority algorithm will make at most c log a m mistakes on that sequence where c is fixed constant

on the symmetric information rate of two dimensional finite state isi channels we derive a pair of bounds upper and lower on the symmetric information rate of a two dimensional finite state intersymbol interference isi channel model for channels with small impulse response support they can be estimated via a modified forward recursion of the bahl cocke jelinek raviv bcjr algorithm the convergence of the bounds is also analyzed to relax the constraint on the size of the impulse response a new upper bound is proposed which allows the tradeoff of the computational complexity and the tightness of the bound these bounds are further extended to d dimensional d 2 isi channels

universal denoising for the finite input general output channel we consider the problem of reconstructing a finite alphabet signal corrupted by a known memoryless channel with a general output alphabet the goodness of the reconstruction is measured by a given loss function we constructively establish the existence of a universal sequence of denoiser s attaining asymptotically the optimum distribution dependent performance for any stationary source that may be generating the noiseless signal we show in fact that there is a whole family of denoiser sequences with this property these schemes are shown to be universal also in a semistochastic setting where the only randomness assumed is that associated with the channel noise the scheme is practical requiring o n sup 1 spl epsiv operations for any spl epsiv 0 and working storage size sublinear in the input data length this extends recent work that presented a discrete universal denoiser for recovering a discrete source corrupted by a discrete memoryless channel dmc

twice universal denoising we propose a sequence of universal denoisers motivated by the goal of extending the notion of twice universality from universal data compression theory to the sliding window denoising setting given a sequence length n and a denoiser the kth order regret of the latter is the maximum excess expected denoising loss relative to sliding window denoisers with window length 2k 1 where for a given clean sequence the expectation is over all channel realizations and the maximum is over all clean sequences of length n we define the twice universality penalty of a denoiser as its excess kth order regret when compared to a bound on the kth order regret of the denoising algorithm dude with parameter k and we are interested in denoisers with a negligible penalty for all k simultaneously we consider a class of denoisers that apply one of a number of constituent denoisers based on minimizing an estimated denoising loss and establish a formal relationship between the error in the estimated denoising loss and the twice universality penalty of the resulting denoiser given a sequence of window parameters kn increasing in n sufficiently fast we use this approach to construct and analyze a specific sequence of denoisers that achieves a much smaller twice universality penalty for k kn than the sequence of dude denoisers with parameter kn

markov random field texture models we consider a texture to be a stochastic possibly periodic two dimensional image field a texture model is a mathematical procedure capable of producing and describing a textured image we explore the use of markov random fields as texture models the binomial model where each point in the texture has a binomial distribution with parameter controlled by its neighbors and number of tries equal to the number of gray levels was taken to be the basic model for the analysis a method of generating samples from the binomial model is given followed by a theoretical and practical analysis of the method s convergence examples show how the parameters of the markov random field control the strength and direction of the clustering in the image the power of the binomial model to produce blurry sharp line like and blob like textures is demonstrated natural texture samples were digitized and their parameters were estimated under the markov random field model a hypothesis test was used for an objective assessment of goodness of fit under the markov random field model overall microtextures fit the model well the estimated parameters of the natural textures were used as input to the generation procedure the synthetic microtextures closely resembled their real counterparts while the regular and inhomogeneous textures did not

on the metric properties of discrete space filling curves a space filling curve is a linear traversal of a discrete finite multidimensional space in order for this traversal to be useful in many applications the curve should preserve locality we quantify locality and bound the locality of multidimensional space filling curves classic hilbert space filling curves come close to achieving optimal locality

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

an empirical study of the simulation of various models used for images markov random fields are typically used as priors in bayesian image restoration methods to represent spatial information in the image commonly used markov random fields are not in fact capable of representing the moderate to large scale clustering present in naturally occurring images and can also be time consuming to simulate requiring iterative algorithms which can take hundreds of thousands of sweeps of the image to converge markov mesh models a causal subclass of markov random fields are however readily simulated we describe an empirical study of simulated realizations from various models used in the literature and we introduce some new mesh type models we conclude however that while large scale clustering may be represented by such models strong directional effects are also present for all but very limited parameterizations it is emphasized that the results do not detract from the use of markov random fields as representers of local spatial properties which is their main purpose in the implementation of bayesian statistical approaches to image analysis brief allusion is made to the issue of parameter estimation

bounding sup norms of cusp forms of large level let f be an l2 normalized weight zero hecke maas cusp form of square free level n character and laplacian eigenvalue 1 4 it is shown that f n 1 37 from which the hybrid bound f 1 4 n for some 0 is derived the first bound holds also for f yk 2f where f is a holomorphic cusp form of weight k with the implied constant now depending on k

inference for discrete markov fields the simplest nontrivial case abstract markov fields provide a general context for describing the strength and structure of spatial interactions the gibbs markov equivalence theorem preston 1974 parameterizes markov fields via their neighborhood structures yielding exponential families in canonical form likelihood inference is therefore apparently straightforward the requisite normalizing constants however are obstreperous even when asymptotic characterizations can be obtained substantial location errors arise during implementation moreover markov fields can exhibit phase transitions and long range interactions thereby creating identifiability problems these issues are illustrated in the simplest nontrivial case the classical ising model of ferromagnetism

prediction of spatial cumulative distribution functions using subsampling abstract the spatial cumulative distribution function scdf is a random function that provides a statistical summary of a random field over a spatial domain of interest in this article we develop a spatial subsampling method for predicting an scdf based on observations made on a hexagonal grid similar to the one used in the environmental monitoring and assessment program of the u s environmental protection agency we show that under quite general conditions the proposed subsampling method provides accurate data based approximations to the sampling distributions of various functionals of the scdf predictor in particular it produces estimators of different population characteristics such as the quantiles and weighted mean integrated squared errors of the empirical predictor as an illustration we apply the subsampling method to construct large sample prediction bands for the scdf of an ecological index for foliage condition of red maple trees in the state of maine

structural information in two dimensional patterns entropy convergence and excess entropy we develop information theoretic measures of spatial structure and pattern in more than one dimension as is well known the entropy density of a two dimensional configuration can be efficiently and accurately estimated via a converging sequence of conditional entropies we show that the manner in which these conditional entropies converge to their asymptotic value serves as a measure of global correlation and structure for spatial systems in any dimension we compare and contrast entropy convergence with mutual information and structure factor techniques for quantifying and detecting spatial structure

the similarity metric a new class of distances appropriate for measuring similarity relations between sequences say one type of similarity per distance is studied we propose a new normalized information distance based on the noncomputable notion of kolmogorov complexity and show that it is in this class and it minorizes every computable distance in the class that is it is universal in that it discovers all computable similarities we demonstrate that it is a metric and call it the similarity metric this theory forms the foundation for a new practical tool to evidence generality and robustness we give two distinctive applications in widely divergent areas using standard compression programs like gzip and gencompress first we compare whole mitochondrial genomes and infer their evolutionary history this results in a first completely automatic computed whole mitochondrial phylogeny tree secondly we fully automatically compute the language tree of 52 different languages

bergman kernels on punctured riemann surfaces in this paper we consider a punctured riemann surface endowed with a hermitian metric that equals the poincare metric near the punctures and a holomorphic line bundle that polarizes the metric we show that the bergman kernel can be localized around the singularities and its local model is the bergman kernel of the punctured unit disc endowed with the standard poincare metric one of the technical tools is a new weighted elliptic estimate near the punctures which is uniform with respect to the tensor power as a consequence we obtain an optimal uniform estimate of the supremum norm of the bergman kernel involving a fractional growth order of the tensor power this holds in particular for the bergman kernel of cusp forms of high weight of non cocompact geometrically finite fuchsian groups of first kind without elliptic elements

stationary markov random fields on a finite rectangular lattice this paper provides a complete characterization of stationary markov random fields on a finite rectangular non toroidal lattice in the basic case of a second order neighborhood system equivalently it characterizes stationary markov fields on z sup 2 whose restrictions to finite rectangular subsets are still markovian i e even on the boundaries until now pickard random fields formed the only known class of such fields first we derive a necessary and sufficient condition for markov random fields on a finite lattice to be stationary it is shown that their joint distribution factors in terms of the marginal distribution on a generic 2 spl times 2 cell which must fulfil some consistency constraints second we solve the consistency constraints and provide a complete characterization of such measures in three cases symmetric measures and gaussian measures are shown to necessarily belong to the pickard class whereas binary measures belong either to the pickard class or to a new nontrivial class which is further studied in particular the corresponding fields admit a simple parameterization and may be simulated in a simple although nonunilateral manner

a local update strategy for iterative reconstruction from projections a method for bayesian reconstruction which relies on updates of single pixel values rather than the entire image at each iteration is presented the technique is similar to gauss seidel gs iteration for the solution of differential equations on finite grids the computational cost per iteration of the gs approach is found to be approximately equal to that of gradient methods for continuously valued images gs is found to have significantly better convergence at modes representing high spatial frequencies in addition gs is well suited to segmentation when the image is constrained to be discretely valued it is shown that bayesian segmentation using gs iteration produces useful estimates at much lower signal to noise ratios than required for continuously valued reconstruction the convergence properties of gradient ascent and gs for reconstruction from integral projections are analyzed and simulations of both maximum likelihood and maximum a posteriori cases are included

compression of two dimensional data distortion free compressibility of individual pictures i e two dimensional arrays of data by finite state encoders is investigated for every individual infinite picture i a quantity rho i is defined called the compressibility of i which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for i by any finite state information lossless encoder this is demonstrated by means of a constructive coding theorem and its converse that apart from their asymptotic significance might also provide useful criteria for finite and practical data compression tasks the proposed picture compressibility is also shown to possess the properties that one would expect and require of a suitably defined concept of two dimensional entropy for arbitrary probabilistic ensembles of infinite pictures while the definition of rho i allows the use of different machines for different pictures the constructive coding theorem leads to a universal compression scheme that is asymptotically optimal for every picture the results are readily extendable to data arrays of any finite dimension

bounds on faltings s delta function through covers let x be a compact riemann surface of genus gx 1 in 1984 g faltings introduced a new invariant fal x associated to x in this paper we give explicit bounds for fal x in terms of fundamental dierential geometric invariants arising from x when gx 1 as an application we are able to give bounds for faltings s delta function for the family of modular curves x0 n in terms of the genus only in combination with work of a abbes p michel and e ullmo this leads to an asymptotic formula for the faltings height of the jacobian j0 n associated to x0 n

scanning and sequential decision making for multidimensional data part i the noiseless case we investigate the problem of scanning and prediction ldquoscandiction rdquo for short of multidimensional data arrays this problem arises in several aspects of image and video processing such as predictive coding for example where an image is compressed by coding the error sequence resulting from scandicting it thus it is natural to ask what is the optimal method to scan and predict a given image what is the resulting minimum prediction loss and whether there exist specific scandiction schemes which are universal in some sense specifically we investigate the following problems first modeling the data array as a random field we wish to examine whether there exists a scandiction scheme which is independent of the field s distribution yet asymptotically achieves the same performance as if this distribution were known this question is answered in the affirmative for the set of all spatially stationary random fields and under mild conditions on the loss function we then discuss the scenario where a nonoptimal scanning order is used yet accompanied by an optimal predictor and derive bounds on the excess loss compared to optimal scanning and prediction this paper is the first part of a two part paper on sequential decision making for multidimensional data it deals with clean noiseless data arrays the second part deals with noisy data arrays namely with the case where the decision maker observes only a noisy version of the data yet it is judged with respect to the original clean data

scan predictive vector quantization of multispectral images conventional vector quantization vq based techniques partition an image into nonoverlapping blocks that are then raster scanned and quantized image blocks that contain an edge result in high frequency vectors the coarse representation of such vectors leads to visually annoying degradations in the reconstructed image the authors present a solution to the edge degradation problem based on some earlier work on scan models the approach reduces the number of vectors with abrupt intensity variations by using an appropriate scan to partition an image into vectors they show how their techniques can be used to enhance the performance of vq of multispectral data sets comparisons with standard techniques are presented and shown to give substantial improvements

computational mechanics pattern and prediction structure and simplicity computational mechanics an approach to structural complexity defines a process s causal states and gives a procedure for finding them we show that the causal state representation an machine is the minimal one consistent with accurate prediction we establish several results on machine optimality and uniqueness and on how machines compare to alternative representations further results relate measures of randomness and structural complexity obtained from machines to those from ergodic and information theories

how to filter an individual sequence with feedback we consider causally estimating filtering the components of a noise corrupted sequence relative to a reference class of filters the noiseless sequence to be filtered is designed by a ldquowell informed antagonist rdquo meaning it may evolve according to an arbitrary law unknown to the filter based on past noiseless and noisy sequence components we show that this setting is more challenging than that of an individual noiseless sequence a k a the ldquosemi stochasticrdquo setting in the sense that any deterministic filter even one guaranteed to do well on every noiseless individual sequence fails under some well informed antagonist on the other hand we constructively establish the existence of a randomized filter which successfully competes with an arbitrary given finite reference class of filters under every antagonist thus unlike in the semi stochastic setting randomization is crucial in the antagonist framework our noise model allows for channels whose noisy output depends on the l past channel outputs in addition to the noiseless channel input symbol memoryless channels are obtained as a special case of our model by taking i 0 in this case our scheme coincides with one that was recently shown to compete with an arbitrary reference class when the underlying noiseless sequence is an individual sequence hence our results show that the latter scheme is universal not only for the semi stochastic setting in which it was originally proposed but also under the well informed antagonist

an analysis of some common scanning techniques for lossless image coding though most image coding techniques use a raster scan to order pixels prior to coding hilbert and other scans have been proposed as having better performance due to their superior locality preserving properties however a general understanding of the merits of various scans has been lacking this paper develops an approach for quantitatively analyzing the effect of pixel scan order for context based predictive lossless image compression and uses it to compare raster hilbert random and hierarchical scans specifically for a quantized gaussian image model and a given scan order it shows how the encoding rate can be estimated from the frequencies with which various pixel configurations are available as previously scanned contexts and from the corresponding conditional differential entropies formulas are derived for such context frequencies and entropies assuming an isotropic image model and contexts consisting of previously scanned adjacent pixels it is found that the raster scan is better than the hilbert scan which is often used in compression applications due to its locality preserving properties the hierarchical scan is better still though it is based on nonadjacent contexts the random scan is the worst of the four considered extensions and implications of the results to lossy coding are also discussed

a universal data compression system a universal data compression algorithm is described which is capable of compressing long strings generated by a finitely generated source with a near optimum per symbol length without prior knowledge of the source this class of sources may be viewed as a generalization of markov sources to random fields moreover the algorithm does not require a working storage much larger than that needed to describe the source generating parameters

universal minimax discrete denoising under channel uncertainty the goal of a denoising algorithm is to recover a signal from its noise corrupted observations perfect recovery is seldom possible and performance is measured under a given single letter fidelity criterion for discrete signals corrupted by a known discrete memoryless channel dmc the discrete universal denoiser dude was recently shown to perform this task asymptotically optimally without knowledge of the statistical properties of the source in the present work we address the scenario where in addition to the lack of knowledge of the source statistics there is also uncertainty in the channel characteristics we propose a family of discrete denoisers and establish their asymptotic optimality under a minimax performance criterion which we argue is appropriate for this setting as we show elsewhere the proposed schemes can also be implemented computationally efficiently

removing mismatches for retinal image registration via multi attribute driven regularized mixture model in order to address the problem of retinal image registration this paper proposes and analyzes a novel and general matching algorithm called multi attribute driven regularized mixture model mad rmm mismatches removal can play a key role in image registration which refers to establish reliable matches between two point sets here the presented approach starts from multi feature attributes which are used to guide the feature matching to identify inliers correct matches from outliers incorrect matches and then estimates the spatial transformation in this paper motivated by the problem of feature matching that the initial correspondence is always contaminated by outliers thereby we formulate this issue as a probability deformable mixture model which consists of gaussian components for inliers and uniform components for outliers moreover the algorithm takes full advantage of using multiple attributes for better general matching performance here we are assuming all inliers are mapped into a high dimensional feature space namely reproducing kernel hilbert space rkhs and the closed form solution to the mapping function is given by the representation theorem with l2 norm regularization under the expectation maximization em algorithm finally we evaluate the performance of the algorithm by applying it to retinal image registration on several datasets where experimental results demonstrate that the mad rmm outperforms current state of the art methods and shows the robustness to outliers on real retinal images

coding for a binary independent piecewise identically distributed source two weighting procedures are presented for compaction of output sequences generated by binary independent sources whose unknown parameter may occasionally change the resulting codes need no knowledge of the sequence length t i e they are strongly sequential and also the number of parameter changes is unrestricted the additional transition redundancy of the first method was shown to achieve the merhav lower bound i e log t bits per transition for the second method we could prove that additional transition redundancy is not more than 3 2 log t bits per transition which is more than the merhav bound however the storage and computational complexity of this method are also more interesting than those of the first method simulations show that the difference in redundancy performance between the two methods is negligible

using the generalized likelihood ratio statistic for sequential detection of a change point we study sequential detection of a change point using the generalized likelihood ratio statistic for the special case of detecting a change in a normal mean with known variance we give approximations to the average run lengths and compare our procedure to standard cusum tests and combined cusum shewhart tests several examples indicating extensions to problems involving multiple parameters are discussed

on the wyner ziv problem for individual sequences we consider a variation of the wyner ziv w z problem pertaining to lossy compression of individual sequences using finite state encoders and decoders there are two main results in this paper the first characterizes the relationship between the performance of the best m state encoder decoder pair to that of the best block code of size lscr for every input sequence and shows that the loss of the latter relative to the former in terms of both rate and distortion never exceeds the order of logm lscr independently of the input sequence thus in the limit of large m the best rate distortion performance of every infinite source sequence can be approached universally by a sequence of block codes which are also implementable by finite state machines while this result assumes an asymptotic regime where the number of states is fixed and only the length n of the input sequence grows without bound we then consider the case where the number of states m mn is allowed to grow concurrently with n our second result is then about the critical growth rate of mn such that the rate distortion performance of mn state encoder decoder pairs can still be matched by a universal code we show that this critical growth rate of mn is linear in n

constrained monte carlo maximum likelihood for dependent data maximum likelihood estimates mles in autologistic models and other exponential family models for dependent data can be calculated with markov chain monte carlo methods the metropolis algorithm or the gibbs sampler which simulate ergodic markov chains having equilibrium distributions in the model from one realization of such a markov chain a monte carlo approximant to the whole likelihood function can be constructed the parameter value if any maximizing this function approximates the mle

stein s method for concentration inequalities we introduce a version of stein s method for proving concentration and moment inequalities in problems with dependence simple illustrative examples from combinatorics physics and mathematical statistics are provided

writing sequences on the plane the problem of arranging two dimensional arrays of data into one dimensional sequences comes up in image processing color quantization and optical and magnetic data recording a good arrangement should enable the one dimensional sequences to be modeled as markov chains or shifts of finite type since this is not possible in general two dimensional data is most commonly scanned by rows columns or diagonals we look into three unusual ways to write a sequence in the plane by penrose tilings by space filling curves and by cylindrical and spiral lattices we show how penrose tilings can be used to record information and how some spiral lattices can be used for quantization of color spaces

color quantization and processing by fibonacci lattices color quantization is sampling of three dimensional 3 d color spaces such as rgb or lab which results in a discrete subset of colors known as a color codebook or palette it is extensively used for display transfer and storage of natural images in internet based applications computer graphics and animation we propose a sampling scheme which provides a uniform quantization of the lab space the idea is based on several results from number theory and phyllotaxy the sampling algorithm is very much systematic and allows easy design of universal image independent color codebooks for a given set of parameters the codebook structure allows fast quantization and ordered dither of color images the display quality of images quantized by the proposed color codebooks is comparable with that of image dependent quantizers most importantly the quantized images are more amenable to the type of processing used for grayscale ones methods for processing grayscale images cannot be simply extended to color images because they rely on the fact that each gray level is described by a single number and the fact that a relation of full order can be easily established on the set of those numbers color spaces such as rgb or lab are on the other hand 3 d the proposed color quantization i e color space sampling and numbering of sampled points makes methods for processing grayscale images extendible to color images we illustrate possible processing of color images by first introducing the basic average and difference operations and then implementing edge detection and compression of color quantized images