spatially adaptive wavelet denoising using the minimum description length principle this paper presents a new spatially adaptive wavelet denoising method based on a doubly stochastic process model of wavelet coefficients the method gives a new threshold which varies spatially according to the variances of the coefficients using the minimum description length mdl principle the new threshold is not only easier to analyze since it is in a closed form but also provides more facility for future compression than several other methods almost without deteriorating mean square error risk

a review of geometric transformations for nonrigid body registration this paper provides a comprehensive and quantitative review of spatial transformations models for nonrigid image registration it explains the theoretical foundation of the models and classifies them according to this basis this results in two categories physically based models described by partial differential equations of continuum mechanics e g linear elasticity and fluid flow and basis function expansions derived from interpolation and approximation theory e g radial basis functions b splines and wavelets recent work on constraining the transformation so that it preserves the topology or is diffeomorphic is also described the final section reviews some recent evaluation studies the paper concludes by explaining under what conditions a particular transformation model is appropriate

meaningful information the information in an individual finite object like a binary string is commonly measured by its kolmogorov complexity one can divide that information into two parts the information accounting for the useful regularity present in the object and the information accounting for the remaining accidental information there can be several ways model classes in which the regularity is expressed kolmogorov has proposed the model class of finite sets generalized later to computable probability mass functions the resulting theory known as algorithmic statistics analyzes the algorithmic sufficient statistic when the statistic is restricted to the given model class however the most general way to proceed is perhaps to express the useful information as a total recursive function the resulting measure has been called the sophistication of the object we develop the theory of recursive functions statistic the maximum and minimum value the existence of absolutely nonstochastic objects that have maximal sophistication all the information in them is meaningful and there is no residual randomness determine its relation with the more restricted model classes of finite sets and computable probability distributions in particular with respect to the algorithmic kolmogorov minimal sufficient statistic the relation to the halting problem and further algorithmic properties

use of mathematical logical concepts in quantum mechanics an example the representation of numbers by product states in quantum mechanics can be extended to the representation of words and word sequences in languages by product states this can be used to study quantum systems that generate text that has meaning a simple example of such a system based on an example described by smullyan is studied here based on a path interpretation for some word states definitions of truth validity consistency and completeness are given and their properties studied it is also shown that the relation between the potential meaning if any of word states and the quantum algorithmic complexity of the process generating the word states must be quite complex or nonexistent

rigorous properties and uncertainty like relationships on product complexity measures application to atomic systems most of research on complexities and the corresponding conclusions have been obtained by numerically quantifying their values and little attention has been paid to their theoretical properties and the exact meaning within an statistical framework valid for any arbitrary n dimensional distribution in the present work different product complexities constructed as a product of two relevant information quantities associated to the concepts of uncertainty and localization are investigated different results on product complexities such as rigorous bounds uncertainty like inequalities relationships among themselves and also statistical interpretations are given for the sake of completeness some of these analytical results on product complexities are numerically checked for the one particle densities of atomic systems in both conjugated spaces

the proper formula for relative entropy and its asymptotics in quantum probability umegaki s relative entropys trd logd logd of states and with density operatorsd andd respectively is shown to be an asymptotic exponent considered from the quantum hypothesis testing viewpoint it is also proved that some other versions of the relative entropy give rise to the same asymptotics as umegaki s one as a byproduct the inequality tra logab tra loga logb is obtained for positive definite matricesa andb

fisher information and stochastic complexity by taking into account the fisher information and removing an inherent redundancy in earlier two part codes a sharper code length as the stochastic complexity and the associated universal process are derived for a class of parametric processes the main condition required is that the maximum likelihood estimates satisfy the central limit theorem the same code length is also obtained from the so called maximum likelihood code

generalized tunstall codes for sources with memory tunstall codes are variable to fixed length codes that maximize the expected number of source letters per dictionary string for discrete memoryless sources we analyze a generalization of tunstall coding to sources with memory and demonstrate that as the dictionary size increases the number of code letters per source symbol comes arbitrarily close to the minimum among all variable to fixed length codes of the same size we also find the asymptotic relationship between the dictionary size and the average length of a dictionary entry

application of kolmogorov complexity and universal codes to identity testing and nonparametric testing of serial independence for time series we show that kolmogorov complexity and such its estimators as universal codes or data compression methods can be applied for hypotheses testing in a framework of classical mathematical statistics the methods for identity testing and nonparametric testing of serial independence for time series are suggested

asymptotical growth of a class of random trees we study three rules for the development of a sequence of finite subtrees t n of an infinite m ary tree t independent realizations omega n of a stationary ergodic process omega on m letters are used to trace out paths in t in the first rule t n is formed by adding a node to t n 1 at the first location where the path defined by omega n leaves t n 1 the second and third rules are similar but more complicated for each rule the height l n of the added node is shown to grow in probability as ln n divided by h the entropy per symbol of the generic process a typical retrieval time has the same behavior on the other hand lim inf nl n ln n sigma 1 lim sup n l n ln n sigma 2 a s where the constants sigma 1 sigma 2 are in general different depend on the rule in use and sigma 1 1 h sigma 2 it is proven along the way that the height of t n grows as sigma 2 ln n with probability one

quantifying registration uncertainty with sparse bayesian modelling we investigate uncertainty quantification under a sparse bayesian model of medical image registration bayesian modelling has proven powerful to automate the tuning of registration hyperparameters such as the trade off between the data and regularization functionals sparsity inducing priors have recently been used to render the parametrization itself adaptive and data driven the sparse prior on transformation parameters effectively favors the use of coarse basis functions to capture the global trends in the visible motion while finer highly localized bases are introduced only in the presence of coherent image information and motion in earlier work approximate inference under the sparse bayesian model was tackled in an efficient variational bayes vb framework in this paper we are interested in the theoretical and empirical quality of uncertainty estimates derived under this approximate scheme vs under the exact model we implement an asymptotically exact inference scheme based on reversible jump markov chain monte carlo mcmc sampling to characterize the posterior distribution of the transformation and compare the predictions of the vb and mcmc based methods the true posterior distribution under the sparse bayesian model is found to be meaningful orders of magnitude for the estimated uncertainty are quantitatively reasonable the uncertainty is higher in textureless regions and lower in the direction of strong intensity gradients

bootstrap resampling for image registration uncertainty estimation without ground truth we address the problem of estimating the uncertainty of pixel based image registration algorithms given just the two images to be registered for cases when no ground truth data is available our novel method uses bootstrap resampling it is very general applicable to almost any registration method based on minimizing a pixel based similarity criterion we demonstrate it using the ssd sad correlation and mutual information criteria we show experimentally that the bootstrap method provides better estimates of the registration accuracy than the state of the art cramer rao bound method additionally we evaluate also a fast registration accuracy estimation frae method which is based on quadratic sensitivity analysis ideas and has a negligible computational overhead frae mostly works better than the cramer rao bound method but is outperformed by the bootstrap method

ockham s razor empirical complexity and truth finding efficiency the nature of empirical simplicity and its relationship to scientific truth are long standing puzzles in this paper empirical simplicity is explicated in terms of empirical effects which are defined in terms of the structure of the inference problem addressed problem instances are classified according to the number of empirical effects they present simple answers are satisfied by simple worlds an efficient solution achieves the optimum worst case cost over each complexity class with respect to such costs as the number of retractions or errors prior to convergence and elapsed time to convergence it is shown that always choosing the simplest theory compatible with experience and hanging on to it while it remains the simplest is both necessary and sufficient for efficiency

a strong version of the redundancy capacity theorem of universal coding the capacity of the channel induced by a given class of sources is well known to be an attainable lower bound on the redundancy of universal codes with respect to this class both in the minimax sense and in the bayesian maximin sense we show that this capacity is essentially a lower bound also in a stronger sense that is for most sources in the class this result extends rissanen s 1984 1986 lower bound for parametric families we demonstrate the applicability of this result in several examples e g parametric families with growing dimensionality piecewise fixed sources arbitrarily varying sources and noisy samples of learnable functions finally we discuss implications of our results to statistical inference

tests for distributed nonfocal brain activations abstract most approaches to detecting changes in functional brain images assume that activations are focal or very localized however the brain s response to cognitive of sensorimotor challenge may be spatially or anatomically distributed in this paper we consider the use fulness of a test based on the mean sum of squares of statistical parametric maps the performance of this test is evaluated using simulated and real data and is compared to the 2 test a test of the size of the activated region and a focal activation test based on the intensity of local maxima we demonstrate that the mean sum of squares test is more sensitive to nonfocal signals and propose that it could be used to complement approaches that are more sensitive to focal activations

applying mdl to learn best model granularity abstract the minimum description length mdl principle is solidly based on a provably ideal method of inference using kolmogorov complexity we test how the theory behaves in practice on a general problem in model selection that of learning the best model granularity the performance of a model depends critically on the granularity for example the choice of precision of the parameters too high precision generally involves modeling of accidental noise and too low precision may lead to confusion of models that should be distinguished this precision is often determined ad hoc in mdl the best model is the one that most compresses a two part code of the data set this embodies occam s razor in two quite different experimental settings the theoretical value determined using mdl coincides with the best value found experimentally in the first experiment the task is to recognize isolated handwritten characters in one subject s handwriting irrespective of size and orientation based on a new modification of elastic matching using multiple prototypes per character the optimal prediction rate is predicted for the learned parameter length of sampling interval considered most likely by mdl which is shown to coincide with the best value found experimentally in the second experiment the task is to model a robot arm with two degrees of freedom using a three layer feed forward neural network where we need to determine the number of nodes in the hidden layer giving best modeling performance the optimal model the one that extrapolizes best on unseen examples is predicted for the number of nodes in the hidden layer considered most likely by mdl which again is found to coincide with the best value found experimentally

convergence and loss bounds for bayesian sequence prediction the probability of observing x sub t at time t given past observations x sub 1 x sub t 1 can be computed if the true generating distribution spl mu of the sequences x sub 1 x sub 2 x sub 3 is known if spl mu is unknown but known to belong to a class spl mscr one can base one s prediction on the bayes mix spl xi defined as a weighted sum of distributions spl nu spl isin spl mscr various convergence results of the mixture posterior spl xi sub t to the true posterior spl mu sub t are presented in particular a new elementary derivation of the convergence spl xi sub t spl mu sub t spl rarr 1 is provided which additionally gives the rate of convergence a general sequence predictor is allowed to choose an action y sub t based on x sub 1 x sub t 1 and receives loss spl lscr sub x t y t if x sub t is the next symbol of the sequence no assumptions are made on the structure of spl lscr apart from being bounded and spl mscr the bayes optimal prediction scheme spl lambda sub spl xi based on mixture spl xi and the bayes optimal informed prediction scheme spl lambda sub spl mu are defined and the total loss l sub spl xi of spl lambda sub spl xi is bounded in terms of the total loss l sub spl mu of spl lambda sub spl mu it is shown that l sub spl xi is bounded for bounded l sub spl mu and l sub spl xi l sub spl mu spl rarr 1 for l sub spl mu spl rarr spl infin convergence of the instantaneous losses is also proven

linked independent component analysis for multimodal data fusion in recent years neuroimaging studies have increasingly been acquiring multiple modalities of data and searching for task or disease related changes in each modality separately a major challenge in analysis is to find systematic approaches for fusing these differing data types together to automatically find patterns of related changes across multiple modalities when they exist independent component analysis ica is a popular unsupervised learning method that can be used to find the modes of variation in neuroimaging data across a group of subjects when multimodal data is acquired for the subjects ica is typically performed separately on each modality leading to incompatible decompositions across modalities using a modular bayesian framework we develop a novel linked ica model for simultaneously modelling and discovering common features across multiple modalities which can potentially have completely different units signal and contrast to noise ratios voxel counts spatial smoothnesses and intensity distributions furthermore this general model can be configured to allow tensor ica or spatially concatenated ica decompositions or a combination of both at the same time linked ica automatically determines the optimal weighting of each modality and also can detect single modality structured components when present this is a fully probabilistic approach implemented using variational bayes we evaluate the method on simulated multimodal data sets as well as on a real data set of alzheimer s patients and age matched controls that combines two very different types of structural mri data morphological data grey matter density and diffusion data fractional anisotropy mean diffusivity and tensor mode

kolmogorov s contributions to information theory and algorithmic complexity briefly information theory says a random object x p x has complexity entropy h xp x log p x with the attendant interpretation that h bits are sufficient to describe x on the average algorithmic complexity says an object x has a complexity k x equal to the length of the shortest binary program that describes x it is a beautiful fact that these ideas are much the same in fact it is roughly true that ek x h moreover if we let pu x pr u prints x be the probability that a given computer u prints x when given a random program it can be shown that log 1 pu x k x for all x thus establishing a vital link between the universal probability measure pu and the universal complexity k more on this later the relationship of these ideas to probability theory was summed up in kolmogorov s 1983 paper which was based on his 1970 talk in nice perhaps only

new results on the size of tries a precise asymptotic expansion of the variance of the size of a trie built on random binary strings is presented this data structure appears in some hashing schemes and communications protocols the variance is asymptotically linear and numerical results are given the reader is referred to an earlier work for formal proofs

renewal theory in the analysis of tries and strings we give a survey of a number of simple applications of renewal theory to problems on random strings and tries insertion depth size insertion mode and imbalance of tries variations for b tries and patricia tries khodak and tunstall codes

towards a coherent statistical framework for dense deformable template estimation summary the problem of estimating probabilistic deformable template models in the field of computer vision or of probabilistic atlases in the field of computational anatomy has not yet received a coherent statistical formulation and remains a challenge we provide a careful definition and analysis of a well defined statistical model based on dense deformable templates for grey level images of deformable objects we propose a rigorous bayesian framework for which we prove asymptotic consistency of the maximum a posteriori estimate and which leads to an effective iterative estimation algorithm of the geometric and photometric parameters in the small sample setting the model is extended to mixtures of finite numbers of such components leading to a fine description of the photometric and geometric variations of an object class we illustrate some of the ideas with images of handwritten digits and apply the estimated models to classification through maximum likelihood

quantum algorithmic entropy we extend algorithmic information theory to quantum mechanics taking a universal semicomputable density matrix universal probability as a starting point and define complexity an operator as its negative logarithm a number of properties of kolmogorov complexity extend naturally to the new domain approximately a quantum state is simple if it is within a small distance from a low dimensional subspace of low kolmogorov complexity the von neumann entropy of a computable density matrix is within an additive constant from the average complexity some of the theory of randomness translates to the new domain we explore the relations of the new quantity to the quantum kolmogorov complexity defined by vitanyi we show that the latter is sometimes as large as 2n 2 log n and the qubit complexity defined by berthiaume dam and laplante the cloning properties of our complexity measure are similar to those of qubit complexity

quantum kolmogorov complexity based on classical descriptions we develop a theory of the algorithmic information in bits contained in an individual pure quantum state this extends classical kolmogorov complexity to the quantum domain retaining classical descriptions quantum kolmogorov complexity coincides with the classical kolmogorov complexity on the classical domain quantum kolmogorov complexity is upper bounded and can be effectively approximated from above under certain conditions with high probability a quantum object is incompressible upper and lower bounds of the quantum complexity of multiple copies of individual pure quantum states are derived and may shed some light on the no cloning properties of quantum states in the quantum situation complexity is not subadditive we discuss some relations with no cloning and approximate cloning properties

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

estimation and inference by compact coding summary the systematic variation within a set of data as represented by a usual statistical model may be used to encode the data in a more compact form than would be possible if they were considered to be purely random the encoded form has two parts the first states the inferred estimates of the unknown parameters in the model the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure in its strict form it has great generality and several nice properties but is computationally infeasible an approximate form is developed and its relation to other methods is explored

approximating rate distortion graphs of individual data experiments in lossy compression and denoising classical rate distortion theory requires specifying a source distribution instead we analyze rate distortion properties of individual objects using the recently developed algorithmic rate distortion theory the latter is based on the noncomputable notion of kolmogorov complexity to apply the theory we approximate the kolmogorov complexity by standard data compression techniques and perform a number of experiments with lossy compression and denoising of objects from different domains we also introduce a natural generalization to lossy compression with side information to maintain full generality we need to address a difficult searching problem while our solutions are therefore not time efficient we do observe good denoising and compression performance

computing large deformation metric mappings via geodesic flows of diffeomorphisms this paper examine the euler lagrange equations for the solution of the large deformation diffeomorphic metric mapping problem studied in dupuis et al 1998 and trouve 1995 in which two images i 0 i 1 are given and connected via the diffeomorphic change of coordinates i 0 1 i 1 where 1 is the end point at t 1 of curve t t 0 1 satisfying t v t t t 0 1 with 0 id the variational problem takes the form mathop arg text m in limits upsilon dot phi t upsilon t left dot phi right left int 0 1 left upsilon t right 2 text d t left i 0 circ phi 1 1 i 1 right l 2 2 right where v t v is an appropriate sobolev norm on the velocity field v t and the second term enforces matching of the images with l 2 representing the squared error norm in this paper we derive the euler lagrange equations characterizing the minimizing vector fields v t t 0 1 assuming sufficient smoothness of the norm to guarantee existence of solutions in the space of diffeomorphisms we describe the implementation of the euler equations using semi lagrangian method of computing particle flows and show the solutions for various examples as well we compute the metric distance on several anatomical configurations as measured by 0 1 v t v dt on the geodesic shortest paths

gacs quantum algorithmic entropy in infinite dimensional hilbert spaces we extend the notion of gacs quantum algorithmic entropy originally formulated for finitely many qubits to infinite dimensional quantum spin chains and investigate the relation of this extension with two quantum dynamical entropies that have been proposed in recent years

on the gaussianity of kolmogorov complexity of mixing sequences it has been proved that for all stationary and ergodic processes the average kolmogorov complexity of the first n observations converges almost surely to its shannon s conditional entropy this paper studies the convergence rate of this asymptotic result in particular we show that if the process satisfies certain mixing conditions then a central limit theorem will be respected furthermore we show that under slightly stronger mixing conditions one may obtain non asymptotic concentration bounds for the kolmogorov complexity

deformable templates using large deformation kinematics a general automatic approach is presented for accommodating local shape variation when mapping a two dimensional 2 d or three dimensional 3 d template image into alignment with a topologically similar target image local shape variability is accommodated by applying a vector field transformation to the underlying material coordinate system of the template while constraining the transformation to be smooth globally positive definite jacobian smoothness is guaranteed without specifically penalizing large magnitude deformations of small subvolumes by constraining the transformation on the basis of a stokesian limit of the fluid dynamical navier stokes equations this differs fundamentally from quadratic penalty methods such as those based on linearized elasticity or thin plate splines in that stress restraining the motion relaxes over time allowing large magnitude deformations kinematic nonlinearities are inherently necessary to maintain continuity of structures during large magnitude deformations and are included in all results after initial global registration final mappings are obtained by numerically solving a set of nonlinear partial differential equations associated with the constrained optimization problem automatic regridding is performed by propagating templates as the nonlinear transformations evaluated on a finite lattice become singular application of the method to intersubject registration of neuroanatomical structures illustrates the ability to account for local anatomical variability

entropy and quantum kolmogorov complexity a quantum brudno s theorem in classical information theory entropy rate and algorithmic complexity per symbol are related by a theorem of brudno in this paper we prove a quantum version of this theorem connecting the von neumann entropy rate and two notions of quantum kolmogorov complexity both based on the shortest qubit descriptions of qubit strings that run by a universal quantum turing machine reproduce them as outputs

on different facets of regularization theory this review provides a comprehensive understanding of regularization theory from different perspectives emphasizing smoothness and simplicity principles using the tools of operator theory and fourier analysis it is shown that the solution of the classical tikhonov regularization problem can be derived from the regularized functional defined by a linear differential integral operator in the spatial fourier domain state of the art research relevant to the regularization theory is reviewed covering occam s razor minimum length description bayesian theory pruning algorithms informational entropy theory statistical learning theory and equivalent regularization the universal principle of regularization in terms of kolmogorov complexity is discussed finally some prospective studies on regularization theory and beyond are suggested

variational bayes inference of spatial mixture models for segmentation mixture models are commonly used in the statistical segmentation of images for example they can be used for the segmentation of structural medical images into different matter types or of statistical parametric maps into activating and nonactivating brain regions in functional imaging spatial mixture models have been developed to augment histogram information with spatial regularization using markov random fields mrfs in previous work an approximate model was developed to allow adaptive determination of the parameter controlling the strength of spatial regularization inference was performed using markov chain monte carlo mcmc sampling however this approach is prohibitively slow for large datasets in this work a more efficient inference approach is presented this combines a variational bayes approximation with a second order taylor expansion of the components of the posterior distribution which would otherwise be intractable to variational bayes this provides inference on fully adaptive spatial mixture models an order of magnitude faster than mcmc we examine the behavior of this approach when applied to artificial data with different spatial characteristics and to functional magnetic resonance imaging statistical parametric maps

an analytic approach to the asymptotic variance of trie statistics and related structures we develop analytic tools for the asymptotics of general trie statistics which are particularly advantageous for clarifying the asymptotic variance many concrete examples are discussed for which new fourier expansions are given the tools are also useful for other splitting processes with an underlying binomial distribution we specially highlight philippe flajolet s contribution in the analysis of these random structures

diffeomorphic registration using geodesic shooting and gauss newton optimisation this paper presents a nonlinear image registration algorithm based on the setting of large deformation diffeomorphic metric mapping lddmm but with a more efficient optimisation scheme both in terms of memory required and the number of iterations required to reach convergence rather than perform a variational optimisation on a series of velocity fields the algorithm is formulated to use a geodesic shooting procedure so that only an initial velocity is estimated a gauss newton optimisation strategy is used to achieve faster convergence the algorithm was evaluated using freely available manually labelled datasets and found to compare favourably with other inter subject registration algorithms evaluated using the same data

second order noiseless source coding theorems shannon s celebrated source coding theorem can be viewed as a one sided law of large numbers we formulate second order noiseless source coding theorems for the deviation of the codeword lengths from the entropy for a class of sources that includes markov chains we prove a one sided central limit theorem and a law of the iterated logarithm

kolmogorov s structure functions and model selection in 1974 kolmogorov proposed a nonprobabilistic approach to statistics and model selection let data be finite binary strings and models be finite sets of binary strings consider model classes consisting of models of given maximal kolmogorov complexity the structure function of the given data expresses the relation between the complexity level constraint on a model class and the least log cardinality of a model in the class containing the data we show that the structure function determines all stochastic properties of the data for every constrained model class it determines the individual best fitting model in the class irrespective of whether the true model is in the model class considered or not in this setting this happens with certainty rather than with high probability as is in the classical case we precisely quantify the goodness of fit of an individual model with respect to individual data we show that within the obvious constraints every graph is realized by the structure function of some data we determine the un computability properties of the various functions contemplated and of the algorithmic minimal sufficient statistic

the shannon mcmillan theorem for ergodic quantum lattice systems we formulate and prove a quantum shannon mcmillan theorem the theorem demonstrates the significance of the von neumann entropy for translation invariant ergodic quantum spin systems on lattices the entropy gives the logarithm of the essential number of eigenvectors of the system on large boxes the one dimensional case covers quantum information sources and is basic for coding theorems

tunstall code khodak variations and random walks a variable to fixed length encoder partitions the source string into variable length phrases that belong to a given and fixed dictionary tunstall and independently khodak designed variable to fixed length codes for memoryless sources that are optimal under certain constraints in this paper we study the tunstall and khodak codes using variety of techniques ranging from stopping times for sums of independent random variables to tauberian theorems and mellin transform after proposing an algebraic characterization of the tunstall and khodak codes we present new results on the variance and a central limit theorem for dictionary phrase lengths this analysis also provides a new argument for obtaining asymptotic results about the mean dictionary phrase length and average redundancy rates

minimum description length induction bayesianism and kolmogorov complexity the relationship between the bayesian approach and the minimum description length approach is established we sharpen and clarify the general modeling principles minimum description length mdl and minimum message length mml abstracted as the ideal mdl principle and defined from bayes s rule by means of kolmogorov complexity the basic condition under which the ideal principle should be applied is encapsulated as the fundamental inequality which in broad terms states that the principle is valid when the data are random relative to every contemplated hypothesis and also these hypotheses are random relative to the universal prior the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized if we restrict the model class to finite sets then application of the ideal principle turns into kolmogorov s minimal sufficient statistic in general we show that data compression is almost always the best strategy both in model selection and prediction

probabilistic inference of regularisation in non rigid registration a long standing issue in non rigid image registration is the choice of the level of regularisation regularisation is necessary to preserve the smoothness of the registration and penalise against unnecessary complexity the vast majority of existing registration methods use a fixed level of regularisation which is typically hand tuned by a user to provide nice results however the optimal level of regularisation will depend on the data which is being processed lower signal to noise ratios require higher regularisation to avoid registering image noise as well as features and different pairs of images require registrations of varying complexity depending on their anatomical similarity in this paper we present a probabilistic registration framework that infers the level of regularisation from the data an additional benefit of this proposed probabilistic framework is that estimates of the registration uncertainty are obtained this framework has been implemented using a free form deformation transformation model although it would be generically applicable to a range of transformation models we demonstrate our registration framework on the application of inter subject brain registration of healthy control subjects from the nirep database in our results we show that our framework appropriately adapts the level of regularisation in the presence of noise and that inferring regularisation on an individual basis leads to a reduction in model over fitting as measured by image folding while providing a similar level of overlap

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

stationary quantum source coding in this article the quantum version of the source coding theorem is obtained for a completely ergodic source this result extends schumacher s quantum noiseless coding theorem for memoryless sources the control of the memory effects requires some earlier results of hiai and petz on high probability subspaces our result is equivalently considered as a compression theorem for noiseless stationary channels

constructing perfect steganographic systems abstract we propose steganographic systems for the case when covertexts containers are generated by a finite memory source with possibly unknown statistics the probability distributions of covertexts with and without hidden information are the same this means that the proposed stegosystems are perfectly secure i e an observer cannot determine whether hidden information is being transmitted the speed of transmission of hidden information can be made arbitrary close to the theoretical limit the shannon entropy of the source of covertexts an interesting feature of the suggested stegosystems is that they do not require any secret or public key at the same time we outline some principled computational limitations on steganography we show that there are such sources of covertexts that any stegosystem that has linear in the length of the covertext speed of transmission of hidden text must have an exponential kolmogorov complexity this shows in particular that some assumptions on the sources of covertext are necessary

patricia tries again revisited the patricia trie is a simple modification of a regular trie by eliminating unary branching nodes the patricia achieves better performance than regular tries however the question is how much on the average is the patricia better this paper offers a thorough answer to this question by considering some statistics of the number of nodes examined in a successful search and an unsuccessful search in the patricia tries it is shown that for the patricia containing n records the average of the successful search length s n asymptotically becomes 1 h 1 ln n o 1 and the variance of s n is either var s n c ln n 0 1 for an asymmetric patricia or var s n 0 1 for a symmetric patricia where h 1 is the entropy of the alphabet over which the patricia is built and c is an explicit constant higher moments of s n are also assessed the number of nodes examined in an unsuccessful search u n is studied only for binary symmetric patricia tries we prove that the m th moment of the unsuccessful search length eu m n satisfies lim n eu m n log m 2 n 1 and the variance of u n is var u n 0 87907 these results suggest that patricia tries are very well balanced trees in the sense that a random shape of patriciatries resembles the shape of complete trees that are ultimately balanced trees

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

computational mechanics pattern and prediction structure and simplicity computational mechanics an approach to structural complexity defines a process s causal states and gives a procedure for finding them we show that the causal state representation an machine is the minimal one consistent with accurate prediction we establish several results on machine optimality and uniqueness and on how machines compare to alternative representations further results relate measures of randomness and structural complexity obtained from machines to those from ergodic and information theories

asymptotics of discrete mdl for online prediction minimum description length mdl is an important principle for induction and prediction with strong relations to optimal bayesian learning this paper deals with learning processes which are independent and identically distributed i i d by means of two part mdl where the underlying model class is countable we consider the online learning framework i e observations come in one by one and the predictor is allowed to update its state of mind after each time step we identify two ways of predicting by mdl for this setup namely a static and a dynamic one a third variant hybrid mdl will turn out inferior we will prove that under the only assumption that the data is generated by a distribution contained in the model class the mdl predictions converge to the true values almost surely this is accomplished by proving finite bounds on the quadratic the hellinger and the kullback leibler loss of the mdl learner which are however exponentially worse than for bayesian prediction we demonstrate that these bounds are sharp even for model classes containing only bernoulli distributions we show how these bounds imply regret bounds for arbitrary loss functions our results apply to a wide range of setups namely sequence prediction pattern classification regression and universal induction in the sense of algorithmic information theory among others

constrained linear basis sets for hrf modelling using variational bayes abstract fmri modelling requires flexible haemodynamic response function hrf modelling with the hrf being allowed to vary spatially and between subjects to achieve this flexibility voxelwise parameterised hrfs have been proposed however inference on such models is very slow an alternative approach is to use basis functions allowing inference to proceed in the more manageable general linear model glm framework however a large amount of the subspace spanned by the basis functions produces nonsensical hrf shapes in this work we propose a technique for choosing a basis set and then the means to constrain the subspace spanned by the basis set to only include sensible hrf shapes penny et al neuroimage 2003 showed how variational bayes can be used to infer on the glm for fmri here we extend the work of penny et al to give inference on the glm with constrained hrf basis functions and with spatial markov random fields on the autoregressive noise parameters constraining the subspace spanned by the basis set allows for far superior separation of activating voxels from nonactivating voxels in fmri data we use spatial mixture modelling to produce final probabilities of activation and demonstrate increased sensitivity on an fmri dataset

rounding of continuous random variables and oscillatory asymptotics we study the characteristic function and moments of the integer valued random variable x alpha where x is a continuous random variables the results can be regarded as exact versions of sheppard s correction rounded variables of this type often occur as subsequence limits of sequences of integer valued random variables this leads to oscillatory terms in asymptotics for these variables something that has often been observed for example in the analysis of several algorithms we give some examples including applications to tries digital search trees and patricia tries

strongly universal quantum turing machines and invariance of kolmogorov complexity we show that there exists a universal quantum turing machine uqtm that can simulate every other qtm until the other qtm has halted and then halt itself with probability one this extends work by bernstein and vazirani who have shown that there is a uqtm that can simulate every other qtm for an arbitrary but preassigned number of time steps as a corollary to this result we give a rigorous proof that quantum kolmogorov complexity as defined by berthiaume is invariant i e depends on the choice of the uqtm only up to an additive constant our proof is based on a new mathematical framework for qtms including a thorough analysis of their halting behavior we introduce the notion of mutually orthogonal halting spaces and show that the information encoded in an input qubit string can always be effectively decomposed into a classical and a quantum part

analysis of digital tries with markovian dependency a complete characterization of a digital tree also called a trie is presented from the depth viewpoint in a markovian framework that is under the assumption that symbols in a key are markov dependent the main findings show that asymptotically as the number of keys n tends to infinity the average depth becomes ed sub n approximately 1 h sub 1 log n c and the variance is var d sub n approximately alpha log n c where h sub 1 is the entropy of the markovian dependent alphabet alpha is a parameter of the probabilistic model and c and c are constants the symmetric independent model has alpha 0 hence in this case var d sub n o 1 limiting distribution is also derived for the depth d sub n and in particular it is shown that d sub n tends to the normal distribution in all cases except the symmetric independent model these results extend all previous analyses since most of them have been limited to independent models

inferring markov chains bayesian estimation model comparison entropy rate and out of class modeling markov chains are a natural and well understood tool for describing one dimensional patterns in time or space we show how to infer kth order markov chains for arbitrary k from finite data by applying bayesian methods to both parameter estimation and model order selection extending existing results for multinomial models of discrete data we connect inference to statistical mechanics through information theoretic type theory techniques we establish a direct relationship between bayesian evidence and the partition function which allows for straightforward calculation of the expectation and variance of the conditional relative entropy and the source entropy rate finally we introduce a method that uses finite data size scaling with model order comparison to infer the structure of out of class processes

a fast diffeomorphic image registration algorithm abstract this paper describes dartel which is an algorithm for diffeomorphic image registration it is implemented for both 2d and 3d image registration and has been formulated to include an option for estimating inverse consistent deformations nonlinear registration is considered as a local optimisation problem which is solved using a levenberg marquardt strategy the necessary matrix solutions are obtained in reasonable time using a multigrid method a constant eulerian velocity framework is used which allows a rapid scaling and squaring method to be used in the computations dartel has been applied to intersubject registration of 471 whole brain images and the resulting deformations were evaluated in terms of how well they encode the shape information necessary to separate male and female subjects and to predict the ages of the subjects

probabilistic model for robust affine and non rigid point set matching in this work we propose a combinative strategy based on regression and clustering for solving point set matching problems under a bayesian framework in which the regression estimates the transformation from the model to the scene and the clustering establishes the correspondence between two point sets the point set matching model is illustrated by a hierarchical directed graph and the matching uncertainties are approximated by a coarse to fine variational inference algorithm furthermore two gaussian mixtures are proposed for the estimation of heteroscedastic noise and spurious outliers and an isotropic or anisotropic covariance can be imposed on each mixture in terms of the transformed model points the experimental results show that the proposed approach achieves comparable performance to state of the art matching or registration algorithms in terms of both robustness and accuracy

variational bayes for generalized autoregressive models we describe a variational bayes vb learning algorithm for generalized autoregressive gar models the noise is modeled as a mixture of gaussians rather than the usual single gaussian this allows different data points to be associated with different noise levels and effectively provides robust estimation of ar coefficients the vb framework is used to prevent overfitting and provides model order selection criteria both for ar order and noise model order we show that for the special case of gaussian noise and uninformative priors on the noise and weight precisions the vb framework reduces to the bayesian evidence framework the algorithm is applied to synthetic and real data with encouraging results

optimality of universal bayesian sequence prediction for general loss and alphabet various optimality properties of universal sequence predictors based on bayes mixtures in general and solomonoff s prediction scheme in particular will be studied the probability of observing xt at time t given past observations x1 xt 1 can be computed with the chain rule if the true generating distribution of the sequences x1x2x3 is known if is unknown but known to belong to a countable or continuous class one can base ones prediction on the bayes mixture defined as a w weighted sum or integral of distributions the cumulative expected loss of the bayes optimal universal prediction scheme based on is shown to be close to the loss of the bayes optimal but infeasible prediction scheme based on we show that the bounds are tight and that no other predictor can lead to significantly smaller bounds furthermore for various performance measures we show pareto optimality of and give an occam s razor argument that the choice w sim 2 k for the weights is optimal where k is the length of the shortest program describing the results are applied to games of chance defined as a sequence of bets observations and rewards the prediction schemes and bounds are compared to the popular predictors based on expert advice extensions to infinite alphabets partial delayed and probabilistic prediction classification and more active systems are briefly discussed