chaos in one dimensional collision complexes the classical scattering of a particle by an oscillating potential well is investigated it is shown that the singularities of the scattering map s form a cantor set for this system this result is obtained from an analysis of the particle s return map which contains a smale s horseshoe and consequently a hyperbolic invariant set with chaotic dynamics the cantor set structure of the s singularities involves arbitrarily complex scattering behaviour

fisher information and stochastic complexity by taking into account the fisher information and removing an inherent redundancy in earlier two part codes a sharper code length as the stochastic complexity and the associated universal process are derived for a class of parametric processes the main condition required is that the maximum likelihood estimates satisfy the central limit theorem the same code length is also obtained from the so called maximum likelihood code

finding the number of clusters in a dataset one of the most difficult problems in cluster analysis is identifying the number of groups in a dataset most previously suggested approaches to this problem are either somewhat ad hoc or require parametric assumptions and complicated calculations in this article we develop a simple yet powerful nonparametric method for choosing the number of clusters based on distortion a quantity that measures the average distance per dimension between each observation and its closest cluster center our technique is computationally efficient and straightforward to implement we demonstrate empirically its effectiveness not only for choosing the number of clusters but also for identifying underlying structure on a wide range of simulated and real world datasets in addition we give a rigorous theoretical justification for the method based on information theoretic ideas specifically results from the subfield of electrical engineering known as rate distortion theory allow us to describe the behavior of the dist

some applications of symbolic dynamics techniques to toral skew endomorphisms a class of dynamical systems of the 2 torus is considered these systems have the form of a skew product between the bernoulli endomorphism defined on the 1 torus and a translation on itself symbolic dynamics techniques allow one to single out wide classes of observables which show an exponential decay of correlations for some observables the rate of correlation decay can be explicitly estimated

bayesian sequential parameter estimation by cognitive radar with multiantenna arrays in this paper we consider the problem of bayesian sequential parameter estimation of extended targets for cognitive radar with multi antenna arrays using adaptive waveforms the target is modeled as a complex gaussian random process using iterative waveform transmission the cognitive radar estimates the target s characteristic parameters and updates its probabilistic model based on new measurements the adaptive waveform is designed by minimizing the conditional entropy from the posterior density of the model parameters we analyze the performance of the developed bayesian sequential estimation algorithm and derive expressions for the signal to noise ratio gain the asymp totical posterior cramer rao bound and the mutual information gain the analysis and numerical simulations demonstrate that the adaptive sequential bayesian estimator yields accelerated convergence of the estimate towards its true value and a smaller estimation error compared with the conventional bayesian esti mator that uses fixed waveform transmission under gaussian or non gaussian noise

devil staircase behavior of dynamical invariants in chaotic scattering a crisis in chaotic scattering is characterized by the merging of two or more nonattracting chaotic saddles the fractal dimension of the resulting chaotic saddle increases through the crisis we present a rigorous analysis for the behavior of dynamical invariants associated with chaotic scattering by utilizing a representative model system that captures the essential dynamical features of crisis our analysis indicates that the fractal dimension and other dynamical invariants are a devil staircase type of function of the system parameter our results can also provide insight for similar devil staircase behaviors observed in the parametric evolution of chaotic saddles of general dissipative dynamical systems and in communicating with chaos 2000 elsevier science b v all rights reserved

model selection and the principle of minimum description length this article reviews the principle of minimum description length mdl for problems of model selection by viewing statistical modeling as a means of generating descriptions of observed data the mdl framework discriminates between competing models based on the complexity of each description this approach began with kolmogorov s theory of algorithmic complexity matured in the literature on information theory and has recently received renewed attention within the statistics community here we review both the practical and the theoretical aspects of mdl as a tool for model selection emphasizing the rich connections between information theory and statistics at the boundary between these two disciplines we find many interesting interpretations of popular frequentist and bayesian procedures as we show mdl provides an objective umbrella under which rather disparate approaches to statistical modeling can coexist and be compared we illustrate the mdl principle by considering problems in regression nonpar

unified optimization framework for multi static radar code design using information theoretic criteria in this paper we study the problem of code design to improve the detection performance of multi static radar in the presence of clutter i e a signal dependent interference to this end we briefly present a discrete time formulation of the problem as well as the optimal detector in the presence of gaussian clutter due to the lack of analytical expression for receiver operation characteristic roc code design based on roc is not feasible therefore we consider several popular information theoretic criteria including bhattacharyya distance kullback leibler kl divergence j divergence and mutual information mi as design metrics the code optimization problems associated with different information theoretic criteria are obtained and cast under a unified framework we propose two general methods based on majorization minimization to tackle the optimization problems in the framework the first method provides optimal solutions via successive majorizations whereas the second one consists of a majorization step a relaxation and a synthesis stage moreover derivations of the proposed methods are extended to tackle the code design problems with a peak to average ratio power par constraint using numerical investigations a general analysis of the coded system performance computational efficiency of the proposed methods and the behavior of the information theoretic criteria is provided

cantor set structures in the singularities of classical potential scattering a classical mechanical system is analysed which exhibits complicated scattering behaviour in the set of all incoming asymptotes there is a fractal subset on which the scattering angle is singular though in the complement of this cantor set the deflection function is regular one can choose impact parameter intervals leading to arbitrarily complicated trajectories the authors show how the complicated scattering behaviour is caused by unstable periodic orbits having homoclinic and heteroclinic connections thereby a hyperbolic invariant set is created leading to horseshoe chaos in the flow this invariant set contains infinitely many unstable localised orbits periodic and aperiodic ones the stable manifolds of these orbits reach out into the asymptotic region and create the singularities of the scattering function

fractal properties of scattering singularities we discuss properties of the set of scattering singularities in regions of irregular scattering we show how a symbolic organisation of the set can be used to determine the fractal dimension and the scaling function this yields information on the distribution of lyapunov exponents of bounded orbits the specific model studied is the motion of a particle in a plane elastically reflected by three circular discs centred on the corners of an equilateral triangle

density estimation by stochastic complexity the results by p hall and e j hannan 1988 on optimization of histogram density estimators with equal bin widths by minimization of the stochastic complexity are extended and sharpened in two separate ways as the first contribution two generalized histogram estimators are constructed the first has unequal bin widths which together with the number of the bins are determined by minimization of the stochastic complexity using dynamic programming the other estimator consists of a mixture of equal bin width estimators each of which is defined by the associated stochastic complexity as the main contribution in the present work two theorems are proved which together extend the universal coding theorems to a large class of data generating densities the first gives an asymptotic upper bound for the code redundancy in the order of magnitude achieved with a special predictive type of histogram estimator which sharpens a related bound the second theorem states that this bound cannot be improved upon by any code whatsoever

information optimality and bayesian modelling the general approach of treating a statistical problem as one of information processing led to the bayesian method of moments reference priors minimal information likelihoods and stochastic complexity these techniques rest on quantities that have physical interpretations from information theory current work includes the role of prediction the emergence of data dependent priors the role of information measures in model selection and the use of conditional mutual information to incorporate partial information

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

universal properties of the transition from quasi periodicity to chaos in dissipative systems abstract an exact renormalization group transformation is developed for dissipative systems which describes how the transition to chaos may occur in a continuous and universal manner if the frequency ratio in the quasi periodic regime is held at a fixed irrational value our approach is a natural extension of k a m theory to strong coupling most of our analysis is for analytic circle maps we have found a strong coupling fixed point where invertibility is lost which describes the universal features of the transition to chaos we find numerically that any two such critical maps with the same winding number are c 1 conjugate it follows that the low frequency peaks in an experimental spectrum are universal and we determine how their envelope scales with frequency when the winding number has a periodic continued fraction our renormalization transform has a fixed point and spectra are self similar in addition for a set of non periodic winding numbers with full measure our renormalization transformation yields an ergodic trajectory in a sub space of all critical maps physically one finds singular and universal spectra that do not scale

self similarity of binary quasiperiodic sequences self similarity in binary quasiperiodic sequences generated by a projection method is shown to exist when and only when it is associated with quadratic irrational numbers and the explicit self similarity transformation for an arbitrary quadratic number is obtained the self similarity transformation is shown not to be reducible to the simplest form for a class of quadratic numbers

a finite size scaling analysis of the localization properties of one dimensional quasiperiodic systems the authors show that the phenomenological renormalization group analysis based on the finite size scaling hypothesis is very effective in analysing the critical properties of the localization delocalization transition ldt of the wavefunctions in quasiperiodic systems they have applied it successfully to the ldt of the discrete schrodinger equation with an incommensurate modulation potential characterized by a quadratic irrational showing that different models with a common incommensurate ratio belong to the same universality class if their potential functions are smooth moreover there exists a one to one correspondence between the whole universality classes of the ldt and the whole equivalence classes of quadratic irrationals with respect to the modular transformation

numerical study of a billiard in a gravitational field abstract billiards have always been used as models for mechanical systems in this paper we describe a very simple billiard which over a range of one continous parameter only exhibits the characteristics of hamiltonian systems having two degrees of freedom and a discontinuity the relationship between this billiard and the well known one dimensional self gravitating system with n 3 is given this billiard consists of a mass point moving in a symmetric wedge of angle 2 under the influence of a constant gravitational field for 45 complete chaos is obtained suggesting k system behavior the general properties of the mapping and some numerical results obtained are discussed of special interest are invariant curves which cross a line of discontinuity and a new universality class for lyapunov numbers

weakly noisy chaotic scattering received 30 april 2013 published 23 september 2013 the effect of a weak source of noise on the chaotic scattering is relevant to situations of physical interest we investigate how a weak source of additive uncorrelated gaussian noise affects both the dynamics and thetopology of a paradigmatic chaotic scattering problem as the one taking place in the open nonhyperbolic regimeof the h enon heiles hamiltonian system we have found long transients for the time escape distributions forcritical values of the noise intensity for which the particles escape slower as compared with the noiseless case an analysis of the survival probability of the scattering function versus the gaussian noise intensity shows asmooth curve with one local maximum and with one local minimum which are related to those long transientsand with the basin structure in phase space on the other hand the computation of the exit basins in phase spaceshows a quadratic curve for which the basin boundaries lose their fractal like structure as noise turned on doi 10 1103 physreve 88 032914 pacs number s 05

pruning induced phase transition observed by a scattering method in hyperbolic systems transient chaos is associated with an underlying chaotic saddle in phase space the structure of the chaotic saddle of a class of piecewise linear area preserving two dimensional maps with overall constant lyapunov exponents has been observed by a scattering method the free energy obtained in this way displays a phase transition at 0 in spite of the fact that no phase transition occurs in the free energy dedcued from the spectrum of lyapunov exponents this is possible because pruning introduces a second effective scaling exponent by creating at each level of the approximation particular small pieces in the incomplete cantor set approximating the saddle the second scaling arises for a subset of values of the control parameter that is dense in the parameter interval

some aspects of the history of bayesian information processing for many years traditional bayesian tb and information theoretic it procedures for learning from data were viewed as distinctly different approaches derivations of the tb and it learning models are reviewed and compared then the synthesis by zellner in 1988 of the tb and it learning models and generalizations of them are described along with descriptions of selected applications included are learning procedures that do not require use of likelihood functions and or priors works by leading bayesians and information theorists are cited and related to tb it issues

hyperbolicity in a double well potential we investigate the dynamics of a particle in a double well potential there is a critical energy greater than zero beyond which no localized motion is possible for certain energies smaller than the critical energy the system can be shown to have a nonempty bounded invariant set conditions are found under which this invariant set is hyperbolic

chaotic scattering by steep repelling potentials consider a classical two dimensional scattering problem a ray is scattered by a potential composed of several tall repelling steep mountains of arbitrary shape we study when the traditional approximation of this nonlinear far from integrable problem by the corresponding simpler billiard problem of scattering by hard wall obstacles of similar shape is justified for one class of chaotic scatterers named here regular sinai scatterers the scattering properties of the smooth system indeed limit to those of the billiards for another class the singular sinai scatterers these two scattering problems have essential differences though the invariant set of such singular scatterers is hyperbolic possibly with singularities that of the smooth flow may have stable periodic orbits even when the potential is arbitrarily steep it follows that the fractal dimension of the scattering function of the smooth flow may be significantly altered by changing the ratio between the steepness parameter and a parameter which measures the billiards deviation from a singular scatterer thus even in this singular case the billiard scattering problem is utilized as a skeleton for studying the properties of the smooth flow finally we see that corners have nontrivial and significant impact on the scattering functions

chaotic scattering modelled by an inclined billiard abstract chaotic scattering as observed in satellite encounters and in other problems is modelled by a simple billiard a point particle bounces on two disks and in addition is subjected to a constant acceleration in an appropriate limit this problem reduces to a piecewise linear mapping we consider a one parameter family of orbits obtained by starting the particle at rest from a constant height we find that this family exhibits chaotic scattering it has an infinite number of discontinuities forming a self similar cantor set with the help of symbolic dynamics the structure of the family can be completely elucidated

information distinguishability with application to analysis of failure data abstract in maximum entropy me modeling the information discrepancy between two distributions is measured in terms of their entropy difference in discrimination information statistics the information discrepancy between two distributions is measured in terms of the kullback leibler function i e relative entropy or cross entropy this article presents an equivalence between kullback leibler functions and entropy differences involving an me distribution based on this equivalence the concept of information discrimination id distinguishability is introduced as a unifying framework for the two methods of measuring information discrepancy between distributions applications of id distinguishability as diagnostics for examining robustness of parametric procedures and sensitivity of nonparametric statistics across parametric families of distributions is proposed the equivalence result facilitates estimation of kullback leibler functions in terms of entropy estimates application of the id distinguish

asymptotic inference for mixture models by using data dependent priors for certain mixture models improper priors are undesirable because they yield improper posteriors however proper priors may be undesirable because they require subjective input we propose the use of specially chosen data dependent priors we show that in some cases data dependent priors are the only priors that produce intervals with second order correct frequentist coverage the resulting posterior also has another interpretation it is the product of a fixed prior and a pseudolikelihood

fractal properties of escape from a two dimensional potential abstract this paper summarizes a numerical investigation of the escape of particles from the two dimensional potential v x y 1 2 x 2 1 2 y 2 x 2 y 2 for variable but fixed energy h 0 12 for 1 1 4h 2 08 escape is impossible but for 1 particles will escape for such larger the final outcome as characterized by the time and direction in which particles escape is a smooth function of initial conditions in certain phase space regions however in other regions the evolution evidences an extremely sensitive dependence on initial conditions indicative of a complex microscopic evolution for sufficiently large values of 2 4 90 0 01 these sensitive regions appear to exhibit a fractal structure with simple scaling properties implying that at a coarse grained level the late time behavior is quite regular specifically the evolution is asymptotically markovian in the sense that for particles that have not yet escaped at late times the coarse grained escape probabilities per unit time tend toward time independent values p 0 independent of the initial conditions the asymptotic values p and the rate of convergence towards these values were examined as a function of and the scale of the coarse graining and were observed to exhibit simple scaling behaviour at least for values of 5 7 p 2 with a critical exponent 0 49 0 05 for a coarse graining corresponding to a linear scale r 0 05 the time t required for convergence towards this value scales as t 2 with 0 39 0 14 0 06 the difference 0 09 0 04 0 02 the value p appears independent of r but the convergence time scales as t r r with 0 08 0 03 seemingly independent of to within statistical uncertainties 0

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

on the practical implication of mutual information for statistical decisionmaking a basic mathematical function that conjoins the two key conceptions of mutual information and bayes risk is defined based on that function some asymptotic theorems that verify an important implication of mutual information in the context of practical bayesian decisionmaking are proven

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

capturing the intangible concept of information abstract the purpose of this article is to discuss the intricacies of quantifying information in some statistical problems the aim is to develop a general appreciation for the meanings of information functions rather than their mathematical use this theme integrates fundamental aspects of the contributions of kullback lindley and jaynes and bridges chaos to probability modeling a synopsis of information theoretic statistics is presented in the form of a pyramid with shannon at the vertex and a triangular base that signifies three distinct variants of quantifying information discrimination information kullback mutual information lindley and maximum entropy information jaynes examples of capturing information by the maximum entropy me method are discussed it is shown that the me approach produces a general class of logit models capable of capturing various forms of sample and nonsample information diagnostics for quantifying information captured by the me logit models are given and decom

information processing and bayesian analysis abstract science involves learning from data herein this process of learning or information processing is considered within the context of optimal information processing as in zellner 1988 1991 1997 information criterion functionals are formulated and optimized to provide optimal information processing rules one of which is bayes theorem by varying the inputs and using alternative side conditions various optimal information processing rules are derived and evaluated generally output information input information for these rules and thus they are 100 efficient learning rules when different weights or costs are associated with alternative inputs anchoring like effects much emphasized in the psychological literature are the results of optimal information processing procedures further dynamic information processing results are reviewed and extensions noted last some implications of the information processing approach for learning from data will be discussed

resonances and transport in the sawtooth map we study transport in a completely chaotic hamiltonian system the hyperbolic sawtooth map analytical expressions are obtained for its cantori and resonances we show that resonances give a complete partition of phase space the flux leaking out of a resonance is given by its turnstiles whose form and areas are obtained analytically when the total flux out of a resonance becomes one third the area of an island the topology of the turnstiles changes at the same parameter value a horseshoe is formed corresponding to the orbits trapped within the resonance based on this a coding scheme for the trapped orbits is introduced and expressions for trapped ordered orbits are obtained the partial flux transferred from one resonance to another is determined by the degree of overlap of their turnstiles we calculate the survival probability within a resonance using the markov model the results are compared with results obtained numerically and from periodic orbit theory

minimax redundancy for the class of memoryless sources let x sup n x sub 1 x sub n be a memoryless source with unknown distribution on a finite alphabet of size k we identify the asymptotic minimax coding redundancy for this class of sources and provide a sequence of asymptotically minimax codes equivalently we determine the limiting behavior of the minimax relative entropy min sub qxn max sub pxn d p sub xn spl par q sub xn where the maximum is over all independent and identically distributed i i d source distributions and the minimum is over all joint distributions we show in this paper that the minimax redundancy minus k 1 2 log n 2 spl pi e converges to log spl int spl radic det i spl theta d spl theta log spl gamma 1 2 sup k spl gamma k 2 where i spl theta is the fisher information and the integral is over the whole probability simplex the bayes strategy using jeffreys prior is shown to be asymptotically maximin but not asymptotically minimax in our setting the boundary risk using jeffreys prior is higher than that of interior points we provide a sequence of modifications of jeffreys prior that put some prior mass near the boundaries of the probability simplex to pull down that risk to the asymptotic minimax level in the limit

an information criterion for likelihood selection for a given source distribution we establish properties of the conditional density achieving the rate distortion function lower bound as the distortion parameter varies in the limit as the distortion tolerated goes to zero the conditional density achieving the rate distortion function lower bound becomes degenerate in the sense that the channel it defines becomes error free as the permitted distortion increases to its limit the conditional density achieving the rate distortion function lower bound defines a channel which no longer depends on the source distribution in addition to the data compression motivation we establish two results one asymptotic one nonasymptotic showing that the the conditional densities achieving the rate distortion function lower bound make relatively weak assumptions on the dependence between the source and its representation this corresponds in bayes estimation to choosing a likelihood which makes relatively weak assumptions on the data generating mechanism if the source is regarded as a prior taken together these results suggest one can use the conditional density obtained from the rate distortion function in data analysis that is when it is impossible to identify a true parametric family on the basis of physical modeling our results provide both data compression and channel coding justification for using the conditional density achieving the rate distortion function lower bound as a likelihood

estimating a product of means bayesian analysis with reference priors abstract suppose that we observe x n 1 and independently y n 1 and are concerned with inference mainly estimation and confidence statements about the product of means this problem arises most obviously in situations of determining area based on measurements of length and width it also arises in other practical contexts however for instance in gypsy moth studies the hatching rate of larvae per unit area can be estimated as the product of the mean of egg masses per unit area times the mean number of larvae hatching per egg mass approximately independent samples can be obtained for each mean see southwood 1978 noninformative prior bayesian approaches to the problem are considered in particular the reference prior approach of bernardo 1979 an appropriate reference prior for the problem is developed and relatively easily implementable formulas for posterior moments e g the posterior mean and variance and credible sets are derived comparisons with alternative noninf

toward a fundamental theory of optimal feature selection part i several authors have studied the problem of dimensionality reduction or feature selection using statistical distance measures e g the chernoff coefficient bhattacharyya distance i divergence and j divergence because they generally felt that direct use of the probability of classification error expression was either computationally or mathematically intractable we show that for the difficult problem of testing one weakly stationary gaussian stochastic process against another when the mean vectors are similar and the covariance matrices patterns differ the probability of error expression may be dealt with directly using a combination of classical methods and distribution function theory the results offer a new and accurate finite dimensionality information theoretic strategy to feature selection and are shown by use of examples to be superior to the well known kadota shepp approach which employs distance measures and asymptotics in its formulation the present part i deals with the theory part ii deals with the implementation of a computer based real time pattern classifier which takes into account a realistic quasi stationarity of the patterns

reference priors with partial information summary in this paper reference priors are derived for three cases where partial information is available if a subjective conditional prior is given two reasonable methods are proposed for finding the marginal reference prior if instead a subjective marginal prior is available a method for defining the conditional reference prior is proposed a sufficient condition is then given under which this conditional reference prior agrees with the conditional reference prior derived in the first stage of the reference prior algorithm of berger bernardo 1989 1992 finally under the assumption of independence a method for finding marginal reference priors is also proposed various examples are given to illustrate the methods

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection

default priors for bayesian and frequentist inference summary we investigate the choice of default priors for use with likelihood for bayesian and frequentist inference such a prior is a density or relative density that weights an observed likelihood function leading to the elimination of parameters that are not of interest and then a density type assessment for a parameter of interest for independent responses from a continuous model we develop a prior for the full parameter that is closely linked to the original bayes approach and provides an extension of the right invariant measure to general contexts we then develop a modified prior that is targeted on a component parameter of interest and by targeting avoids the marginalization paradoxes of dawid and co workers this modifies jeffreys s prior and provides extensions to the development of welch and peers these two approaches are combined to explore priors for a vector parameter of interest in the presence of a vector nuisance parameter examples are given to illustrate the computation of the priors