new approach to the correlation spectrum near intermittency a quantum mechanical analogy the correlation spectrum of fully developed one dimensional mappings is studied near and at a weakly intermittent situation using a suitable infinitematrix representation the eigenvalue equation of the frobenius perron operator is approximately reduced to the radial schrodinger equation of the hydrogen atom corrections are calculated by quantum mechanical perturbation theory analytical expressions for the spectral properties and correlation functions are derived and checked numerically compared to our previous work the accuracy of the present results is significantly higher owing to the controlled and systematic approximation scheme

color image watermark extraction based on support vector machines abstract this work proposes a novel watermarking technique called svm based color image watermarking sciw based on support vector machines svms for the authentication of color images to protect the copyright of a color image a signature a watermark which is represented by a sequence of binary data is embedded in the color image the watermark extraction issue can be treated as a classification problem involving binary classes the sciw method constructs a set of training patterns with the use of binary labels by employing three image features which are the differences between a local image statistic and the luminance value of the center pixel in a sliding window with three distinct shapes this set of training patterns is gathered from a pair of images an original image and its corresponding watermarked image in the spatial domain a quasi optimal hyperplane a binary classifier can be realized by an svm the sciw method utilizes this set of training patterns to train the svm and then applies the trained svm to classify a set of testing patterns following the results produced by the classifier the trained svm the sciw method retrieves the hidden signature without the original image during watermark extraction experimental results have demonstrated that the sciw method is sufficiently robust against several color image manipulations and that it outperforms other proposed methods considered in this work

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

perceptually adaptive spread transform image watermarking scheme using hadamard transform the present paper proposes a digital image watermarking scheme using the characteristics of the human visual system hvs spread transform technique and statistical information measure spread transform st scheme is implemented using the transform coefficients of both the host and the watermark signal watermark embedding strength is adaptively adjusted using frequency sensitivity luminance contrast and entropy masking of hvs model the choice of hadamard transform as watermark embedding domain offers several advantages such as low loss in image information higher image fidelity greater reliability of watermark detection and higher data hiding capacity at high degree of compression performance of the proposed method is compared with a number of recently reported watermarking schemes based on spread spectrum ss and quantization index modulation qim

on the estimation of the order of a markov chain and universal data compression the authors estimate the order of a finite markov source based on empirically observed statistics the performance criterion adopted is to minimize the probability of underestimating the model order while keeping the overestimation probability exponent at a prescribed level a universal asymptotically optimal test in the sense just defined is proposed for the case where a given integer is known to be the upper bound of the true order for the case where such a bound is unavailable an alternative rule based on the lempel ziv data compression algorithm is shown to be asymptotically optimal also and computationally more efficient

optimal image watermarking using hybrid optimization algorithm image watermarking is a process of embedding secret information into cover image to secure transmission of secret data literature presents several image watermarking techniques are based on different transformation such as wavelet transform fourier transform and cosine transform most of the authors have been developed an embedding and extraction algorithm newly with the transformed image then the optimal location for embedding the secret data was identified by using optimization algorithm accordingly the authors have developed an optimal robust watermarking technique using genetic algorithm and wavelet transform in the previous work watermarks were embedded into the wavelet coefficients of hl and lh band after searching the optimal locations in order to improve both quality of watermarked image and robustness of the watermark in this work the authors have developed to improve the genetic algorithm by combining it with artificial bee colony algorithm abc algorithm here they have used hybrid algorithm for finding of optimal location in watermarking process finally the comparative evaluation of the hybrid algorithm will be done with the existing and previous technique using different images and the performance of the extended algorithm will be analyzed using the psnr nc with convergence rate

estimating the number of states of a finite state source the problem of estimating the number of states of a finite alphabet finite state source is investigated an estimator is developed that asymptotically attains the minimum probability of understanding the number of states among all estimators with a prescribed exponential decay rate of overestimation probability the proposed estimator relies on the lempel ziv data compression algorithm in an intuitively appealing manner

joint universal lossy coding and identification of stationary mixing sources with general alphabets in this paper we consider the problem of joint universal variable rate lossy coding and identification for parametric classes of stationary beta mixing sources with general polish alphabets compression performance is measured in terms of lagrangians while identification performance is measured by the variational distance between the true source and the estimated source provided that the sources are mixing at a sufficiently fast rate and satisfy certain smoothness and vapnik chervonenkis vc learnability conditions it is shown that for bounded metric distortions there exist universal schemes for joint lossy compression and identification whose lagrangian redundancies converge to zero as radic vn logn n as the block length n tends to infinity where vn is the vc dimension of a certain class of decision regions defined by the n dimensional marginal distributions of the sources furthermore for each n the decoder can identify n dimensional marginal of the active source up to a ball of radius o radic vnlogn n in variational distance eventually with probability one the results are supplemented by several examples of parametric sources satisfying the regularity conditions

eigenvalue spectrum of the frobenius perron operator near intermittency the spectral properties of the frobenius perron operator of one dimensional maps are studied when approaching a weakly intermittent situation numerical investigation of a particular family of maps shows that the spectrum becomes extremely dense and the eigenfunctions become concentrated in the vicinity of the intermittent fixed point analytical considerations generalize the results to a broader class of maps near and at weak intermittency and show that one branch of the map is dominant in the determination of the spectrum explicit approximate expressions are derived for both the eigenvalues and the eigenfunctions and are compared with the numerical results textcopyright 1996 the american physical society

correlation properties of exactly solvable chaotic oscillators we derive exact expressions for the autocorrelation and cross correlation functions of wave forms generated by two exactly solvable chaotic systems for each system an analytic expression exists for chaotic solutions which we use to evaluate correlation integrals explicitly for some specific parameter values we calculate the mean and variance of the correlation functions averaged over all possible solutions the mean autocorrelation is shown to decay at a rate equal to the kolmogorov sinai entropy all derived results are shown to agree with numerical calculations of the corresponding quantities

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

strongly consistent code based identification and order estimation for constrained finite state model classes observations are made of data generated by a stationary ergodic finite alphabet information source according to an unknown statistical model two modeling problems the identification problem and the order estimation problem are considered it is required that the given model class in the identification problem and each given model class in the order estimation problem be a constrained finite state model class which is a type that includes many model classes of information theoretic interest strongly consistent decision rules are exhibited in both the identification problem and the order estimation problem the decision rules are code based in that a model class is chosen based on how well a certain code for that class encodes the observed data the code used for a model class is based on the maximum likelihood code for that class and asymptotic code performance is gauged by means of a key property of divergence rate distance

on strong consistency of model selection in classification this paper considers model selection in classification in many applications such as pattern recognition probabilistic inference using a bayesian network prediction of the next in a sequence based on a markov chain the conditional probability p y y x x of class yisiny given attribute value xisinx is utilized by model we mean the equivalence relation in x for x x isinxx x harrp y y x x p y y x x forall yisiny by classification we mean the number of such equivalence classes is finite we estimate the model from n samples zn xi yi i 1 n isin xtimesy n using information criteria in the form empirical entropy h plus penalty term k 2 dn the model such that h k 2 dn is minimized is the estimated model where k is the number of independent parameters in the model and dn n 1 infin is a real nonnegative sequence such that lim supndn n 0 for autoregressive processes although the definitions of h and k are different it is known that the estimated model almost surely coincides with the true model as nrarrinfin if dn n 1 infin 2loglogn n 1 infin and that it does not if dn n 1 infin 2loglogn n 1 infin hannan and quinn the problem whether the same property is true for classification was open this paper solves the problem in the affirmative

hidden markov processes an overview of statistical and information theoretic aspects of hidden markov processes hmps is presented an hmp is a discrete time finite state homogeneous markov chain observed through a discrete time memoryless invariant channel in recent years the work of baum and petrie 1966 on finite state finite alphabet hmps was expanded to hmps with finite as well as continuous state spaces and a general alphabet in particular statistical properties and ergodic theorems for relative entropy densities of hmps were developed consistency and asymptotic normality of the maximum likelihood ml parameter estimator were proved under some mild conditions similar results were established for switching autoregressive processes these processes generalize hmps new algorithms were developed for estimating the state parameter and order of an hmp for universal coding and classification of hmps and for universal decoding of hidden markov channels these and other related topics are reviewed

nonparametric statistical inference for ergodic processes in this work a method for statistical analysis of time series is proposed which is used to obtain solutions to some classical problems of mathematical statistics under the only assumption that the process generating the data is stationary ergodic namely three problems are considered goodness of fit or identity testing process classification and the change point problem for each of the problems a test is constructed that is asymptotically accurate for the case when the data is generated by stationary ergodic processes the tests are based on empirical estimates of distributional distance

optimal error exponents in hidden markov models order estimation we consider the estimation of the number of hidden states the order of a discrete time finite alphabet hidden markov model hmm the estimators we investigate are related to code based order estimators penalized maximum likelihood ml estimators and penalized versions of the mixture estimator introduced by liu and narayan 1994 we prove strong consistency of those estimators without assuming any a priori upper bound on the order and smaller penalties than previous works we prove a version of stein s lemma for hmm order estimation and derive an upper bound on underestimation exponents then we prove that this upper bound can be achieved by the penalized ml estimator and by the penalized mixture estimator the proof of the latter result gets around the elusive nature of the ml in hmm by resorting to large deviation techniques for empirical processes finally we prove that for any consistent hmm order estimator for most hmm the overestimation exponent is null

consistent order estimation and minimal penalties consider an i i d sequence of random variables whose distribution f lies in one of the nested families of models mq q 1 the smallest index q such that mq contains f is called the model order the aim of this paper is to explore the consistency properties of penalized likelihood model order estimators such as bayesian information criterion we show in a general setting that the minimal strongly consistent penalty is of order q loglogn where q is a dimensional quantity in contrast to previous work an a priori upper bound on the model order is not assumed the results rely on a sharp characterization of the pathwise fluctuations of the generalized likelihood ratio statistic under entropy assumptions on the model classes our results are applied to the geometrically complex problem of location mixture order estimation which is widely used but poorly understood

error exponents for ar order testing this paper is concerned with error exponents in testing problems raised by autoregressive ar modeling the tests to be considered are variants of generalized likelihood ratio testing corresponding to traditional approaches to autoregressive moving average arma modeling estimation in several related problems such as markov order or hidden markov model order estimation optimal error exponents have been determined thanks to large deviations theory ar order testing is specially challenging since the natural tests rely on quadratic forms of gaussian processes in sharp contrast with empirical measures of markov chains the large deviation principles ldps satisfied by gaussian quadratic forms do not always admit an information theoretic representation despite this impediment we prove the existence of nontrivial error exponents for gaussian ar order testing and furthermore we exhibit situations where the exponents are optimal these results are obtained by showing that the log likelihood process indexed by ar models of a given order satisfy an ldp upper bound with a weakened information theoretic representation

resonances for axiom bf a flows etant donne un flot d axiome a sur m et des fonctions lisses b c m r on montre que la fonction de correlation en temps bc pour un etat de gibbs a une transformee de fourier bc mesomorphe dans une bande on etudie les residus des poles de bc

increasing selective pressure towards the best compromise in evolutionary multiobjective optimization the extended nosga method most current approaches in the evolutionary multiobjective optimization literature concentrate on adapting an evolutionary algorithm to generate an approximation of the pareto frontier however finding this set does not solve the problem the decision maker still has to choose the best compromise solution out of that set here we introduce a new characterization of the best compromise solution of a multiobjective optimization problem by using a relational system of preferences based on a multicriteria decision aid way of thinking and an outranked based dominance generalization we derive some necessary and sufficient conditions which describe satisfactory approximations to the best compromise such conditions define a lexicographic minimum of a bi objective optimization problem which is a map of the original one the nosga ii method is a nsga ii inspired efficient way of solving the resulting mapped problem

performance analysis of grammar based codes revisited the compression performance of grammar based codes is revisited from a new perspective previously the compression performance of grammar based codes was evaluated against that of the best arithmetic coding algorithm with finite contexts in this correspondence we first define semifinite state sources and finite order semi markov sources based on the definitions of semifinite state sources and finite order semi markov sources and the idea of run length encoding rle we then extend traditional rle algorithms to context based rle algorithms rle algorithms with k contexts and rle algorithms of order k where k is a nonnegative integer for each individual sequence x let r sup sub sr k x and r sup sub sr k x be the best compression rate given by rle algorithms with k contexts and by rle algorithms of order k respectively it is proved that for any x r sup sub sr k is no greater than the best compression rate among all arithmetic coding algorithms with k contexts furthermore it is shown that there exist stationary ergodic semi markov sources for which the best rle algorithms without any context outperform the best arithmetic coding algorithms with any finite number of contexts finally we show that the worst case redundancies of grammar based codes against r sup sub sr k x and r sup sub sr k x among all length n individual sequences x from a finite alphabet are upper bounded by d sub 1 loglogn logn and d sub 2 loglogn logn respectively where d sub 1 and d sub 2 are constants this redundancy result is stronger than all previous corresponding results

an optimal image watermarking approach based on a multi objective genetic algorithm in accordance with the multi objective nature of image watermarking an optimal image watermarking approach using a multi objective genetic algorithm is presented in this paper both watermarking parameters and embedding positions are often important factors affecting the performance of watermarking systems the proposed multi objective watermarking method can automatically optimize system parameters and a variable length mechanism is specially designed to search the most suitable positions for embedding watermarks the method can also remove the difficult issue of determining optimal watermarking parameters from previous watermarking algorithms the proposed multi objective watermarking method directly deals with the problem of optimizing watermarking under non dominated meaning thus it can effectively avoid the difficulty of determining the optimally weighted factor in existing single objective watermarking schemes in addition a pareto optimal set generated by multi objective optimization can provide flexibility in selecting the most suitable watermarking parameters according to practical requirements

large scale multiple testing under dependence summary the paper considers the problem of multiple testing under dependence in a compound decision theoretic framework the observed data are assumed to be generated from an underlying two state hidden markov model we propose oracle and asymptotically optimal datadriven procedures that aim to minimize the false non discovery rate fnr subject to a constraint on the false discovery rate fdr it is shown that the performance of a multiple testing procedure can be substantially improved by adaptively exploiting the dependence structure among hypotheses and hence conventional fdr procedures that ignore this structural information are inefficient both theoretical properties and numerical performances of the procedures proposed are investigated it is shown that the procedures proposed control fdr at the desired level enjoy certain optimality properties and are especially powerful in identifying clustered non null cases the new procedure is applied to an influenza like illness surveillance study for detecting the timing of epidemic periods

universal filtering via hidden markov modeling the problem of discrete universal filtering in which the components of a discrete signal emitted by an unknown source and corrupted by a known discrete memoryless channel dmc are to be causally estimated is considered a family of filters are derived and are shown to be universally asymptotically optimal in the sense of achieving the optimum filtering performance when the clean signal is stationary ergodic and satisfies an additional mild positivity condition our schemes are comprised of approximating the noisy signal using a hidden markov process hmp via maximum likelihood ml estimation followed by the use of the forward recursions for hmp state estimation it is shown that as the data length increases and as the number of states in the hmp approximation increases our family of filters attains the performance of the optimal distribution dependent filter an extension to the case of channels with memory is also established

autocorrelation function and spectral linewidth of spiral chaos in a physical experiment we present results of physical experiments where we measure the autocorrelation function acf and the spectral linewidth of the basic frequency of a spiral chaotic attractor in a generator with inertial nonlinearity both without and in the presence of external noise it is shown that the acf of spiral attractors decays according to an exponential law with a decrement which is defined by the phase diffusion coefficient it is also established that the evolution of the instantaneous phase can be approximated by a wiener random process

markov approximations and decay of correlations for anosov flows we develop markov approximations for very general suspension flows based on this we obtain a stretched exponential bound on time correlation functions for 3 d anosov flows that verify uniform nonintegrability of foliations these include contact anosov flows and geodesic flows on compact surfaces of variable negative curvature our bound on correlations is stable under small smooth perturbations

on the choice of a model to fit data from an exponential family let x 1 cdots x n be iid observations coming from an exponential family the problem of interest is this given a finite number of models m j smoothly curved manifolds in mathbb r k choose the best model to fit the observations with some penalty for choosing models with dimensions which are too large a result of schwarz is made more specific and is extended to the case where the models are curved manifolds if s y n j is up to a constant c n independent of the model the log of the posterior probability of the j th model where the sample mean y n 1 n sum n i 1 x i has been replaced by y schwarz suggested an asymptotic expansion of s y n j whose leading terms are gamma y n j n sup psi in m j cap theta y psi b psi frac 1 2 k j log n in the case where the models are affine subspaces of mathbb r k we establish a similar asymptotic expansion including the next term with uniform bounds for y in a compact neighborhood of nabla b theta where theta is the true value of the parameter we suggest a criterion for the choice of the best model that consists of maximizing the three leading terms in the expansion s y n j we show that the criterion gives the correct model with probabilities p n theta rightarrow 1 as n rightarrow infty

on decay of correlations in anosov flows there is some disagreement about the meaning of the phrase chaotic flow however there is no doubt that mixing anosov flows provides an example of such systems anosov systems were introduced and extensively studied in his classical memoir a among other things he proved the following fact known now as anosov alternative for flows either every strong stable and strong unstable manifold is everywhere dense or the flow gt is a suspension over an anosov diffeomorphism by a constant roof function if the first alternative holds gt is mixing with respect to every gibbs measure see pp2 now the natural question is what is the estimated rate of mixing this is certainly one of the simplest questions concerning correlation decay in continuous time systems nevertheless the only results obtained until recently dealt with the case when the system discussed had an additional algebraic structure the easier case of anosov diffeomorphisms can be treated by the methods of thermodynamic formalism of sinai ruelle and bowen b2 namely one uses markov partitions to construct an isomorphism between the diffeomorphism and a subshift of finite type and then proves that all such subshifts are exponentially mixing this method would succeed also for flows if any suspension over a subshift of finite type had exponentially decaying correlations however the simplest example suspensions with locally constant roof functions never have such a property ri one can use the above observation to produce examples of axiom a flows with arbitrary slow correlation decay it became clear therefore that some additional geometric properties should be taken into account in recent work chernov chl ch2 has employed a uniform nonintegrability condition to get a subexponential estimate for correlation functions for geodesic flows on surfaces of variable negative curvature his method relies on the technique of markov approximation developed in chl the aim of this paper is to combine geometric considerations of chernov with the thermodynamic formalism approach the later method seems to be more appropriate than markov approximations since it gives simpler

correlation analysis of dynamical chaos abstract we study correlation and spectral properties of chaotic self sustained oscillations of different types it is shown that some classical models of stochastic processes can be used to describe behavior of autocorrelation functions of chaos the influence of noise on chaotic systems is also considered

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

a new approach for optimization in image watermarking by using genetic algorithms in this paper the authors propose the spread spectrum image watermarking algorithm using the discrete multiwavelet transform performance improvement with respect to existing algorithms is obtained by genetic algorithms optimization in the proposed optimization process the authors search for parameters that consist of threshold values and the embedding strength to improve the visual quality of watermarked images and the robustness of the watermark these parameters are varied to find the most suitable for images with different characteristics the experimental results show that the proposed algorithm yields a watermark that is invisible to human eyes and robust to various image manipulations the authors also compare their experimental results with the results of previous work using various test images

order estimation and sequential universal data compression of a hidden markov source by the method of mixtures we consider first the estimation of the order i e the number of states of a discrete time finite alphabet stationary ergodic hidden markov source hms our estimator uses a description of the observed data in terms of a uniquely decodable code with respect to a mixture distribution obtained by suitably mixing a parametric family of distributions on the observation space this procedure avoids maximum likelihood calculations the order estimator is shown to be strongly consistent with the probability of underestimation decaying exponentially fast in the number n of observations while the probability of overestimation does not exceed cn sup 3 where c is a constant next we present a sequential algorithm for the uniquely decodable universal data compression of the hms which performs an on line estimation of source order followed by arithmetic coding this code asymptotically attains optimum average redundancy

decay of correlations dedicated to micheline ishay i would like to thank p boyland l chierchia v donnay g de martino c gole j l lebowitz m lyubich m rychlik i g schwarz s vaienti and especially g gallavotti for helpful and enlightening discussions particularly warm thanks go to n chernov for carefully reading and finding a mistake in an early version the present paper benefits from several improvements due to his sharp criticism in addition i am indebted to p collet and most of all m wojtkowski for introducing me to the magical world of cones finally i thank j milnor director of the institute for mathematical sciences at stony brook university where i was visiting during part of this work and the italian c n r gnfm for providing travel funds

multi objective evolutionary algorithms based on the summation of normalized objectives and diversified selection most multi objective evolutionary algorithms moeas use the concept of dominance in the search process to select the top solutions as parents in an elitist manner however as moeas are probabilistic search methods some useful information may be wasted if the dominated solutions are completely disregarded in addition the diversity may be lost during the early stages of the search process leading to a locally optimal or partial pareto front beside this the non domination sorting process is complex and time consuming to overcome these problems this paper proposes multi objective evolutionary algorithms based on summation of normalized objective values and diversified selection snov ds the performance of this algorithm is tested on a set of benchmark problems using both multi objective evolutionary programming moep and multi objective differential evolution mode with the proposed method the performance metric has improved significantly and the speed of the parent selection process has also increased when compared with the non domination sorting in addition the proposed algorithm also outperforms ten other algorithms

on the minimal penalty for markov order estimation we show that large scale typicality of markov sample paths implies that the likelihood ratio statistic satisfies a law of iterated logarithm uniformly to the same scale as a consequence the penalized likelihood markov order estimator is strongly consistent for penalties growing as slowly as log log n when an upper bound is imposed on the order which may grow as rapidly as log n our method of proof using techniques from empirical process theory does not rely on the explicit expression for the maximum likelihood estimator in the markov case and could therefore be applicable in other settings