space time multiscale model for wave propagation in heterogeneous media abstract a space time multiscale model for wave propagation in heterogeneous media is developed the model builds on the authors previous work on the higher order mathematical homogenization theory with multiple spatial and temporal scales and is aimed at addressing the issues of stability and mathematical consistency starting from the weak forms of homogenized macroscopic equations of motion terms causing the solution secularity are identified and enforced to vanish this condition recovers the missing boundary conditions and gives rise to two secularity constrains imposing the uniform validity of asymptotic expansions finite element semi discretization in space along with an analytical solution in time are employed to incorporate the secularity constrains in the leading order solution and account for the slow time dependence of the leading solution pade approximation is utilized to develop the time stepping schemes on the fast time scale the formulation is verified for wave propagation problems in semi infinite and finite domains

on the limiting distribution of lempel ziv 78 redundancy for memoryless sources we study the lempel ziv 78 algorithm and show that its normalized redundancy rate tends to a gaussian distribution for memoryless sources we accomplish it by extending findings from our 1995 paper in particular by presenting a new simplified proof of the central limit theorem clt for the number of phrases in the lz 78 algorithm we first analyze the asymptotic behavior of the total path length in the associated digital search tree built from independent sequences then a renewal theory type argument yields clt for lz 78 scheme here we extend our analysis of lz 78 algorithm to present new results on the convergence of moments moderate and large deviations and clt for the normalized redundancy in particular we confirm that the average redundancy rate decays as 1 log n and we find that the variance is of order 1 n where n is the length of the text

counting markov types balanced matrices and eulerian graphs the method of types is one of the most popular techniques in information theory and combinatorics two sequences of equal length have the same type if they have identical empirical distributions in this paper we focus on markov types that is sequences generated by a markov source of order one we note that sequences having the same markov type share the same so called balanced frequency matrix that counts the number of distinct pairs of symbols we enumerate the number of markov types for sequences of length over an alphabet of size this turns out to be asymptotically equivalent to estimating the number of the balanced frequency matrices the number of integer solutions of a system of linear diophantine equations and the number of connected eulerian multigraphs for fixed we prove that the number of markov types is asymptotically equal to d m nm2 m m2 m where we give an integral representation for d m for m we conclude that asymptotically the number of types is equivalent to 2m3m 2em2 m2m22m m 2 nm2 m provided that m o n1 4 these findings are derived by analytical techniques ranging from analytic combinatorics to multidimensional generating functions to the saddle point method

an adaptive gmsfem for high contrast flow problems in this paper we derive an a posteriori error indicator for the generalized multiscale finite element method gmsfem framework this error indicator is further used to develop an adaptive enrichment algorithm for the linear elliptic equation with multiscale high contrast coefficients the gmsfem which has recently been introduced in 13 allows solving multiscale parameter dependent problems at a reduced computational cost by constructing a reduced order representation of the solution on a coarse grid the main idea of the method consists of 1 the construction of snapshot space 2 the construction of the offline space and 3 the construction of the online space the latter for parameter dependent problems in 13 it was shown that the gmsfem provides a flexible tool to solve multiscale problems with a complex input space by generating appropriate snapshot offline and online spaces in this paper we study an adaptive enrichment procedure and derive an a posteriori error indicator which gives an estimate of the local error over coarse grid regions we consider two kinds of error indicators where one is based on the l 2 norm of the local residual and the other is based on the weighted h 1 norm of the local residual where the weight is related to the coefficient of the elliptic equation we show that the use of weighted h 1 norm residual gives a more robust error indicator which works well for cases with high contrast media the convergence analysis of the method is given in our analysis we do not consider the error due to the fine grid discretization of local problems and only study the errors due to the enrichment numerical results are presented that demonstrate the robustness of the proposed error indicators

isolating segments fixed point index and symbolic dynamics ii homoclinic solutions we present a topological method for detecting homoclinic solutions in nonautonomous ode s as an application we show that there exist infinitely many geometrically distinct homoclinic solutions to the trivial 0 solution in the planar systemz z 1 ei t z 2 for 0 0 495

the asymptotic redundancy of bayes rules for markov chains we derive the asymptotics of the redundancy of bayes rules for markov chains of fixed order over a finite alphabet extending the work of barron and clarke 1990 on independent and identically distributed i i d sources the asymptotics are derived when the actual source is the class of spl phi mixing sources which strictly includes markov chains these results can be used to derive minimax asymptotic rates of convergence for universal codes when a markov chain of fixed order is used as a model

the method of types information theory the method of types is one of the key technical tools in shannon theory and this tool is valuable also in other fields in this paper some key applications are presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method and a wide selection of further applications are surveyed these range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems while the method of types is suitable primarily for discrete memoryless models its extensions to certain models with memory are also discussed

separation of random number generation and resolvability we consider the problem of determining when a given source can be used to approximate the output due to any input to a given channel we provide achievability and converse results for a general source and channel for the special case of a full rank discrete memoryless channel we give a stronger converse result than we can give for a general channel

dynamic data driven bayesian gmsfem abstract in this paper we propose a bayesian approach for multiscale problems with the availability of dynamic observational data our method selects important degrees of freedom probabilistically in a generalized multiscale finite element method framework due to scale disparity in many multiscale applications computational models cannot resolve all scales dominant modes in the generalized multiscale finite element method are used as permanent basis functions which we use to compute an inexpensive multiscale solution and the associated uncertainties through our bayesian framework we can model approximate solutions by selecting the unresolved scales probabilistically we consider parabolic equations in heterogeneous media the temporal domain is partitioned into subintervals using residual information and given dynamic data we design appropriate prior distribution for modeling missing subgrid information the likelihood is designed to minimize the residual in the underlying pde problem and the mismatch of observational data using the resultant posterior distribution the sampling process identifies important degrees of freedom beyond permanent basis functions the method adds important degrees of freedom in resolving subgrid information and ensuring the accuracy of the observations

isolating segments fixed point index and symbolic dynamics iii applications abstract we prove the existence of complicated dynamics in some class of periodic systems on the plane the proof is based on the continuation of the considered system to the model one

the performance of universal encoding universal coding theory is surveyed from the viewpoint of the interplay between delay and redundancy the price for universality turns out to be acceptably small

interval algorithm for random number generation the problem of generating a random number with an arbitrary probability distribution by using a general biased m coin is studied an efficient and very simple algorithm based on the successive refinement of partitions of the unit interval 0 1 which we call the interval algorithm is proposed a fairly tight evaluation on the efficiency is given generalizations of the interval algorithm to the following cases are investigated 1 output sequence is independent and identically distributed i i d 2 output sequence is markov 3 input sequence is markov 4 input sequence and output sequence are both subject to arbitrary stochastic processes

remark on complicated dynamics of some planar system abstract in this paper we study the complicated dynamics generated by the planar periodic system z z 1 e i t z 2 for 0 0 495 we prove the existence of infinitely many geometrically distinct periodic solutions with some special properties

analysis of the heterogeneous multiscale method for parabolic homogenization problems the heterogeneous multiscale method hmm is applied to various parabolic problems with multiscale coefficients these problems can be either linear or nonlinear optimal estimates are proved for the error between the hmm solution and the homogenized solution

finding structure with randomness probabilistic algorithms for constructing approximate matrix decompositions low rank matrix approximations such as the truncated singular value decomposition and the rank revealing qr decomposition play a central role in data analysis and scientific computing this work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low rank matrix approximation these techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets this paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions these methods use random sampling to identify a subspace that captures most of the action of a matrix the input matrix is then compressed either explicitly or implicitly to this subspace and the reduced matrix is manipulated deterministically to obtain the desired low rank factorization in many cases this approach beats its classical competitors in terms of accuracy robustness and or speed these claims are supported by extensive numerical experiments and a detailed error analysis the specific benefits of randomized techniques depend on the computational environment consider the model problem of finding the k dominant components of the singular value decomposition of an m times n matrix i for a dense input matrix randomized algorithms require bigo mn log k floating point operations flops in contrast to bigo mnk for classical algorithms ii for a sparse input matrix the flop count matches classical krylov subspace methods but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures iii for a matrix that is too large to fit in fast memory the randomized techniques require only a constant number of passes over the data as opposed to bigo k passes for classical algorithms in fact it is sometimes possible to perform matrix approximation with a single pass over the data

a geometric method for detecting chaotic dynamics a new method of detection of chaos in dynamical systems generated by time periodic nonautonomous differential equations is presented it is based on the existence of some sets called periodic isolating segments in the extended phase space satisfying some topological conditions by chaos we mean the existence of a compact invariant set such that the poincare map is semiconjugated to the shift on two symbols and the counterimage by the semiconjugacy of any periodic point in the shift contains a periodic point of the poincare map as an application we prove that the planar equationz 1 ei t z 2 zgenerates chaotic dynamics provided 0 1 288

estimating the number of states of a finite state source the problem of estimating the number of states of a finite alphabet finite state source is investigated an estimator is developed that asymptotically attains the minimum probability of understanding the number of states among all estimators with a prescribed exponential decay rate of overestimation probability the proposed estimator relies on the lempel ziv data compression algorithm in an intuitively appealing manner

homogenization of parabolic equations with a continuum of space and time scales this paper addresses the issue of the homogenization of linear divergence form parabolic operators in situations where no ergodicity and no scale separation in time or space are available namely we consider divergence form linear parabolic operators in omega subset mathbb r n with l infty omega times 0 t coefficients it appears that the inverse operator maps the unit ball of l 2 omega times 0 t into a space of functions which at small time and space scales are close in h 1 norm to a functional space of dimension n it follows that once one has solved these equations at least n times it is possible to homogenize them both in space and in time reducing the number of operation counts necessary to obtain further solutions in practice we show under a cordes type condition that the first order time derivatives and second order space derivatives of the solution of these operators with respect to caloric coordinates are in l 2 instead of h 1 with euclidean coordinates if the medium is time independent then it is sufficient to solve n times the associated elliptic equation in order to homogenize the parabolic equation

universal delay limited simulation universal delay limited simulation of an unknown information source of a certain parametric family e g the family of memoryless sources or markov sources of a given order given a training sequence from that source and a stream of independent and uniformly distributed bits is considered the goal of universal simulation is that the probability law of the generated sequence be identical to that of the training sequence with minimum mutual information between the random processes generating both sequences in the delay limited setting the simulation algorithm generates a random sequence sequentially by delivering one symbol for each training symbol that is made available after a given initial delay whereas the random bits are assumed to be available on demand in this paper the optimal universal delay limited simulation scheme is characterized for broad parametric families and the mutual information achieved by the proposed scheme is analyzed the results are extended to a setting of variable delay

isolating segments fixed point index and symbolic dynamics abstract an extension of the recently introduced srzednicki wojcik method for detecting chaotic dynamics in periodically forced ordinary differential equations is presented as an application of the method we construct a topological model for the planar equation z 1 e i t z 2 z z c 1 and we show by a continuation argument that the symbolic dynamics on three symbols for the topological model continues to eq 1 for 0

a space time formulation for multiscale phenomena abstract we develop subgrid scale models for a class of nonsymmetric linear evolution operators by applying the variational multiscale method in space time the results generalize those of hughes 14 which were confined to the steady case the subgrid scale models are shown to be a paradigm for bubble function finite element methods and provide a theoretical and practical framework for the development of so called stabilized methods

generating random bits from an arbitrary source fundamental limits suppose we are given a random source and want to use it as a random number generator at what rate can we generate fair bits from it we address this question in an information theoretic setting by allowing for some arbitrarily small but nonzero deviation from ideal random bits we prove our results with three different measures of approximation between the ideal and the obtained probability distributions the variational distance the d bar distance and the normalized divergence two different contexts are studied fixed length and variable length random number generation the fixed length results of this paper provide an operational characterization of the inf entropy rate of a source defined in han and verdu see ibid vol 39 no 3 p 752 772 1993 and the variable length results characterize the liminf of the entropy rate thereby establishing a pleasing duality with the fundamental limits of source coding a feature of our results is that we do not restrict ourselves to ergodic or to stationary sources

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

generalized multiscale finite element methods gmsfem in this paper we propose a general approach called generalized multiscale finite element method gmsfem for performing multiscale simulations for problems without scale separation over a complex input space as in multiscale finite element methods msfems the main idea of the proposed approach is to construct a small dimensional local solution space that can be used to generate an efficient and accurate approximation to the multiscale solution with a potentially high dimensional input parameter space in the proposed approach we present a general procedure to construct the offline space that is used for a systematic enrichment of the coarse solution space in the online stage the enrichment in the online stage is performed based on a spectral decomposition of the offline space in the online stage for any input parameter a multiscale space is constructed to solve the global problem on a coarse grid the online space is constructed via a spectral decomposition of the offline space and by choosing the eigenvectors corresponding to the largest eigenvalues the computational saving is due to the fact that the construction of the online multiscale space for any input parameter is fast and this space can be re used for solving the forward problem with any forcing and boundary condition compared with the other approaches where global snapshots are used the local approach that we present in this paper allows us to eliminate unnecessary degrees of freedom on a coarse grid level we present various examples in the paper and some numerical results to demonstrate the effectiveness of our method

addendum to on universal simulation of information sources using training data in a recent paper merhav and weinberger ieee trans inf theory vol 50 no 1 p 5 20 2004 we studied the problem of universal simulation of an unknown information source of a certain parametric family given a training sequence from that source and given a limited budget of purely random bits the goal was to generate another random sequence of the same length or shorter whose probability law is identical to that of the given training sequence but with minimum statistical dependency minimum mutual information between the input training sequence and the output sequence in this addendum we point out a concrete optimal simulation scheme that is easy to implement as opposed to the nonconstructive existence result in that paper and we make a number of additional observations on the universal simulation problem

multiscale direction splitting algorithms for parabolic equations with highly heterogeneous coefficients in this paper we discuss two methods for upscaling of highly heterogeneous data for parabolic problems in the context of a direction splitting time approximation the first method is a direct application of the idea of jenny et al 2003 in the context of the direction splitting approach the second method devises the approximation from the schur complement corresponding to the interface unknowns of the coarse grid by applying a proper l 2 projection operator to it the spatial discretization employed in this paper is based on a mac finite volume stencil but the same approach can be implemented within a proper finite element discretization a key feature of the present approach is that it can extend to 3d problems with very little computational overhead the properties of the resulting approximations are demonstrated numerically on some benchmark coefficient data available in the literature

channel simulation by interval algorithm a performance analysis of interval algorithm this article deals with the problem of simulating a discrete memoryless channel and proposes two algorithms for channel simulation by using the interval algorithm the first algorithm provides exact channel simulation and the number of fair random bits per input sample approaches the conditional resolvability of the channel with probability one the second algorithm provides approximate channel simulation and the approximation error measured by the variational distance vanishes exponentially as the block length tends to infinity when the number of fair random bits per input sample is above the conditional resolvability further some asymptotic properties of these algorithms as well as the original interval algorithm for random number generation are clarified

redundancy of the lempel ziv incremental parsing rule the lempel ziv codes are universal variable to fixed length codes that have become virtually standard in practical lossless data compression for any given source output string from a markov or unifilar source we upper bound the difference between the number of binary digits needed to encode the string and the self information of the string we use this result to demonstrate that for unifilar or markov sources the redundancy of encoding the first n letters of the source output with the lempel ziv incremental parsing rule lz 78 the welch modification lzw or a new variant is o ln n sup 1 and we upper bound the exact form of convergence we conclude by considering the relationship between the code length and the empirical entropy associated with a string

simulation of random processes and rate distortion theory we study the randomness necessary for the simulation of a random process with given distributions on terms of the finite precision resolvability of the process finite precision resolvability is defined as the minimal random bit rate required by the simulator as a function of the accuracy with which the distributions are replicated the accuracy is quantified by means of various measures variational distance divergence orstein 1973 prohorov 1956 and related measures of distance between the distributions of random process in the case of ornstein prohorov and other distances of the kantorovich vasershtein type we show that the finite precision resolvability is equal to the rate distortion function with a fidelity criterion derived from the accuracy measure this connection leads to new results on nonstationary rate distortion theory in the case of variational distance the resolvability of stationary ergodic processes is shown to equal entropy rate regardless of the allowed accuracy in the case of normalized divergence explicit expressions for finite precision resolvability are obtained in many cases of interest and connections with data compression with minimum probability of block error are shown

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

a parametric texture model based on joint statistics of complex wavelet coefficients we present a universal statistical model for texture images in the context of an overcomplete complex wavelet transform the model is parameterized by a set of statistics computed on pairs of coefficients corresponding to basis functions at adjacent spatial locations orientations and scales we develop an efficient algorithm for synthesizing random images subject to these constraints by iteratively projecting onto the set of images satisfying each constraint and we use this to test the perceptual validity of the model in particular we demonstrate the necessity of subgroups of the parameter set by showing examples of texture synthesis that fail when those parameters are removed from the set we also demonstrate the power of our model by successfully synthesizing examples drawn from a diverse collection of artificial and natural textures

periodic points of the planar area preserving poincare map inside isolating segment abstract the aim of this note is to study the structure of the set of periodic points of the area preserving poincare map p associated to t periodic local planar process we will use the approach based on the method of periodic isolating segments isolating chains and the fixed point index theory the number of the applications to the processes generated by periodic in time non autonomous planar ode s are given

a sequential algorithm for the universal coding of finite memory sources the estimation and universal compression of discrete sources are considered and a sequential algorithm for the universal coding of finite memory sources attaining asymptotically minimum redundancy is presented the algorithm performs an online estimation of the source states and uses an arithmetic code

multiscale analysis of diffusion processes in composite media the goal of this paper is to present some homogenization results for a nonlinear problem arising in the modeling of diffusion in a periodic structure formed by two media with different properties separated by an active interface our setting is relevant for modeling heat diffusion in composite materials with imperfect interfaces or electrical conduction in biological tissues the approach we follow is based on the periodic unfolding method which allows us to deal with general media

on the average redundancy rate of the lempel ziv code in this paper we settle a long standing open problem concerning the average redundancy r sub n of the lempel ziv 78 lz78 code we prove that for a memoryless source the average redundancy rate attains asymptotically er sub n a spl delta n log n o log log n log sup 2 n where a is an explicitly given constant that depends on source characteristics and spl delta x is a fluctuating function with a small amplitude we also derive the leading term for the kth moment of the number of phrases we conclude by conjecturing a precise formula on the expected redundancy for a markovian source the main result of this paper is a consequence of the second order properties of the lempel ziv algorithm obtained by jacquet and szpankowski 1995 these findings have been established by analytical techniques of the precise analysis of algorithms we give a brief survey of these results since they are interesting in their own right and shed some light on the probabilistic behavior of pattern matching based data compression

markov types and minimax redundancy for markov sources redundancy of universal codes for a class of sources determines by how much the actual code length exceeds the optimal code length in the minimax scenario one designs the best code for the worst source within the class such minimax redundancy comes in two flavors average minimax or worst case minimax we study the worst case minimax redundancy of universal block codes for markovian sources of any order we prove that the maximal minimax redundancy for markov sources of order r is asymptotically equal to 1 2m sup r m 1 log sub 2 n log sub 2 a sub m sup r lnlnm sup 1 m 1 lnm o 1 where n is the length of a source sequence m is the size of the alphabet and a sub m sup r is an explicit constant e g we find that for a binary alphabet m 2 and markov of order r 1 the constant a sub 2 sup 1 16 spl middot g spl ap 14 655449504 where g is the catalan number unlike previous attempts we view the redundancy problem as an asymptotic evaluation of certain sums over a set of matrices representing markov types the enumeration of markov types is accomplished by reducing it to counting eulerian paths in a multigraph in particular we propose exact and asymptotic formulas for the number of strings of a given markov type all of these findings are obtained by analytic and combinatorial tools of analysis of algorithms

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

generalized multiscale finite element methods for space time heterogeneous parabolic equations abstract in this paper we consider local multiscale model reduction for problems with multiple scales in space and time we developed our approaches within the framework of the generalized multiscale finite element method gmsfem using space time coarse cells the main idea of gmsfem is to construct a local snapshot space and a local spectral decomposition in the snapshot space previous research in developing multiscale spaces within gmsfem focused on constructing multiscale spaces and relevant ingredients in space only in this paper our main objective is to develop a multiscale model reduction framework within gmsfem that uses space time coarse cells we construct space time snapshot and offline spaces we compute these snapshot solutions by solving local problems a complete snapshot space will use all possible boundary conditions however this can be very expensive we propose using randomized boundary conditions and oversampling cf calo et al 2016 we construct the local spectral decomposition based on our analysis as presented in the paper we present numerical results to confirm our theoretical findings and to show that using our proposed approaches we can obtain an accurate solution with low dimensional coarse spaces we discuss using online basis functions constructed in the online stage and using the residual information online basis functions use global information via the residual and provide fast convergence to the exact solution provided a sufficient number of offline basis functions we present numerical studies for our proposed online procedures we remark that the proposed method is a significant extension compared to existing methods which use coarse cells in space only because of 1 the parabolic nature of cell solutions 2 extra degrees of freedom associated with space time cells and 3 local boundary conditions in space time cells

twice universal simulation of markov sources and individual sequences the problem of universal simulation given a training sequence is studied both in a stochastic setting and for individual sequences in the stochastic setting the training sequence is assumed to be emitted by a markov source of unknown order extending previous work where the order is assumed known and leading to the notion of twice universal simulation a simulation scheme which partitions the set of sequences of a given length into classes is proposed for this setting and shown to be asymptotically optimal this partition extends the notion of type classes to the twice universal setting in the individual sequence scenario the same simulation scheme is shown to generate sequences which are statistically similar in a strong sense to the training sequence for statistics of any order while essentially maximizing the uncertainty on the output

type classes of context trees it is well known that a tree model does not always admit a finite state machine fsm representation with the same minimal number of parameters therefore known characterizations of type classes for fsms do not apply in general to tree models in this paper the type class of a sequence with respect to a given context tree is studied an exact formula is derived for the size of the class extending whittle s formula for type classes with respect to fsms the derivation is more intricate than in the fsm case since some basic properties of fsm types do not hold in general for tree types the derivation also yields an efficient enumeration of the tree type class a formula for the number of type classes with respect to is also derived the formula is asymptotically tight up to a multiplicative constant and also extends the corresponding result for fsms the asymptotic behavior of the number of type classes and of the size of a class is expressed in terms of the so called minimal canonical extension of t a tree that is generally larger than but smaller than its fsm closure

a space time least square finite element scheme for advection diffusion equations abstract a space time least square finite element scheme is presented for the advection diffusion problems at moderate to high peclet numbers this scheme is designed to eliminate spurious oscillations and can be used to define the steady state solution as the asymptotic transient solution for large time numerical results using linear elements in a 1d space and bilinear elements in a 2d space demonstrate the accuracy and the stability of the new scheme

performance of lz algorithms on individual sequences the performance of three versions of the lempel ziv 1976 algorithm on individual sequences is investigated it is shown that as the restart length goes to infinity each compresses an individual sequence as well as any block to variable finite state information lossless algorithm and that the same conclusion holds for sliding window lz as the window width goes to infinity examples are given showing that an infinite memory version outperforms such finite memory forms and that such finite memory forms can compress more than the ziv 1978 entropy which is the best compression attainable by finite state block to block codes that have vanishing probability of error

generalized multiscale finite element method symmetric interior penalty coupling motivated by applications to numerical simulations of flows in highly heterogeneous porous media we develop multiscale finite element methods for second order elliptic equations we discuss a multiscale model reduction technique in the framework of the discontinuous galerkin finite element method we propose two different finite element spaces on the coarse mesh the first space is based on a local eigenvalue problem that uses an interior weighted l 2 norm and a boundary weighted l 2 norm for computing the mass matrix the second choice is based on generation of a snapshot space and subsequent selection of a subspace of a reduced dimension the approximation with these multiscale spaces is based on the discontinuous galerkin finite element method framework we investigate the stability and derive error estimates for the methods and further experimentally study their performance on a representative number of numerical examples

universal modeling and coding the problems arising in the modeling and coding of strings for compression purposes are discussed the notion of an information source that simplifies and sharpens the traditional one is axiomatized and adaptive and nonadaptive models are defined with a measure of complexity assigned to the models a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all a general class of so called first in first out fifo arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models because the coding parameters are the probabilities that define the model their design is easy and the application of the code is straightforward even with adaptively changing source models

source and channel simulation using arbitrary randomness necessary and sufficient conditions for approximation of a general channel by a general source are proved for the special case of a deterministic channel input which corresponds to source simulation we prove a stronger necessary condition as the approximation criteria vanishing variational distance between the original and the approximated quantity is used for both of the problems both necessary and sufficient conditions for the two problems are based on some individual properties of the sources and the channel and are relatively easy to evaluate in particular unlike prior results for this problem our results do not require solving an optimization problem to test simulatability the results are illustrated with several nonergodic examples

perfectly secure encryption of individual sequences in analogy to the well known notion of finite state compressibility of individual sequences due to lempel and ziv we define a similar notion of finite state encryptability of an individual plain text sequence as the minimum asymptotic key rate that must be consumed by finite state encrypters so as to guarantee perfect secrecy in a well defined sense our main basic result is that the finite state encryptability is equal to the finite state compressibility for every individual sequence this is in parallelism to shannon s classical probabilistic counterpart result asserting that the minimum required key rate is equal to the entropy rate of the source however the redundancy defined as the gap between the upper bound direct part and the lower bound converse part in the encryption problem turns out to decay at a different rate in fact much slower than the analogous redundancy associated with the compression problem we also extend our main theorem in several directions allowing 1 availability of side information si at the encrypter decrypter eavesdropper 2 lossy reconstruction at the decrypter and 3 the combination of both lossy reconstruction and si in the spirit of the wyner ziv problem

the error exponent for the noiseless encoding of finite ergodic markov sources a new approach to the classical fixed length noiseless source coding problem is proposed for the case of finite ergodic markov sources this approach is based on simple counting arguments the central notion of markov type a set containing all the source sequences having the same transition counts from letter to letter is introduced and the cardinality of such a set is evaluated via graph theoretical tools the error exponent is shown to be a weighted average of informational divergences and the universal character of the result is stressed as a corollary the classical source coding theorem determining the achievable rates is rederived

universal coding with minimum probability of codeword length overflow lossless block to variable length source coding is studied for finite state finite alphabet sources the aim is to minimize the probability that the normalized length of the codeword will exceed a given threshold b subject to the kraft inequality it is shown that the lempel ziv algorithm 1978 asymptotically attains the optimal performance in the sense just defined independently of the source and the value of b for the subclass of unifilar markov sources faster convergence to the asymptotic optimum performance can be accomplished by using the minimum description length universal code for this subclass it is demonstrated that these universal codes are also nearly optimal in the sense of minimizing buffer overflow probability and asymptotically optimal in a competitive sense

compression of two dimensional data distortion free compressibility of individual pictures i e two dimensional arrays of data by finite state encoders is investigated for every individual infinite picture i a quantity rho i is defined called the compressibility of i which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for i by any finite state information lossless encoder this is demonstrated by means of a constructive coding theorem and its converse that apart from their asymptotic significance might also provide useful criteria for finite and practical data compression tasks the proposed picture compressibility is also shown to possess the properties that one would expect and require of a suitably defined concept of two dimensional entropy for arbitrary probabilistic ensembles of infinite pictures while the definition of rho i allows the use of different machines for different pictures the constructive coding theorem leads to a universal compression scheme that is asymptotically optimal for every picture the results are readily extendable to data arrays of any finite dimension

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

types of markov fields and tilings the method of types is one of the most popular techniques in information theory and combinatorics however thus far the method has been mostly applied to 1 d markov processes and it has not been thoroughly studied for general markov fields markov fields over a finite alphabet of size m ge 2 can be viewed as models for multidimensional systems with local interactions the locality of these interactions is represented by a shape mathcal s while its marking by symbols of the underlying alphabet is called a tile two assignments in a markov field have the same type if they have the same empirical distribution i e if they have the same number of tiles of a given type our goal is to study the growth of the number of possible markov field types in either a d dimensional box of lengths n 1 ldots n d or its cyclic counterpart a d dimensional torus we relate this question to the enumeration of nonnegative integer solutions of a large system of diophantine linear equations called the conservation laws we view a markov type as a vector in a d m mathcal s dimensional space and count the number of such vectors satisfying the conservation laws which turns out to be the number of integer points in a certain polytope for the torus this polytope is of dimension mu d 1 mathrm rk mathrm mathrm c where mathrm rk mathrm mathrm c is the number of linearly independent conservation laws mathrm mathrm c this provides an upper bound on the number of types then we construct a matching lower bound leading to the conclusion that the number of types in the torus markov field is theta n mu where n n 1 ldots n d these results are derived by geometric tools including ideas of discrete and convex multidimensional geometry

residual driven online generalized multiscale finite element methods the construction of local reduced order models via multiscale basis functions has been an area of active research in this paper we propose online multiscale basis functions which are constructed using the offline space and the current residual online multiscale basis functions are constructed adaptively in some selected regions based on our error indicators we derive an error estimator which shows that one needs to have an offline space with certain properties to guarantee that additional online multiscale basis function will decrease the error this error decrease is independent of physical parameters such as the contrast and multiple scales in the problem the offline spaces are constructed using generalized multiscale finite element methods gmsfem we show that if one chooses a sufficient number of offline basis functions one can guarantee that additional online multiscale basis functions will reduce the error independent of contrast we note that the construction of online basis functions is motivated by the fact that the offline space construction does not take into account distant effects using the residual information we can incorporate the distant information provided the offline approximation satisfies certain properties in the paper theoretical and numerical results are presented our numerical results show that if the offline space is sufficiently large in terms of the dimension such that the coarse space contains all multiscale spectral basis functions that correspond to small eigenvalues then the error reduction by adding online multiscale basis function is independent of the contrast we discuss various ways computing online multiscale basis functions which include a use of small dimensional offline spaces

universal simulation with fidelity criteria we consider the problem of universal simulation of a memoryless source with some partial extensions to markov sources based on a training sequence emitted from the source the objective is to maximize the conditional entropy of the simulated sequence given the training sequence subject to a certain distance constraint between the probability distribution of the output sequence and the probability distribution of the input training sequence we derive for several distance criteria single letter expressions for the maximum attainable conditional entropy as well as corresponding universal simulation schemes that asymptotically attain these maxima

application of a conservative generalized multiscale finite element method to flow models in this paper we propose a method for the construction of locally conservative flux fields from generalized multiscale finite element method gmsfem pressure solutions the flux values are obtained from an element based postprocessing procedure in which an independent set of 4x4 linear systems need to be solved to test the performance of the method we consider two heterogeneous permeability coefficients and couple the resulting fluxes to a two phase flow model the increase in accuracy associated with the computation of the gmsfem pressure solutions is inherited by the postprocessed flux fields and saturation solutions and is closely correlated to the size of the reduced order systems in particular the addition of more basis functions to the enriched coarse space yields solutions that more accurately capture the behavior of the fine scale model a number of numerical examples are offered to validate the performance of the method

asymptotic optimality and asymptotic equipartition properties of log optimum investment on montre comment un investisseur peut repartir ses fonds entre plusieurs opportunites d investissement pour maximiser le taux de croissance de son capital

a universal data compression system a universal data compression algorithm is described which is capable of compressing long strings generated by a finitely generated source with a near optimum per symbol length without prior knowledge of the source this class of sources may be viewed as a generalization of markov sources to random fields moreover the algorithm does not require a working storage much larger than that needed to describe the source generating parameters

factor maps and invariant distributional chaos abstract the main aim of this article is to show that maps with the specification property have invariant distributionally scrambled sets and that this kind of scrambled set can be transferred from factor to extension under finite to one factor maps this solves some open questions in the literature of the topic we also show how our method can be applied in practice taking as example poincare map of time periodic nonautonomous planar differential equation z v t z 1 e i t z 2 z 3 n where and n are sufficiently small positive real numbers

a space time galerkin least squares finite element formulation of the navier stokes equations for moving domain problems a space time galerkin least squares finite element formulation of the navier stokes equations is presented for the analysis of free surface flows moving spatial configurations and deforming fluid structure interfaces the variational equation is based on the time discontinuous galerkin method employing the physical entropy variables the space time elements are oriented in time to accommodate the spatial deformations if the elements are oriented along the particle paths the formulation is lagrangian and if they are fixed in time it is eulerian consequently this formulation is analogous to the arbitrary lagrangian eulerian ale technique a novel mesh rezoning strategy is presented to orient the elements in time and adapt the fluid mesh to the changing spatial configuration numerical results are presented to show the performance of the method

universal portfolios with side information we present a sequential investment algorithm the spl mu weighted universal portfolio with side information which achieves to first order in the exponent the same wealth as the best side information dependent investment strategy the best state constant rebalanced portfolio determined in hindsight from observed market and side information outcomes this is an individual sequence result which shows the difference between the exponential growth wealth of the best state constant rebalanced portfolio and the universal portfolio with side information is uniformly less than d 2n log n 1 k n log 2 for every stock market and side information sequence and for all time n here d k m 1 is the number of degrees of freedom in the state constant rebalanced portfolio with k states of side information and m stocks the proof of this result establishes a close connection between universal investment and universal data compression

optimal algorithms for universal random number generation from finite memory sources we study random number generators rngs both in the fixed to variable length fvr and the variable to fixed length vfr regimes in a universal setting in which the input is a finite memory source of arbitrary order and unknown parameters with arbitrary input and output finite alphabet sizes applying the method of types we characterize essentially unique optimal universal rngs that maximize the expected output respectively minimize the expected input length in the fvr respectively vfr case for the fvr case the rng studied is a generalization of elias s scheme while in the vfr case the general scheme is new we precisely characterize up to an additive constant the corresponding expected lengths which include second order terms similar to those encountered in universal data compression and universal simulation furthermore in the fvr case we consider also a twice universal setting in which the markov order k of the input source is also unknown

universal prediction of individual sequences the problem of predicting the next outcome of an individual binary sequence using finite memory is considered the finite state predictability of an infinite sequence is defined as the minimum fraction of prediction errors that can be made by any finite state fs predictor it is proven that this fs predictability can be achieved by universal sequential prediction schemes an efficient prediction procedure based on the incremental parsing procedure of the lempel ziv data compression algorithm is shown to achieve asymptotically the fs predictability some relations between compressibility and predictability are discussed and the predictability is proposed as an additional measure of the complexity of a sequence

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection