the sets of divergence points of self similar measures are residual abstract let be a self similar measure supported on a self similar set k with the open set condition for x k let a d x be the set of accumulation points of d r x log b x r log r as r 0 in this paper we show that for any closed non singleton subinterval i r the set of points x for which the set a d x equals i is either empty or residual

asymmetric evaluations of erasure and undetected error probabilities the problem of channel coding with the erasure option is revisited for discrete memoryless channels the interplay between the code rate the undetected and total error probabilities is characterized using the information spectrum method a sequence of codes of increasing blocklengths n is designed to illustrate this tradeoff furthermore for additive discrete memoryless channels with uniform input distribution we establish that our analysis is tight with respect to the ensemble average this is done by analyzing the ensemble performance in terms of a tradeoff between the code rate the undetected and total errors this tradeoff is parameterized by the threshold in a generalized likelihood ratio test two asymptotic regimes are studied first the code rate tends to the capacity of the channel at a rate slower than n 1 2 corresponding to the moderate deviations regime in this case both error probabilities decay subexponentially and asymmetrically the precise decay rates are characterized second the code rate tends to capacity at a rate of n 1 2 in this case the total error probability is asymptotically a positive constant while the undetected error probability decays as exp bn 1 2 for some b 0 the proof techniques involve the applications of a modified or shifted version of the gartner ellis theorem and the type class enumerator method to characterize the asymptotic behavior of a sequence of cumulant generating functions

a model based initial guess for estimating parameters in systems of ordinary differential equations summary the inverse problem of parameter estimation from noisy observations is a major challenge in statistical inference for dynamical systems parameter estimation is usually carried out by optimizing some criterion function over the parameter space unless the optimization process starts with a good initial guess the estimation may take an unreasonable amount of time and may converge to local solutions if at all in this article we introduce a novel technique for generating good initial guesses that can be used by any estimation method we focus on the fairly general and often applied class of systems linear in the parameters the new methodology bypasses numerical integration and can handle partially observed systems we illustrate the performance of the method using simulations and apply it to real data

second order resolvability intrinsic randomness and fixed length source coding for mixed sources information spectrum approach the second order achievable asymptotics in typical random number generation problems such as resolvability intrinsic randomness and fixed length source coding are considered in these problems several researchers have derived the first order and the second order achievability rates for general sources using the information spectrum methods although these formulas are general their computations are quite hard hence an attempt to address explicit computation problems of achievable rates is meaningful in particular for i i d sources the second order achievable rates have earlier been determined simply by using the asymptotic normality in this paper we consider mixed sources of two i i d sources the mixed source is a typical case of nonergodic sources and whose self information does not have the asymptotic normality nonetheless we can explicitly compute the second order achievable rates for these sources on the basis of two peak asymptotic normality in addition extensions of our results to more general mixed sources such as a mixture of countably infinite i i d sources or markov sources and a continuous mixture of i i d sources are considered

second order asymptotics for the gaussian mac with degraded message sets this paper studies the second order asymptotics of the gaussian multiple access channel with degraded message sets for a fixed average error probability varepsilon in 0 1 and an arbitrary point on the boundary of the capacity region we characterize the speed of convergence of rate pairs that converge to that boundary point for codes that have asymptotic error probability no larger than varepsilon as a stepping stone to this local notion of the second order asymptotics we study a global notion and establish relationships between the two we provide a numerical example to illustrate how the angle of approach to a boundary point affects the second order coding rate this is the first conclusive characterization of the second order asymptotics of a network information theory problem in which the capacity region is not a polygon

optimal lossless data compression non asymptotics and asymptotics this paper provides an extensive study of the behavior of the best achievable rate and other related fundamental limits in variable length strictly lossless compression in the non asymptotic regime the fundamental limits of fixed to variable lossless compression with and without prefix constraints are shown to be tightly coupled several precise quantitative bounds are derived connecting the distribution of the optimal code lengths to the source information spectrum and an exact analysis of the best achievable rate for arbitrary sources is given fine asymptotic results are proved for arbitrary not necessarily prefix compressors on general mixing sources nonasymptotic explicit gaussian approximation bounds are established for the best achievable rate on markov sources the source dispersion and the source varentropy rate are defined and characterized together with the entropy rate the varentropy rate serves to tightly approximate the fundamental nonasymptotic limits of fixed to variable compression for all but very small block lengths

uniform random number generation from markov chains non asymptotic and asymptotic analyses in this paper we derive non asymptotic achievability and converse bounds on the random number generation with without side information our bounds are efficiently computable in the sense that the computational complexity does not depend on the block length we also characterize the asymptotic behaviors of the large deviation regime and the moderate deviation regime by using our bounds which implies that our bounds are asymptotically tight in those regimes we also show the second order rates of those problems and derive single letter forms of the variances characterizing the second order rates furthermore we address the relative entropy rate and the modified mutual information rate for these problems

new bounds on the expected length of one to one codes we provide new bounds on the expected length l of a binary one to one code for a discrete random variable x with entropy h we prove that l spl ges h log h 1 hlog 1 1 h this bound improves on previous results furthermore we provide upper bounds on the expected length of the best code as function of h and the most likely source letter probability

approximation properties on invariant measure and oseledec splitting in non uniformly hyperbolic systems we prove that each invariant measure in a non uniformly hyperbolic system can be approximated by atomic measures on hyperbolic periodic orbits this contributes to our main result that the mean angle definition 1 10 independence number definition 1 6 and oseledec splitting for an ergodic hyperbolic measure with simple spectrum can be approximated by those for atomic measures on hyperbolic periodic orbits respectively combining this result with the approximation property of lyapunov exponents by wang and sun 2005 theorem 1 9 we strengthen katok s closing lemma 1980 by presenting more extensive information not only about the state system but also its linearization in the present paper we also study an ergodic theorem and a variational principle for mean angle independence number and liao s style number definition 1 3 which are bases for discussing the approximation properties in the main result

minimum expected length of fixed to variable lossless compression without prefix constraints the minimum expected length for fixed to variable length encoding of an n block memoryless source with entropy h grows as nh o 1 where the term o 1 lies between 0 and 1 however this well known performance is obtained under the implicit constraint that the code assigned to the whole n block is a prefix code dropping the prefix constraint which is rarely necessary at the block level we show that the minimum expected length for a finite alphabet memoryless source with known distribution grows as nh 1 2 log n o 1 unless the source is equiprobable we also refine this result up to o 1 for those memoryless sources whose log probabilities do not reside on a lattice

analytic variations on redundancy rates of renewal processes csiszar and shields 1996 proved that the minimax redundancy for a class of stationary renewal processes is spl theta spl radic n where n is the block length this interesting result provides a nontrivial bound on redundancy for a nonparametric family of processes the present paper gives a precise estimate of the redundancy rate for such nonstationary renewal sources namely 2 log2 spl radic spl pi sup 2 6 1 n o log n this asymptotic expansion is derived by complex analytic methods that include generating function representations mellin transforms singularity analysis and saddle point estimates this work places itself within the framework of analytic information theory

universal channel coding for general output alphabet we propose two classes of universal codes that are suited to two asymptotic regimes when the output alphabet is possibly continuous the first class has the property that the error probability decays exponentially fast and we identify an explicit lower bound on the error exponent the other class attains the epsilon capacity of the channel and we also identify the second order term in the asymptotic expansion the proposed encoder is essentially based on the packing lemma of the method of types for the decoder we first derive a renyi relative entropy version of clarke and barron s formula the distance between the true distribution and the bayesian mixture which is of independent interest the universal decoder is stated in terms of this formula and quantities used in the information spectrum method the methods contained herein allow us to analyze universal codes for channels with continuous and discrete output alphabets in a unified manner and to analyze their performances in terms of the exponential decay of the error probability and the second order coding rate

security analysis of varepsilon almost dual universal 2 hash functions smoothing of min entropy versus smoothing of renyi entropy of order 2 recently varepsilon almost dual universal2 hash functions have been proposed as a new and wider class of hash functions using this class of hash functions several efficient hash functions were proposed this paper evaluates the security performance when we apply this kind of hash functions we evaluate the security in several kinds of setting based on the l 1 distinguishability criterion and the modified mutual information criterion the obtained evaluation is based on smoothing of renyi entropy of order 2 and or min entropy we clarify the difference between these two methods

large deviation analysis for quantum security via smoothing of renyi entropy of order 2 it is known that the security evaluation can be done by smoothing of renyi entropy of order 2 in the classical and quantum settings when we apply universal 2 hash functions using the smoothing of renyi entropy of order 2 we derive security bounds for l 1 distinguishability and modified mutual information criterion under the classical and quantum setting and have derived these exponential decreasing rates these results are extended to the case when we apply e almost dual universal 2 hash functions furthermore we apply this analysis to the secret key generation with error correction

parameter estimation for differential equations a generalized smoothing approach summary we propose a new method for estimating parameters in models that are defined by a system of non linear differential equations such equations represent changes in system outputs by linking the behaviour of derivatives of a process to the behaviour of the process itself current methods for estimating parameters in differential equations from noisy data are computationally intensive and often poorly suited to the realization of statistical objectives such as inference and interval estimation the paper describes a new method that uses noisy measurements on a subset of variables to estimate the parameters defining a system of non linear differential equations the approach is based on a modification of data smoothing methods along with a generalization of profiled estimation we derive estimates and confidence intervals and show that these have low bias and good coverage properties respectively for data that are simulated from models in chemical engineering and neurobiology the performance of the method is demonstrated by using real world data from chemistry and from the progress of the autoimmune disease lupus

on the asymptotic redundancy of lossless block coding with two codeword lengths with the additional constraint of requiring only two codeword lengths lossless codes of blocks of size n generated by stationary memoryless binary sources are studied for arbitrary spl delta 0 classical large deviation inequalities imply the existence of codes attaining an expected redundancy of the order o n sup 1 2 spl delta it is shown that it is not possible to construct lossless codes with two codeword lengths having rate of order better or equal to o n sup 1 2

on a 145 32 binary cyclic code we discovered that a 145 32 binary cyclic code generated by the product of an irreducible polynomial of degree 28 with exponent 145 and the polynomial x sup 4 x sup 3 x sup 2 x 1 has a minimum distance of 44

optimal binary search trees abstract we consider the problem of building optimal binary search trees the binary search tree is a widely used data structure for information storage and retrieval a binary search tree t for a set of keys from a total order is a binary tree in which each node has a key value and all the keys of the left subtree are less than the key at the root and all the keys of the right subtree are greater than the key at the root this property holding recursively for the left and right subtrees of the tree t suppose we are given n keys and the probabilities of accessing each key and those occurring in the gap between two successive keys the optimal binary search tree problem is to construct a binary search tree on these n keys that minimizes the expected access time one variant of this problem is when only the gaps have nonzero access probabilities and is called the optimal alphabetic tree problem another related problem is when there is no order between the keys and there are probabilities associated only with the gaps and the objective is to build a binary tree with minimum expected weighted path length from the root this is called the huffman tree problem in this survey we assess known results on the structural properties of the optimal trees algorithms and lower bounds to construct and to verify optimal trees and heuristics to construct nearly optimal trees and other related results

decoding quantum information via the petz recovery map we obtain a lower bound on the maximum number of qubits qn e n which can be transmitted over n uses of a quantum channel n for a given non zero error threshold e to obtain our result we first derive a bound on the one shot entanglement transmission capacity of the channel and then compute its asymptotic expansion up to the second order in our method to prove this achievability bound the decoding map used by the receiver on the output of the channel is chosen to be the petz recovery map also known as the transpose channel our result in particular shows that this choice of the decoder can be used to establish the coherent information as an achievable rate for quantum information transmission applying our achievability bound to the 50 50 erasure channel which has zero quantum capacity we find that there is a sharp error threshold above which qn e n scales as n

two sided bounds for nonuniform rates of convergence in the central limit theorem two sided bounds are derived for nonuniform rates of convergence in the central limit theorem and are applied to solve several problems on rates of convergence in nonuniform metrics in particular an expression for the fastest rate of convergence is obtained and rates of convergence using different norming constants are compared a useful estimate of the rate of convergence in the 1 x 2 metric is derived which does not require the assumption of 2 th order moments

new bounds on the expected length of optimal one to one codes in this correspondence we consider one to one encodings for a discrete memoryless source which are one shot encodings associating a distinct codeword with each source symbol such encodings could be employed when only a single source symbol rather than a sequence of source symbols needs to be transmitted for example such a situation can arise when the last message must be acknowledged before the next message can be transmitted we consider two slightly different types of one to one encodings depending on whether the empty codeword is used or not and obtain lower and upper bounds on the expected length of optimal one to one codes we first give an extension of a known tight lower bound on the expected length of optimal one to one codes for the case that the the size of the source alphabet is finite and partial information about the source symbol probabilities is available as expected our lower bound is no less than the previously known lower bound obtained without side information about the source symbol probabilities we then consider the case that the source entropy is available and derive arbitrarily tight lower bounds on the expected length of optimal one to one codes we also derive arbitrarily tight lower bounds for the case that the source entropy and the probability of the most likely source symbol are available finally given that the probability of the most likely source symbol is available we obtain an upper bound on the expected length of optimal one to one codes our upper bound is tighter than the best upper bound known in the literature

on unique decodability in this paper we propose a revisitation of the topic of unique decodability and of some fundamental theorems of lossless coding it is widely believed that for any discrete source x every ldquouniquely decodablerdquo block code satisfies e l x 1 x 2 x n gesh x 1 x 2 x n where x 1 x 2 x n are the first n symbols of the source e l x 1 x 2 x n is the expected length of the code for those symbols and h x 1 x 2 x n is their joint entropy we show that for certain sources with memory the above inequality only holds when a limiting definition of ldquouniquely decodable coderdquo is considered in particular the above inequality is usually assumed to hold for any ldquopractical coderdquo due to a debatable application of mcmillan s theorem to sources with memory we thus propose a clarification of the topic also providing an extended version of mcmillan s theorem to be used for markovian sources

tight lower bounds for optimum code length corresp a new lower bound for the mean code length of all one to one codes for a random variable with n outcomes is derived the bound which is tight improves an earlier one due to leung yan cheong and cover another bound for one to one codes for binary information sources is derived

generating random bits from an arbitrary source fundamental limits suppose we are given a random source and want to use it as a random number generator at what rate can we generate fair bits from it we address this question in an information theoretic setting by allowing for some arbitrarily small but nonzero deviation from ideal random bits we prove our results with three different measures of approximation between the ideal and the obtained probability distributions the variational distance the d bar distance and the normalized divergence two different contexts are studied fixed length and variable length random number generation the fixed length results of this paper provide an operational characterization of the inf entropy rate of a source defined in han and verdu see ibid vol 39 no 3 p 752 772 1993 and the variable length results characterize the liminf of the entropy rate thereby establishing a pleasing duality with the fundamental limits of source coding a feature of our results is that we do not restrict ourselves to ergodic or to stationary sources

optimum overflow thresholds in variable length source coding allowing non vanishing error probability the variable length source coding problem allowing the error probability up to some constant is considered for general sources in this problem the optimum mean codeword length of variable length codes has already been determined on the other hand in this paper we focus on the overflow or excess codeword length probability instead of the mean codeword length the infimum of overflow thresholds under the constraint that both of the error probability and the overflow probability are smaller than or equal to some constant is called the optimum overflow threshold in this paper we first derive finite length upper and lower bounds on these probabilities so as to analyze the optimum overflow thresholds then by using these bounds we determine the general formula of the optimum overflow thresholds in both of the first order and second order forms next we consider another expression of the derived general formula so as to reveal the relationship with the optimum coding rate in the fixed length source coding problem finally we apply the general formula derived in this paper to the case of stationary memoryless sources

edgeworth expansion for u statistics under minimal conditions berry esseen bounds for u statistics under the optimal moment conditions were derived by koroljuk and borovskich and friedrich under the same optimal moment assumptions with an additional nonlattice condition we establish a one term edgeworth expansion with remainder o n 1 2 for u statistics

sieve estimation of constant and time varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error this article considers estimation of constant and time varying coefficients in nonlinear ordinary differential equation ode models where analytic closed form solutions are not available the numerical solution based nonlinear least squares nls estimator is investigated in this study a numerical algorithm such as the runge kutta method is used to approximate the ode solution the asymptotic properties are established for the proposed estimators considering both numerical error and measurement error the b spline is used to approximate the time varying coefficients and the corresponding asymptotic theories in this case are investigated under the framework of the sieve approach our results show that if the maximum step size of the p order numerical algorithm goes to zero at a rate faster than n 1 p 4 the numerical error is negligible compared to the measurement error this result provides a theoretical guidance in selection of the step size for numerical evaluations of odes moreover we have shown that the numerical solution based nls estimator and the sieve nls estimator are strongly consistent the sieve estimator of constant parameters is asymptotically normal with the same asymptotic co variance as that of the case where the true ode solution is exactly known while the estimator of the time varying parameter has the optimal convergence rate under some regularity conditions the theoretical results are also developed for the case when the step size of the ode numerical solver does not go to zero fast enough or the numerical error is comparable to the measurement error we illustrate our approach with both simulation studies and clinical data on hiv viral dynamics

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

asymptotically optimal multistage tests of simple hypotheses a family of variable stage size multistage tests of simple hypotheses is described based on efficient multistage sampling procedures using a loss function that is a linear combination of sampling costs and error probabilities these tests are shown to minimize the integrated risk to second order as the costs per stage and per observation approach zero a numerical study shows significant improvement over group sequential tests in a binomial testing problem

ergodic properties of systems with asymptotic average shadowing property abstract in this paper we explore a topological system f m m with asymptotic average shadowing property and extend sigmund s results from bowen s specification case we show that every non empty compact and connected subset v m inv f coincides with some v f y moreover we show that the set m v y m v f y v is dense in v v supp in particular if max m inv f supp coincides with m then m max y v f y m inv f is residual in m as consequences we have several corollaries one is that every invariant measure has generic points another is that the set consisting of those points for which the birkhoff ergodic average does not exist called irregular set is either dense in max residual provided that max m or empty in particular we give an uncountable division of irregular set and obtain a refined characterization which can be as a substantial generalization of 18 with a different method remark that for systems with asymptotic average shadowing property this article studies birkhoff ergodic average from topological viewpoint but it is still unknown how about the theory from the perspective of dimension theory

asymptotics and non asymptotics for universal fixed to variable source coding universal fixed to variable lossless source coding for memoryless sources is studied in the finite blocklength and higher order asymptotic regimes optimal three term fixed error asymptotic expressions are derived for general fixed to variable codes and for prefix codes it is shown that the non prefix type size code in which codeword lengths are chosen in ascending order of type class size achieves the optimal third order term and outperforms classical two stage codes converse results are proved making use of a result on the distribution of the empirical entropy and laplace s approximation finally the fixed to variable coding problem without a prefix constraint is shown to be essentially the same as the universal guessing problem

generic points of invariant measures for an amenable residually finite group actions with the weak specification property abstract we prove that every measure invariant for an amenable residually finite group action satisfying the weak specification property has a generic point this extends the sigmund theorem and completes the result of ren

variable length compression allowing errors this paper studies the fundamental limits of the minimum average length of lossless and lossy variable length compression allowing a nonzero error probability epsilon for lossless compression we give nonasymptotic bounds on the minimum average length in terms of erokhin s rate distortion function and we use those bounds to obtain a gaussian approximation on the speed of approach to the limit which is quite accurate for all but small blocklengths 1 epsilon k h mathsf s k v mathsf s 2 pi 1 2 mathop rm exp q 1 left epsilon right 2 2 where q 1 left cdot right is the functional inverse of the standard gaussian complementary cumulative distribution function and v mathsf s is the source dispersion a nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of 1 epsilon but this asymptotic limit is approached from below i e larger source dispersions and shorter blocklengths are beneficial variable length lossy compression under an excess distortion constraint is shown to exhibit similar properties

second order asymptotics of visible mixed quantum source coding via universal codes the simplest example of a quantum information source with memory is a mixed source which emits signals entirely from one of two memoryless quantum sources with given a priori probabilities considering a mixed source consisting of a general one parameter family of memoryless sources we derive the second order asymptotic rate for fixed length visible source coding furthermore we specialize our main result to a mixed source consisting of two memoryless sources our results provide the first example of the second order asymptotics for a quantum information processing task employing a resource with memory for the case of a classical mixed source using a finite alphabet our results reduce to those obtained by nomura and han to prove the achievability part of our main result we introduce universal quantum source codes achieving the second order asymptotic rates these are obtained by an extension of hayashi s construction of their classical counterparts

an information spectrum approach to weak variable length source coding with side information this paper studies variable length vl source coding of general sources with side information novel one shot coding bounds for slepian wolf sw coding which give nonasymptotic tradeoff between the error probability and the codeword length of vl sw coding are established one shot results are applied to asymptotic analysis and a general formula for the optimal coding rate achievable by weakly lossless vl sw coding i e vl sw coding with vanishing error probability is derived our general formula reveals how the encoder side information and or vl coding improve the optimal coding rate in the general setting in addition it is shown that if the encoder side information is useless in weakly lossless vl coding then it is also useless even in the case where the error probability may be positive asymptotically

general formulas for fixed length quantum entanglement concentration in this paper we derive general formulas for the amount of entanglement that can be concentrated from general sequences of partially entangled bipartite pure states the formulas are obtained by using an information spectrum approach the formulas obtained express the optimal rate under constant error constraints or exponential error constraints in the asymptotic framework since the formulas treat general sequences they can be applied to i i d sequences as well as sequences with correlations

simulation of random processes and rate distortion theory we study the randomness necessary for the simulation of a random process with given distributions on terms of the finite precision resolvability of the process finite precision resolvability is defined as the minimal random bit rate required by the simulator as a function of the accuracy with which the distributions are replicated the accuracy is quantified by means of various measures variational distance divergence orstein 1973 prohorov 1956 and related measures of distance between the distributions of random process in the case of ornstein prohorov and other distances of the kantorovich vasershtein type we show that the finite precision resolvability is equal to the rate distortion function with a fidelity criterion derived from the accuracy measure this connection leads to new results on nonstationary rate distortion theory in the case of variational distance the resolvability of stationary ergodic processes is shown to equal entropy rate regardless of the allowed accuracy in the case of normalized divergence explicit expressions for finite precision resolvability are obtained in many cases of interest and connections with data compression with minimum probability of block error are shown

tight exponential analysis of universally composable privacy amplification and its applications motivated by the desirability of universal composability we analyze in terms of l1 distinguishability the task of secret key generation from a joint random variable under this secrecy criterion using the renyi entropy of order 1 s for s 0 1 we derive a new upper bound of eve s distinguishability under the application of the universal2 hash functions it is also shown that this bound gives the tight exponential rate of decrease in the case of independent and identical distributions the result is applied to the wiretap channel model and to secret key generation distillation by public discussion

new nonasymptotic channel coding theorems for structured codes new nonasymptotic random coding theorems with error probability epsilon and finite block length n based on gallager parity check ensemble and general parity check ensembles are derived in this paper the resulting nonasymptotic achievability bounds when combined with nonasymptotic equipartition properties developed in this paper can be easily computed analytically these nonasymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as n goes to infinity with either constant or subexponentially decreasing epsilon in the case of gallager parity check ensemble and to imply that low density parity check ldpc codes be capacity achieving in the case of ldpc ensembles numerically they are also compared favorably for finite n and epsilon of practical interest with existing nonasymptotic achievability bounds in the literature

second order analysis for joint source channel coding with general channel and markovian source we derive the optimal second order rates in joint source channel coding when the channel is a general discrete memoryless channel and the source is an irreducible and ergodic markov process in contrast previous studies solved it only when the channel satisfies a certain unique variance condition and the source is subject to an independent and identical distribution we also compare the joint source channel scheme with the separation scheme in the second order regime while a previous study made a notable comparison with numerical calculation to discuss these two topics we introduce two kinds of new distribution families switched gaussian convolution distribution and product distribution which are defined by modifying the gaussian distribution

precise minimax redundancy and regret recent years have seen a resurgence of interest in redundancy of lossless coding the redundancy regret of universal fixed to variable length coding for a class of sources determines by how much the actual code length exceeds the optimal ideal over the class code length in a minimax scenario one finds the best code for the worst source either in the worst case called also maximal minimax or on average we first study the worst case minimax redundancy over a class of stationary ergodic sources and replace shtarkov s bound by an exact formula among others we prove that a generalized shannon code minimizes the worst case redundancy derive asymptotically its redundancy and establish some general properties this allows us to obtain precise redundancy for memoryless markov and renewal sources for example we present the exact constant of the redundancy for memoryless and markov sources by showing that the integer nature of coding contributes log logm m 1 logm o 1 where m is the size of the alphabet then we deal with the average minimax redundancy and regret our approach here is orthogonal to most recent research in this area since we aspire to show that asymptotically the average minimax redundancy is equivalent to the worst case minimax redundancy for some classes of sources after formulating some general bounds relating these two redundancies we prove our assertion for memoryless and markov sources nevertheless we provide evidence that maximal redundancy of renewal processes does not have the same leading term as the average minimax redundancy however our general results show that maximal and average regrets are asymptotically equivalent

convergence properties of the nelder mead simplex method in low dimensions the nelder mead simplex algorithm first published in 1965 is an enormously popular direct search method for multidimensional unconstrained minimization despite its widespread use essentially no theoretical results have been proved explicitly for the nelder mead algorithm this paper presents convergence properties of the nelder mead algorithm applied to strictly convex functions in dimensions 1 and 2 we prove convergence to a minimizer for dimension 1 and various limited convergence results for dimension 2 a counterexample of mckinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the nelder mead algorithm converges to a nonminimizer it is not yet known whether the nelder mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions

folklore in source coding information spectrum approach information theory has several traditional folklore problems about data compression or channel coding with reference to random number generation problems here we focus on and reasonably formulate one of them from the viewpoint of information spectra specifically we verify the validity of the folklore that the output from any source encoder working at the optimal coding rate with asymptotically vanishing probability of error looks like almost completely random

un critere sur les petites boules dans le theoreme limite central we show that a banach space valued random variablex such that mathop lim limits x to infty t 2 mathbb p left left x right t right 0 satisfies the central limit theorem if and only if the following criterion on small balls is fulfilled mathop lim limits x to infty t 2 mathbb p left left x right t right 0

equivocations exponents and second order coding rates under various renyi information measures we evaluate the asymptotics of equivocations their exponents as well as their second order coding rates under various renyi information measures specifically we consider the effect of applying a hash function on a source and we quantify the level of non uniformity and dependence of the compressed source from another correlated source when the number of copies of the sources is large unlike previous works that use shannon information measures to quantify randomness information or uniformity we define our security measures in terms of a more general class of information measures the renyi information measures and their gallager type counterparts a special case of these renyi information measure is the class of shannon information measures we prove tight asymptotic results for the security measures and their exponential rates of decay we also prove bounds on the second order asymptotics and show that these bounds match when the magnitudes of the second order coding rates are large we do so by establishing new classes non asymptotic bounds on the equivocation and evaluating these bounds using various probabilistic limit theorems asymptotically

strong secrecy from channel resolvability we analyze physical layer security based on the premise that the coding mechanism for secrecy over noisy channels is tied to the notion of channel resolvability instead of considering capacity based constructions which associate to each message a subcode that operates just below the capacity of the eavesdropper s channel we consider channel resolvability based constructions which associate to each message a subcode that operates just above the resolvability of the eavesdropper s channel building upon the work of csiszar and hayashi we provide further evidence that channel resolvability is a powerful and versatile coding mechanism for secrecy by developing results that hold for strong secrecy metrics and arbitrary channels specifically we show that at least for symmetric wiretap channels random capacity based constructions fail to achieve the strong secrecy capacity while channel resolvability based constructions achieve it we then leverage channel resolvability to establish the secrecy capacity region of arbitrary broadcast channels with confidential messages and a cost constraint for strong secrecy metrics finally we specialize our results to study the secrecy capacity of wireless channels with perfect channel state information csi mixed channels and compound channels with receiver csi as well as the secret key capacity of source models for secret key agreement by tying secrecy to channel resolvability we obtain achievable rates for strong secrecy metrics with simple proofs

streaming universal distortion free entanglement concentration this paper presents a streaming sequential protocol for universal entanglement concentration at the shannon bound alice and bob begin with n identical but unknown two qubit pure states each containing e ebits of entanglement they each run a reversible algorithm on their qubits and end up with y perfect epr pairs where y ne o n our protocol is streaming so the n input systems are fed in one at a time and perfect epr pairs start popping out almost immediately it matches the optimal block protocol exactly at each stage so the average yield after n inputs is y ne o logn so somewhat surprisingly there is no tradeoff between yield and lag our protocol optimizes both in contrast the optimal n qubit block protocol achieves the same yield but since no epr pairs are produced until the entire input block is read its lag is o n finally our algorithm runs in o log n space so a lot of entanglement can be efficiently concentrated using a very small e g current or near future technology quantum processor along the way we find an optimal streaming protocol for extracting randomness from classical i i d sources and a more space efficient implementation of the schur transform

specification on the interval we study the consequences of discontinuities on the specification property for interval maps after giving a necessary and sufficient condition for a piecewise monotonic piecewise continuous map to have this property we show that for a large and natural class of families of such maps including the 3 transformations the set of parameters for which the specification property holds though dense has zero lebesgue measure thus regarding the specification property the general case is at the opposite of the continuous case solved by a m blokh russian math surveys 38 1983 133 134 for which we give a proof

an information spectrum approach to classical and quantum hypothesis testing for simple hypotheses the information spectrum analysis made by han for classical hypothesis testing for simple hypotheses is extended to a unifying framework including both classical and quantum hypothesis testing the results are also applied to fixed length source coding when loosening the normalizing condition for probability distributions and for quantum states we establish general formulas for several quantities relating to the asymptotic optimality of tests codes in terms of classical and quantum information spectra

new lower bounds on the cost of binary search trees abstract in this paper we provide new lower bounds on the cost c of binary search trees the bounds are expressed in terms of the entropy h of the probability distribution the number of elements and the probability that a search is successful most of our lower bounds are derived by means of a new technique which exploits the relation between trees and codes our lower bounds compare favorably with known limitations we also provide an achievable upper bound on the kraft sum generalized to the internal nodes of a tree this improves on a previous result

exact convergence rate and leading term in central limit theorem for student s t statistic the leading term in the normal approximation to the distribution of student s t statistic is derived in a general setting with the sole assumption being that the sampled distribution is in the domain of attraction of a normal law the form of the leading term is shown to have its origin in the way in which extreme data influence properties of the studentized sum the leading term approximation is used to give the exact rate of convergence in the central limit theorem up to order n 1 2 where n denotes sample size it is proved that the exact rate uniformly on the whole real line is identical to the exact rate on sets of just three points moreover the exact rate is identical to that for the non studentized sum when the latter is normalized for scale using a truncated form of variance but when the corresponding truncated centering constant is omitted examples of characterizations of convergence rates are also given it is shown that in some instances their validity uniformly on the whole real line is equivalent to their validity on just two symmetric points

differential equation modeling of hiv viral fitness experiments model identification model selection and multimodel inference many biological processes and systems can be described by a set of differential equation de models however literature in statistical inference for de models is very sparse we propose statistical estimation model selection and multimodel averaging methods for hiv viral fitness experiments in vitro that can be described by a set of nonlinear ordinary differential equations ode the parameter identifiability of the ode models is also addressed we apply the proposed methods and techniques to experimental data of viral fitness for hiv 1 mutant 103n we expect that the proposed modeling and inference approaches for the de models can be widely used for a variety of biomedical studies

parameter estimation for differential equation models using a framework of measurement error in regression models differential equation de models are widely used in many scientific fields including engineering physics and biomedical sciences the so called forward problem the problem of simulations and predictions of state variables for given parameter values in the de models has been extensively studied by mathematicians physicists engineers and other scientists however the inverse problem the problem of parameter estimation based on the measurements of output variables has not been well explored using modern statistical methods although some least squares based approaches have been proposed and studied in this article we propose parameter estimation methods for ordinary differential equation ode models based on the local smoothing approach and a pseudo least squares psls principle under a framework of measurement error in regression models the asymptotic properties of the proposed psls estimator are established we also compare the psls method to the corresponding simulation extrapolation

the asymptotic average shadowing property and transitivity abstract we introduce the notion of the asymptotic average shadowing property abbrev aasp and investigate the relation between the aasp and transitivity it is shown that a continuous map on x with the aasp is chain transitive and a l hyperbolic homeomorphism on x with the aasp is topologically transitive where x is a compact metric space

on one to one codes for memoryless cost channels a new twist to the classical problem of compressing a discrete source over noiseless channel alphabets where different code symbols may have different transmission costs is considered one to one or nonsingular codes map each letter from the source alphabet into a distinct codeword consisting of a string of channel alphabet letters they need not be uniquely decodable they have been studied for binary code alphabets with equal letter costs since the early 1970s and are extended here to memoryless cost channel alphabets the study of optimal one to one codes is linked in this paper to three other well known source coding problems the algorithm to construct the optimal one to one code is closely related to the varn code and the tunstall code these relationships and convexity arguments are used to establish some upper and lower bounds on the expected cost per source symbol of the optimal one to one code finally the average cost per symbol of the optimal one to one code is demonstrated to offer a new lower bound to the average cost per symbol of the optimal uniquely decodable code which is sometimes tighter than the classic bound from shannon s landmark 1948 paper

second order noiseless source coding theorems shannon s celebrated source coding theorem can be viewed as a one sided law of large numbers we formulate second order noiseless source coding theorems for the deviation of the codeword lengths from the entropy for a class of sources that includes markov chains we prove a one sided central limit theorem and a law of the iterated logarithm

optimistic shannon coding theorems for arbitrary single user systems the conventional definitions of the source coding rate and of channel capacity require the existence of reliable codes for all sufficiently large block lengths alternatively if it is required that good codes exist for infinitely many block lengths then optimistic definitions of source coding rate and channel capacity are obtained in this work formulas for the optimistic minimum achievable fixed length source coding rate and the minimum spl epsi achievable source coding rate for arbitrary finite alphabet sources are established the expressions for the optimistic capacity and the optimistic spl epsi capacity of arbitrary single user channels are also provided the expressions of the optimistic source coding rate and capacity are examined for the class of information stable sources and channels respectively finally examples for the computation of optimistic capacity are presented

on the asymptotic equivalence of l p metrics for convergence to normality this paper is concerned with the rate of convergence to zero of the l p metrics np l p constructed out of differences between distribution functions for departure from normality for normed sums of independent and identically distributed random variables with zero mean and unit variance it is shown that the np are under broad conditions asymptotically equivalent in the strong sense that for l p p np np is universally bounded away from zero and infinity as n

asymptotic average shadowing property on compact metric spaces abstract we prove that if for a continuous map f on a compact metric space x the chain recurrent set r f has more than one chain component then f does not satisfy the asymptotic average shadowing property we also show that if a continuous map f on a compact metric space x has the asymptotic average shadowing property and if a is an attractor for f then a is the single attractor for f and we have a r f we also study diffeomorphisms with asymptotic average shadowing property and prove that if m is a compact manifold which is not finite with dim m 2 then the c 1 interior of the set of all c 1 diffeomorphisms with the asymptotic average shadowing property is characterized by the set of stable diffeomorphisms

on the redundancy of lossy source coding with abstract alphabets the redundancy problem of lossy source coding with abstract source and reproduction alphabets is considered for coding at a fixed rate level it is shown that for any fixed rate r 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the rate r such that the distortion redundancy of c sub n defined as the difference between the performance of c sub n and the distortion rate function d p r of p is upper bounded by spl part d p r spl part r ln n 2n o ln n n for coding at a fixed distortion level it is demonstrated that for any d 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the fixed distortion d such that the rate redundancy of c sub n defined as the difference between the performance of c sub n and the rate distortion function r p d of p is upper bounded by 7ln n 6n o ln n n these results strengthen the traditional berger s 1968 1971 abstract alphabet source coding theorem and extend the positive redundancy results of zhang yang and wei see ibid vol 43 no 1 p 71 91 1997 and ibid vol 42 p 803 21 1996 on lossy source coding with finite alphabets and the redundancy result of wyner see ibid vol 43 p 1452 64 1997 on block coding of memoryless gaussian sources

universal coding with minimum probability of codeword length overflow lossless block to variable length source coding is studied for finite state finite alphabet sources the aim is to minimize the probability that the normalized length of the codeword will exceed a given threshold b subject to the kraft inequality it is shown that the lempel ziv algorithm 1978 asymptotically attains the optimal performance in the sense just defined independently of the source and the value of b for the subclass of unifilar markov sources faster convergence to the asymptotic optimum performance can be accomplished by using the minimum description length universal code for this subclass it is demonstrated that these universal codes are also nearly optimal in the sense of minimizing buffer overflow probability and asymptotically optimal in a competitive sense

solving slp recourse problems with arbitrary multivariate distributions the dependent case bounding the expectation of a convex function of a multivariate random variable is of great importance for solving stochastic linear programming problems slp with recourse the classic bounds are those of jensen and edmundson madansky whereby jensen s inequality is available in the independent as well as in the dependent case in contrast with the edmundson madansky e m inequality madansky a 1959 bounds on the expectation of a convex function of multivariate random variable ann math statist 30 743 746 which is valid only in the independent case this paper presents a straightforward extension of the e m inequality for the dependent case yielding a discrete distribution that is extremal with respect to a partial ordering of a set of distribution functions further we prove that the bounds for the expectation behave monotonically by applying the obtained inequality to smaller and smaller subintervals as this requires the evaluation of conditional probabilities and expectations we replace the given distribution by a discrete one resulting from sampling and determine the sampling size that keeps the statistical error negligible finally we conclude with the application to slp recourse problems and state some computational results illustrating the effort for solving recourse problems

the ugc hardness threshold of the lp grothendieck problem for p 2 we consider the problem of given an n n matrix a aij whose diagonal entries vanish approximating in polynomial time the number rm opt p a max left sum i j 1 na ij x ix j x 1 ldots x n in mathbb r n wedge left sum i 1 n x i p right 1 p le 1 right when p 2 this is simply the problem of computing the maximum eigenvalue of a whereas for p actually it suffices to take p log n it is the grothendieck problem on the complete graph which was shown to have a o log n approximation algorithm in nemirovski et al nemirovski a c roos t terlaky 1999 on maximization of quadratic form over intersection of ellipsoids with common center math program ser a86 3 463 473 megretski megretski a 2001 relaxations of quadratic programs in operator theory and system analysis systems approximation singular integral operators and related topics bordeaux 2000 vol 129 operator theory advances and applications birkhauser basel 365 392 charikar and wirth charikar m a wirth 2004 maximizing quadratic programs extending grothendieck s inequality proc 45th annual sympos foundations comput sci ieee computer society 54 60 and was used in the work of charikar and wirth noted above to design the best known algorithm for the problem of computing the maximum correlation in correlation clustering thus the problem of approximating optp a interpolates between the spectral p 2 case and the correlation clustering p case from a physics point of view this problem corresponds to computing the ground states of spin glasses in a hard wall potential well we design a polynomial time algorithm which given p 2 and an n n matrix a aij with zeros on the diagonal computes optp a up to a factor p e 30 log p on the other hand assuming the unique games conjecture ugc we show that it is np hard to approximate optp a up to a factor smaller than p e hence as p the ugc hardness threshold for computing optp a is exactly p e 1 o 1

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

asymptotic efficiency and finite sample properties of the generalized profiling estimation of parameters in ordinary differential equations ordinary differential equations odes are commonly used to model dynamic behavior of a system because many parameters are unknown and have to be estimated from the observed data there is growing interest in statistics to develop efficient estimation procedures for these parameters among the proposed methods in the literature the generalized profiling estimation method developed by ramsay and colleagues is particularly promising for its computational efficiency and good performance in this approach the ode solution is approximated with a linear combination of basis functions the coefficients of the basis functions are estimated by a penalized smoothing procedure with an ode defined penalty however the statistical properties of this procedure are not known in this paper we first give an upper bound on the uniform norm of the difference between the true solutions and their approximations then we use this bound to prove the consistency and asymptotic normality of this estimation procedure we show that the asymptotic covariance matrix is the same as that of the maximum likelihood estimation therefore this procedure is asymptotically efficient for a fixed sample and fixed basis functions we study the limiting behavior of the approximation when the smoothing parameter tends to infinity we propose an algorithm to choose the smoothing parameters and a method to compute the deviation of the spline approximation from solution without solving the odes

second order asymptotics in fixed length source coding and intrinsic randomness there is a difference between the optimal rates of fixed length source coding and intrinsic randomness when we care about the second order asymptotics we prove this difference for general information sources and then investigate independent and identically distributed i i d random variables and markovian variables as examples the difference is demonstrated through an investigation of the second order asymptotic behavior of the rates a universal fixed length source code attaining the second order optimal rate is also proposed the difference between the rates of fixed length source coding and intrinsic randomness proves that the outputs of fixed length source codes are not uniformly distributed

on the effect of random norming on the rate of convergence in the central limit theorem it is shown that studentizing i e normalizing by the sample standard deviation rather than the population standard deviation can improve the rate of convergence in the central limit theorem this provides concise confirmation of one feature of the folklore that a studentized sum is in some sense more robust than a normed sum the case of infinite population standard deviation is also examined 1 introduction our purpose in this paper is to show that studentizing the operation of replacing a population standard deviation in a norming constant by its sample estimate can improve the rate of convergence in the central limit theorem thus we provide concise confirmation of one feature of the folklore that a studentized sum is in some sense more robust than a normed sum the reason is that large deviations of summands which are responsible for slow rates of convergence when ordinary norming constants are used cancel from the sample mean and sample standard deviation remark 3 7 in section 3 will give an intuitive explanation of the mechanism behind this cancellation note that the large deviations have a particularly subtle influence since under the condition that the distribution is attracted to the normal law the largest summand is negligible in comparison with the sum itself section 2 will briefly survey results on rates of convergence in the ordinary i e nonstudentized central limit theorem and will discuss their connection with our work section 3 will state our main results and summarize their consequences and section 4 will give proofs

achievable moderate deviations asymptotics for streaming compression of correlated sources motivated by streaming multi view video coding and wireless sensor networks we consider the problem of blockwise streaming compression of a pair of correlated sources which we term streaming slepian wolf coding we study the moderate deviations regime in which the rate pairs of a sequence of codes converge along a straight line to various points on the boundary of the slepian wolf region at a speed slower than the inverse square root of the blocklength n while the error probability decays subexponentially fast in n our main result focuses on the directions of approaches to corner points of the slepian wolf region it states that for each correlated source and all corner points there exists a non empty subset of directions of approaches such that the moderate deviations constant the constant of proportionality for the subexponential decay of the error probability is enhanced over the non streaming case by at least a factor of t the block delay of decoding source block pairs we specialize our main result to the setting of streaming lossless source coding and generalize this result to the setting where we have different delay requirements for each of the two source blocks the proof of our main result involves the use of various analytical tools and amalgamates several ideas from the recent information theoretic streaming literature we adapt the so called truncated memory encoding idea from draper and khisti 2011 and lee tan and khisti 2016 to ensure that the effect of error accumulation is nullified in the limit of large block lengths we also adapt the use of the so called minimum weighted empirical suffix entropy decoder which was used by draper chang and sahai 2014 to derive achievable error exponents for symbolwise streaming slepian wolf coding

forcing function diagnostics for nonlinear dynamics this article investigates the problem of model diagnostics for systems described by nonlinear ordinary differential equations odes i propose modeling lack of fit as a time varying correction to the right hand side of a proposed differential equation this correction can be described as being a set of additive forcing functions estimated from data representing lack of fit in this manner allows us to graphically investigate model inadequacies and to suggest model improvements i derive lack of fit tests based on estimated forcing functions model building in partially observed systems of odes is particularly difficult and i consider the problem of identification of forcing functions in these systems the methods are illustrated with examples from computational neuroscience

edgeworth expansion of a function of sample means many important statistics can be written as functions of sample means of vector variables a fundamental contribution to the edgeworth expansion for functions of sample means was made by bhattacharya and ghosh in their work the crucial cramer c condition is assumed on the joint distribution of all the components of the vector variable however in many practical situations only one or a few of the components satisfy conditionally this condition while the rest do not such a case is referred to as satisfying the partial cramer c condition the purpose of this paper is to establish edgeworth expansions for functions of sample means when only the partial cramer c condition is satisfied

dual universality of hash functions and its applications to quantum cryptography in this paper we introduce the concept of dual universality of hash functions and present its applications to quantum cryptography we begin by establishing the one to one correspondence between a linear function family f and a code family c and thereby defining e almost dual universal2 hash functions as a generalization of the conventional universal2 hash functions then we show that this generalized and thus broader class of hash functions is in fact sufficient for the security of quantum cryptography this result can be explained in two different formalisms first by noting its relation to the biased family introduced by dodis and smith we demonstrate that renner s two universal hashing lemma is generalized to our class of hash functions next we prove that the proof technique by shor and preskill can be applied to quantum key distribution qkd systems that use our generalized class of hash functions for privacy amplification while shor preskill formalism requires an implementer of a qkd system to explicitly construct a linear code of the calderbank shor steane css type this result removes the existing difficulty of the construction of a linear code of css code by replacing it by the combination of an ordinary classical error correcting code and our proposed hash function we also show that a similar result applies to the quantum wire tap channel finally we compare our results in the two formalisms and show that in typical qkd scenarios the shor preskill type argument gives better security bounds in terms of the trace distance and holevo information than the method based on the biased family

information spectrum approach to second order coding rate in channel coding in this paper second order coding rate of channel coding is discussed for general sequence of channels the optimum second order transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method we apply this result to the discrete memoryless case the discrete memoryless case with a cost constraint the additive markovian case and the gaussian channel case with an energy constraint we also clarify that the gallager bound does not give the optimum evaluation in the second order coding rate

a case where interference does not affect the channel dispersion in 1975 carleial presented a special case of an interference channel called the very strong interference regime in which the interference does not reduce the capacity of the constituent point to point gaussian channels in this paper we show that in the strictly very strong interference regime the dispersions are similarly unaffected more precisely in this paper we characterize the second order coding rates of the gaussian interference channel in the strictly very strong interference regime in other words we characterize the speed of convergence of rates of optimal block codes toward a boundary point of the rectangular capacity region these second order coding rates are expressed in terms of the average probability of error and variances of appropriately defined information densities which coincide with the dispersion of the single user gaussian channel this allows us to conclude that the dispersions are unaffected by interference in this channel model

convergence determining sets in the central limit theorem a complete characterisation is given of the class of all doublets which determine the rate of convergence in the central limit theorem this enables a number of important properties of convergence determining sets to be deduced in particular it is shown that no singleton can be convergence determining and any set consisting of four or more distinct points is convergence determining numerical and analytic methods are used to derive the geometry of the class of all convergence determining doublets

an achievable bound for optimal noiseless coding of a random variable corresp for a discrete n valued random variable n possibly denumerably infinite leung yan cheong and cover have given bounds for the minimal expected length of a one to one not necessarily uniquely decodable code l 1 1 sum i 1 n p i log left frac 1 2 1 right it is shown that the best possible case occurs for certain denumerably infinite sets of nonzero probabilities this absolute bound is related to the shannon entropy h of the distribution by h cdot is the binary entropy function