asymmetric evaluations of erasure and undetected error probabilities the problem of channel coding with the erasure option is revisited for discrete memoryless channels the interplay between the code rate the undetected and total error probabilities is characterized using the information spectrum method a sequence of codes of increasing blocklengths n is designed to illustrate this tradeoff furthermore for additive discrete memoryless channels with uniform input distribution we establish that our analysis is tight with respect to the ensemble average this is done by analyzing the ensemble performance in terms of a tradeoff between the code rate the undetected and total errors this tradeoff is parameterized by the threshold in a generalized likelihood ratio test two asymptotic regimes are studied first the code rate tends to the capacity of the channel at a rate slower than n 1 2 corresponding to the moderate deviations regime in this case both error probabilities decay subexponentially and asymmetrically the precise decay rates are characterized second the code rate tends to capacity at a rate of n 1 2 in this case the total error probability is asymptotically a positive constant while the undetected error probability decays as exp bn 1 2 for some b 0 the proof techniques involve the applications of a modified or shifted version of the gartner ellis theorem and the type class enumerator method to characterize the asymptotic behavior of a sequence of cumulant generating functions

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection

fine asymptotics for universal one to one compression of parametric sources universal source coding at short blocklengths is considered for an i i d exponential family of distributions the type size code has previously been shown to be optimal up to the third order rate for universal compression of all memoryless sources over finite alphabets the type size code assigns sequences ordered based on their type class sizes to binary strings ordered lexicographically to generalize this type class approach for parametric sources a natural scheme is to define two sequences to be in the same type class if and only if they are equiprobable under any model in the parametric class this natural approach however is shown to be suboptimal a variation of the type size code is introduced where type classes are defined based on neighborhoods of minimal sufficient statistics the asymptotics of the overflow rate of this variation are derived and a converse result establishes its optimality up to the third order term

optimal lossless data compression non asymptotics and asymptotics this paper provides an extensive study of the behavior of the best achievable rate and other related fundamental limits in variable length strictly lossless compression in the non asymptotic regime the fundamental limits of fixed to variable lossless compression with and without prefix constraints are shown to be tightly coupled several precise quantitative bounds are derived connecting the distribution of the optimal code lengths to the source information spectrum and an exact analysis of the best achievable rate for arbitrary sources is given fine asymptotic results are proved for arbitrary not necessarily prefix compressors on general mixing sources nonasymptotic explicit gaussian approximation bounds are established for the best achievable rate on markov sources the source dispersion and the source varentropy rate are defined and characterized together with the entropy rate the varentropy rate serves to tightly approximate the fundamental nonasymptotic limits of fixed to variable compression for all but very small block lengths

new bounds on the expected length of one to one codes we provide new bounds on the expected length l of a binary one to one code for a discrete random variable x with entropy h we prove that l spl ges h log h 1 hlog 1 1 h this bound improves on previous results furthermore we provide upper bounds on the expected length of the best code as function of h and the most likely source letter probability

guessing under source uncertainty this paper considers the problem of guessing the realization of a finite alphabet source when some side information is provided the only knowledge the guesser has about the source and the correlated side information is that the joint source is one among a family a notion of redundancy is first defined and a new divergence quantity that measures this redundancy is identified this divergence quantity shares the pythagorean property with the kullback leibler divergence good guessing strategies that minimize the supremum redundancy over the family are then identified the min sup value measures the richness of the uncertainty set the min sup redundancies for two examples the families of discrete memoryless sources and finite state arbitrarily varying sources are then determined

improved bounds on lossless source coding and guessing moments via renyi measures this paper provides upper and lower bounds on the optimal guessing moments of a random variable taking values on a finite set when side information may be available these moments quantify the number of guesses required for correctly identifying the unknown object and similarly to arikan s bounds they are expressed in terms of the arimoto renyi conditional entropy although arikan s bounds are asymptotically tight the improvement of the bounds in this paper is significant in the non asymptotic regime relationships between moments of the optimal guessing function and the map error probability are also established characterizing the exact locus of their attainable values the bounds on optimal guessing moments serve to improve non asymptotic bounds on the cumulant generating function of the codeword lengths for fixed to variable optimal lossless source coding without prefix constraints non asymptotic bounds on the reliability function of discrete memoryless sources are derived as well relying on these techniques lower bounds on the cumulant generating function of the codeword lengths are derived by means of the smooth renyi entropy for source codes that allow decoding errors

goodness of fit and related inference processes for quantile regression abstract we introduce a goodness of fit process for quantile regression analogous to the conventional r2 statistic of least squares regression several related inference processes designed to test composite hypotheses about the combined effect of several covariates over an entire range of conditional quantile functions are also formulated the asymptotic behavior of the inference processes is shown to be closely related to earlier p sample goodness of fit theory involving bessel processes the approach is illustrated with some hypothetical examples an application to recent empirical models of international economic growth and some monte carlo evidence

least absolute deviation estimation for fractionally integrated autoregressive moving average time series models with conditional heteroscedasticity summary we consider a unified least absolute deviation estimator for stationary and nonstationary fractionally integrated autoregressive moving average models with conditional heteroscedasticity its asymptotic normality is established when the second moments of errors and innovations are finite several other alternative estimators are also discussed and are shown to be less efficient and less robust than the proposed approach a diagnostic tool consisting of two portmanteau tests is designed to check whether or not the estimated models are adequate the simulation experiments give further support to our model and the results for the absolute returns of the dow jones industrial average index daily closing price demonstrate their usefulness in modelling time series exhibiting the features of long memory conditional heteroscedasticity and heavy tails

minimum expected length of fixed to variable lossless compression without prefix constraints the minimum expected length for fixed to variable length encoding of an n block memoryless source with entropy h grows as nh o 1 where the term o 1 lies between 0 and 1 however this well known performance is obtained under the implicit constraint that the code assigned to the whole n block is a prefix code dropping the prefix constraint which is rarely necessary at the block level we show that the minimum expected length for a finite alphabet memoryless source with known distribution grows as nh 1 2 log n o 1 unless the source is equiprobable we also refine this result up to o 1 for those memoryless sources whose log probabilities do not reside on a lattice

source coding using families of universal hash functions this correspondence is concerned with new connections between source coding and two kinds of families of hash functions known as the families of universal hash functions and n strongly universal hash functions where n ges 2 is an integer first it is pointed out that such families contain classes of well known source codes such as bin codes and linear codes next performance of a source coding scheme using either of the two kinds of families is evaluated an upper bound on the expectation of the decoding error probability is obtained for each family the expectation of the decoding error probability is analyzed in detail for the cases of discrete memoryless sources and sources without the memoryless assumption under a certain class of decoders

superlinear systems of second order ode s abstract we discuss the existence of positive solutions of the system u f t u v u v in 0 1 v g t u v u v in 0 1 u 0 u 1 v 0 v 1 0 where the nonlinearities f and g satisfy a superlinearity condition at both 0 and our main result is the proof of a priori bounds for the eventual solutions as an application we consider the dirichlet problem in an annulus for systems of semilinear elliptic equations with nonlinearities depending on the gradient as well as a second application we consider fourth order elastic beam equations with dependence also on the derivatives u u u

positive solutions for one dimensional p laplacian boundary value problems with dependence on the first order derivative abstract this paper presents sufficient conditions for the existence and multiplicity of positive solutions to the one dimensional p laplacian differential equation p u a t f u u 0 subject to some boundary conditions we show that it has at least one or two or three positive solutions under some assumptions by applying the fixed point theorem

the performance of universal encoding universal coding theory is surveyed from the viewpoint of the interplay between delay and redundancy the price for universality turns out to be acceptably small

kolmogorov s contributions to information theory and algorithmic complexity briefly information theory says a random object x p x has complexity entropy h xp x log p x with the attendant interpretation that h bits are sufficient to describe x on the average algorithmic complexity says an object x has a complexity k x equal to the length of the shortest binary program that describes x it is a beautiful fact that these ideas are much the same in fact it is roughly true that ek x h moreover if we let pu x pr u prints x be the probability that a given computer u prints x when given a random program it can be shown that log 1 pu x k x for all x thus establishing a vital link between the universal probability measure pu and the universal complexity k more on this later the relationship of these ideas to probability theory was summed up in kolmogorov s 1983 paper which was based on his 1970 talk in nice perhaps only

new bounds on the expected length of optimal one to one codes in this correspondence we consider one to one encodings for a discrete memoryless source which are one shot encodings associating a distinct codeword with each source symbol such encodings could be employed when only a single source symbol rather than a sequence of source symbols needs to be transmitted for example such a situation can arise when the last message must be acknowledged before the next message can be transmitted we consider two slightly different types of one to one encodings depending on whether the empty codeword is used or not and obtain lower and upper bounds on the expected length of optimal one to one codes we first give an extension of a known tight lower bound on the expected length of optimal one to one codes for the case that the the size of the source alphabet is finite and partial information about the source symbol probabilities is available as expected our lower bound is no less than the previously known lower bound obtained without side information about the source symbol probabilities we then consider the case that the source entropy is available and derive arbitrarily tight lower bounds on the expected length of optimal one to one codes we also derive arbitrarily tight lower bounds for the case that the source entropy and the probability of the most likely source symbol are available finally given that the probability of the most likely source symbol is available we obtain an upper bound on the expected length of optimal one to one codes our upper bound is tighter than the best upper bound known in the literature

strong converses in source coding relative to a fidelity criterion two strong converses are obtained for an abstract alphabet stationary ergodic source coded relative to an appropriate fidelity criterion it is shown that given a distortion rate point d r that lies below the rate distortion curve 1 block codes that operate at rate level r must encode sample source blocks at a rate exceeding d with probability tending to one as the block length tends to infinity and 2 variable rate codes that operate at distortion level d must encode sample source blocks at a rate exceeding r with probability tending to one as the block length tends to infinity the previously known weak converses guarantee only that the indicated probabilities remain bounded away from zero as block length tends to infinity the proofs of the strong converses involve sample converses in source coding theory

strong converses for group testing from finite blocklength results we prove new strong converse results in a variety of group testing settings generalizing a result of baldassini et al first in the non adaptive case we mimic the hypothesis testing argument introduced in the finite blocklength channel coding regime by polyanskiy et al and using joint source channel coding arguments of kostina and verdu in the adaptive case we combine this approach with a novel model formulation based on causal probability and directed information theory in both cases we prove results which are valid for finite sized problems and imply capacity results in the asymptotic regime these results are illustrated graphically for a range of models

existence of positive solutions for the p laplacian with p gradient term abstract in this paper we study the existence of positive solutions for the p laplacian involving a p gradient term due to the non variational structure and the fact that the nonlinearity may be critical or supercritical the variational method is no longer valid taking advantage of global c 1 estimates and the liouville type theorems we employ the blow up argument to obtain the a priori estimates on solutions and finally obtain the existence result based on the krasnoselskii fixed point theorem

tight lower bounds for optimum code length corresp a new lower bound for the mean code length of all one to one codes for a random variable with n outcomes is derived the bound which is tight improves an earlier one due to leung yan cheong and cover another bound for one to one codes for binary information sources is derived

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

pointwise redundancy in lossy data compression and universal lossy data compression we characterize the achievable pointwise redundancy rates for lossy data compression at a fixed distortion level pointwise redundancy refers to the difference between the description length achieved by an nth order block code and the optimal nr d bits for memoryless sources we show that the best achievable redundancy rate is of order o spl radic n in probability this follows from a second order refinement to the classical source coding theorem in the form of a one sided central limit theorem moreover we show that along almost any source realization the description lengths of any sequence of block codes operating at distortion level d exceed nr d by at least as much as c spl radic nloglogn infinitely often corresponding direct coding theorems are also given showing that these rates are essentially achievable the above rates are in sharp contrast with the expected redundancy rates of order o log n reported by various authors our approach is based on showing that the compression performance of an arbitrary sequence of codes is essentially bounded below by the performance of shannon s random code we obtain partial generalizations of the above results for arbitrary sources with memory and we prove lossy analogs of barron s lemma barron 1985

weak variable length slepian wolf coding with linked encoders for mixed sources coding problems for correlated information sources were first investigated by slepian and wolf they considered the data compression system called the sw system where two sequences emitted from correlated sources are separately encoded to codewords and sent to a single decoder which has to output the original sequence pairs with a small probability or error in this paper we investigate the coding problem of a modified sw system allowing two encoders to communicate with zero rate first we consider the fixed length coding and clarify that the admissible rate region for general sources is equal to that of the original sw system next we investigate the variable length coding having the asymptotically vanishing probability of error we clarify the admissible rate region for mixed sources characterized by two ergodic sources and show that this region is strictly wider than that for fixed length codes further we investigate the universal coding problem for memoryless sources in the system and show that the sw system with linked encoders has much more flexibility than the original sw system

estimation and inference by compact coding summary the systematic variation within a set of data as represented by a usual statistical model may be used to encode the data in a more compact form than would be possible if they were considered to be purely random the encoded form has two parts the first states the inferred estimates of the unknown parameters in the model the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure in its strict form it has great generality and several nice properties but is computationally infeasible an approximate form is developed and its relation to other methods is explored

guessing subject to distortion we investigate the problem of guessing a random vector x within distortion level d our aim is to characterize the best attainable performance in the sense of minimizing in some probabilistic sense the number of required guesses g x until the error falls below d the underlying motivation is that g x is the number of candidate codewords to be examined by a rate distortion block encoder until a satisfactory codeword is found in particular for memoryless sources we provide a single letter characterization of the least achievable exponential growth rate of the spl rho th moment of g x as the dimension of the random vector x grows without bound in this context we propose an asymptotically optimal guessing scheme that is universal both with respect to the information source and the value of spl rho we then study some properties of the exponent function e d spl rho along with its relation to the source coding exponents finally we provide extensions of our main results to the gaussian case guessing with side information and sources with memory

second order sturm liouville problems with asymmetric superlinear nonlinearities abstract we consider the nonlinear sturm liouville problem p x u x q x u x f x u x h x in 0 c 00 u 0 c 01 u 0 0 c 10 u c 11 u 0 where p c 1 0 q c 0 0 with p x 0 for all x 0 ci02 ci12 0 i 0 1 h l2 0 we suppose that f 0 r r is continuous and there exist increasing functions l u 0 r and positive constants a b such that lim t l t and a l f x a u 0 f x a b 0 for all x 0 thus the nonlinearity is superlinear as u x but linearly bounded as u x existence and non existence results are obtained for the above problem similar results have been obtained before for problems in which f is linearly bounded as and these results have been expressed in terms of half eigenvalues of the problem the results obtained here for the superlinear case are expressed in terms of certain asymptotes of these half eigenvalues

optimum overflow thresholds in variable length source coding allowing non vanishing error probability the variable length source coding problem allowing the error probability up to some constant is considered for general sources in this problem the optimum mean codeword length of variable length codes has already been determined on the other hand in this paper we focus on the overflow or excess codeword length probability instead of the mean codeword length the infimum of overflow thresholds under the constraint that both of the error probability and the overflow probability are smaller than or equal to some constant is called the optimum overflow threshold in this paper we first derive finite length upper and lower bounds on these probabilities so as to analyze the optimum overflow thresholds then by using these bounds we determine the general formula of the optimum overflow thresholds in both of the first order and second order forms next we consider another expression of the derived general formula so as to reveal the relationship with the optimum coding rate in the fixed length source coding problem finally we apply the general formula derived in this paper to the case of stationary memoryless sources

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

variable length compression allowing errors this paper studies the fundamental limits of the minimum average length of lossless and lossy variable length compression allowing a nonzero error probability epsilon for lossless compression we give nonasymptotic bounds on the minimum average length in terms of erokhin s rate distortion function and we use those bounds to obtain a gaussian approximation on the speed of approach to the limit which is quite accurate for all but small blocklengths 1 epsilon k h mathsf s k v mathsf s 2 pi 1 2 mathop rm exp q 1 left epsilon right 2 2 where q 1 left cdot right is the functional inverse of the standard gaussian complementary cumulative distribution function and v mathsf s is the source dispersion a nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of 1 epsilon but this asymptotic limit is approached from below i e larger source dispersions and shorter blocklengths are beneficial variable length lossy compression under an excess distortion constraint is shown to exhibit similar properties

asymptotics and non asymptotics for universal fixed to variable source coding universal fixed to variable lossless source coding for memoryless sources is studied in the finite blocklength and higher order asymptotic regimes optimal three term fixed error asymptotic expressions are derived for general fixed to variable codes and for prefix codes it is shown that the non prefix type size code in which codeword lengths are chosen in ascending order of type class size achieves the optimal third order term and outperforms classical two stage codes converse results are proved making use of a result on the distribution of the empirical entropy and laplace s approximation finally the fixed to variable coding problem without a prefix constraint is shown to be essentially the same as the universal guessing problem

a one to one code and its anti redundancy one to one codes are ldquoone shotrdquo codes that assign a distinct codeword to source symbols and are not necessarily prefix codes more generally uniquely decodable interestingly as wyner proved in 1972 for such codes the average code length can be smaller than the source entropy by how much we call this difference the anti redundancy various authors over the years have shown that the anti redundancy can be as big as minus the logarithm of the source entropy however to the best of our knowledge precise estimates do not exist in this note we consider a block code of length n generated for a binary memoryless source and prove that the average anti redundancy is 1 2 log2 n c f n o 1 where c is a constant and either f n 0 if log2 1 p p is irrational where p is the probability of generating a ldquo0rdquo or f n is a fluctuating function as the code length increases this relatively simple finding requires a combination of analytic tools such as precise evaluation of bernoulli sums the saddle point method and theory of distribution of sequences modulo 1

minimum guesswork with an unreliable oracle we study a guessing game where alice holds a discrete random variable x and bob tries to sequentially guess its value before the game begins bob can obtain side information about x by asking an oracle carole any binary question of his choosing carole s answer is however unreliable and is incorrect with probability epsilon we show that bob should always ask carole whether the index of x is odd or even with respect to a descending order of probabilities this question simultaneously minimizes all the guessing moments for any value of epsilon in particular this result settles a conjecture of burin and shayevitz we further consider a more general setup where bob can ask a multiple choice m ary question and then observe carole s answer through a noisy channel when the channel is completely symmetric i e when carole decides whether to lie regardless of bob s question and has no preference when she lies a similar question about the ordered index of x modulo m is optimal interestingly however the problem of testing whether a given question is optimal appears to be generally difficult in other symmetric channels we provide supporting evidence for this difficulty by showing that a core property required in our proofs becomes np hard to test in the general m ary case we establish this hardness result via a reduction from the problem of testing whether a system of modular difference disequations has a solution which we prove to be np hard for m geq 3

quantile correlations and quantile autoregressive modeling in this article we propose two important measures quantile correlation qcor and quantile partial correlation qpcor we then apply them to quantile autoregressive qar models and introduce two valuable quantities the quantile autocorrelation function qacf and the quantile partial autocorrelation function qpacf this allows us to extend the box jenkins three stage procedure model identification model parameter estimation and model diagnostic checking from classical autoregressive models to quantile autoregressive models specifically the qpacf of an observed time series can be employed to identify the autoregressive order while the qacf of residuals obtained from the fitted model can be used to assess the model adequacy we not only demonstrate the asymptotic properties of qcor and qpcor but also show the large sample results of qacf qpacf and the quantile version of the box pierce test moreover we obtain the bootstrap approximations to the distributions of parameter estimators and p

equivocations exponents and second order coding rates under various renyi information measures we evaluate the asymptotics of equivocations their exponents as well as their second order coding rates under various renyi information measures specifically we consider the effect of applying a hash function on a source and we quantify the level of non uniformity and dependence of the compressed source from another correlated source when the number of copies of the sources is large unlike previous works that use shannon information measures to quantify randomness information or uniformity we define our security measures in terms of a more general class of information measures the renyi information measures and their gallager type counterparts a special case of these renyi information measure is the class of shannon information measures we prove tight asymptotic results for the security measures and their exponential rates of decay we also prove bounds on the second order asymptotics and show that these bounds match when the magnitudes of the second order coding rates are large we do so by establishing new classes non asymptotic bounds on the equivocation and evaluating these bounds using various probabilistic limit theorems asymptotically

fixed length lossy compression in the finite blocklength regime this paper studies the minimum achievable source coding rate as a function of blocklength n and probability that the distortion exceeds a given level d tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength for stationary memoryless sources with separable distortion the minimum rate achievable is shown to be closely approximated by r d v d n q 1 where r d is the rate distortion function v d is the rate dispersion a characteristic of the source which measures its stochastic variability and q 1 is the inverse of the standard gaussian complementary cumulative distribution function

strict stationarity testing and glad estimation of double autoregressive models abstract in this article we develop a tractable procedure for testing strict stationarity in a double autoregressive model and formulate the problem as testing if the top lyapunov exponent is negative without strict stationarity assumption we construct a consistent estimator of the associated top lyapunov exponent and employ a random weighting approach for its variance estimation which in turn are used in a t type test we also propose a glad estimation for parameters of interest relaxing key assumptions on the commonly used qmle all estimators except for the intercept are shown to be consistent and asymptotically normal in both stationary and explosive situations the finite sample performance of the proposed procedures is evaluated via monte carlo simulation studies and a real dataset of interest rates is analyzed

second order noiseless source coding theorems shannon s celebrated source coding theorem can be viewed as a one sided law of large numbers we formulate second order noiseless source coding theorems for the deviation of the codeword lengths from the entropy for a class of sources that includes markov chains we prove a one sided central limit theorem and a law of the iterated logarithm

generalized autoregressive conditional heteroskedasticity a natural generalization of the arch autoregressive conditional heteroskedastic process introduced in engle 1982 to allow for past conditional variances in the current conditional variance equation is proposed stationarity conditions and autocorrelation structure for this new class of parametric models are derived maximum likelihood estimation and testing are also considered finally an empirical example relating to the uncertainty of the inflation rate is presented

feedback in the non asymptotic regime without feedback the backoff from capacity due to non asymptotic blocklength can be quite substantial for blocklengths and error probabilities of interest in many practical applications in this paper novel achievability bounds are used to demonstrate that in the non asymptotic regime the maximal achievable rate improves dramatically thanks to variable length coding and feedback for example for the binary symmetric channel with capacity 1 2 the blocklength required to achieve 90 of the capacity is smaller than 200 compared to at least 3100 for the best fixed blocklength code even with noiseless feedback virtually all the advantages of noiseless feedback are shown to be achievable even if the feedback link is used only to send a single signal informing the encoder to terminate the transmission stop feedback it is demonstrated that the non asymptotic behavior of the fundamental limit depends crucially on the particular model chosen for the end of packet control signal fixed blocklength codes and related questions concerning communicating with a guaranteed delay are discussed in which situation feedback is demonstrated to be almost useless even non asymptotically

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

variable length source dispersions differ under maximum and average error criteria variable length compression without prefix free constraints and with side information available at both encoder and decoder is considered instead of requiring the code to be error free we allow for it to have a non vanishing error probability we derive one shot bounds on the optimal average codeword length by proposing two new information quantities namely the conditional and unconditional varepsilon cutoff entropies using these one shot bounds we obtain the second order asymptotics of the problem under two different formalisms the average and maximum probabilities of error over the realization of the side information while the first order terms in the asymptotic expansions for both formalisms are identical we find that the source dispersion under the average error formalism is in most cases strictly smaller than its maximum error counterpart applications to a certain class of guessing problems previously studied by kuzuoka 2020 are also discussed

on the redundancy of lossy source coding with abstract alphabets the redundancy problem of lossy source coding with abstract source and reproduction alphabets is considered for coding at a fixed rate level it is shown that for any fixed rate r 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the rate r such that the distortion redundancy of c sub n defined as the difference between the performance of c sub n and the distortion rate function d p r of p is upper bounded by spl part d p r spl part r ln n 2n o ln n n for coding at a fixed distortion level it is demonstrated that for any d 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the fixed distortion d such that the rate redundancy of c sub n defined as the difference between the performance of c sub n and the rate distortion function r p d of p is upper bounded by 7ln n 6n o ln n n these results strengthen the traditional berger s 1968 1971 abstract alphabet source coding theorem and extend the positive redundancy results of zhang yang and wei see ibid vol 43 no 1 p 71 91 1997 and ibid vol 42 p 803 21 1996 on lossy source coding with finite alphabets and the redundancy result of wyner see ibid vol 43 p 1452 64 1997 on block coding of memoryless gaussian sources

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

second and third order asymptotics of the continuous time poisson channel the paper derives the optimal second order coding rate for the continuous time poisson channel we also obtain bounds on the third order coding rate this is the first instance of a second order result for a continuous time channel the converse proof hinges on a novel construction of an output distribution induced by wyner s discretized channel and the construction of an appropriate epsilon net of the input probability simplex while the achievability proof follows the general program to prove the third order term for non singular discrete memoryless channels put forth by polyanskiy several non standard techniques such as new definitions and bounds on the probabilities of typical sets using logarithmic sobolev inequalities are employed to handle the continuous nature of the channel

universal coding with minimum probability of codeword length overflow lossless block to variable length source coding is studied for finite state finite alphabet sources the aim is to minimize the probability that the normalized length of the codeword will exceed a given threshold b subject to the kraft inequality it is shown that the lempel ziv algorithm 1978 asymptotically attains the optimal performance in the sense just defined independently of the source and the value of b for the subclass of unifilar markov sources faster convergence to the asymptotic optimum performance can be accomplished by using the minimum description length universal code for this subclass it is demonstrated that these universal codes are also nearly optimal in the sense of minimizing buffer overflow probability and asymptotically optimal in a competitive sense

conditional quantile estimation for generalized autoregressive conditional heteroscedasticity models conditional quantile estimation is an essential ingredient in modern risk management although generalized autoregressive conditional heteroscedasticity garch processes have proven highly successful in modeling financial data it is generally recognized that it would be useful to consider a broader class of processes capable of representing more flexibly both asymmetry and tail behavior of conditional returns distributions in this article we study estimation of conditional quantiles for garch models using quantile regression quantile regression estimation of garch models is highly nonlinear we propose a simple and effective two step approach of quantile regression estimation for linear garch time series in the first step we use a quantile autoregression sieve approximation for the garch model by combining information over different quantiles then second stage estimation for the garch model is carried out based on the first stage minimum distance estimation of the scale process of the time series

composite quantile regression and the oracle model selection theory coefficient estimation and variable selection in multiple linear regression is routinely done in the penalized least squares ls framework the concept of model selection oracle introduced by fan and li j amer statist assoc 96 2001 1348 1360 characterizes the optimal behavior of a model selection procedure however the least squares oracle theory breaks down if the error variance is infinite in the current paper we propose a new regression method called composite quantile regression cqr we show that the oracle model selection theory using the cqr oracle works beautifully even when the error variance is infinite we develop a new oracular procedure to achieve the optimal properties of the cqr oracle when the error variance is finite cqr still enjoys great advantages in terms of estimation efficiency we show that the relative efficiency of cqr compared to the least squares is greater than 70 regardless the error distribution moreover cqr could be much more efficient and sometimes arbitrarily more efficient than the least squares the same conclusions hold when comparing a cqr oracular estimator with a ls oracular estimator

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

uniformly powerful goodness of fit tests the simple hypothesis is tested that the distribution of independent random variables x1 x2 xn is a given probability measure po let 7r be any sequence of partitions the alternative hypothesis is the set of probability measures p with ea ip a po a i 8 where 8 0 note the dependence of this set of alternatives on the sample size it is shown that if the effective cardinality of the partitions is of the same order as the sample size then sequences of tests exist with uniformly exponentially small probabilities of error conversely if the effective cardinality is of larger order than the sample size then no such sequence of tests exists the effective cardinality is the number of sets in the partition which exhaust all but a negligible portion of the probability under the null hypothesis

critical behavior in lossy source coding the following critical phenomenon was recently discovered when a memoryless source is compressed using a variable length fixed distortion code the fastest convergence rate of the pointwise compression ratio to r d is either o spl radic n or o log n we show it is always o spl radic n except for discrete uniformly distributed sources

existence of solutions of semilinear systems with gradient dependence via eigenvalue criteria abstract in this paper new criteria are established for the existence of positive radial solutions of a semilinear elliptic system depending on the gradient these criteria are determined by some relationships between the upper and lower bounds on suitable strips of r n of the nonlinearities of the system and the principal characteristic values of some associated linear hammerstein integral operators moreover using smoothing tools the totality of the involved cone is established

strong functional representation lemma and applications to coding theorems this paper shows that for any random variables x and y it is possible to represent y as a function of x z such that z is independent of x and i x z y le log i x y 1 4 bits we use this strong functional representation lemma sfrl to establish a bound on the rate needed for one shot exact channel simulation for general discrete or continuous random variables strengthening the results by harsha et al and braverman and garg and to establish new and simple achievability results for one shot variable length lossy source coding multiple description coding and gray wyner system we also show that the sfrl can be used to reduce the channel with state noncausally known at the encoder to a point to point channel which provides a simple achievability proof of the gelfand pinsker theorem

second order asymptotics in fixed length source coding and intrinsic randomness there is a difference between the optimal rates of fixed length source coding and intrinsic randomness when we care about the second order asymptotics we prove this difference for general information sources and then investigate independent and identically distributed i i d random variables and markovian variables as examples the difference is demonstrated through an investigation of the second order asymptotic behavior of the rates a universal fixed length source code attaining the second order optimal rate is also proposed the difference between the rates of fixed length source coding and intrinsic randomness proves that the outputs of fixed length source codes are not uniformly distributed

asymptotic inference for a nonstationary double ar 1 model we investigate the nonstationary double ar 1 model y t y t 1 t y 2 t 1 where 0 a 0 the t are independent standard normal random variables and e log t 0 we show that the maximum likelihood estimator of is consistent and asymptotically normal combination of this result with that in ling 2004 for the stationary case gives the asymptotic normality of the maximum likelihood estimator of for any in the real line with a root n rate of convergence this is in contrast to the results for the classical ar 1 model corresponding to a 0

joint source channel coding with feedback this paper quantifies the fundamental limits of variable length transmission of a general possibly analog source over a memoryless channel with noiseless feedback under a distortion constraint we consider excess distortion average distortion and guaranteed distortion d semifaithful codes in contrast to the asymptotic fundamental limit a general conclusion is that allowing variable length codes and feedback leads to a sizable improvement in the fundamental delay distortion tradeoff in addition we investigate the minimum energy required to reproduce k source samples with a given fidelity after transmission over a memoryless gaussian channel and we show that the required minimum energy is reduced with feedback and an average rather than maximal power constraint

linear double autoregression abstract this paper proposes the linear double autoregression a conditional heteroscedastic model with a conditional mean structure but compatible with the quantile regression the existence of a strictly stationary solution is discussed for which a necessary and sufficient condition is established a doubly weighted quantile regression estimation procedure is introduced where the first set of weights ensures the asymptotic normality of the estimator and the second set improves its efficiency through balancing individual quantile regression estimators across multiple quantile levels bayesian information criteria are proposed for model selection and two goodness of fit tests are constructed to check the adequacy of the fitted conditional mean and conditional scale structures simulation studies indicate that the proposed inference tools perform well in finite samples and an empirical example illustrates the usefulness of the new model

minimax redundancy for the class of memoryless sources let x sup n x sub 1 x sub n be a memoryless source with unknown distribution on a finite alphabet of size k we identify the asymptotic minimax coding redundancy for this class of sources and provide a sequence of asymptotically minimax codes equivalently we determine the limiting behavior of the minimax relative entropy min sub qxn max sub pxn d p sub xn spl par q sub xn where the maximum is over all independent and identically distributed i i d source distributions and the minimum is over all joint distributions we show in this paper that the minimax redundancy minus k 1 2 log n 2 spl pi e converges to log spl int spl radic det i spl theta d spl theta log spl gamma 1 2 sup k spl gamma k 2 where i spl theta is the fisher information and the integral is over the whole probability simplex the bayes strategy using jeffreys prior is shown to be asymptotically maximin but not asymptotically minimax in our setting the boundary risk using jeffreys prior is higher than that of interior points we provide a sequence of modifications of jeffreys prior that put some prior mass near the boundaries of the probability simplex to pull down that risk to the asymptotic minimax level in the limit

nonlinear singular boundary value problems in the study of symmetric plasma abstract this paper concerns the solvability of the singular differential equation r t x p t x q t f t x r t x p t x with some boundary conditions and improves some known results

quasilinear equations with dependence on the gradient abstract we discuss the existence of positive solutions of the problem q t u t f t u t u t for t 0 1 and u 0 u 1 0 where the nonlinearity f satisfies a superlinearity condition at 0 and a local superlinearity condition at this general quasilinear differential operator involves a weight q and a main differentiable part which is not necessarily a power due to the superlinearity of f and its dependence on the derivative a condition of the bernstein nagumo type is assumed also involving the differential operator our main result is the proof of a priori bounds for the eventual solutions the presence of the derivative in the right hand side of the equation requires a priori bounds not only on the solutions themselves but also on their derivatives which brings additional difficulties as an application we consider a quasilinear dirichlet problem in an annulus div a u u f x u u in r 1 x r 2 u x 0 on x r 1 and x r 2

the redundancy of source coding with a fidelity criterion 1 known statistics the problem of redundancy of source coding with respect to a fidelity criterion is considered for any fixed rate r 0 and any memoryless source with finite source and reproduction alphabets and a common distribution p the nth order distortion redundancy d sub n r of fixed rate coding is defined as the minimum of the difference between the expected distortion per symbol of any block code with length n and rate r and the distortion rate function d p r of the source p it is demonstrated that for sufficiently large n d sub n r is equal to spl part spl part r d p r ln n 2n o ln n n where spl part spl part r d p r is the partial derivative of d p r evaluated at r and assumed to exist for any fixed distortion level d 0 and any memoryless source p the nth order rate redundancy r sub n d of coding at fixed distortion level d or by using d semifaithful codes is defined as the minimum of the difference between the expected rate per symbol of any d semifaithful code of length n and the rate distortion function r p d of p evaluated at d it is proved that for sufficiently large n r sub n d is upper bounded by ln n n o ln n n and lower bounded by in n 2n o in n n as a by product the lower bound of r sub n d derived in this paper gives a positive answer to a conjecture proposed by yu and speed 1993

asymptotic properties on codeword lengths of an optimal fv code for general sources this correspondence is concerned with asymptotic properties on the codeword length of a fixed to variable length code fv code for a general source x sup n sub n 1 sup spl infin with a finite or countably infinite alphabet suppose that for each n spl ges 1 x sup n is encoded to a binary codeword spl phi sub n x sup n of length l spl phi sub n x sup n letting spl epsiv sub n denote the decoding error probability we consider the following two criteria on fv codes i spl epsiv sub n 0 for all n spl ges 1 and ii lim sup sub n spl rarr spl infin spl epsiv sub n spl les spl epsiv for an arbitrarily given spl epsiv spl isin 0 1 under criterion i we show that if x sup n is encoded by an arbitrary prefix free fv code asymptotically achieving the entropy 1 nl spl phi sub n x sup n 1 nlog sub 2 1 px sup n x sup n spl rarr 0 in probability as n spl rarr spl infin under a certain condition where p sub x sup n denotes the probability distribution of x sup n under criterion ii we first determine the minimum rate achieved by fv codes next we show that 1 nl spl phi sub n x sup n of an arbitrary fv code achieving the minimum rate in a certain sense has a property similar to the lossless case

strict stationarity of generalized autoregressive processes in this paper we consider the multivariate equation x n 1 a n 1 x n b n 1 with i i d coefficients which have only a logarithmic moment we give a necessary and sufficient condition for existence of a strictly stationary solution independent of the future as an application we characterize the multivariate arma equations with general noise which have such a solution

global self weighted and local quasi maximum exponential likelihood estimators for arma garch igarch models this paper investigates the asymptotic theory of the quasi maximum exponential likelihood estimators qmele for arma garch models under only a fractional moment condition the strong consistency and the asymptotic normality of the global self weighted qmele are obtained based on this self weighted qmele the local qmele is showed to be asymptotically normal for the arma model with garch finite variance and igarch errors a formal comparison of two estimators is given for some cases a simulation study is carried out to assess the performance of these estimators and a real example on the world crude oil price is given

self weighted least absolute deviation estimation for infinite variance autoregressive models summary how to undertake statistical inference for infinite variance autoregressive models has been a long standing open problem to solve this problem we propose a self weighted least absolute deviation estimator and show that this estimator is asymptotically normal if the density of errors and its derivative are uniformly bounded furthermore a wald test statistic is developed for the linear restriction on the parameters and it is shown to have non trivial local power simulation experiments are carried out to assess the performance of the theory and method in finite samples and a real data example is given the results are entirely different from other published results and should provide new insights for future research on heavy tailed time series

the dispersion of nearest neighbor decoding for additive non gaussian channels we study the second order asymptotics of information transmission using random gaussian codebooks and nearest neighbor decoding over a power limited stationary memoryless additive non gaussian noise channel we show that the dispersion term depends on the non gaussian noise only through its second and fourth moments thus complementing the capacity result lapidoth 1996 which depends only on the second moment furthermore we characterize the second order asymptotics of point to point codes over k sender interference networks with non gaussian additive noise specifically we assume that each user s codebook is gaussian and that nn decoding is employed i e that interference from the k 1 unintended users gaussian interfering signals is treated as noise at each decoder we show that while the first order term in the asymptotic expansion of the maximum number of messages depends on the power of the interfering codewords only through their sum this does not hold for the second order term

weak variable length source coding given a general source x x sup n sub n 1 sup spl infin source coding is characterized by a pair spl phi sub n spl psi sub n of encoder spl phi sub n and decoder spl psi sub n together with the probability of error spl epsi sub n spl equiv pr spl psi sub n spl phi sub n x sup n spl ne x sup n if the length of the encoder output spl phi sub n x sup n is fixed then it is called fixed length source coding while if the length of the encoder output spl phi sub n x sup n is variable then it is called variable length source coding usually in the context of fixed length source coding the probability of error spl epsi sub n is required to asymptotically vanish i e lim sub n spl rarr spl infin spl epsi sub n 0 whereas in the context of variable length source coding the probability of error spl epsi sub n is required to be exactly zero i e spl epsi sub n 0 spl forall n 1 2 in contrast to these we consider the problem of variable length source coding with asymptotically vanishing probability of error i e lim sub n spl rarr spl infin spl epsi sub n 0 and establish several fundamental theorems on this new subject

an achievable bound for optimal noiseless coding of a random variable corresp for a discrete n valued random variable n possibly denumerably infinite leung yan cheong and cover have given bounds for the minimal expected length of a one to one not necessarily uniquely decodable code l 1 1 sum i 1 n p i log left frac 1 2 1 right it is shown that the best possible case occurs for certain denumerably infinite sets of nonzero probabilities this absolute bound is related to the shannon entropy h of the distribution by h cdot is the binary entropy function