asymmetric evaluations of erasure and undetected error probabilities the problem of channel coding with the erasure option is revisited for discrete memoryless channels the interplay between the code rate the undetected and total error probabilities is characterized using the information spectrum method a sequence of codes of increasing blocklengths n is designed to illustrate this tradeoff furthermore for additive discrete memoryless channels with uniform input distribution we establish that our analysis is tight with respect to the ensemble average this is done by analyzing the ensemble performance in terms of a tradeoff between the code rate the undetected and total errors this tradeoff is parameterized by the threshold in a generalized likelihood ratio test two asymptotic regimes are studied first the code rate tends to the capacity of the channel at a rate slower than n 1 2 corresponding to the moderate deviations regime in this case both error probabilities decay subexponentially and asymmetrically the precise decay rates are characterized second the code rate tends to capacity at a rate of n 1 2 in this case the total error probability is asymptotically a positive constant while the undetected error probability decays as exp bn 1 2 for some b 0 the proof techniques involve the applications of a modified or shifted version of the gartner ellis theorem and the type class enumerator method to characterize the asymptotic behavior of a sequence of cumulant generating functions

polygonal approximation to the flow on the critical surface for the bistable equation abstract we present a method to follow the evolution of the flow on the stable manifold of the stationary wave for the bistable equation previous calculations indicated that a large class of initial data those whose support above threshold is kept away from 0 and on this surface are characterized by a constant value of their area the results presented here show that this is not sufficient to characterize data on threshold we have observed that the areas converge smoothly under the flow to the area of the standing wave we conjecture that the critical surface m is divided into three distinct regions in the first the critical multiplier is minimal in the second data has constant area and the third is a region containing the standing wave the procedure is an adaptation of euler s polygonal method for the solution of ordinary differential equations

composite scheme lr th for decoding with erasures and its effective equivalence to forney s rule for decoding with erasures forney s scheme is known to be optimal in the sense that no other scheme can make the erasure probability p sub ers and undetected error probability p sub uer simultaneously smaller we propose a scheme for erasure decision which tests the likelihood ratio as well as the likelihood itself and show that the attainable upper bounds on p sub ers and p sub uer are the same as those proved for the optimal scheme up to a constant factor we also show that the scheme gives when applied to convolutional codes a bound which is related to the block coding bound via forney s inverse concatenation construction we show that this bound is the same as the one which naturally arises when we apply raghavan and baum s 1998 optimal scheme to convolutional code

maximum entropy and bayesian data analysis entropic prior distributions the problem of assigning probability distributions which reflect the prior information available about experiments is one of the major stumbling blocks in the use of bayesian methods of data analysis in this paper the method of maximum relative entropy me is used to translate the information contained in the known form of the likelihood into a prior distribution for bayesian inference the argument is inspired and guided by intuition gained from the successful use of me methods in statistical mechanics for experiments that cannot be repeated the resulting entropic prior is formally identical with the einstein fluctuation formula for repeatable experiments however the expected value of the entropy of the likelihood turns out to be relevant information that must be included in the analysis the important case of a gaussian likelihood is treated in detail

gradient descent learning in and out of equilibrium relations between the off thermal equilibrium dynamical process of on line learning and the thermally equilibrated off line learning are studied for potential gradient descent learning the approach of opper to study on line bayesian algorithms is used for potential based or maximum likelihood learning we look at the on line learning algorithm that best approximates the off line algorithm in the sense of least kullback leibler information loss the closest on line algorithm works by updating the weights along the gradient of an effective potential which is different from the parent off line potential a few examples are analyzed and the origin of the potential annealing is discussed

banach lie poisson spaces and reduction the category of banach lie poisson spaces is introduced and studied it is shown that the category of w algebras can be considered as one of its subcategories examples and applications of banach lie poisson spaces to quantization and integration of hamiltonian systems are given the relationship between classical and quantum reduction is discussed

multiple impulse solutions to mckean s caricature of the nerve equation ii stability we study mckean s caricature of a nerve conduction equation where h is the heaviside function it is proved that an n ple impulse solution resembling the superposition of n unstable solitary impulses has at most 2n 1 and at least n unstable modes exactly n unstable modes corresponding to the amplitudes and the rest of them corresponding to the spacings the n amplitude modes always exist we prove also that for an n ple impulse solution resembling the superposition of n stable solitary impulses there are at most n 1 unstable modes and all of them are of spacing type

large deviations behavior of the logarithmic error probability of random codes this work studies the deviations of the error exponent of the constant composition code ensemble around its expectation known as the error exponent of the typical random code trc in particular it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the trc exponent is exponentially small upper and lower bounds for this exponent are given which coincide in some cases in addition the probability of randomly drawing a codebook whose error exponent is larger than the trc exponent is shown to be double exponentially small upper and lower bounds to the double exponential exponent are given the results suggest that codebooks whose error exponent is larger than the error exponent of the trc are extremely rare the key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables

bracket formulation of diffusion convection equations abstract an abstract diffusion convection equation is introduced it is shown that many well known and experimentally tested time evolution equations of physical systems e g the boltzmann and the enskog kinetic equations and the navier stokes fourier fluid mechanics equations are identical to or well approximated by realizations of the abstract diffusion convection equation the physical content of the abstract diffusion convection equation is compatibility with equilibrium thermodynamics it means that properties of solutions of the abstract diffusion convection equation are mathematical formulations of the experience on which equilibrium thermodynamics is based the requirement that all admissible time evolution equations are realizations of the abstract diffusion convection equation is used to find admissible closures in the bbgky and the maxwell grad hierarchies and admissible constitutive relations in generalized hydrodynamics

minimax universal decoding with an erasure option motivated by applications of rateless coding decision feedback and automatic repeat request arq we study the problem of universal decoding for unknown channels in the presence of an erasure option specifically we harness the competitive minimax methodology developed in earlier studies in order to derive a universal version of forney s classical erasure list decoder which in the erasure case optimally trades off between the probability of erasure and the probability of undetected error the proposed universal erasure decoder guarantees universal achievability of a certain fraction xi of the optimum error exponents of these probabilities in a sense to be made precise in the sequel a single letter expression for xi which depends solely on the coding rate and the neyman pearson threshold to be defined is provided the example of the binary symmetric channel is studied in full detail and some conclusions are drawn

erasure list random coding error exponents are not universally achievable we study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure list option at the decoder in the random coding regime in particular we harness a universal version of forney s classical erasure list decoder developed in earlier studies which is based on the competitive minimax methodology and guarantees universal achievability of a certain fraction of the optimum random coding error exponents in this paper we derive an exact single letter expression for the maximum achievable fraction examples are given in which the maximal achievable fraction is strictly less than unity which imply that in general there is no universal erasure list decoder which achieves the same random coding error exponents as the optimal decoder for a known channel this is in contrast to the situation in ordinary decoding without the erasure list option where optimum exponents are universally achievable as is well known it is also demonstrated that previous lower bounds derived for the maximal achievable fraction are not tight in general we then analyze a generalized random coding ensemble which incorporate a training sequence in conjunction with a suboptimal practical decoder plug in decoder which first estimates the channel using the available training sequence and then decodes the remaining symbols of the codeword using the estimated channel one of the implications of our results is setting the stage for a reasonable criterion of optimal training finally we compare the performance of the plug in decoder and the universal decoder in terms of the achievable error exponents and show that the latter is noticeably better than the former

error exponents of erasure list decoding revisited via moments of distance enumerators the analysis of random coding error exponents pertaining to erasure list decoding due to forney is revisited instead of using jensen s inequality as well as some other inequalities in the derivation we demonstrate that an exponentially tight analysis can be carried out by assessing the relevant moments of certain distance enumerators the resulting bound has the following advantages i it is at least as tight as forney s bound ii under certain symmetry conditions associated with the channel and the random coding distribution it is simpler than forney s bound in the sense that it involves an optimization over one parameter only rather than two and iii in certain special cases like the binary symmetric channel bsc the optimum value of this parameter can be found in closed form and so there is no need to conduct a numerical search we have not found yet a numerical example where this new bound is strictly better than forney s bound and this may provide an additional evidence to support forney s conjecture that his bound is tight for the average code however when applying the proposed analysis technique to a certain universal decoder with erasures we demonstrate that it may yield significantly tighter exponential error bounds we believe that this technique can be useful in simplifying and improving exponential error bounds in other problem settings as well

essential equivalence of the general equation for the nonequilibrium reversible irreversible coupling generic and steepest entropy ascent models of dissipation for nonequilibrium thermodynamics by reformulating the steepest entropy ascent sea dynamical model for nonequilibrium thermodynamics in the mathematical language of differential geometry we compare it with the primitive formulation of the general equation for the nonequilibrium reversible irreversible coupling generic model and discuss the main technical differences of the two approaches in both dynamical models the description of dissipation is of the entropy gradient type sea focuses only on the dissipative i e entropy generating component of the time evolution chooses a sub riemannian metric tensor as dissipative structure and uses the local entropy density field as potential generic emphasizes the coupling between the dissipative and nondissipative components of the time evolution chooses two compatible degenerate structures poisson and degenerate co riemannian and uses the global energy and entropy functionals as potentials as an illustration we rewrite the known generic formulation of the boltzmann equation in terms of the square root of the distribution function adopted by the sea formulation we then provide a formal proof that in more general frameworks whenever all degeneracies in the generic framework are related to conservation laws the sea and generic models of the dissipative component of the dynamics are essentially interchangeable provided of course they assume the same kinematics as part of the discussion we note that equipping the dissipative structure of generic with the leibniz identity makes it automatically sea on metric leaves

error exponents of optimum decoding for the interference channel exponential error bounds for the finite alphabet interference channel ifc with two transmitter receiver pairs are investigated under the random coding regime our focus is on optimum decoding as opposed to heuristic decoding rules that have been used in previous works like joint typicality decoding decoding based on interference cancellation and decoding that considers the interference as additional noise indeed the fact that the actual interfering signal is a codeword and not an independent and identically distributed i i d noise process complicates the application of conventional techniques to the performance analysis of the optimum decoder using analytical tools rooted in statistical physics we derive a single letter expression for error exponents achievable under optimum decoding and demonstrate strict improvement over error exponents obtainable using suboptimal decoding rules but which are amenable to more conventional analysis

exact random coding exponents for erasure decoding random coding of channel decoding with an erasure option is studied by analyzing the large deviations behavior of the code ensemble we obtain exact single letter formulas for the error exponents in lieu of forney s lower bounds the analysis technique we use is based on an enhancement and specialization of tools for assessing the statistical properties of certain distance enumerators we specialize our results to the setup of the binary symmetric channel case with uniform random coding distribution and derive an explicit expression for the error exponent which unlike forney s bounds does not involve optimization over two parameters we also establish the fact that for this setup the difference between the exact error exponent corresponding to the probability of undetected decoding error and the exponent corresponding to the erasure event is equal to the threshold parameter numerical calculations indicate that for this setup as well as for a z channel forney s bound coincides with the exact random coding exponent

exact correct decoding exponent of the wiretap channel decoder the performance of the achievability scheme for wyner s wiretap channel model is examined from the perspective of the probability of correct decoding pc at the wiretap channel decoder in particular for finite alphabet memoryless channels the exact random coding exponent of pc is derived as a function of the total coding rate r1 and the rate of each subcode r2 two different representations are given for this function and its basic properties are provided we also characterize the region of pairs of rates r1 r2 of full security in the sense of the random coding exponent of pc in other words the region where the exponent of this achievability scheme is the same as that of blind guessing at the eavesdropper side finally an analogous derivation of the correct decoding exponent is outlined for the case of the gaussian channel

generalization of the dynamical lack of fit reduction from generic to generic the lack of fit statistical reduction developed and formulated first by bruce turkington is a general method taking liouville equation for probability density detailed level and transforming it to reduced dynamics of projected quantities less detailed level in this paper the method is generalized the hamiltonian liouville equation is replaced by an arbitrary hamiltonian evolution combined with gradient dynamics generic the boltzmann entropy is replaced by an arbitrary entropy and the kinetic energy by an arbitrary energy the gradient part is a generalized gradient dynamics generated by a dissipation potential the reduced evolution of the projected state variables is shown to preserve the generic structure of the original detailed level evolution the dissipation potential is obtained by solving a hamilton jacobi equation in summary the lack of fit reduction can start with generic and obtain generic for the reduced state variables

the generalized random energy model and its application to the statistical physics of ensembles of hierarchical codes in an earlier work the statistical physics associated with finite temperature decoding of code ensembles along with the relation to their random coding error exponents were explored in a framework that is analogous to derrida s random energy model rem of spin glasses according to which the energy levels of the various spin configurations are independent random variables the generalized rem grem extends the rem in that it introduces correlations between energy levels in an hierarchical structure in this paper we explore some analogies between the behavior of the grem and that of code ensembles which have parallel hierarchical structures in particular in analogy to the fact that the grem may have different types of phase transition effects depending on the parameters of the model then the above mentioned hierarchical code ensembles behave substantially differently in the various domains of the design parameters of these codes we make an attempt to explore the insights that can be imported from the statistical mechanics of the grem and be harnessed to serve for code design considerations and guidelines

achievable error exponents for the private fingerprinting game fingerprinting systems in the presence of collusive attacks are analyzed as a game between a fingerprinter and a decoder on the one hand and a coalition of two or more attackers on the other hand the fingerprinter distributes to different users different fingerprinted copies of a host data covertext drawn from a memoryless stationary source embedded with different fingerprints the coalition members create a forgery of the data while aiming at erasing the fingerprints in order not to be detected their action is modeled by a multiple access channel mac we analyze the performance of two classes of decoders associated with different kinds of error events the decoder of the first class aims at detecting the entire coalition whereas the second is satisfied with the detection of at least one member of the coalition both decoders have access to the original covertext data and observe the forgery in order to identify member s of the coalition motivated by a worst case approach we assume that the coalition of attackers is informed of the hiding strategy taken by the fingerprinter and the decoder while they are uninformed of the attacking scheme achievable single letter expressions for the two kinds of error exponents are obtained single letter lower bounds are also derived for the subclass of constant composition codes these lower and the upper bounds coincide for the error exponent of the first class further for the error of the first kind a decoder that is optimal is introduced and the worst case attack channel is characterized