second order and moderate deviations asymptotics for successive refinement we derive the optimal second order coding region and moderate deviations constant for successive refinement source coding with a joint excess distortion probability constraint we consider two scenarios 1 a discrete memoryless source dms and arbitrary distortion measures at the decoders and 2 a gaussian memoryless source gms and quadratic distortion measures at the decoders for a dms with arbitrary distortion measures we prove an achievable second order coding region using type covering lemmas by kanlis and narayan and by no ingber and weissman we prove the converse using the perturbation approach by gu and effros when the dms is successively refinable the expressions for the second order coding region and the moderate deviations constant are simplified and easily computable for this case we also obtain new insights on the second order behavior compared with the scenario where separate excess distortion proabilities are considered for example we describe a dms for which the optimal second order region transitions from being characterizable by a bivariate gaussian to a univariate gaussian as the distortion levels are varied we then consider a gms with quadratic distortion measures to prove the direct part we make use of the sphere covering theorem by verger gaugry together with appropriately defined gaussian type classes to prove the converse we generalize kostina and verdu s one shot converse bound for point to point lossy source coding we remark that this proof is applicable to general successively refinable sources in the proofs of the moderate deviations results for both scenarios we follow a strategy similar to that for the second order asymptotics and use the moderate deviations principle

second order resolvability intrinsic randomness and fixed length source coding for mixed sources information spectrum approach the second order achievable asymptotics in typical random number generation problems such as resolvability intrinsic randomness and fixed length source coding are considered in these problems several researchers have derived the first order and the second order achievability rates for general sources using the information spectrum methods although these formulas are general their computations are quite hard hence an attempt to address explicit computation problems of achievable rates is meaningful in particular for i i d sources the second order achievable rates have earlier been determined simply by using the asymptotic normality in this paper we consider mixed sources of two i i d sources the mixed source is a typical case of nonergodic sources and whose self information does not have the asymptotic normality nonetheless we can explicitly compute the second order achievable rates for these sources on the basis of two peak asymptotic normality in addition extensions of our results to more general mixed sources such as a mixture of countably infinite i i d sources or markov sources and a continuous mixture of i i d sources are considered

second order asymptotics for the gaussian mac with degraded message sets this paper studies the second order asymptotics of the gaussian multiple access channel with degraded message sets for a fixed average error probability varepsilon in 0 1 and an arbitrary point on the boundary of the capacity region we characterize the speed of convergence of rate pairs that converge to that boundary point for codes that have asymptotic error probability no larger than varepsilon as a stepping stone to this local notion of the second order asymptotics we study a global notion and establish relationships between the two we provide a numerical example to illustrate how the angle of approach to a boundary point affects the second order coding rate this is the first conclusive characterization of the second order asymptotics of a network information theory problem in which the capacity region is not a polygon

exponential strong converse for content identification with lossy recovery we revisit the high dimensional content identification with lossy recovery problem tuncel and gunduz 2014 and establish an exponential strong converse theorem as a corollary of the exponential strong converse theorem we derive an upper bound on the joint identification error and excess distortion exponent for the problem our main results can be specialized to the biometrical identification problem willems 2003 and the content identification problem tuncel 2009 since these two problems are both special cases of the content identification with lossy recovery problem we leverage the information spectrum method introduced by oohama and adapt the strong converse techniques therein to be applicable to the problem at hand

neyman pearson test for zero rate multiterminal hypothesis testing the problem of zero rate multiterminal hypothesis testing is revisited from the perspective of information spectrum approach and finite blocklength analysis a neyman pearson like test is proposed and its non asymptotic performance is clarified for a short block length it is numerically determined that the proposed test is superior to the previously reported hoeffding like test proposed by han kobayashi for a large deviation regime it is shown that our proposed test achieves an optimal trade off between the type i and type ii exponents presented by han kobayashi among the class of symmetric type based testing schemes when the type i error probability is non vanishing the proposed test is optimal up to the second order term of the type ii error exponent the latter term is characterized in terms of the variance of the projected relative entropy density the information geometry method plays an important role in the analysis as well as the construction of the test

minimum expected length of fixed to variable lossless compression without prefix constraints the minimum expected length for fixed to variable length encoding of an n block memoryless source with entropy h grows as nh o 1 where the term o 1 lies between 0 and 1 however this well known performance is obtained under the implicit constraint that the code assigned to the whole n block is a prefix code dropping the prefix constraint which is rarely necessary at the block level we show that the minimum expected length for a finite alphabet memoryless source with known distribution grows as nh 1 2 log n o 1 unless the source is equiprobable we also refine this result up to o 1 for those memoryless sources whose log probabilities do not reside on a lattice

nonasymptotic noisy lossy source coding this paper shows new general nonasymptotic achievability and converse bounds and performs their dispersion analysis for the lossy compression problem in which the compressor observes the source through a noisy channel while this problem is asymptotically equivalent to a noiseless lossy source coding problem with a modified distortion function nonasymptotically there is a noticeable gap in how fast their minimum achievable coding rates approach the common rate distortion function as evidenced both by the refined asymptotic analysis dispersion and the numerical results the size of the gap between the dispersions of the noisy problem and the asymptotically equivalent noiseless problem depends on the stochastic variability of the channel through which the compressor observes the source

asymptotic controllability implies feedback stabilization it is shown that every asymptotically controllable system can be globally stabilized by means of some discontinuous feedback law the stabilizing strategy is based on pointwise optimization of a smoothed version of a control lyapunov function iteratively sending trajectories into smaller and smaller neighborhoods of a desired equilibrium a major technical problem and one of the contributions of the present paper concerns the precise meaning of solution when using a discontinuous controller

on the dispersions of the gel fand pinsker channel and dirty paper coding this paper studies the second order coding rates for memoryless channels with a state sequence known non causally at the encoder in the case of finite alphabets an achievability result is obtained using constant composition random coding and by using a small fraction of the block to transmit the empirical distribution of the state sequence for error probabilities less than 0 5 it is shown that the second order rate improves on an existing one based on independent and identically distributed random coding in the gaussian case dirty paper coding with an almost sure power constraint an achievability result is obtained using random coding over the surface of a sphere and using a small fraction of the block to transmit a quantized description of the state power it is shown that the second order asymptotics are identical to the single user gaussian channel of the same input power without a state

nonasymptotic and second order achievability bounds for coding with side information we present a novel nonasymptotic or finite blocklength achievability bounds for three side information problems in network information theory these include 1 the wyner ahlswede korner wak problem of almost lossless source coding with rate limited side information 2 the wyner ziv wz problem of lossy source coding with side information at the decoder and 3 the gel fand pinsker gp problem of channel coding with noncausal state information available at the encoder the bounds are proved using ideas from channel simulation and channel resolvability our bounds for all three problems improve on all previous nonasymptotic bounds on the error probability of the wak wz and gp problems in particular those derived by verdu using our novel nonasymptotic bounds we recover the general formulas for the optimal rates of these side information problems finally we also present achievable second order coding rates by applying the multidimensional berry esseen theorem to our new nonasymptotic bounds numerical results show that the second order coding rates obtained using our nonasymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds

a recursive approach to low complexity codes a method is described for constructing long error correcting codes from one or more shorter error correcting codes referred to as subcodes and a bipartite graph a graph is shown which specifies carefully chosen subsets of the digits of the new codes that must be codewords in one of the shorter subcodes lower bounds to the rate and the minimum distance of the new code are derived in terms of the parameters of the graph and the subeodes both the encoders and decoders proposed are shown to take advantage of the code s explicit decomposition into subcodes to decompose and simplify the associated computational processes bounds on the performance of two specific decoding algorithms are established and the asymptotic growth of the complexity of decoding for two types of codes and decoders is analyzed the proposed decoders are able to make effective use of probabilistic information supplied by the channel receiver e g reliability information without greatly increasing the number of computations required it is shown that choosing a transmission order for the digits that is appropriate for the graph and the subcodes can give the code excellent burst error correction abilities the construction principles

the dynamics of group codes dual abelian group codes and systems fundamental results concerning the dynamics of abelian group codes behaviors and their duals are developed duals of sequence spaces over locally compact abelian lca groups may be defined via pontryagin duality dual group codes are orthogonal subgroups of dual sequence spaces the dual of a complete code or system is finite and the dual of a laurent code or system is anti laurent if c and c sup spl perp are dual codes then the state spaces of c act as the character groups of the state spaces of c sup spl perp the controllability properties of c are the observability properties of c sup spl perp in particular c is strongly controllable if and only if c sup spl perp is strongly observable and the controller memory of c is the observer memory of c sup spl perp the controller granules of c act as the character groups of the observer granules of c sup spl perp examples of minimal observer form encoder and syndrome former constructions are given finally every observer granule of c is an end around controller granule of c

normal factor graphs and holographic transformations this paper stands at the intersection of two distinct lines of research one line is holographic algorithms a powerful approach introduced by valiant for solving various counting problems in computer science the other is normal factor graphs an elegant framework proposed by forney for representing codes defined on graphs we introduce the notion of holographic transformations for normal factor graphs and establish a very general theorem called the generalized holant theorem which relates a normal factor graph to its holographic transformation we show that the generalized holant theorem on the one hand underlies the principle of holographic algorithms and on the other hand reduces to a general duality theorem for normal factor graphs a special case of which was first proved by forney in the course of our development we formalize a new semantics for normal factor graphs which highlights various linear algebraic properties that potentially enable the use of normal factor graphs as a linear algebraic tool

strong converses in source coding relative to a fidelity criterion two strong converses are obtained for an abstract alphabet stationary ergodic source coded relative to an appropriate fidelity criterion it is shown that given a distortion rate point d r that lies below the rate distortion curve 1 block codes that operate at rate level r must encode sample source blocks at a rate exceeding d with probability tending to one as the block length tends to infinity and 2 variable rate codes that operate at distortion level d must encode sample source blocks at a rate exceeding r with probability tending to one as the block length tends to infinity the previously known weak converses guarantee only that the indicated probabilities remain bounded away from zero as block length tends to infinity the proofs of the strong converses involve sample converses in source coding theory

characteristic generators and dualization for tail biting trellises this paper focuses on dualizing tail biting trellises particularly kv trellises these trellises are based on characteristic generators as introduced by koetter vardy 2003 and may be regarded as a natural generalization of minimal conventional trellises even though they are not necessarily minimal two dualization techniques will be investigated the local dualization introduced by forney 2001 for general normal graphs and a linear algebra based dualization tailored to the specific class of tail biting bahl cocke jelinek raviv bcjr trellises introduced by nori shankar 2006 it turns out that in general the bcjr dual is a subtrellis of the local dual while for kv trellises these two coincide furthermore making use of both the bcjr construction and the local dualization it will be shown that for each complete set of characteristic generators of a code there exists a complete set of characteristic generators of the dual code such that their resulting kv trellises are dual to each other if paired suitably this proves a stronger version of a conjecture formulated by koetter vardy

pointwise redundancy in lossy data compression and universal lossy data compression we characterize the achievable pointwise redundancy rates for lossy data compression at a fixed distortion level pointwise redundancy refers to the difference between the description length achieved by an nth order block code and the optimal nr d bits for memoryless sources we show that the best achievable redundancy rate is of order o spl radic n in probability this follows from a second order refinement to the classical source coding theorem in the form of a one sided central limit theorem moreover we show that along almost any source realization the description lengths of any sequence of block codes operating at distortion level d exceed nr d by at least as much as c spl radic nloglogn infinitely often corresponding direct coding theorems are also given showing that these rates are essentially achievable the above rates are in sharp contrast with the expected redundancy rates of order o log n reported by various authors our approach is based on showing that the compression performance of an arbitrary sequence of codes is essentially bounded below by the performance of shannon s random code we obtain partial generalizations of the above results for arbitrary sources with memory and we prove lossy analogs of barron s lemma barron 1985

nonpathological lyapunov functions and discontinuous caratheodory systems differential equations with discontinuous right hand side and solutions intended in caratheodory sense are considered for these equations sufficient conditions which guarantee both lyapunov stability and asymptotic stability in terms of nonsmooth lyapunov functions are given an invariance principle is also proven

existence of carathe spl acute odory solutions in nonlinear systems with discontinuous switching feedback controllers in this note we consider the existence of a carathe spl acute odory solution in a single input single output nonlinear system with a discontinuous switching feedback controller the main contribution is to show that if the nonlinear system can be transformed into a global normal form then we can specify the value of the discontinuous switching feedback controller on the switching hypersurface so that the closed loop system has a carathe spl acute odory solution

lossy joint source channel coding in the finite blocklength regime this paper finds new tight finite blocklength bounds for the best achievable lossy joint source channel code rate and demonstrates that joint source channel code design brings considerable performance advantage over a separate one in the nonasymptotic regime a joint source channel code maps a block of k source symbols onto a length n channel codeword and the fidelity of reproduction at the receiver end is measured by the probability e that the distortion exceeds a given threshold d for memoryless sources and channels it is demonstrated that the parameters of the best joint source channel code must satisfy nc kr d nv k v d q 1 e where c and v are the channel capacity and channel dispersion respectively r d and v d are the source rate distortion and rate dispersion functions and q is the standard gaussian complementary cumulative distribution function symbol by symbol uncoded transmission is known to achieve the shannon limit when the source and channel satisfy a certain probabilistic matching condition in this paper we show that even when this condition is not satisfied symbol by symbol transmission is in some cases the best known strategy in the nonasymptotic regime

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

discrete lossy gray wyner revisited second order asymptotics large and moderate deviations in this paper we revisit the discrete lossy gray wyner problem in particular we derive its optimal second order coding rate region its error exponent reliability function and its moderate deviations constant under mild conditions on the source to obtain the second order asymptotics we extend some ideas from watanabe s work in particular we leverage the properties of an appropriate generalization of the conditional distortion tilted information density which was first introduced by kostina and verdu the converse part uses a perturbation argument by gu and effros in their strong converse proof of the discrete gray wyner problem the achievability part uses two novel elements 1 a generalization of various type covering lemmas and 2 the uniform continuity of the conditional rate distortion function in both the source joint distribution and the distortion level to obtain the error exponent for the achievability part we use the same generalized type covering lemma and for the converse we use the strong converse together with a change of measure technique finally to obtain the moderate deviations constant we apply the moderate deviations theorem to probabilities defined in terms of information spectrum quantities

a nonsmooth lyapunov function and stability for ode s of caratheodory type abstract the problem of lyapunov stability for systems of ode s of caratheodory type is considered it is proved that without the necessity of calculating dini or clarke generalized gradients the locally lipschitz lyapunov function can follow different kinds of stability

strong successive refinability and rate distortion complexity tradeoff we investigate the second order asymptotics source dispersion of the successive refinement problem similar to the classical definition of a successively refinable source we say that a source is strongly successively refinable if successive refinement coding can achieve the second order optimum rate including the dispersion terms at both decoders we establish a sufficient condition for strong successive refinability we show that any discrete source under hamming distortion and the gaussian source under quadratic distortion are strongly successively refinable we also demonstrate how successive refinement ideas can be used in point to point lossy compression problems in order to reduce complexity we give two examples the binary hamming and gaussian quadratic cases in which a layered code construction results in a low complexity scheme that attains optimal performance for example when the number of layers grows with the block length n we show how to design an o n log n algorithm that asymptotically achieves the rate distortion bound

local irreducibility of tail biting trellises this paper investigates tail biting trellis realizations for linear block codes intrinsic trellis properties are used to characterize irreducibility on given intervals of the time axis it proves beneficial to always consider the trellis and its dual simultaneously a major role is played by trellis properties that amount to observability and controllability of trellis fragments of various lengths for fragments of length less than the minimum span length of the code it is shown that fragment observability and fragment controllability are equivalent to irreducibility for reducible trellises a constructive reduction procedure is presented the considerations also lead to a characterization for when the dual of a trellis allows a product factorization into elementary atomic trellises

smoothing brascamp lieb inequalities and strong converses of coding theorems the brascamp lieb inequality in functional analysis can be viewed as a measure of the uncorrelatedness of a joint probability distribution we define the smooth brascamp lieb bl divergence as the infimum of the best constant in the brascamp lieb inequality under a perturbation of the joint probability distribution an information spectrum upper bound on the smooth bl divergence is proved using properties of the subgradient of a certain convex functional in particular in the i i d setting such an infimum converges to the best constant in a certain mutual information inequality we then derive new single shot converse bounds for the omniscient helper common randomness generation problem and the gray wyner source coding problem in terms of the smooth bl divergence where the proof relies on the functional formulation of the brascamp lieb inequality exact second order rates are thus obtained in the stationary memoryless and nonvanishing error setting these offer rare instances of strong converses second order converses for continuous sources when the rate region involves auxiliary random variables

on the dispersions of three network information theory problems we analyze the dispersions of distributed lossless source coding the slepian wolf problem the multiple access channel and the asymmetric broadcast channel for the two encoder slepian wolf problem we introduce a quantity known as the entropy dispersion matrix which is analogous to the scalar dispersions that have gained interest recently we prove a global dispersion result that can be expressed in terms of this entropy dispersion matrix and provides intuition on the approximate rate losses at a given blocklength and error probability to gain better intuition about the rate at which the nonasymptotic rate region converges to the slepian wolf boundary we define and characterize two operational dispersions 1 the local dispersion and 2 the weighted sum rate dispersion the former represents the rate of convergence to a point on the slepian wolf boundary whereas the latter represents the fastest rate for which a weighted sum of the two rates converges to its asymptotic fundamental limit interestingly when we approach either of the two corner points the local dispersion is characterized not by a univariate gaussian but a bivariate one as well as a subset of off diagonal elements of the aforementioned entropy dispersion matrix finally we demonstrate the versatility of our achievability proof technique by providing inner bounds for the multiple access channel and the asymmetric broadcast channel in terms of dispersion matrices all our proofs are unified by a so called vector rate redundancy theorem which is proved using the multidimensional berry esseen theorem

secret key agreement general capacity and second order asymptotics we revisit the problem of secret key agreement using interactive public communication for two parties and propose a new secret key agreement protocol the protocol attains the secret key capacity for general observations and attains the second order asymptotic term in the maximum length of a secret key for independent and identically distributed observations in contrast to the previously suggested secret key agreement protocols the proposed protocol uses interactive communication in fact the standard one way communication protocol used prior to this paper fails to attain the asymptotic results above our converse proofs rely on a recently established upper bound for secret key lengths both our lower and upper bounds are derived in a single shot setup and the asymptotic results are obtained as corollaries

second order region for gray wyner network the coding problem over the gray wyner network is studied from the second order coding rates perspective a tilted information density for this network is introduced in the spirit of kostina verdu and under a certain regularity condition the second order region is characterized in terms of the variance of this tilted information density and the tangent vector of the first order region the second order region is proved by the type method the achievability part is proved by the type covering argument and the converse part is proved by a refinement of the perturbation approach that was used by gu effros to show the strong converse of the gray wyner network this is the first instance that the second order region is characterized for a multi terminal problem where the characterization of the first order region involves an auxiliary random variable

second order rate region of constant composition codes for the multiple access channel this paper studies the second order asymptotics of coding rates for the discrete memoryless multiple access channel mac with a fixed target error probability using constant composition random coding coded time sharing and a variant of hoeffding s combinatorial central limit theorem an inner bound on the set of locally achievable second order coding rates is given for each point on the boundary of the capacity region it is shown that the inner bound for constant composition random coding includes that recovered by independent identically distributed random coding and that the inclusion may be strict the inner bound is extended to the gaussian mac via an increasingly fine quantization of the inputs

lasalle yoshizawa corollaries for nonsmooth systems in this technical note two generalized corollaries to the lasalle yoshizawa theorem are presented for nonautonomous systems described by nonlinear differential equations with discontinuous right hand sides lyapunov based analysis methods that achieve asymptotic convergence when the candidate lyapunov derivative is upper bounded by a negative semi definite function in the presence of differential inclusions are presented a design example illustrates the utility of the corollaries

fixed length lossy compression in the finite blocklength regime this paper studies the minimum achievable source coding rate as a function of blocklength n and probability that the distortion exceeds a given level d tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength for stationary memoryless sources with separable distortion the minimum rate achievable is shown to be closely approximated by r d v d n q 1 where r d is the rate distortion function v d is the rate dispersion a characteristic of the source which measures its stochastic variability and q 1 is the inverse of the standard gaussian complementary cumulative distribution function

minimal realizations of linear systems the shortest basis approach given a discrete time linear system c a shortest basis for c is a set of linearly independent generators for c with the least possible lengths a basis b is a shortest basis if and only if it has the predictable span property i e has the predictable delay and degree properties and is non catastrophic or alternatively if and only if it has the subsystem basis property for any interval j the generators in b whose span is in j is a basis for the subsystem cj the dimensions of the minimal state spaces and minimal transition spaces of c are simply the numbers of generators in a shortest basis b that are active at any given state or symbol time respectively a minimal linear realization for c in controller canonical form follows directly from a shortest basis for c and a minimal linear realization for c in observer canonical form follows directly from a shortest basis for the orthogonal system c this approach seems conceptually simpler than that of classical minimal realization theory

generalized lyapunov approach for functional differential inclusions abstract this paper is concerned with the generalized lyapunov approach for functional differential inclusions fdi at first we prove a class of generalized halanay s inequalities then by applying the generalization of halanay s inequalities we investigate the uniformly ultimate boundedness and uniformly asymptotic stability of fdi

generalized lyapunov method for discontinuous systems abstract nonlinear dynamical systems described by differential equations with discontinuous right hand side and solutions intended in filippov sense are considered based on filippov differential inclusion and a new chain rule for differentiating regular functions along filippov solution trajectories different kinds of stability and convergence results are presented moreover we investigate the stability and convergence for the corresponding perturbation system of the discontinuous system and some new criteria are addressed

codes on graphs constraint complexity of cycle free realizations of linear codes cycle free graphical realizations of linear codes generalize trellis realizations given a linear code c and a cycle free graph topology there exists a well defined minimal realization for c on that graph in which each constraint is a linear code with a well defined length and dimension the constraint complexity of the realization is defined as maximum dimension of any constraint code there exists a graph that minimizes constraint complexity in which all internal nodes have degree 3 and all interface nodes have degree 2 and which moreover can be put in the form of a tree structured trellis realization the constraint complexity of a general cycle free graph realization can be less than that of any conventional trellis realization but not by very much such realizations can yield reductions in decoding complexity even when they do not reduce constraint complexity

codes on graphs duality and macwilliams identities a conceptual framework involving partition functions of normal factor graphs is introduced paralleling a similar recent development by al bashabsheh and mao the partition functions of dual normal factor graphs are shown to be a fourier transform pair whether or not the graphs have cycles the original normal graph duality theorem follows as a corollary within this framework macwilliams identities are found for various local and global weight generating functions of general group or linear codes on graphs this generalizes and provides a concise proof of the macwilliams identity for linear time invariant convolutional codes that was recently found by gluesing luerssen and schneider further macwilliams identities are developed for terminated convolutional codes particularly for tail biting codes similar to those studied recently by bocharova hug johannesson and kudryashov

smooth lyapunov functions and robustness of stability for difference inclusions we demonstrate the equivalence of robust global asymptotic stability gas of the origin and the existence of a smooth lyapunov function for difference inclusions defined by upper semicontinuous set valued maps sufficient conditions for robust gas are given as an application of these results we give conditions for robust gas of difference equations defined by discontinuous right hand sides

wyner s common information under renyi divergence measures we study a generalized version of wyner s common information problem also coined the distributed source simulation problem the original common information problem consists in understanding the minimum rate of the common input to independent processors to generate an approximation of a joint distribution when the distance measure used to quantify the discrepancy between the synthesized and target distributions is the normalized relative entropy our generalization involves changing the distance measure to the unnormalized and normalized renyi divergences of order alpha 1 s in 0 2 we show that the minimum rate needed to ensure the renyi divergences between the distribution induced by a code and the target distribution vanishes remains the same as the one in wyner s setting except when the order alpha 1 s 0 this implies that wyner s common information is rather robust to the choice of distance measure employed as a byproduct of the proofs used to the establish the above results the exponential strong converse for the common information problem under the total variation distance measure is established

the dynamics of group codes state spaces trellis diagrams and canonical encoders a group code c over a group g is a set of sequences of group elements that itself forms a group under a component wise group operation a group code has a well defined state space sigma sub k at each time k each code sequence passes through a well defined state sequence the set of all state sequences is also a group code the state code of c the state code defines an essentially unique minimal realization of c the trellis diagram of c is defined by the state code of c and by labels associated with each state transition the set of all label sequences forms a group code the label code of c which is isomorphic to the state code of c if c is complete and strongly controllable then a minimal encoder in controller canonical feedbackfree form may be constructed from certain sets of shortest possible code sequences called granules the size of the state space sigma sub k is equal to the size of the state space of this canonical encoder which is given by a decomposition of the input groups of c at each time k if c is time invariant and nu controllable then mod sigma sub k mod pi sub 1

channels with cost constraints strong converse and dispersion this paper shows the strong converse and the dispersion of memoryless channels with cost constraints and performs a refined analysis of the third order term in the asymptotic expansion of the maximum achievable channel coding rate showing that it is equal to 1 2 log n n in most cases of interest the analysis is based on a nonasymptotic converse bound expressed in terms of the distribution of a random variable termed the mathsf b tilted information density which plays a role similar to that of the mathsf d tilted information in lossy source coding we also analyze the fundamental limits of lossy joint source channel coding over channels with cost constraints

non asymptotic converse bounds and refined asymptotics for two source coding problems in this paper we revisit two multi terminal lossy source coding problems the lossy source coding problem with side information available at the encoder and one of the two decoders which we term as the kaspi problem kaspi 1994 and the multiple description coding problem with one semi deterministic distortion measure which we refer to as the fu yeung problem fu and yeung 2002 for the kaspi problem we first present the properties of optimal test channels subsequently we generalize the notion of the distortion tilted information density for the lossy source coding problem to the kaspi problem and prove a non asymptotic converse bound using the properties of optimal test channels and the well defined distortion tilted information density finally for discrete memoryless sources we derive refined asymptotics which includes the second order large and moderate deviations asymptotics in the converse proof of second order asymptotics we apply the berry esseen theorem to the derived non asymptotic converse bound the achievability proof follows by first proving a type covering lemma tailored to the kaspi problem then properly taylor expanding the well defined distortion tilted information densities and finally applying the berry esseen theorem we then generalize the methods used in the kaspi problem to the fu yeung problem as a result we obtain the properties of optimal test channels for the minimum sum rate function a non asymptotic converse bound and refined asymptotics for discrete memoryless sources since the successive refinement problem is a special case of the fu yeung problem as a by product we obtain a non asymptotic converse bound for the successive refinement problem which is a strict generalization of the non asymptotic converse bound for successively refinable sources zhou tan and motani 2017

generalized lyapunov approach for convergence of neural networks with discontinuous or non lipschitz activations the paper considers a class of additive neural networks where the neuron activations are modeled by discontinuous functions or by continuous non lipschitz functions some tools are developed which enable us to apply a lyapunov like approach to differential equations with discontinuous right hand side modeling the neural network dynamics the tools include a chain rule for computing the time derivative along the neural network solutions of a nondifferentiable lyapunov function and a comparison principle for this time derivative which yields conditions for exponential convergence or convergence in finite time by means of the lyapunov like approach a general result is proved on global exponential convergence toward a unique equilibrium point of the neural network solutions moreover new results on global convergence in finite time are established which are applicable to neuron activations with jump discontinuities or neuron activations modeled by means of continuous non lipschitz holder functions

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

a case where interference does not affect the channel dispersion in 1975 carleial presented a special case of an interference channel called the very strong interference regime in which the interference does not reduce the capacity of the constituent point to point gaussian channels in this paper we show that in the strictly very strong interference regime the dispersions are similarly unaffected more precisely in this paper we characterize the second order coding rates of the gaussian interference channel in the strictly very strong interference regime in other words we characterize the speed of convergence of rates of optimal block codes toward a boundary point of the rectangular capacity region these second order coding rates are expressed in terms of the average probability of error and variances of appropriately defined information densities which coincide with the dispersion of the single user gaussian channel this allows us to conclude that the dispersions are unaffected by interference in this channel model

a rate of convergence result for a universal d semifaithful code the problem of optimal rate universal coding is considered in the context of rate distortion theory a d semifaithful universal coding scheme for discrete memoryless sources is given the main result is a refined covering lemma based on the random coding argument and the method of types the average codelength of the code is shown to approach its lower bound the rate distortion function at a rate o n sup 1 log n and this is conjectured to be optimal based on a result of a j pilc 1968 issues of constructiveness and universality are addressed

asymptotic stability and smooth lyapunov functions abstract we establish that differential inclusions corresponding to upper semicontinuous multifunctions are strongly asymptotically stable if and only if there exists a smooth lyapunov function since well known concepts of generalized solutions of differential equations with discontinuous right hand side can be described in terms of solutions of certain related differential inclusions involving upper semicontinuous multifunctions this result gives a lyapunov characterization of asymptotic stability of either filippov or krasovskii solutions for differential equations with discontinuous right hand side in the study of weak as opposed to strong asymptotic stability the existence of a smooth lyapunov function is rather exceptional however the methods employed in treating the strong case of asymptotic stability are applied to yield a necessary condition for the existence of a smooth lyapunov function for weakly asymptotically stable differential inclusions this is an extension to the context of lyapunov functons of brockett s celebrated covering condition from continuous feedback stabilization theory

stability theorems for delay differential inclusions this technical note addresses stability problems of delay differential inclusions a chain rule is proposed for a wide class of lyapunov krasovskii functionals which are not necessarily invariantly differentiable we also obtain an invariance like theorem where the derivative of the candidate functional is bounded above by a continuous negative semidefinite function several examples are given to show the effectiveness of the results

critical behavior in lossy source coding the following critical phenomenon was recently discovered when a memoryless source is compressed using a variable length fixed distortion code the fastest convergence rate of the pointwise compression ratio to r d is either o spl radic n or o log n we show it is always o spl radic n except for discrete uniformly distributed sources

second order asymptotics in fixed length source coding and intrinsic randomness there is a difference between the optimal rates of fixed length source coding and intrinsic randomness when we care about the second order asymptotics we prove this difference for general information sources and then investigate independent and identically distributed i i d random variables and markovian variables as examples the difference is demonstrated through an investigation of the second order asymptotic behavior of the rates a universal fixed length source code attaining the second order optimal rate is also proposed the difference between the rates of fixed length source coding and intrinsic randomness proves that the outputs of fixed length source codes are not uniformly distributed

strong converse using change of measure arguments the strong converse for a coding theorem shows that the optimal asymptotic rate possible with vanishing error cannot be improved by allowing a fixed error building on a method introduced by gu and effros for centralized coding problems we develop a general and simple recipe for proving strong converse that is applicable for distributed problems as well heuristically our proof of strong converse mimics the standard steps for proving a weak converse except that we apply those steps to a modified distribution obtained by conditioning the original distribution on the event that no error occurs a key component of our recipe is the replacement of the hard markov constraints implied by the distributed nature of the problem with a soft information cost using a variational formula introduced by oohama we illustrate our method by providing a short proof of the strong converse for the wyner ziv problem and strong converse theorems for interactive function computation common randomness and secret key agreement and the wiretap channel the latter three strong converse problems were open prior to this work

semiconcave control lyapunov functions and stabilizing feedbacks we study the general problem of stabilization of globally asymptotically controllable systems we construct discontinuous feedback laws and particularly we make it possible to choose these continuous outside a small set closed with measure zero of discontinuity in the case of control systems which are affine in the control moreover this set of singularities is shown to be repulsive for the caratheodory solutions of the closed loop system under an additional assumption

linear tail biting trellises characteristic generators and the bcjr construction this paper investigates the constructions of tail biting trellises for linear block codes as introduced by koetter and vardy 2003 and nori and shankar 2006 for a given code the sets of characteristic generators are defined slightly more generally than by koetter and vardy in particular they are not uniquely determined by the code the effect of the choice of characteristic generators on the resulting product trellises called kv trellises is discussed in detail it is shown that each kv trellis is a span based bcjr trellis and that the latter are always nonmergeable finally a duality conjecture posed by koetter and vardy is addressed by making use of a dualization technique of bcjr trellises the conjecture is proven for minimal trellises

a new asymptotic stability criterion for nonlinear time variant differential equations a new sufficient condition for asymptotic stability of ordinary differential equations is proposed unlike classical lyapunov theory the time derivative along solutions of the lyapunov function may have positive and negative values the classical lyapunov approach may be regarded as an infinitesimal version of the present theorem verification in practical problems is harder than in the classical case an example is included in order to indicate how the present theorem may be applied

asymptotic stability of nonautonomous systems by liapunov s direct method abstract this paper discusses the asymptotic stability of nonautonomous systems by means of the direct method of liapunov we assume existence of a positive time invariant liapunov function with negative semi definite derivative we focus on the extra conditions needed in order to guarantee asymptotic stability and present a theorem which offers a new criterion for asymptotic stability the theorem is illustrated first by recovering barbashin s principle for autonomous and periodic systems and second by studying the nonlinear pendulum with time varying damping for which we derive a new asymptotic stability result

information spectrum approach to second order coding rate in channel coding in this paper second order coding rate of channel coding is discussed for general sequence of channels the optimum second order transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method we apply this result to the discrete memoryless case the discrete memoryless case with a cost constraint the additive markovian case and the gaussian channel case with an energy constraint we also clarify that the gallager bound does not give the optimum evaluation in the second order coding rate

lyapunov stability theory of nonsmooth systems this paper develops nonsmooth lyapunov stability theory and lasalle s invariance principle for a class of nonsmooth lipschitz continuous lyapunov functions and absolutely continuous state trajectories computable tests based on filipov s differential inclusion and clarke s generalized gradient are derived the primary use of these results is in analyzing the stability of equilibria of differential equations with discontinuous right hand side such as in nonsmooth dynamic systems or variable structure control

codes on graphs observability controllability and local reducibility this paper investigates properties of realizations of linear or group codes on general graphs that lead to local reducibility trimness and properness are dual properties of constraint codes a linear or group realization with a constraint code that is not both trim and proper is locally reducible a linear or group realization on a finite cycle free graph is minimal if and only if every local constraint code is trim and proper a realization is called observable if there is a one to one correspondence between codewords and configurations and controllable if it has independent constraints a linear or group realization is observable if and only if its dual is controllable a simple counting test for controllability is given an unobservable or uncontrollable realization is locally reducible parity check realizations are controllable if and only if they have independent parity checks in an uncontrollable tail biting trellis realization the behavior partitions into disconnected sub behaviors but this property does not hold for nontrellis realizations on a general graph the support of an unobservable configuration is a generalized cycle

the redundancy of source coding with a fidelity criterion 1 known statistics the problem of redundancy of source coding with respect to a fidelity criterion is considered for any fixed rate r 0 and any memoryless source with finite source and reproduction alphabets and a common distribution p the nth order distortion redundancy d sub n r of fixed rate coding is defined as the minimum of the difference between the expected distortion per symbol of any block code with length n and rate r and the distortion rate function d p r of the source p it is demonstrated that for sufficiently large n d sub n r is equal to spl part spl part r d p r ln n 2n o ln n n where spl part spl part r d p r is the partial derivative of d p r evaluated at r and assumed to exist for any fixed distortion level d 0 and any memoryless source p the nth order rate redundancy r sub n d of coding at fixed distortion level d or by using d semifaithful codes is defined as the minimum of the difference between the expected rate per symbol of any d semifaithful code of length n and the rate distortion function r p d of p evaluated at d it is proved that for sufficiently large n r sub n d is upper bounded by ln n n o ln n n and lower bounded by in n 2n o in n n as a by product the lower bound of r sub n d derived in this paper gives a positive answer to a conjecture proposed by yu and speed 1993