design of capacity approaching irregular low density parity check codes we design low density parity check ldpc codes that perform at rates extremely close to the shannon capacity the codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides our theoretical analysis of the codes is based on the work of richardson and urbanke see ibid vol 47 no 2 p 599 618 2000 assuming that the underlying communication channel is symmetric we prove that the probability densities at the message nodes of the graph possess a certain symmetry using this symmetry property we then show that under the assumption of no cycles the message densities always converge as the number of iterations tends to infinity furthermore we prove a stability condition which implies an upper bound on the fraction of errors that a belief propagation decoder can correct when applied to a code induced from a bipartite graph with a given degree distribution our codes are found by optimizing the degree structure of the underlying graphs we develop several strategies to perform this optimization we also present some simulation results for the codes found which show that the performance of the codes is very close to the asymptotic theoretical bounds

on improved bounds on the decoding error probability of block codes over interleaved fading channels with applications to turbo like codes we derive here improved upper bounds on the decoding error probability of block codes which are transmitted over fully interleaved rician fading channels coherently detected and maximum likelihood ml decoded we assume that the fading coefficients during each symbol are statistically independent due to a perfect channel interleaver and that perfect estimates of these fading coefficients are provided to the receiver the improved upper bounds on the block and bit error probabilities are derived for fully interleaved fading channels with various orders of space diversity and are found by generalizing some previously introduced upper bounds for the binary input additive white gaussian nose awgn channel the advantage of these bounds over the ubiquitous union bound is demonstrated for some ensembles of turbo codes and low density parity check ldpc codes and it is especially pronounced in a portion of the rate region exceeding the cutoff rate our generalization of the duman and salehi bound duman and salehi 1998 duman 1998 which is based on certain variations of gallager s 1965 bounding technique is demonstrated to be the tightest reported upper bound we therefore apply it to calculate numerically upper bounds on the thresholds of some ensembles of turbo like codes referring to the optimal ml decoding for certain ensembles of uniformly interleaved turbo codes the upper bounds derived here also indicate good match with computer simulation results of efficient iterative decoding algorithms

improved dynamic dictionary matching in the dynamic dictionary matching problem a dictionary d contains a set of patterns that can change over time by insertion and deletion of individual patterns the user also presents text strings and asks for all occurrences of any patterns in the text the two main contributions of this paper are 1 a faster algorithm for dynamic string dictionary matching with bounded alphabets and 2 a dynamic dictionary matching algorithm for two dimensional texts and patterns the first contribution is based on an algorithm that solves the general problem of maintaining a sequence of well balanced parentheses under the operations insert delete and find nearest enclosing parenthesis pair the main new idea behind the second contribution is a novel method to efficiently manipulate failure links for two dimensional patterns

reversible low density parity check codes low density parity check ldpc codes may be decoded using a circuit implementation of the sum product algorithm which maps the factor graph of the code by reusing the decoder for encoding both tasks can be performed using the same circuit thus reducing area and verification requirements motivated by this iterative encoding techniques based upon the graphical representation of the code are proposed code design constraints which ensure encoder convergence are presented and then used to design iteratively encodable codes while also preventing 4 cycle creation we show how the jacobi method for iterative matrix inversion can be applied to finite field matrices viewed as message passing and employed as the core of an iterative encoder we present an algebraic construction of 4 cycle free iteratively encodable codes using circulant matrices analysis of these codes identifies a weakness in their structure due to a repetitive pattern in the factor graph the graph supports pseudo codewords of low pseudo weight in order to remove the repetitive pattern in the graph we propose a recursive technique for generating iteratively encodable codes the new codes offer flexibility in the choice of code length and rate and performance that compares well to randomly generated quasi cyclic and extended euclidean geometry codes

performance bounds for erasure list and decision feedback schemes with linear block codes a message independence property and some new performance upper bounds are derived in this work for erasure list and decision feedback schemes with linear block codes transmitted over memoryless symmetric channels similar to the classical work of forney this work is focused on the derivation of some gallager type bounds on the achievable tradeoffs for these coding schemes where the main novelty is the suitability of the bounds for both random and structured linear block codes or code ensembles the bounds are applicable to finite length codes and to the asymptotic case of infinite block length and they are applied to low density parity check code ensembles

efficient erasure correcting codes we introduce a simple erasure recovery algorithm for codes derived from cascades of sparse bipartite graphs and analyze the algorithm by analyzing a corresponding discrete time random process as a result we obtain a simple criterion involving the fractions of nodes of different degrees on both sides of the graph which is necessary and sufficient for the decoding process to finish successfully with high probability by carefully designing these graphs we can construct for any given rate r and any given real number spl epsiv a family of linear codes of rate r which can be encoded in time proportional to ln 1 spl epsiv times their block length n furthermore a codeword can be recovered with high probability from a portion of its entries of length 1 spl epsiv rn or more the recovery algorithm also runs in time proportional to n ln 1 spl epsiv our algorithms have been implemented and work well in practice various implementation issues are discussed

protograph based raptor like ldpc codes this paper proposes protograph based raptor like pbrl codes as a class of rate compatible low density parity check codes for binary input awgn channels as with the raptor codes exclusive or operations on precoded bits produce additional parity bits providing extensive rate compatibility unlike raptor codes each additional parity bit in the protograph is explicitly designed to optimize the density evolution threshold during the lifting process approximate cycle extrinsic message degree ace and circulant progressive edge growth cpeg constraints are used to avoid undesirable graphical structures some density evolution performance is sacrificed to obtain lower error floors particularly at short blocklengths simulation results are shown for information block sizes of k 1032 hbox and 16 384 for a target frame error rate of 10 5 at each rate the k 1032 hbox and 16 384 code families perform within 1 db and 0 4 db of both the gallager bound and the normal approximation respectively the 16 384 code family outperforms the best known standardized code family namely the ar4ja codes the pbrl codes also outperform dvb s2 codes that have the advantages of longer blocklengths and outer bch codes performance is similar to rc code families designed by nguyen et al that do not constrain codes to have the pbrl structure and involve simulation in the optimization process at each rate

average stopping set weight distributions of redundant random ensembles in this paper redundant random ensembles are defined and their average stopping set ss weight distributions are analyzed a redundant random ensemble consists of a set of binary matrices with linearly dependent rows these linearly dependent rows significantly reduce the number of stopping sets ss of small size upper and lower bounds on the average ss weight distribution of the redundant random ensembles are proved based on a combinatorial argument asymptotic forms of these bounds reveal asymptotic behavior of the average ss weight distributions from these bounds a tradeoff between the number of redundant rows corresponding to decoding complexity of belief propagation on binary erasure channel and the average ss weight distribution corresponding to decoding performance can be derived

on ensembles of low density parity check codes asymptotic distance distributions we derive expressions for the average distance distributions in several ensembles of regular low density parity check codes ldpc among these ensembles are the standard one defined by matrices having given column and row sums ensembles defined by matrices with given column sums or given row sums and an ensemble defined by bipartite graphs

approximately lower triangular ensembles of ldpc codes with linear encoding complexity the complexity of brute force encoding of low density parity check ldpc codes is proportional to the square value of the block length richardson and urbanke have proposed efficient encoding algorithms for ldpc codes these algorithms permute the parity check matrix of the code iteratively such that it becomes approximately lower triangular we propose a new approach for efficient encoding of ldpc codes in which we modify the code ensemble to force an approximate lower triangular structure thus eliminating the need to apply the algorithms of richardson and urbanke in this ensemble we prove that the new ensemble has the same asymptotic threshold as the corresponding standard ensemble the new ensemble can be used for linear time encoding of an arbitrary code profile computer simulations confirm that the performances of the standard and new ensembles are also very similar when using finite length codes

a recursive approach to low complexity codes a method is described for constructing long error correcting codes from one or more shorter error correcting codes referred to as subcodes and a bipartite graph a graph is shown which specifies carefully chosen subsets of the digits of the new codes that must be codewords in one of the shorter subcodes lower bounds to the rate and the minimum distance of the new code are derived in terms of the parameters of the graph and the subeodes both the encoders and decoders proposed are shown to take advantage of the code s explicit decomposition into subcodes to decompose and simplify the associated computational processes bounds on the performance of two specific decoding algorithms are established and the asymptotic growth of the complexity of decoding for two types of codes and decoders is analyzed the proposed decoders are able to make effective use of probabilistic information supplied by the channel receiver e g reliability information without greatly increasing the number of computations required it is shown that choosing a transmission order for the digits that is appropriate for the graph and the subcodes can give the code excellent burst error correction abilities the construction principles

weight distribution of low density parity check codes we derive the average weight distribution function and its asymptotic growth rate for low density parity check ldpc code ensembles we show that the growth rate of the minimum distance of ldpc codes depends only on the degree distribution pair it turns out that capacity achieving sequences of standard unstructured ldpc codes under iterative decoding over the binary erasure channel bec known to date have sublinearly growing minimum distance in the block length

bounds on the decoding error probability of binary linear codes via their spectra bounds on the error probability of maximum likelihood decoding of a binary linear code are considered the bounds derived use the weight spectrum of the code and they are tighter than the conventional union bound in the case of large noise in the channel the bounds derived are applied to a code with an average spectrum and the result is compared to the random coding exponent the author shows that the bound considered for the binary symmetrical channel case coincides asymptotically with the random coding bound for the case of awgn channel the author shows that berlekamp s 1980 tangential bound can be improved but even this improved bound does not coincide with the random coding bound although it can be very close to it

efficient encoding of low density parity check codes low density parity check ldpc codes can be considered serious competitors to turbo codes in terms of performance and complexity and they are based on a similar philosophy constrained random code ensembles and iterative decoding algorithms we consider the encoding problem for ldpc codes more generally we consider the encoding problem for codes specified by sparse parity check matrices we show how to exploit the sparseness of the parity check matrix to obtain efficient encoders for the 3 6 regular ldpc code for example the complexity of encoding is essentially quadratic in the block length however we show that the associated coefficient can be made quite small so that encoding codes even of length n spl sime 100000 is still quite practical more importantly we show that optimized codes actually admit linear time encoding

distance distributions in ensembles of irregular low density parity check codes we derive asymptotic expressions for the average distance distributions in ensembles of irregular low density parity check ldpc codes the ensembles are defined by matrices with given profiles of column and row sums

expander codes using expander graphs we construct a new family of asymptotically good linear error correcting codes these codes have linear time sequential decoding algorithms and logarithmic time parallel decoding algorithms that use a linear number of processors we present both randomized and explicit constructions of these codes experimental results demonstrate the good performance of the randomly chosen codes

results on the improved decoding algorithm for low density parity check codes over the binary erasure channel in this correspondence we first investigate some analytical aspects of the recently proposed improved decoding algorithm for low density parity check ldpc codes over the binary erasure channel bec we derive a necessary and sufficient condition for the improved decoding algorithm to successfully complete decoding when the decoder is initialized to guess a predetermined number of guesses after the standard message passing terminates at a stopping set furthermore we present improved bounds on the number of bits to be guessed for successful completion of the decoding process when a stopping set is encountered under suitable conditions we derive a lower bound on the number of iterations to be performed for complete decoding of the stopping set we then present a superior novel improved decoding algorithm for ldpc codes over the binary erasure channel bec the proposed algorithm combines the observation that a considerable fraction of unsatisfied check nodes in the neighborhood of a stopping set are of degree two and the concept of guessing bits to perform simple and intuitive graph theoretic manipulations on the tanner graph the proposed decoding algorithm has a complexity similar to previous improved decoding algorithms finally we present simulation results of short length codes over bec that demonstrate the superiority of our algorithm over previous improved decoding algorithms for a wide range of bit error rates

optimal off line detection of repetitions in a string an algorithm is presented to detect within optimal time o n log n and space o n off line on a ram all of the distinct repetitions in a given textstring on a finite alphabet the proposed strategy is self contained as it depends more heavily on algorithmic design considerations than on the combinatorial properties of the output it is based on a new data structure the leaf tree which is particularly suited to exploit simple properties of the suffix tree associated with the string to be analyzed

on the error correction of regular ldpc codes using the flipping algorithm the iterative bit flipping algorithm is applied to the standard regular low density parity check ldpc code ensemble in the past it was shown for a typical code in the ensemble with left degree at least five and block length sufficiently large that this algorithm can correct a linear in the block length number of worst case errors in this paper this result is extended to the case where the left degree is at least four for the case where the left degree is larger than four an improvement compared to existing results of several orders of magnitude is obtained on the fraction of worst case errors that can be corrected it is also shown how the results can be further improved when random errors produced by the channel as opposed to worst case errors are considered

asymptotic enumeration methods for analyzing ldpc codes we show how asymptotic estimates of powers of polynomials with nonnegative coefficients can be used in the analysis of low density parity check ldpc codes in particular we show how these estimates can be used to derive the asymptotic distance spectrum of both regular and irregular ldpc code ensembles we then consider the binary erasure channel bec using these estimates we derive lower bounds on the error exponent under iterative decoding of ldpc codes used over the bec both regular and irregular code structures are considered these bounds are compared to the corresponding bounds when optimal maximum likelihood ml decoding is applied

enumerators for protograph based ensembles of ldpc and generalized ldpc codes protograph based ldpc and generalized ldpc g ldpc codes have the advantages of a simple design procedure and highly structured encoders and decoders the design of such protograph based codes relies on what is effectively a computer based search as such following gallager it is prudent to restrict the search to a good ensemble for example an ensemble whose minimum distance grows linearly with codeword length a good ensemble can also mean one with good stopping set trapping set or pseudocodeword properties in this paper ensemble codeword weight enumerators for finite length ldpc and g ldpc codes based on protographs were derived and then the asymptotic case was considered the asymptotic results allow us to determine whether or not the typical relative minimum distance in the ensemble grows linearly with codeword length then the codeword weight enumerator technique is adapted to yield ensemble stopping set trapping set and pseudocodeword enumerators for protograph ldpc and g ldpc codes in this case the asymptotic results allow us to determine whether or not the typical relative smallest stopping set size trapping set size and pseudoweight grows linearly with codeword length trapping set enumerators for g ldpc code ensembles represent a more complex problem which we do not consider here

the ml decoding performance of ldpc ensembles over z sub q we derive the asymptotic spectra of low density parity check ldpc ensembles over z sub q we consider two ensembles of ldpc matrices one is binary and the other q ary we also show that for modulo additive noise channels both ensembles achieve the random coding error exponent for graphs with sufficiently large connectivity

the approximate maximum likelihood certificate the confidence in the reliability of a codeword output by some not necessarily optimal decoding algorithm is discussed a new property which relies on the linear programming lp decoder the approximate maximum likelihood certificate amlc is introduced to address this issue as follows first the channel output vector is decoded by some symmetric decoder d e g belief propagation or min sum algorithm decoding second the channel output vector is decoded by lp decoding third if the decoding result of d is a codeword its lp value is compared to the lp value of the lp decoding result the latter need not be a codeword if these two values are close the amlc holds using upper bounding techniques we show that the conditional frame error probability given that the amlc holds is with some degree of confidence below a threshold in channels with low noise this threshold is orders of magnitude lower than the simulated frame error rate and our bound holds with a very high degree of confidence this is in stark contrast with standard monte carlo simulation which would require excessively long runs to demonstrate like performance when the amlc holds our approach thus provides the decoder with extra error detection capability which is especially important in applications requiring high data integrity

on finding bipartite graphs with a small number of short cycles and large girth the problem of finding bipartite tanner graphs with given degree sequences that have large girth and few short cycles is of great interest in many applications including construction of good low density parity check ldpc codes in this paper we prove that for given integers alpha beta and gamma and degree sequences pi and pi the problem of determining whether there exists a simple bipartite graph with degree sequences pi pi that has at most alpha beta and gamma cycles of length four six and eight respectively is np complete this is proved by a two step polynomial time reduction from the 3 partition problem on the other hand using connections to linear hypergraphs we prove that given the degree sequence pi a polynomial time algorithm can be devised to determine whether there exists a bipartite graph whose degree sequence on one side of the bipartition is pi and has a girth of at least six in addition to these complexity results we devise a quasi polynomial time algorithm that can construct a bipartite graph with given irregular degree sequences and a girth that increases logarithmically with the size of the graph compared to the well known progressive edge growth peg algorithm the proposed method in some cases results in tanner graphs with larger girth particularly for scenarios where the degree sequences of the graph are strictly enforced

constructing short length irregular ldpc codes with low error floor trapping sets tss are known to cause error floors in regular and irregular low density parity check ldpc codes by avoiding major error contributing tss during the code construction process codes with low error floors can effectively be built in it has been shown that tss labeled as w u are considered as being equivalent under the automorphism of the graph and are therefore contributing equally to the error floor however tss with the same label w u are not identical in general particularly for the case of irregular ldpc codes in this paper we introduce a parameter e that can identify the number of distinguishable cycles in the connected subgraph induced by an elementary trapping set further we propose a code construction algorithm namely the progressive edge growth approximate minimum cycle set extrinsic message degree peg acse method that aims to avoid small elementary trapping sets etss particularly detrimental etss we also develop theorems evaluating the minimum possible etss formed by peg construction algorithms in general we compare the characteristics of the codes built using the proposed method and those built using peg only or peg approximate minimum cycle extrinsic message degree peg ace methods results from simulations show that the codes constructed using the proposed peg acse method produce lower error rates particularly at the high signal to noise snr region compared with codes constructed using other peg based algorithms

on decoding of low density parity check codes over the binary erasure channel this paper investigates decoding of low density parity check ldpc codes over the binary erasure channel bec we study the iterative and maximum likelihood ml decoding of ldpc codes on this channel we derive bounds on the ml decoding of ldpc codes on the bec we then present an improved decoding algorithm the proposed algorithm has almost the same complexity as the standard iterative decoding however it has better performance simulations show that we can decrease the error rate by several orders of magnitude using the proposed algorithm we also provide some graph theoretic properties of different decoding algorithms of ldpc codes over the bec which we think are useful to better understand the ldpc decoding methods in particular for finite length codes

upper bounds on the rate of ldpc codes as a function of minimum distance new upper bounds on the rate of low density parity check ldpc codes as a function of the minimum distance of the code are derived the bounds apply to regular ldpc codes and sometimes also to right regular ldpc codes their derivation is based on combinatorial arguments and linear programming the new bounds improve upon the previous bounds due to burshtein et al it is proved that at least for high rates regular ldpc codes with full rank parity check matrices have worse relative minimum distance than the one guaranteed by the gilbert varshamov bound

a generalization of the suffix tree to square matrices with applications we describe a new data structure the lsuffix tree which generalizes mccreight s suffix tree for a string j assoc comput mach 23 1976 pp 262 272 to a square matrix all matrices have entries from a totally ordered alphabet sigma based on the lsuffix tree we give efficient algorithms for the static versions of the following dual problems that arise in low level image processing and visual databases two dimensional pattern retrieval we have a library of texts s text 1 cdots text r where text i is an n i times n i matrix 1 leq i leq r we may preprocess the library then given an m times m m leq n i 1 leq i leq r pattern matrix pat we want to find all occurrences of pat in text for all text in s let t s sigma i 1 r n i 2 be the size of the library the preprocessing step builds the lsuffix tree for the matrices in s and then transforms it into an index a trie defined over sigma it takes o t s log sigma log t s time and o t s space the index can be queried directly in o m 2 log sigma totocc time where totocc is the total number of occurrences of pat in text for all text in s two dimensional dictionary matching we have a dictionary of patterns dc pat 1 cdots pat s where pat i is of dimension m i times m i 1 leq i leq s we may preprocess the dictionary then given an n times n text matrix text we want to search for all occurrences of patterns in the dictionary in the text let t dc sigma i 1 s m 2 i be the size of the dictionary and let overline t dc be the sum of the m i s the preprocessing consists of building the lsuffix tree for the matrices in dc it takes o t dc log sigma overline t dc log overline t dc time and o t dc space the search step takes o n 2 log sigma log overline t dc totocc time where totocc is the total number of occurrences of patterns in the text both problems have a dynamic version in which the library and the dictionary respectively can be updated by insertion or deletion of square matrices in them in a companion paper we will provide algorithms for the dynamic version

results on parity check matrices with optimal stopping and or dead end set enumerators the performance of iterative decoding techniques for linear block codes correcting erasures depends very much on the sizes of the stopping sets associated with the underlying tanner graph or equivalently the parity check matrix representing the code in this correspondence we introduce the notion of dead end sets to explicitly demonstrate this dependency the choice of the parity check matrix entails a tradeoff between performance and complexity we give bounds on the complexity of iterative decoders achieving optimal performance in terms of the sizes of the underlying parity check matrices further we fully characterize codes for which the optimal stopping set enumerator equals the weight enumerator

capacity achieving codes with bounded graphical complexity and maximum likelihood decoding in this paper the existence of capacity achieving codes for memoryless binary input output symmetric mbios channels under maximum likelihood ml decoding with bounded graphical complexity is investigated graphical complexity of a code is defined as the number of edges in the graphical representation of the code per information bit and is proportional to the decoding complexity per information bit per iteration under iterative decoding irregular repeat accumulate ira codes are studied first utilizing the asymptotic average weight distribution aawd of these codes and invoking divsalar s bound on the binary input additive white gaussian noise biawgn channel it is shown that simple nonsystematic ira ensembles outperform systematic ira and regular low density parity check ldpc ensembles with the same graphical complexity and are at most 0 124 db away from the shannon limit however a conclusive result as to whether these nonsystematic ira codes can really achieve capacity cannot be reached motivated by this inconclusive result a new family of codes is proposed called low density parity check and generator matrix ldpc gm codes which are serially concatenated codes with an outer ldpc code and an inner low density generator matrix ldgm code it is shown that these codes can achieve capacity on any mbios channel using ml decoding and also achieve capacity on any bec using belief propagation bp decoding both with bounded graphical complexity moreover it is shown that under certain conditions these capacity achieving codes have linearly increasing minimum distances and achieve the asymptotic gilbert varshamov bound for all rates

average coset weight distributions of gallager s ldpc code ensemble in this correspondence the average coset weight distributions of gallager s low density parity check ldpc code ensemble are investigated gallager s ldpc code ensemble consists of regular mtimesn ldpc matrices with column weight j and row weight k the average coset weight distribution can be derived by enumerating the number of parity check matrices in the ensemble satisfying certain conditions based on combinatorial arguments a formula for the average coset weight distribution will be proved from the formula we can show some properties of the average coset weight distributions such as equivalence classes of syndromes symmetry of the distributions and a lower bound on coset weight

coding for parallel channels gallager bounds and applications to turbo like codes the transmission of coded communication systems is widely modeled to take place over a set of parallel channels this model is used for transmission over block fading channels rate compatible puncturing of turbo like codes multicarrier signaling multilevel coding etc new upper bounds on the maximum likelihood ml decoding error probability are derived in the parallel channel setting we focus on the generalization of the gallager type bounds and discuss the connections between some versions of these bounds the tightness of these bounds for parallel channels is exemplified for structured ensembles of turbo codes repeat accumulate ra codes and some of their recent variations e g punctured accumulate repeat accumulate codes the bounds on the decoding error probability of an ml decoder are compared to computer simulations of iterative decoding the new bounds show a remarkable improvement over the union bound and some other previously reported bounds for independent parallel channels this improvement is exemplified for relatively short block lengths and it is pronounced when the block length is increased in the asymptotic case where we let the block length tend to infinity inner bounds on the attainable channel regions of modern coding techniques under ml decoding are obtained based solely on the asymptotic growth rates of the average distance spectra of these code ensembles

on updating suffix tree labels abstract we investigate the problem of maintaining the arc labels in the suffix tree data structure gusfield et al 1992 amir et al 1994 when it undergoes string insertions and deletions in current literature this problem is solved either by a simple accounting strategy to obtain amortized bounds fiala and green 1989 larson 1996 or by a periodical suffix tree reconstruction to obtain worst case bounds according to the global rebuilding technique in overmars 1983 unfortunately the former approach is simple and space efficient at the cost of attaining amortized bounds for the single update the latter is space consuming in practice because it needs to keep two extra suffix tree copies in this paper we obtain a simple realtime algorithm that achieves worst case bounds and only requires small additional space i e a bi directional pointer per suffix tree label we analyze it by introducing a combinatorial coloring problem on the suffix tree arcs

finite length analysis of low density parity check codes on the binary erasure channel in this paper we are concerned with the finite length analysis of low density parity check ldpc codes when used over the binary erasure channel bec the main result is an expression for the exact average bit and block erasure probability for a given regular ensemble of ldpc codes when decoded iteratively we also give expressions for upper bounds on the average bit and block erasure probability for regular ldpc ensembles and the standard random ensemble under maximum likelihood ml decoding finally we present what we consider to be the most important open problems in this area

new nonasymptotic channel coding theorems for structured codes new nonasymptotic random coding theorems with error probability epsilon and finite block length n based on gallager parity check ensemble and general parity check ensembles are derived in this paper the resulting nonasymptotic achievability bounds when combined with nonasymptotic equipartition properties developed in this paper can be easily computed analytically these nonasymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as n goes to infinity with either constant or subexponentially decreasing epsilon in the case of gallager parity check ensemble and to imply that low density parity check ldpc codes be capacity achieving in the case of ldpc ensembles numerically they are also compared favorably for finite n and epsilon of practical interest with existing nonasymptotic achievability bounds in the literature

asymptotic average multiplicity of structures within different categories of trapping sets absorbing sets and stopping sets in random regular and irregular ldpc code ensembles the performance of low density parity check ldpc codes in the error floor region is closely related to some substructures of the code s tanner graph collectively referred to as trapping sets tss in this paper we study the asymptotic average number of different types of trapping sets such as elementary tss ets leafless etss lets absorbing sets abs elementary abss eabs and stopping sets ss in random variable regular and irregular ldpc code ensembles we demonstrate that regardless of the type of the ts as the code s length tends to infinity the average number of a given structure tends to infinity to a positive constant or to zero if the structure contains no cycle only one cycle or more than one cycle respectively for the case where the structure contains a single cycle we derive the asymptotic expected multiplicity of the structure by counting the average number of its constituent cycles and all the possible ways that the structure can be constructed from the cycle this in general involves computing the expected number of cycles of a certain length with a certain given combination of node degrees or computing the expected number of cycles of a certain length expanded to the desired structure by the connection of trees to its nodes the asymptotic results obtained in this work which are independent of the block length and only depend on the code s degree distributions are shown to be accurate even for finite length codes

weight distributions of regular low density parity check codes over finite fields the average weight distribution of a regular low density parity check ldpc code ensemble over a finite field is thoroughly analyzed in particular a precise asymptotic approximation of the average weight distribution is derived for the small weight case and a series of fundamental qualitative properties of the asymptotic growth rate of the average weight distribution are proved based on this analysis a general result including all previous results as special cases is established for the minimum distance of individual codes in a regular ldpc code ensemble

interactive encoding and decoding based on binary ldpc codes with syndrome accumulation interactive encoding and decoding based on binary low density parity check codes with syndrome accumulation sa ldpc ied is proposed and investigated assume that the source alphabet is gf 2 and the side information alphabet is finite it is first demonstrated how to convert any classical universal lossless code cn with block length n and side information available to both the encoder and decoder into a universal sa ldpc ied scheme it is then shown that with the word error probability approaching 0 subexponentially with n the compression rate including both the forward and backward rates of the resulting sa ldpc ied scheme is upper bounded by a functional of that of cn which in turn approaches the compression rate of cn for each and every individual sequence pair xn yn and the conditional entropy rate h x y for any stationary ergodic source and side information x y as the average variable node degree l of the underlying ldpc code increases without bound when applied to the class of binary source and side information x y correlated through a binary symmetrical channel with crossover probability unknown to both the encoder and decoder the resulting sa ldpc ied scheme can be further simplified yielding even improved rate performance versus the bit error probability when l is not large simulation results coupled with linear time belief propagation decoding on binary source side information pairs confirm the theoretic analysis and further show that the sa ldpc ied scheme consistently outperforms the slepian wolf coding scheme based on the same underlying ldpc code as a by product probability bounds involving ldpc established in the course are also interesting on their own and expected to have implications on the performance of ldpc for channel coding as well

constructing ldpc codes by error minimization progressive edge growth a novel approach to constructing tanner graphs using progressive edge growth peg is introduced it yields ldpc codes providing minimized block error probability in binary erasure channels bec the constructed codes exhibit superior performance over codes generated by previously known algorithms both for bec and awgn channels furthermore an upper bound on the expected block error probability in the error floor region of the generated codes is derived this allows analytical prediction of the codes error floor performance finally the method is generalized for generating simple in implementation ldpc codes based on lifted graphs

performance bounds for nonbinary linear block codes over memoryless symmetric channels the performance of nonbinary linear block codes is studied in this paper via the derivation of new upper bounds on the block error probability under maximum likelihood ml decoding the transmission of these codes is assumed to take place over a memoryless and symmetric channel the new bounds which are based on the gallager bounds and their variations are applied to the gallager ensembles of nonbinary and regular low density parity check ldpc codes these upper bounds are also compared with sphere packing lower bounds this study indicates that the new upper bounds are useful for the performance evaluation of coded communication systems which incorporate nonbinary coding techniques

spectral shape of doubly generalized ldpc codes efficient and exact evaluation this paper analyzes the asymptotic exponent of the weight spectrum for irregular doubly generalized ldpc d gldpc codes in the process an efficient numerical technique for its evaluation is presented involving the solution of a 4 4 system of polynomial equations the expression is consistent with previous results including the case where the normalized weight or stopping set size tends to zero the spectral shape is shown to admit a particularly simple form in the special case where all variable nodes are repetition codes of the same degree a case which includes tanner codes for this case it is also shown how certain symmetry properties of the local weight distribution at the cns induce a symmetry in the overall weight spectral shape function finally using these new results weight and stopping set size spectral shapes are evaluated for some example generalized and doubly generalized ldpc code ensembles

nondensely defined evolution impulsive differential inclusions with nonlocal conditions in this paper we shall establish sufficient conditions for the existence of integral solutions for some nondensely defined evolution impulsive differential inclusions in banach spaces with nonlocal conditions

lp sdp hierarchy lower bounds for decoding random ldpc codes random d v d c regular low density parity check ldpc codes where each variable is involved in d v parity checks and each parity check involves d c variables are well known to achieve the shannon capacity of the binary symmetric channel for sufficiently large d v and d c under exponential time decoding however polynomial time algorithms are only known to correct a much smaller fraction of errors one of the most powerful polynomial time algorithms with a formal analysis is the linear programming lp decoding algorithm of feldman et al which is known to correct an omega 1 d c fraction of errors in this paper we show that fairly powerful extensions of lp decoding based on the sherali adams and lasserre hierarchies fail to correct much more errors than the basic lp decoder in particular we show that 1 for any values of d v and d c a linear number of rounds of the sherali adams lp hierarchy cannot correct more than an o 1 d c fraction of errors on a random d v d c regular ldpc code and 2 for any value of d v and infinitely many values of d c a linear number of rounds of the lasserre sdp hierarchy cannot correct more than an o 1 d c fraction of errors on a random d v d c regular ldpc code our proofs use a new stretching and collapsing technique that allows us to leverage recent progress in the study of the limitations of lp sdp hierarchies for maximum constraint satisfaction problems max csps the problem then reduces to the construction of special balanced pairwise independent distributions for sherali adams and special cosets of balanced pairwise independent subgroups for lasserre our algebraic construction for the lasserre hierarchy is based on designing sets of points in mathbb f q d for q any power of 2 and d 2 3 with special hyperplane incidence properties constructions that may be of independent interest an intriguing consequence of our work is that expansion seems to be both the strength and the weakness of random regular ldpc codes some of our techniques are more generally applicable to a large class of boolean csps called min ones in particular for k hypergraph vertex cover we obtain an improved integrality gap of k 1 epsilon that holds after a linear number of rounds of the lasserre hierarchy for any k q 1 with q an arbitrary prime power the best previous gap for a linear number of rounds was equal to 2 epsilon and due to schoenebeck

performance of low density parity check codes with linear minimum distance this correspondence studies the performance of the iterative decoding of low density parity check ldpc code ensembles that have linear typical minimum distance and stopping set size we first obtain a lower bound on the achievable rates of these ensembles over memoryless binary input output symmetric channels we improve this bound for the binary erasure channel we also introduce a method to construct the codes meeting the lower bound for the binary erasure channel then we give upper bounds on the rate of ldpc codes with linear minimum distance when their right degree distribution is fixed we compare these bounds to the previously derived upper bounds on the rate when there is no restriction on the code ensemble

on the asymptotic input output weight distributions of some accumulate based codes in this letter we show how to compute the asymptotic growth rate of input output weight enumerator agr iowe for some accumulate based codes by using the sharp tools already developed numerical results on the agr iowe for irregular repeat accumulate ira codes systematic regular ra sra codes and concatenated zigzag codes are reported it is observed that the sra code has the same agr iowe as a comparable concatenated zigzag code for both sra and concatenated zigzag codes if keeping the code rate fixed the increase of the grouping factor for the component punctured accumulate code may result in better asymptotic performance under maximum likelihood decoding but often worse performance under iterative sum product decoding

tight exponential upper bounds on the ml decoding error probability of block codes over fully interleaved fading channels we derive tight exponential upper bounds on the decoding error probability of block codes which are operating over fully interleaved rician fading channels coherently detected and maximum likelihood decoded it is assumed that the fading samples are statistically independent and that perfect estimates of these samples are provided to the decoder these upper bounds on the bit and block error probabilities are based on certain variations of the gallager bounds these bounds do not require integration in their final version and they are reasonably tight in a certain portion of the rate region exceeding the cutoff rate of the channel by inserting interconnections between these bounds we show that they are generalized versions of some reported bounds for the binary input additive white gaussian noise channel

average coset weight distribution of combined ldpc matrix ensembles in this paper the average coset weight distribution acwd of structured ensembles of low density parity check ldpc matrices which are called combined ensembles is discussed a combined ensemble is composed of a set of simpler ensembles such as a regular bipartite ensemble two classes of combined ensembles have prime importance a stacked ensemble and a concatenated ensemble the acwd formulas of these ensembles are shown in this paper such formulas play a key role to evaluate the average weight distribution of some classes of combined ensembles

global bounds and asymptotics for a system of reaction diffusion equations on etudie le probleme aux valeurs limites non lineaire u t a u u v dans r v t c u d v u v dans r avec les conditions u v 0 sur r et u 0 x u 0 x v 0 x v 0 x dans on etablit l existence de solutions globales fortes et on etudie leur comportement quand t

on the asymptotic weight and stopping set distribution of regular ldpc ensembles in this correspondence we estimate the variance of weight and stopping set distribution of regular low density parity check ldpc ensembles using this estimate and the second moment method we obtain bounds on the probability that a randomly chosen code from regular ldpc ensemble has its weight distribution and stopping set distribution close to respective ensemble averages we are able to show that a large fraction of total number of codes have their weight and stopping set distribution close to the average

second order weight distributions a fundamental property of codes the second order weight distribution is proposed to solve the problems such as computing second moments of weight distributions of linear code ensembles a series of results parallel to those for weight distributions is established for second order weight distributions in particular an analogue of macwilliams identities is proved the second order weight distributions of regular ldpc code ensembles are then computed as easy consequences the second moments of weight distributions of regular ldpc code ensembles are obtained furthermore the application of second order weight distributions in random coding approach is discussed the second order weight distributions of the ensembles generated by a so called 2 good random generator or parity check matrix are computed where a 2 good random matrix is a kind of generalization of the uniformly distributed random matrix over a finite filed and is very useful for solving problems that involve pairwise or triple wise properties of sequences it is shown that the 2 good property is reflected in the second order weight distribution which thus plays a fundamental role in some well known problems in coding theory and combinatorics an example of linear intersecting codes is finally provided to illustrate this fact

asymptotic distribution of absorbing sets and fully absorbing sets for regular sparse code ensembles it is well recognized that low density parity check ldpc codes can suffer from an error floor when decoded iteratively this performance degradation is often attributed to the class of objects known as trapping sets past work has focused on characterizing the distribution of trapping sets for a variety of code ensembles including regular irregular and structured ldpc codes as a subset of the trapping set collection there exists a class of graphical structures called the absorbing sets an absorbing set is a combinatorially defined object in particular a fully absorbing set is stable under bit flipping decoding by construction there can exist trapping sets that are not stable under such a decoder as a result for finite precision iterative decoding algorithms used over additive channels absorbing sets can describe decoding errors more accurately than the broader class of trapping sets in this paper we compute the normalized logarithmic asymptotic distributions of absorbing sets and fully absorbing sets including elementary fully absorbing sets the calculations are based on the trapping set enumeration method proposed by milenkovic soljanin and whiting in 1 we compare distributions of absorbing and trapping sets for representative code parameters of interest and quantify the lack of discrepancies between the two good absorbing set properties are implied for known structured ldpc codes including repeat accumulate codes and protograph based constructions establishing the distribution of fully absorbing sets especially when the discrepancy with the trapping set distribution is significant allows one to further refine the estimates of the error rates under bit flipping and related decoders

nonadaptive group testing based on sparse pooling graphs an information theoretical analysis of nonadaptive group testing schemes based on sparse pooling graphs is presented a pooling graph is a bipartite graph for which the adjacency matrix is a pooling matrix the binary status of the objects to be tested are modeled by independent identically distributed bernoulli random variables with probability p an l r n regular pooling graph is a bipartite graph in which the left nodes have degree l the right nodes have degree r and n is the number of left nodes the main contribution of this paper is a direct coding theorem that gives the conditions for the existence of an estimator that can achieve an arbitrarily small probability of error an estimator is a function that uses observations to infer the state of an object the direct coding theorem is proved by averaging the upper bound on the probability of the estimation error of the typical set estimator over an l r n regular pooling graph ensemble numerical results indicate sharp threshold behaviors in the asymptotic regime these results can provide a concrete benchmark for nonadaptive group testing of existing and emerging detection algorithms over a noiseless system

expander graph arguments for message passing algorithms we show how expander based arguments may be used to prove that message passing algorithms can correct a linear number of erroneous messages the implication of this result is that when the block length is sufficiently large once a message passing algorithm has corrected a sufficiently large fraction of the errors it will eventually correct all errors this result is then combined with known results on the ability of message passing algorithms to reduce the number of errors to an arbitrarily small fraction for relatively high transmission rates the results hold for various message passing algorithms including gallager s hard decision and soft decision with clipping decoding algorithms our results assume low density parity check ldpc codes based on an irregular bipartite graph

upper bound on error exponent of regular ldpc codes transmitted over the bec the error performance of the ensemble of typical low definition parity check ldpc codes transmitted over the binary erasure channel bec is analyzed in the past lower bounds on the error exponents were derived in this paper a probabilistic upper bound on this error exponent is derived this bound holds with some confidence level

existence theory for perturbed impulsive hyperbolic differential inclusions with variable times in this paper the existence of solutions for a perturbed impulsive hyperbolic differential inclusion with variable times is proved under the mixed generalized lipschitz and caratheodory s conditions

maxwell construction the hidden bridge between iterative and maximum a posteriori decoding there is a fundamental relationship between belief propagation and maximum a posteriori decoding a decoding algorithm which is called the maxwell decoder is introduced and provides a constructive description of this relationship both the algorithm itself and the analysis of the new decoder are reminiscent of the maxwell construction in thermodynamics this paper investigates in detail the case of transmission over the binary erasure channel while the extension to general binary memoryless channels is discussed in a companion paper

density evolution for asymmetric memoryless channels density evolution de is one of the most powerful analytical tools for low density parity check ldpc codes and graph codes with message passing decoding algorithms with channel symmetry as one of its fundamental assumptions density evolution has been widely and successfully applied to different channels including binary erasure channels becs binary symmetric channels bscs binary additive white gaussian noise biawgn channels etc this paper generalizes density evolution for asymmetric memoryless channels which in turn broadens the applications to general memoryless channels e g z channels composite white gaussian noise channels etc the central theorem underpinning this generalization is the convergence to perfect projection for any fixed size supporting tree a new iterative formula of the same complexity is then presented and the necessary theorems for the performance concentration theorems are developed several properties of the new density evolution method are explored including stability results for general asymmetric memoryless channels simulations code optimizations and possible new applications suggested by this new density evolution method are also provided this result is also used to prove the typicality of linear ldpc codes among the coset code ensemble when the minimum check node degree is sufficiently large it is shown that the convergence to perfect projection is essential to the belief propagation bp algorithm even when only symmetric channels are considered hence the proof of the convergence to perfect projection serves also as a completion of the theory of classical density evolution for symmetric memoryless channels

nonlinear impulse evolution systems and applications to population models abstract we apply results on the existence uniqueness and stability of solutions obtained previously for the abstract impulse evolution system to the study of the mathematical model of population subject to abrupt changes of important biological parameters caused by harvesting diseases immigration etc

soft decoding based strategies for relay and interference channels analysis and achievable rates using ldpc codes we provide a rigorous mathematical analysis of two communication strategies soft decode and forward soft df for relay channels and soft partial interference cancelation soft ic for interference channels both strategies involve soft estimation which assists the decoding process we consider ldpc codes not because of their practical benefits but because of their analytic tractability which enables an asymptotic analysis similar to random coding methods of information theory unlike some works on the closely related demodulate and forward we assume non memoryless code structure aware estimation with soft df we develop simultaneous density evolution to bound the decoding error probability at the destination this result applies to erasure relay channels in one variant of soft df the relay applies wyner ziv coding to enhance its communication with the destination borrowing from compress and forward to analyze soft ic we adapt existing techniques for iterative multiuser detection and focus on binary input additive white gaussian noise interference channels we prove that optimal point to point codes are unsuitable for soft ic as well as for all strategies that apply partial decoding to improve upon single user detection and multiuser detection including han kobayashi

lower bounds on the error rate of ldpc code ensembles the ensemble of regular low definition parity check ldpc codes is considered using concentration results on the weight distribution lower bounds on the error rate of a random code in the ensemble are derived these bounds hold with some confidence level combining these results with known lower bounds on the error exponent confidence intervals on the error exponent under maximum likelihood ml decoding are obtained over a large range of channel parameter and transmission rate values when the graph connectivity is sufficiently large the upper bound of the interval approaches the lower bound and the probability that the error exponent is within the interval can be arbitrarily close to one in fact in this case the true error exponent approaches the maximum between the random coding and the expurgated random coding exponents with probability that approaches one

on first order impulsive partial differential inequalities abstract this paper deals with the cauchy problem for nonlinear impulsive partial differential equations of first order theorems on impulsive differential inequalities are obtained comparison results implying uniqueness criteria are proved

random coding techniques for nonrandom codes this work provides techniques to apply the channel coding theorem and the resulting error exponent which was originally derived for totally random block code ensembles to ensembles of codes with less restrictive randomness demands as an example the random coding technique can even be applied for an ensemble that contains a single code for a specific linear code we get an upper bound for the error probability which equals gallager s 1968 random coding bound up to a factor determined by the maximum ratio between the weight distribution of the code and the expected random weight distribution

on the undetected error probability of binary matrix ensembles in this paper an ensemble analysis of the undetected error probability of a standard error detection scheme based on a sparse binary parity check matrix is presented the bernoulli ensemble the members of which are considered to be matrices generated from an i i d bernoulli source is primarily considered herein the main contributions of the present study are i the derivation of the error exponent of the average undetected error probability and ii closed form expressions for the variance of the undetected error probability the behavior of the exponent for an ensemble of sparse matrices is shown to be somewhat different from that for an ensemble of dense matrices furthermore as a byproduct of the proof of the variance formula a simple covariance formula of the weight distribution is derived

low density parity check codes from transversal designs with improved stopping set distributions this paper examines the construction of low density parity check ldpc codes from transversal designs based on sets of mutually orthogonal latin squares mols by transferring the concept of configurations in combinatorial designs to the level of latin squares we thoroughly investigate the occurrence and avoidance of stopping sets for the arising codes stopping sets are known to determine the decoding performance over the binary erasure channel and should be avoided for small sizes based on large sets of simple structured mols we derive powerful constraints for the choice of suitable subsets leading to improved stopping set distributions for the corresponding codes we focus on ldpc codes with column weight 4 but the results are also applicable for the construction of codes with higher column weights finally we show that a subclass of the presented codes has quasi cyclic structure which allows low complexity encoding

improved low density parity check codes using irregular graphs we construct new families of error correcting codes based on gallager s 1973 low density parity check codes we improve on gallager s results by introducing irregular parity check matrices and a new rigorous analysis of hard decision decoding of these codes we also provide efficient methods for finding good irregular structures for such decoding algorithms our rigorous analysis based on martingales our methodology for constructing good irregular codes and the demonstration that irregular structure improves performance constitute key points of our contribution we also consider irregular codes under belief propagation we report the results of experiments testing the efficacy of irregular codes on both binary symmetric and gaussian channels for example using belief propagation for rate 1 4 codes on 16000 bits over a binary symmetric channel previous low density parity check codes can correct up to approximately 16 errors while our codes correct over 17 in some cases our results come very close to reported results for turbo codes suggesting that variations of irregular low density parity check codes may be able to match or beat turbo code performance