design of capacity approaching irregular low density parity check codes we design low density parity check ldpc codes that perform at rates extremely close to the shannon capacity the codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides our theoretical analysis of the codes is based on the work of richardson and urbanke see ibid vol 47 no 2 p 599 618 2000 assuming that the underlying communication channel is symmetric we prove that the probability densities at the message nodes of the graph possess a certain symmetry using this symmetry property we then show that under the assumption of no cycles the message densities always converge as the number of iterations tends to infinity furthermore we prove a stability condition which implies an upper bound on the fraction of errors that a belief propagation decoder can correct when applied to a code induced from a bipartite graph with a given degree distribution our codes are found by optimizing the degree structure of the underlying graphs we develop several strategies to perform this optimization we also present some simulation results for the codes found which show that the performance of the codes is very close to the asymptotic theoretical bounds

map estimation via agreement on trees message passing and linear programming we develop and analyze methods for computing provably optimal maximum a posteriori probability map configurations for a subclass of markov random fields defined on graphs with cycles by decomposing the original distribution into a convex combination of tree structured distributions we obtain an upper bound on the optimal value of the original problem i e the log probability of the map assignment in terms of the combined optimal values of the tree problems we prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common an important implication is that any such shared configuration must also be a map configuration for the original distribution next we develop two approaches to attempting to obtain tight upper bounds a a tree relaxed linear program lp which is derived from the lagrangian dual of the upper bounds and b a tree reweighted max product message passing algorithm that is related to but distinct from the max product algorithm in this way we establish a connection between a certain lp relaxation of the mode finding problem and a reweighted form of the max product min sum message passing algorithm

on expander codes sipser and spielman see ibid vol 42 p 1717 22 nov 1996 have introduced a constructive family of asymptotically good linear error correcting codes expander codes together with a simple parallel algorithm that will always remove a constant fraction of errors we introduce a variation on their decoding algorithm that with no extra cost in complexity provably corrects up to 12 times more errors

analysis and design of tuned turbo codes it has been widely observed that there exists a fundamental tradeoff between the minimum hamming distance properties and the iterative decoding convergence behavior of turbo like codes while capacity achieving code ensembles typically are asymptotically bad in the sense that their minimum distance does not grow linearly with block length and they therefore exhibit an error floor at moderate to high signal to noise ratios asymptotically good codes usually converge further away from channel capacity in this paper we introduce the concept of tuned turbo codes a family of asymptotically good hybrid concatenated code ensembles where asymptotic minimum distance growth rates convergence thresholds and code rates can be tradedoff using two tuning parameters and by decreasing the asymptotic minimum distance growth rate is reduced in exchange for improved iterative decoding convergence behavior while increasing raises the asymptotic minimum distance growth rate at the expense of worse convergence behavior and thus the code performance can be tuned to fit the desired application by decreasing a similar tuning behavior can be achieved for higher rate code ensembles

on the dynamics of the error floor behavior in regular ldpc codes it is shown that dominant trapping sets of regular low density parity check ldpc codes so called absorption sets undergo a two phased dynamic behavior in the iterative message passing decoding algorithm using a linear dynamic model for the iteration behavior of these sets it is shown that they undergo an initial geometric growth phase which stabilizes in a final bit flipping behavior where the algorithm reaches a fixed point this analysis is shown to lead to very accurate numerical calculations of the error floor bit error rates down to error rates that are inaccessible by simulation the topology of the dominant absorption sets of an example code the ieee 802 3an 2048 1723 regular ldpc code are identified and tabulated using topological relationships in combination with search algorithms

low ber performance estimation of ldpc codes via application of importance sampling to trapping sets we introduce an importance sampling is method that successfully simulates the performance of low density parity check ldpc codes in an awgn channel at very low bit error rates bers by effectively finding and biasing bit node combinations that are the dominant sources of error events called trapping sets the developed technique provokes more frequent decoder failures consequently fewer simulation runs and higher simulation gains are achieved

fast decoding of expander codes expander codes are tanner codes defined on sparse graphs that have good expansion properties sipser and spielman 1996 showed that there is a linear time decoding algorithm for expander codes when the vertex expansion is at least 3 4 and the number of errors corrected is a constant fraction of the code length later feldman et al 2007 gave a decoding algorithm that allows the expansion to be 2 3 1 3c where c is the left degree of the underlying bipartite graph at the expense of polynomial time decoding complexity recently viderman 2013 further improved the expansion parameter to 2 3 1 6c and the decoding algorithm runs in linear time these results are for expander codes whose inner codes are parity check codes by using stronger inner codes chilappagari et al 2010 showed that there is a linear time decoding algorithm for every vertex expansion greater than 1 2 in this paper it is shown that for every vertex expansion there is a linear time decoding algorithm for expander codes using inner codes with minimum distance depending on the vertex expansion and that the number of errors corrected is a constant fraction of the code length

guessing facets polytope structure and improved lp decoder we investigate the structure of the polytope underlying the linear programming lp decoder introduced by feldman karger and wainwright we first show that for expander codes every fractional pseudocodeword always has at least a constant fraction of nonintegral bits we then prove that for expander codes the active set of any fractional pseudocodeword is smaller by a constant fraction than that of any codeword we further exploit these geometrical properties to devise an improved decoding algorithm with the same order of complexity as lp decoding that provably performs better the method is very simple it first applies ordinary lp decoding and when it fails it proceeds by guessing facets of the polytope and then resolving the linear program on these facets while the lp decoder succeeds only if the ml codeword has the highest likelihood over all pseudocodewords we prove that the proposed algorithm when applied to suitable expander codes succeeds unless there exists a certain number of pseudocodewords all adjacent to the ml codeword on the lp decoding polytope and with higher likelihood than the ml codeword we then describe an extended algorithm still with polynomial complexity that succeeds as long as there are at most polynomially many pseudocodewords above the ml codeword

ldpc codes based on latin squares cycle structure stopping set and trapping set analysis it is well known that certain combinatorial structures in the tanner graph of a low density parity check ldpc code exhibit a strong influence on its performance under iterative decoding these structures include cycles stopping trapping sets and parameters such as the diameter of the code in general it is very hard to find a complete characterization of such configurations in an arbitrary code and even harder to understand the intricate relationships that exist between these entities it is therefore of interest to identify a simple setting in which all the described combinatorial structures can be enumerated and studied within a joint framework one such setting is developed in this paper for the purpose of analyzing the distribution of short cycles and the structure of stopping and trapping sets in tanner graphs of ldpc codes based on idempotent and symmetric latin squares the parity check matrices of ldpc codes based on latin squares have a special form that allows for connecting combinatorial parameters of the codes with the number of certain subrectangles in the latin squares subrectangles of interest can be easily identified and in certain instances completely enumerated this study can be extended in several different directions one of which is concerned with modifying the code design process in order to eliminate or reduce the number of configurations bearing a negative influence on the performance of the code another application of the results includes determining to which extent a configuration governs the behavior of the bit error rate curve in the waterfall and error floor regions

signal space characterization of iterative decoding by tracing the flow of computations in the iterative decoders for low density parity check codes we formulate a signal space view for a finite number of iterations in a finite length code on a gaussian channel maximum a posteriori map codeword decoding or maximum likelihood decoding decodes to the codeword signal that is closest to the channel output in euclidean distance in contrast we show that iterative decoding decodes to the pseudosignal that has highest correlation with the channel output the set of pseudosignals corresponds to pseudocodewords only a vanishingly small number of which correspond to codewords we show that some pseudocodewords cause decoding errors but that there are also pseudocodewords that frequently correct the deleterious effects of other pseudocodewords

quasi cyclic ldpc codes an algebraic construction rank analysis and codes on latin squares quasi cyclic ldpc codes are the most promising class of structured ldpc codes due to their ease of implementation and excellent performance over noisy channels when decoded with message passing algorithms as extensive simulation studies have shown in this paper an approach for constructing quasi cyclic ldpc codes based on latin squares over finite fields is presented by analyzing the parity check matrices of these codes combinatorial expressions for their ranks and dimensions are derived experimental results show that with iterative decoding algorithms the constructed codes perform very well over the awgn and the binary erasure channels

the admm penalized decoder for ldpc codes linear programming lp decoding for low density parity check codes was introduced by feldman et al and has been shown to have theoretical guarantees in several regimes furthermore it has been reported in the literature via simulation and via instanton analysis that lp decoding displays better error rate performance at high signal to noise ratios snr than does belief propagation bp decoding however at low snrs lp decoding is observed to have worse performance than bp in this paper we seek to improve lp decoding at low snrs while maintaining lp decoding s high snr performance our main contribution is a new class of decoders obtained by applying the alternating direction method of multipliers admm algorithm to a set of non convex optimization problems these non convex problems are constructed by adding a penalty term to the objective of lp decoding the goal of the penalty is to make pseudocodewords which are non integer vertices of the lp relaxation more costly we name this class of decoders admm penalized decoders for low and moderate snrs we simulate admm penalized decoding with ell 1 and ell 2 penalties we find that these decoders can outperform both bp and lp decoding for high snrs where it is difficult to obtain data via simulation we use an instanton analysis and find that asymptotically admm penalized decoding performs better than bp but not as well as lp unfortunately since admm penalized decoding is not a convex program we have not been successful in developing theoretical guarantees however the non convex program can be approximated using a sequence of linear programs an approach that yields a reweighted lp decoder we show that a two round reweighted lp decoder has an improved theoretical recovery threshold when compared with lp decoding in addition we find via simulation that reweighted lp decoding significantly attains lower error rates than lp decoding at low snrs

error correction capability of column weight three ldpc codes under the gallager a algorithm part ii the relation between the girth and the error correction capability of column weight three ldpc codes under the gallager a algorithm is investigated it is shown that a column weight three ldpc code with tanner graph of girth g 10 can correct all error patterns with up to g 2 1 errors in at most g 2 iterations of the gallager a algorithm for codes with tanner graphs of girth g 8 it is shown that girth alone cannot guarantee correction of all error patterns with up to g 2 1 errors under the gallager a algorithm sufficient conditions to correct g 2 1 errors are then established by studying trapping sets

construction of quasi cyclic ldpc codes for awgn and binary erasure channels a finite field approach in the late 1950s and early 1960s finite fields were successfully used to construct linear block codes especially cyclic codes with large minimum distances for hard decision algebraic decoding such as bose chaudhuri hocquenghem bch and reed solomon rs codes this paper shows that finite fields can also be successfully used to construct algebraic low density parity check ldpc codes for iterative soft decision decoding methods of construction are presented ldpc codes constructed by these methods are quasi cyclic qc and they perform very well over the additive white gaussian noise awgn binary random and burst erasure channels with iterative decoding in terms of bit error probability block error probability error floor and rate of decoding convergence collectively particularly they have low error floors since the codes are qc they can be encoded using simple shift registers with linear complexity

construction of low density parity check codes based on balanced incomplete block designs this correspondence presents a method for constructing structured regular low density parity check ldpc codes based on a special type of combinatoric designs known as balance incomplete block designs codes constructed by this method have girths at least 6 and they perform well with iterative decoding furthermore several classes of these codes are quasi cyclic and hence their encoding can be implemented with simple feedback shift registers

a recursive approach to low complexity codes a method is described for constructing long error correcting codes from one or more shorter error correcting codes referred to as subcodes and a bipartite graph a graph is shown which specifies carefully chosen subsets of the digits of the new codes that must be codewords in one of the shorter subcodes lower bounds to the rate and the minimum distance of the new code are derived in terms of the parameters of the graph and the subeodes both the encoders and decoders proposed are shown to take advantage of the code s explicit decomposition into subcodes to decompose and simplify the associated computational processes bounds on the performance of two specific decoding algorithms are established and the asymptotic growth of the complexity of decoding for two types of codes and decoders is analyzed the proposed decoders are able to make effective use of probabilistic information supplied by the channel receiver e g reliability information without greatly increasing the number of computations required it is shown that choosing a transmission order for the digits that is appropriate for the graph and the subcodes can give the code excellent burst error correction abilities the construction principles

finite alphabet iterative decoders part i decoding beyond belief propagation on the binary symmetric channel we introduce a new paradigm for finite precision iterative decoding on low density parity check codes over the binary symmetric channel the messages take values from a finite alphabet and unlike traditional quantized decoders which are quantized versions of the belief propagation bp decoder the proposed finite alphabet iterative decoders faids do not propagate quantized probabilities or log likelihoods and the variable node update functions do not mimic the bp decoder rather the update functions are maps designed using the knowledge of potentially harmful subgraphs that could be present in a given code thereby rendering these decoders capable of outperforming the bp in the error floor region on certain column weight three codes of practical interest we show that there exist faids that surpass the floating point bp decoder in the error floor region while requiring only three bits of precision for the representation of the messages hence faids are able to achieve a superior performance at much lower complexity we also provide a methodology for the selection of faids that is not code specific but gives a set of candidate faids containing potentially good decoders in the error floor region for any column weight three code we validate the code generality of our methodology by providing particularly good three bit precision faids for a variety of codes with different rates and lengths

on the error correction of regular ldpc codes using the flipping algorithm the iterative bit flipping algorithm is applied to the standard regular low density parity check ldpc code ensemble in the past it was shown for a typical code in the ensemble with left degree at least five and block length sufficiently large that this algorithm can correct a linear in the block length number of worst case errors in this paper this result is extended to the case where the left degree is at least four for the case where the left degree is larger than four an improvement compared to existing results of several orders of magnitude is obtained on the fraction of worst case errors that can be corrected it is also shown how the results can be further improved when random errors produced by the channel as opposed to worst case errors are considered

expander codes using expander graphs we construct a new family of asymptotically good linear error correcting codes these codes have linear time sequential decoding algorithms and logarithmic time parallel decoding algorithms that use a linear number of processors we present both randomized and explicit constructions of these codes experimental results demonstrate the good performance of the randomly chosen codes

on mutual information likelihood ratios and estimation error for the additive gaussian channel this paper considers the model of an arbitrarily distributed signal x observed through an added independent white gaussian noise w y x w new relations between the minimal mean square error of the noncausal estimator and the likelihood ratio between y and w are derived this is followed by an extended version of a recently derived relation between the mutual information i x y and the minimal mean square error these results are applied to derive infinite dimensional versions of the fisher information and the de bruijn identity a comparison between the causal and noncausal estimation errors yields a restricted form of the logarithmic sobolev inequality the derivation of the results is based on the malliavin calculus

impact of redundant checks on the lp decoding thresholds of ldpc codes feldman et al 11 asked whether the performance of the linear programming lp decoder can be improved by adding redundant parity checks to tighten the lp relaxation we prove in this paper that for low density parity check codes even if we include all redundant parity checks asymptotically there is no gain in the lp decoder threshold on the binary symmetric channel under certain conditions on the base tanner graph first we show that if the tanner graph has bounded check degree and satisfies a condition which we call asymptotic strength then including high degree redundant parity checks in the lp does not significantly improve the threshold of the lp decoder in the following sense for each constant delta 0 there is a constant k 0 such that the threshold of the lp decoder containing all redundant checks of degree at most k improves by at most delta upon adding to the lp all redundant checks of degree larger than k we conclude that if the graph satisfies an additional condition which we call rigidity then including all redundant checks does not improve the threshold of the base lp we call the graph asymptotically strong if the lp decoder corrects a constant fraction of errors even if the log likelihood ratios of the correct variables are arbitrarily small by building on a construction due feldman et al 9 and its recent improvement by viderman 24 we show that asymptotic strength follows from sufficiently large variable to check expansion we also give a geometric interpretation of asymptotic strength in terms pseudocodewords we call the graph rigid if the minimum weight of a sum of check nodes involving a cycle tends to infinity as the block length tends to infinity under the assumptions that the graph girth is logarithmic and the minimum check degree is at least 3 rigidity is equivalent to the nondegeneracy property that adding at least logarithmically many checks does not give a constant weight check we argue that nondegeneracy is a typical property of random check regular tanner graphs

applications of a kushner and clark lemma to general classes of stochastic algorithms two general classes of stochastic algorithms are considered including algorithms considered by ljung as well as algorithms of the form theta n 1 theta n gamma n 1 v n 1 theta n z where z is a stationary ergodic process it is shown how one can apply a lemma of kushner and clark to obtain properties of these algorithms this is done by using in particular martingale arguments in the generalized ljung case in these various situations the convergence is obtained by the method of the associated ordinary differential equation under the classical boundedness assumptions in the case of linear algorithms the boundedness assumptions are dropped

noise conditions for prespecified convergence rates of stochastic approximation algorithms we develop deterministic necessary and sufficient conditions on individual noise sequences of a stochastic approximation algorithm for the error of the iterates to converge at a given rate specifically suppose spl rho sub n is a given positive sequence converging monotonically to zero consider a stochastic approximation algorithm x sub n 1 x sub n a sub n a sub n x sub n b sub n a sub n e sub n where x sub n is the iterate sequence a sub n is the step size sequence e sub n is the noise sequence and x is the desired zero of the function f x ax b then under appropriate assumptions we show that x sub n x o spl rho sub n if and only if the sequence e sub n satisfies one of five equivalent conditions these conditions are based on well known formulas for noise sequences kushner and clark s 1978 condition chen s see proc ifac world congr p 375 80 1996 condition kulkarni and horn s see ieee trails automat contr vol 41 p 419 24 1996 condition a decomposition condition and a weighted averaging condition our necessary and sufficient condition on e sub n to achieve a convergence rate of spl rho sub n is basically that the sequence e sub n spl rho sub n satisfies any one of the above five well known conditions we provide examples to illustrate our result in particular we easily recover the familiar result that if a sub n a n and e sub n is a martingale difference process with bounded variance then x sub n x o n sup 1 2 log n sup spl beta for any spl beta 1 2

network based consensus averaging with general noisy channels this paper focuses on the consensus averaging problem on graphs under general imperfect communications we study a particular class of distributed consensus algorithms based on damped updates and using the ordinary differential equation method we prove that the updates converge almost surely to the consensus average for various models of perturbation of data exchanged between nodes the convergence is not asymptotic in the size of the network our analysis applies to various types of stochastic disturbances to the updates including errors in update calculations dithered quantization and imperfect data exchange among nodes under a suitable stability condition we prove that the error between estimated and true averages is asymptotically gaussian and we show how the asymptotic covariance is specified by the graph laplacian for additive perturbations we show how the scaling of the asymptotic mse is controlled by the spectral gap of the laplacian

high rate quasi cyclic low density parity check codes derived from finite affine planes this paper shows that several attractive classes of quasi cyclic qc low density parity check ldpc codes can be obtained from affine planes over finite fields one class of these consists of duals of one generator qc codes presented here for codes contained in this class are the exact minimum distance and a lower bound on the multiplicity of the minimum weight codewords further it is shown that the minimum hamming distance of a code in this class is equal to its minimum additive white gaussian noise awgn pseudoweight also discussed is a class consisting of codes from circulant permutation matrices and an explicit formula for the rank of the parity check matrix is presented for these codes additionally it is shown that each of these codes can be identified with a code constructed from a constacyclic maximum distance separable code of dimension 2 the construction is similar to the derivation of reed solomon rs based ldpc codes presented by chen and djurdjevic experimental results show that a number of high rate qc ldpc codes with excellent error performance are contained in these classes

message passing algorithms and improved lp decoding linear programming lp decoding for low density parity check codes and related domains such as compressed sensing has received increased attention over recent years because of its practical performance coming close to that of iterative decoding algorithms and its amenability to finite blocklength analysis several works starting with the work of feldman showed how to analyze lp decoding using properties of expander graphs this line of analysis works for only low error rates about a couple of orders of magnitude lower than the empirically observed performance it is possible to do better for the case of random noise as shown by daskalakis and koetter and vontobel building on work of koetter and vontobel we obtain a novel understanding of lp decoding which allows us to establish a 0 05 fraction of correctable errors for rate codes this comes very close to the performance of iterative decoders and is significantly higher than the best previously noted correctable bit error rate for lp decoding our analysis exploits an explicit connection between lp decoding and message passing algorithms and unlike other techniques directly works with the primal linear program an interesting byproduct of our method is a notion of a locally optimal solution that we show to always be globally optimal i e it is the nearest codeword such a solution can in fact be found in near linear time by a reweighted version of the min sum algorithm obviating the need for lp our analysis implies in particular that this reweighted version of the min sum decoder corrects up to a 0 05 fraction of errors

cyclic and quasi cyclic ldpc codes on constrained parity check matrices and their trapping sets this paper is concerned with construction and structural analysis of both cyclic and quasi cyclic codes particularly low density parity check ldpc codes it consists of three parts the first part shows that a cyclic code given by a parity check matrix in circulant form can be decomposed into descendant cyclic and quasi cyclic codes of various lengths and rates some fundamental structural properties of these descendant codes are developed including the characterization of the roots of the generator polynomial of a cyclic descendant code the second part of the paper shows that cyclic and quasi cyclic descendant ldpc codes can be derived from cyclic finite geometry ldpc codes using the results developed in the first part of the paper this enlarges the repertoire of cyclic ldpc codes the third part of the paper analyzes the trapping set structure of regular ldpc codes whose parity check matrices satisfy a certain constraint on their rows and columns several classes of finite geometry and finite field cyclic and quasi cyclic ldpc codes with large minimum distances are shown to have no harmful trapping sets of size smaller than their minimum distances consequently their error floor performances are dominated by their minimum distances

error floor approximation for ldpc codes in the awgn channel this paper addresses the prediction of error floors of low density parity check codes transmitted over the additive white gaussian noise channel using a linear state space model to estimate the behavior of the sum product algorithm spa decoder in the vicinity of trapping sets tss we study the performance of the spa decoder in the log likelihood ratio llr domain as a function of the llr saturation level when applied to several widely studied codes the model accurately predicts a significant decrease in the error floor as the saturation level is allowed to increase for nonsaturating decoders however we find that the state space model breaks down after a small number of iterations due to the strong correlation of llr messages we then revisit richardson s importance sampling methodology for estimating error floors due to tss when those floors are too low for monte carlo simulation we propose modifications that account for the behavior of a nonsaturating decoder and present the resulting error floor estimates for the margulis code these estimates are much lower significantly steeper and more sensitive to iteration count than those previously reported

using linear programming to decode binary linear codes a new method is given for performing approximate maximum likelihood ml decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel the decoding algorithm is based on a linear programming lp relaxation that is defined by a factor graph or parity check representation of the code the resulting lp decoder generalizes our previous work on turbo like codes a precise combinatorial characterization of when the lp decoder succeeds is provided based on pseudocodewords associated with the factor graph our definition of a pseudocodeword unifies other such notions known for iterative algorithms including stopping sets irreducible closed walks trellis cycles deviation sets and graph covers the fractional distance d sub frac of a code is introduced which is a lower bound on the classical distance it is shown that the efficient lp decoder will correct up to spl lceil d sub frac 2 spl rceil 1 errors and that there are codes with d sub frac spl omega n sup 1 spl epsi an efficient algorithm to compute the fractional distance is presented experimental evidence shows a similar performance on low density parity check ldpc codes between lp decoding and the min sum and sum product algorithms methods for tightening the lp relaxation to improve performance are also provided

computing linear transformations with unreliable components we consider the problem of computing a binary linear transformation when all circuit components are unreliable two models of unreliable components are considered probabilistic errors and permanent errors we introduce the encoded technique that ensures that the error probability of the computation of the linear transformation is kept bounded below a small constant independent of the size of the linear transformation even when all logic gates in the computation are noisy by deriving a lower bound we show that in some cases the computational complexity of the encoded technique achieves the optimal scaling in error probability further we examine the gain in energy efficiency from the use of a voltage scaling scheme where gate energy is reduced by lowering the supply voltage we use a gate energy reliability model to show that tuning gate energy appropriately at different stages of the computation dynamic voltage scaling in conjunction with encoded can lead to orders of magnitude energy savings over the classical uncoded approach finally we also examine the problem of computing a linear transformation when noiseless decoders can be used providing upper and lower bounds to the problem

an alternative proof for convergence of stochastic approximation algorithms an alternative proof for convergence of stochastic approximation algorithms is provided the proof is completely deterministic very elementary involving only basic notions of convergence and direct in that it remains in a discrete setting an alternative form of the kushner clark condition is introduced and utilized and the results are the first to prove necessity for general gain sequences in a hilbert space setting

a unique nonnegative solution to an underdetermined system from vectors to matrices this paper investigates the uniqueness of a nonnegative vector solution and the uniqueness of a positive semidefinite matrix solution to underdetermined linear systems a vector solution is the unique solution to an underdetermined linear system only if the measurement matrix has a row span intersecting the positive orthant focusing on two types of binary measurement matrices bernoulli 0 1 matrices and adjacency matrices of general expander graphs we show that in both cases the support size of a unique nonnegative solution can grow linearly namely o n with the problem dimension n we also provide closed form characterizations of the ratio of this support size to the signal dimension for the matrix case we show that under a necessary and sufficient condition for the linear compressed observations operator there will be a unique positive semidefinite matrix solution to the compressed linear observations we further show that a randomly generated gaussian linear compressed observations operator will satisfy this condition with overwhelmingly high probability

pseudocodeword performance analysis for ldpc convolutional codes message passing iterative decoders for low density parity check ldpc block codes are known to be subject to decoding failures due to so called pseudocodewords these failures can cause the large signal to noise ratio snr performance of message passing iterative decoding to be worse than that predicted by the maximum likelihood ml decoding union bound in this paper we address the pseudocodeword problem from the convolutional code perspective in particular we compare the performance of ldpc convolutional codes with that of their wrapped quasi cyclic block versions and we show that the minimum pseudoweight of an ldpc convolutional code is at least as large as the minimum pseudoweight of an underlying quasi cyclic code this result which parallels a well known relationship between the minimum hamming weight of convolutional codes and the minimum hamming weight of their quasi cyclic counterparts is due to the fact that every pseudocodeword in the convolutional code induces a pseudocodeword in the block code with pseudoweight no larger than that of the convolutional code s pseudocodeword this difference in the weight spectra leads to improved performance at low to moderate snrs for the convolutional code a conclusion supported by simulation results

algebraic construction of quasi cyclic ldpc codes for the awgn and erasure channels this paper is concerned with construction of quasi cyclic qc low density parity check ldpc codes for three different types of channels the additive white gaussian noise the binary random erasure and the binary burst erasure channels two algebraic methods for systematic construction of qc ldpc codes are presented codes constructed perform well over all three types of channels

admm lp decoding of non binary ldpc codes in mathbb f _ 2 m in this paper we develop efficient decoders for non binary low density parity check codes using the alternating direction method of multipliers admm we apply admm to two decoding problems the first problem is linear programming lp decoding in order to develop an efficient algorithm we focus on non binary codes in fields of characteristic two this allows us to transform each constraint in mathbb f 2 m to a set of constraints in mathbb f 2 that has a factor graph representation applying admm to the lp decoding problem results in two types of non trivial sub routines the first type requires us to solve an unconstrained quadratic program we solve this problem efficiently by leveraging new results obtained from studying the above factor graphs the second type requires euclidean projections onto polytopes that are studied in the literature such projections can be solved efficiently using off the shelf techniques which scale linearly in the dimension of the vector to project admm lp decoding scales linearly with block length linearly with check degree and quadratically with field size the second problem we consider is a penalized lp decoding problem this problem is obtained by incorporating a penalty term into the lp decoding objective the purpose of the penalty term is to make non integer solutions pseudocodewords more expensive and hence to improve decoding performance the admm algorithm for the penalized lp problem requires euclidean projection onto a polytope formed by embedding the constraints specified by the non binary single parity check code which can be solved by applying the admm technique to the resulting quadratic program empirically this decoder achieves a much reduced error rate than lp decoding at low signal to noise ratios

correcting a fraction of errors in nonbinary expander codes with linear programming a linear programming decoder for nonbinary expander codes is presented it is shown that the proposed decoder has the nearest neighbor certificate properties it is also shown that this decoder corrects any pattern of errors of a relative weight up to approximately 1 4 a b where a and b are the relative minimum distances of the constituent codes

on trapping sets and guaranteed error correction capability of ldpc codes and gldpc codes the relation between the girth and the guaranteed error correction capability of left regular low density parity check ldpc codes when decoded using the bit flipping serial and parallel algorithms is investigated a lower bound on the size of variable node sets which expand by a factor of at least 3 4 is found based on the moore bound this bound combined with the well known expander based arguments leads to a lower bound on the guaranteed error correction capability the decoding failures of the bit flipping algorithms are characterized using the notions of trapping sets and fixed sets the relation between fixed sets and a class of graphs known as cage graphs is studied upper bounds on the guaranteed error correction capability are then established based on the order of cage graphs the results are extended to left regular and right uniform generalized ldpc codes it is shown that this class of generalized ldpc codes can correct a linear number of worst case errors in the code length under the parallel bit flipping algorithm when the underlying tanner graph is a good expander a lower bound on the size of variable node sets which have the required expansion is established

relative frequencies of generalized simulated annealing we consider a class of nonhomogeneous markov chains arising in simulated annealing and related stochastic search algorithms using only elementary first principles we analyze the convergence and rate of convergence of the relative frequencies of visits to states in the markov chain we describe in detail three examples including the standard simulated annealing algorithm to show how our framework applies to specific stochastic search algorithmsthese examples have not previously been recognized to be sufficiently similar to share common analytical grounds our analysis though elementary provides the strongest sample path convergence results to date for simulated annealing type markov chains our results serve to illustrate that by taking a purely sample path view surprisingly strong statements can be made using only relatively elementary tools

a sub graph expansion contraction method for error floor computation in this paper we present a computationally efficient method for estimating error floors of low density parity check ldpc codes over the binary symmetric channel bsc without any prior knowledge of its trapping sets tss given the tanner graph g of a code and the decoding algorithm mathcal d the method starts from a list of short cycles in g and expands each cycle by including its sufficiently large neighborhood in g variable nodes of the expanded sub graphs mathcal g text exp are then corrupted exhaustively by all possible error patterns and decoded by mathcal d operating on mathcal g text exp union of support of the error patterns for which mathcal d fails on each mathcal g text exp defines a subset of variable nodes that is a ts the knowledge of the minimal error patterns and their strengths in each tss is used to compute an estimation of the frame error rate this estimation represents the contribution of error events localized on tss and therefore serves as an accurate estimation of the error floor performance of mathcal d at low bsc cross over probabilities we also discuss trade offs between accuracy and computational complexity our analysis shows that in some cases the proposed method provides a million fold improvement in computational complexity over standard monte carlo simulation

iterative approximate linear programming decoding of ldpc codes with linear complexity the problem of low complexity linear programming lp decoding of low density parity check ldpc codes is considered an iterative algorithm similar to min sum and belief propagation for efficient approximate solution of this problem was proposed by vontobel and koetter in this paper the convergence rate and computational complexity of this algorithm are studied using a scheduling scheme that we propose in particular we are interested in obtaining a feasible vector in the lp decoding problem that is close to optimal in the following sense the distance normalized by the block length between the minimum and the objective function value of this approximate solution can be made arbitrarily small it is shown that such a feasible vector can be obtained with a computational complexity which scales linearly with the block length combined with previous results that have shown that the lp decoder can correct some fixed fraction of errors we conclude that this error correction can be achieved with linear computational complexity this is achieved by first applying the iterative lp decoder that decodes the correct transmitted codeword up to an arbitrarily small fraction of erroneous bits and then correcting the remaining errors using some standard method these conclusions are also extended to generalized ldpc codes

malliavin derivatives and derivatives of functionals of the wiener process with respect to a scale parameter let f c 0w be a functional of the wiener process with variance parameter c 2 0 and let f cw be an extension of f c 0w to f cw c in 0 c 0 relations are derived between the malliavin derivatives between the derivatives with respect to the scale parameter partial f rho cw partial rho p 1 and noncoherent derivatives such as de f cw sqrt varepsilon c tilde w mid w d varepsilon varepsilon 0 where tilde w is another wiener process independent of w and between the generator of the nontime homogeneous ornstein uhlenbeck process

the minimum distance of turbo like codes worst case upper bounds are derived on the minimum distance of parallel concatenated turbo codes serially concatenated convolutional codes repeat accumulate codes repeat convolute codes and generalizations of these codes obtained by allowing nonlinear and large memory constituent codes it is shown that parallel concatenated turbo codes and repeat convolute codes with sub linear memory are asymptotically bad it is also shown that depth two serially concatenated codes with constant memory outer codes and sublinear memory inner codes are asymptotically bad most of these upper bounds hold even when the convolutional encoders are replaced by general finite state automata encoders in contrast it is proven that depth three serially concatenated codes obtained by concatenating a repetition code with two accumulator codes through random permutations can be asymptotically good

finite length analysis of low density parity check codes on the binary erasure channel in this paper we are concerned with the finite length analysis of low density parity check ldpc codes when used over the binary erasure channel bec the main result is an expression for the exact average bit and block erasure probability for a given regular ensemble of ldpc codes when decoded iteratively we also give expressions for upper bounds on the average bit and block erasure probability for regular ldpc ensembles and the standard random ensemble under maximum likelihood ml decoding finally we present what we consider to be the most important open problems in this area

quasi cyclic ldpc codes an algebraic construction this paper presents two new large classes of qc ldpc codes one binary and one non binary codes in these two classes are constructed by array dispersions of row distance constrained matrices formed based on additive subgroups of finite fields experimental results show that codes constructed perform very well over the awgn channel with iterative decoding based on belief propagation codes of a subclass of the class of binary codes have large minimum distances comparable to finite geometry ldpc codes and they offer effective tradeoff between error performance and decoding complexity when decoded with low complexity reliability based iterative decoding algorithms such as binary message passing decoding algorithms non binary codes decoded with a fast fourier transform based sum product algorithm achieve significantly large coding gains over reed solomon codes of the same lengths and rates decoded with either the hard decision berlekamp massey algorithm or the algebraic soft decision kotter vardy algorithm they have potential to replace reed solomon codes in some communication or storage systems where combinations of random and bursts of errors or erasures occur

low floor decoders for ldpc codes one of the most significant impediments to the use of ldpc codes in many communication and storage systems is the error rate floor phenomenon associated with their iterative decoders the error floor has been attributed to certain subgraphs of an ldpc code s tanner graph induced by so called trapping sets we show in this paper that once we identify the trapping sets of an ldpc code of interest a sum product algorithm spa decoder can be custom designed to yield floors that are orders of magnitude lower than floors of the the conventional spa decoder we present three classes of such decoders 1 a bi mode decoder 2 a bit pinning decoder which utilizes one or more outer algebraic codes and 3 three generalized ldpc decoders we demonstrate the effectiveness of these decoders for two codes the rate 1 2 2640 1320 margulis code which is notorious for its floors and a rate 0 3 640 192 quasi cyclic code which has been devised for this study although the paper focuses on these two codes the decoder design techniques presented are fully generalizable to any ldpc code

an analysis of the block error probability performance of iterative decoding asymptotic iterative decoding performance is analyzed for several classes of iteratively decodable codes when the block length of the codes n and the number of iterations i go to infinity three classes of codes are considered these are gallager s regular low density parity check ldpc codes tanner s generalized ldpc gldpc codes and the turbo codes due to berrou et al it is proved that there exist codes in these classes and iterative decoding algorithms for these codes for which not only the bit error probability p sub b but also the block frame error probability p sub b goes to zero as n and i go to infinity

construction of regular and irregular ldpc codes geometry decomposition and masking two algebraic methods for systematic construction of structured regular and irregular low density parity check ldpc codes with girth of at least six and good minimum distances are presented these two methods are based on geometry decomposition and a masking technique numerical results show that the codes constructed by these methods perform close to the shannon limit and as well as random like ldpc codes furthermore they have low error floors and their iterative decoding converges very fast the masking technique greatly simplifies the random like construction of irregular ldpc codes designed on the basis of the degree distributions of their code graphs

design of ldpc decoders for improved low error rate performance quantization and algorithm choices many classes of high performance low density parity check ldpc codes are based on parity check matrices composed of permutation submatrices we describe the design of a parallel serial decoder architecture that can be used to map any ldpc code with such a structure to a hardware emulation platform high throughput emulation allows for the exploration of the low bit error rate ber region and provides statistics of the error traces which illuminate the causes of the error floors of the 2048 1723 reed solomon based ldpc rs ldpc code and the 2209 1978 array based ldpc code two classes of error events are observed oscillatory behavior and convergence to a class of non codewords termed absorbing sets the influence of absorbing sets can be exacerbated by message quantization and decoder implementation in particular quantization and the log tanh function approximation in sum product decoders strongly affect which absorbing sets dominate in the errorfloor region we show that conventional sum product decoder implementations of the 2209 1978 array based ldpc code allow low weight absorbing sets to have a strong effect and as a result elevate the error floor dually quantized sum product decoders and approximate sum product decoders alleviate the effects of low weight absorbing sets thereby lowering the error floor

lp decoding of regular ldpc codes in memoryless channels we study error bounds for linear programming decoding of regular low density parity check ldpc codes for memoryless binary input output symmetric channels we prove bounds on the word error probability that are inverse doubly exponential in the girth of the factor graph for memoryless binary input awgn channel we prove lower bounds on the threshold for regular ldpc codes whose factor graphs have logarithmic girth under lp decoding specifically we prove a lower bound of 0 735 upper bound of eb n0 2 67 db on the threshold of 3 6 regular ldpc codes whose factor graphs have logarithmic girth our proof is an extension of a recent paper of arora daskalakis and steurer stoc 2009 who presented a novel probabilistic analysis of lp decoding over a binary symmetric channel their analysis is based on the primal lp representation and has an explicit connection to message passing algorithms we extend this analysis to any mbios channel

error rate estimation of low density parity check codes on binary symmetric channels using cycle enumeration the performance of low density parity check ldpc codes decoded by hard decision iterative decoding algorithms can be accurately estimated if the weight j and the number ej of the smallest error patterns that cannot be corrected by the decoder are known to obtain j and ej one would need to perform the direct enumeration of error patterns with weight i les j the complexity of enumeration increases exponentially with j essentially as nj where n is the code block length this limits the application of direct enumeration to codes with small n and j in this letter we approximate j and ej by enumerating and testing the error patterns that are subsets of short cycles in the code s tanner graph this reduces the computational complexity by several orders of magnitude compared to direct enumeration making it possible to estimate the error rates for almost any practical ldpc code to obtain the error rate estimates we propose an algorithm that progressively improves the estimates as larger cycles are enumerated through a number of examples we demonstrate that the proposed method can accurately estimate both the bit error rate ber and the frame error rate fer of regular and irregular ldpc codes decoded by a variety of hard decision iterative decoding algorithms

improved bounds on the word error probability of ra 2 codes with linear programming based decoding this paper deals with the linear programming based decoding algorithm of feldman and karger for repeat accumulate turbo like codes we present a new structural characterization that captures the event that decoding fails based on this structural characterization we develop polynomial algorithms that given an ra 2 code compute upper and lower bounds on the word error probability p sub w for the binary symmetric and the additive white gaussian noise awgn channels our experiments with an implementation of these algorithms for bounding p sub w demonstrate in many interesting cases an improvement in the upper bound on the word error probability by a factor of over 1000 compared to the bounds by feldman et al the experiments also indicate that the improvement in upper bound increases as the codeword length increases and the channel noise decreases the computed lower bounds on the word error probability in our experiments are roughly ten times smaller than the upper bound

lp sdp hierarchy lower bounds for decoding random ldpc codes random d v d c regular low density parity check ldpc codes where each variable is involved in d v parity checks and each parity check involves d c variables are well known to achieve the shannon capacity of the binary symmetric channel for sufficiently large d v and d c under exponential time decoding however polynomial time algorithms are only known to correct a much smaller fraction of errors one of the most powerful polynomial time algorithms with a formal analysis is the linear programming lp decoding algorithm of feldman et al which is known to correct an omega 1 d c fraction of errors in this paper we show that fairly powerful extensions of lp decoding based on the sherali adams and lasserre hierarchies fail to correct much more errors than the basic lp decoder in particular we show that 1 for any values of d v and d c a linear number of rounds of the sherali adams lp hierarchy cannot correct more than an o 1 d c fraction of errors on a random d v d c regular ldpc code and 2 for any value of d v and infinitely many values of d c a linear number of rounds of the lasserre sdp hierarchy cannot correct more than an o 1 d c fraction of errors on a random d v d c regular ldpc code our proofs use a new stretching and collapsing technique that allows us to leverage recent progress in the study of the limitations of lp sdp hierarchies for maximum constraint satisfaction problems max csps the problem then reduces to the construction of special balanced pairwise independent distributions for sherali adams and special cosets of balanced pairwise independent subgroups for lasserre our algebraic construction for the lasserre hierarchy is based on designing sets of points in mathbb f q d for q any power of 2 and d 2 3 with special hyperplane incidence properties constructions that may be of independent interest an intriguing consequence of our work is that expansion seems to be both the strength and the weakness of random regular ldpc codes some of our techniques are more generally applicable to a large class of boolean csps called min ones in particular for k hypergraph vertex cover we obtain an improved integrality gap of k 1 epsilon that holds after a linear number of rounds of the lasserre hierarchy for any k q 1 with q an arbitrary prime power the best previous gap for a linear number of rounds was equal to 2 epsilon and due to schoenebeck

estimation of bit and frame error rates of finite length low density parity check codes on binary symmetric channels a method for estimating the performance of low density parity check ldpc codes decoded by hard decision iterative decoding algorithms on binary symmetric channels bscs is proposed based on the enumeration of the smallest weight error patterns that cannot be all corrected by the decoder this method estimates both the frame error rate fer and the bit error rate ber of a given ldpc code with very good precision for all crossover probabilities of practical interest through a number of examples we show that the proposed method can be effectively applied to both regular and irregular ldpc codes and to a variety of hard decision iterative decoding algorithms compared with the conventional monte carlo simulation the proposed method has a much smaller computational complexity particularly for lower error rates

quasi cyclic ldpc codes on cyclic subgroups of finite fields a new class of quasi cyclic ldpc codes whose parity check matrices are arrays of circulant permutation matrices constructed based on cyclic subgroups of finite fields is presented this class of codes contains several known classes of algebraic quasi cyclic ldpc codes as subclasses experimental results show that the codes constructed perform very well over the awgn channel when decoded with iterative decoding based on belief propagation this class of new qc ldpc codes contains a subclass of codes which have large minimum distances combinatorial expressions for the ranks of the parity check matrices of a subclass of codes constructed based on fields of characteristic two are given

on decoding irregular tanner codes with local optimality guarantees we consider decoding of binary linear tanner codes using message passing iterative decoding and linear programming lp decoding in memoryless binary input output symmetric mbios channels we present new certificates that are based on a combinatorial characterization for the local optimality of a codeword in irregular tanner codes with respect to any mbios channel this characterization is a generalization of arora proc acm symp theory of computing 2009 and vontobel proc inf theory and appl workshop 2010 and is based on a conical combination of normalized weighted subtrees in the computation trees of the tanner graph these subtrees may have any finite height h even equal or greater than half of the girth of the tanner graph in addition the degrees of local code nodes in these subtrees are not restricted to two i e these subtrees are not restricted to skinny trees we prove that local optimality in this new characterization implies maximum likelihood ml optimality and lp optimality and show that a certificate can be computed efficiently we also present a new message passing iterative decoding algorithm called normalized weighted min sum nwms nwms decoding is a belief propagation bp type algorithm that applies to any irregular binary tanner code with single parity check local codes e g low density and high density parity check codes we prove that if a locally optimal codeword with respect to height parameter h exists whereby notably h is not limited by the girth of the tanner graph then nwms decoding finds this codeword in h iterations the decoding guarantee of the nwms decoding algorithm applies whenever there exists a locally optimal codeword because local optimality of a codeword implies that it is the unique ml codeword the decoding guarantee also provides an ml certificate for this codeword finally we apply the new local optimality characterization to regular tanner codes and prove lower bounds on the noise thresholds of lp decoding in mbios channels when the noise is below these lower bounds the probability that lp decoding fails to decode the transmitted codeword decays doubly exponentially in the girth of the tanner graph

a characterization of the kernels associated with the multiple integral representation of some functional of the wiener process in this paper we present a characterization of those wiener functionals that are the likelihood ratio for a signal plus independent noise model the characterization is expressed in terms of the representation of such functionals in a series of multiple wiener integrals

a separation algorithm for improved lp decoding of linear block codes maximum likelihood ml decoding is the optimal decoding algorithm for arbitrary linear block codes and can be written as an integer programming ip problem feldman relaxed this ip problem and presented linear programming lp based decoding in this paper we propose a new separation algorithm to improve the error correcting performance of lp decoding for binary linear block codes we use an ip formulation with indicator variables that help in detecting the violated parity checks we derive gomory cuts from the ip and use them in our separation algorithm an efficient method of finding cuts induced by redundant parity checks rpc is also proposed under certain circumstances we can guarantee that these rpc cuts are valid and cut off the fractional optimal solutions of lp decoding it is demonstrated on three ldpc codes and two bch codes that our separation algorithm performs significantly better than lp decoding and belief propagation bp decoding

hilbert valued perturbed subgradient algorithms we propose a hilbert valued perturbed subgradient algorithm with stochastic noises and we provide a convergence proof for this algorithm under classical assumptions on the descent direction and new assumptions on the stochastic noises instead of requiring the stochastic noises to correspond to martingale increments we only require these noises to be asymptotically so furthermore the variance of these noises is allowed to grow infinitely under the control of a decreasing sequence linked with the subgradient stepsizes this algorithm can be used to solve stochastic closed loop control problems without any a priori discretization of the uncertainty such as linear decision rules or tree representations it can also be used as a way to perform stochastic dynamic programming without state space discretization or a priori functional bases i e approximate dynamic programming both problems arise frequently for example in power systems scheduling or option pricing this article focuses on the theorical foundations of the algorithm and gives references to detailed practical experimentations

error correction capability of column weight three ldpc codes in this paper the error correction capability of column weight three low density parity check ldpc codes when decoded using the gallager a algorithm is investigated it is proved that a necessary condition for a code to correct all error patterns with up to k ges 5 errors is to avoid cycles of length up to 2k in its tanner graph as a consequence of this result it is shown that given any alpha 0 exist n such that forall n n no code in the ensemble of column weight three codes can correct all alphan or fewer errors the results are extended to the bit flipping algorithms

expander graph arguments for message passing algorithms we show how expander based arguments may be used to prove that message passing algorithms can correct a linear number of erroneous messages the implication of this result is that when the block length is sufficiently large once a message passing algorithm has corrected a sufficiently large fraction of the errors it will eventually correct all errors this result is then combined with known results on the ability of message passing algorithms to reduce the number of errors to an arbitrarily small fraction for relatively high transmission rates the results hold for various message passing algorithms including gallager s hard decision and soft decision with clipping decoding algorithms our results assume low density parity check ldpc codes based on an irregular bipartite graph

efficient admm decoding of ldpc codes using lookup tables linear programming decoding with the alternating direction method of multipliers admm is a promising decoding technique for low density parity check ldpc codes where the computational complexity of euclidean projections onto check polytopes becomes a prominent problem in this paper the problem is circumvented by building lookup tables luts and quantizing the inputs to approach approximate euclidean projections at low computational complexities to challenge the huge memory cost of luts we first propose two commutative compositions of euclidean projection and self map and show the existence of a small quantization range which does not alter the euclidean projection then we investigate the design and simplification of the luts by exploiting the commutative compositions and check node decomposition techniques an efficient algorithm for the lut based projection is demonstrated by using one simplification method simulation results show that for both the regular and irregular ldpc codes the admm decoding using lut based projection can substantially reduce the decoding time while maintaining the error rate performance at a comparatively large memory cost

lp decoding corrects a constant fraction of errors we show that for low density parity check ldpc codes whose tanner graphs have sufficient expansion the linear programming lp decoder of feldman karger and wainwright can correct a constant fraction of errors a random graph will have sufficient expansion with high probability and recent work shows that such graphs can be constructed efficiently a key element of our method is the use of a dual witness a zero valued dual solution to the decoding linear program whose existence proves decoding success we show that as long as no more than a certain constant fraction of the bits are flipped by the channel we can find a dual witness this new method can be used for proving bounds on the performance of any lp decoder even in a probabilistic setting our result implies that the word error rate of the lp decoder decreases exponentially in the code length under the binary symmetric channel bsc this is the first such error bound for ldpc codes using an analysis based on pseudocodewords recent work by koetter and vontobel shows that lp decoding and min sum decoding of ldpc codes are closely related by the graph cover structure of their pseudocodewords in their terminology our result implies that that there exist families of ldpc codes where the minimum bsc pseudoweight grows linearly in the block length

dense error correction via ell 1 minimization this paper studies the problem of recovering a sparse signal x n from highly corrupted linear measurements y ax e m where e is an unknown error vector whose nonzero entries may be unbounded motivated by an observation from face recognition in computer vision this paper proves that for highly correlated and possibly overcomplete dictionaries a any sufficiently sparse signal x can be recovered by solving an l1 minimization problem min x 1 e 1 subject to y ax e more precisely if the fraction of the support of the error e is bounded away from one and the support of a is a very small fraction of the dimension m then as m becomes large the above l1 minimization succeeds for all signals x and almost all sign and support patterns of e this result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 of the observations corrupted the proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of independent identically distributed i i d gaussian vectors with nonzero mean and small variance dubbed the cross and bouquet cab model simulations and experiments corroborate the findings and suggest extensions to the result

high dimensional variable selection with sparse random projections measurement sparsity and statistical efficiency we consider the problem of high dimensional variable selection given n noisy observations of a k sparse vector rp estimate the subset of non zero entries of a significant body of work has studied behavior of l1 relaxations when applied to random measurement matrices that are dense e g gaussian bernoulli in this paper we analyze sparsified measurement ensembles and consider the trade off between measurement sparsity as measured by the fraction of non zero entries and the statistical efficiency as measured by the minimal number of observations n required for correct variable selection with probability converging to one our main result is to prove that it is possible to let the fraction on non zero entries 0 at some rate yielding measurement matrices with a vanishing fraction of non zeros per row while retaining the same statistical efficiency as dense ensembles a variety of simulation results confirm the sharpness of our theoretical predictions

iterative linear programming decoding of nonbinary ldpc codes with linear complexity the problem of low complexity linear programming lp decoding of nonbinary low density parity check ldpc codes is considered and an iterative lp decoding algorithm is presented results that were previously derived for binary ldpc codes are extended to the nonbinary case both simple and generalized nonbinary ldpc codes are considered it is shown how the algorithm can be implemented efficiently using a finite field fast fourier transform then the convergence rate of the algorithm is analyzed the complexity of the algorithm scales linearly in the block length and it can approximate up to an arbitrarily small relative error the objective function of the exact lp solution when applied to a typical code from an appropriate nonbinary ldpc code ensemble the algorithm can correct a constant fraction of errors in linear in the block length computational complexity computer experiments with the new iterative lp decoding algorithm show that in the error floor region it can have better performance compared to belief propagation decoding with similar computational requirements

on the construction of structured ldpc codes free of small trapping sets we present a method to construct low density parity check ldpc codes with low error floors on the binary symmetric channel codes are constructed so that their tanner graphs are free of certain small trapping sets these trapping sets are selected from the trapping set ontology for the gallager a b decoder they are selected based on their relative harmfulness for a given decoding algorithm we evaluate the relative harmfulness of different trapping sets for the sum product algorithm by using the topological relations among them and by analyzing the decoding failures on one trapping set in the presence or absence of other trapping sets we apply this method to construct structured ldpc codes to facilitate the discussion we give a new description of structured ldpc codes whose parity check matrices are arrays of permutation matrices this description uses latin squares to define a set of permutation matrices that have disjoint support and to derive a simple necessary and sufficient condition for the tanner graph of a code to be free of four cycles

convergence of weighted min sum decoding via dynamic programming on trees applying the max product and sum product algorithms to loopy graphs is now quite popular for best assignment problems this is largely due to their low computational complexity and impressive performance in practice still there is no general understanding of the conditions required for convergence or optimality of converged solutions or both this paper presents an analysis of both attenuated max product decoding and weighted min sum decoding for low density parity check ldpc codes which guarantees convergence to a fixed point when a weight factor is sufficiently small it also shows that if the fixed point satisfies some consistency conditions then it must be both a linear programming lp and maximum likelihood ml decoding solution for dv dc regular ldpc codes the weight factor must satisfy dv 1 1 are given finally connections are explored with recent work on the threshold of lp decoding

burst decoding of cyclic codes based on circulant parity check matrices an error burst correcting algorithm is developed based on a circulant parity check matrix of a cyclic code the proposed algorithm is more efficient than error trapping if the code rate is less than about 2 3 it is shown that for any n k cyclic code there is an n n circulant parity check matrix such that the algorithm applied to this matrix corrects error bursts of lengths up to the error burst correction limit of the cyclic code this same matrix can be used to efficiently correct erasure bursts of lengths up to n k the error burst correction capabilities of a class of cyclic low density parity check ldpc codes constructed from finite geometries are also considered

low density parity check codes based on finite geometries a rediscovery and new results this paper presents a geometric approach to the construction of low density parity check ldpc codes four classes of ldpc codes are constructed based on the lines and points of euclidean and projective geometries over finite fields codes of these four classes have good minimum distances and their tanner 1981 graphs have girth 6 finite geometry ldpc codes can be decoded in various ways ranging from low to high decoding complexity and from reasonably good to very good performance they perform very well with iterative decoding furthermore they can be put in either cyclic or quasi cyclic form consequently their encoding can be achieved in linear time and implemented with simple feedback shift registers this advantage is not shared by other ldpc codes in general and is important in practice finite geometry ldpc codes can be extended and shortened in various ways to obtain other good ldpc codes several techniques of extension and shortening are presented long extended finite geometry ldpc codes have been constructed and they achieve a performance only a few tenths of a decibel away from the shannon theoretical limit with iterative decoding

eliminating trapping sets in low density parity check codes by using tanner graph covers we discuss error floor asympotics and present a method for improving the performance of low density parity check ldpc codes in the high snr error floor region the method is based on tanner graph covers that do not have trapping sets from the original code the advantages of the method are that it is universal as it can be applied to any ldpc code channel decoding algorithm and it improves performance at the expense of increasing the code length without losing the code regularity without changing the decoding algorithm and under certain conditions without lowering the code rate the proposed method can be modified to construct convolutional ldpc codes also the method is illustrated by modifying tanner mackay and margulis codes to improve performance on the binary symmetric channel bsc under the gallager b decoding algorithm decoding results on awgn channel are also presented to illustrate that optimizing codes for one channel decoding algorithm can lead to performance improvement on other channels

two reliability based iterative majority logic decoding algorithms for ldpc codes this paper presents two novel reliability based iterative majority logic decoding algorithms for ldpc codes both algorithms are binary message passing algorithms and require only logical operations and integer additions consequently they can be implemented with simple combinational logic circuits they either outperform or perform just as well as the existing weighted bit flipping or other reliability based iterative decoding algorithms for ldpc codes in error performance with a faster rate of decoding convergence and less decoding complexity compared to the sum product algorithm for ldpc codes they offer effective trade offs between performance and decoding complexity

linear programming decoding of spatially coupled codes for a given family of spatially coupled codes we prove that the linear programming lp threshold on the binary symmetric channel bsc of the tail biting graph cover ensemble is the same as the lp threshold on the bsc of the derived spatially coupled ensemble this result is in contrast with the fact that spatial coupling significantly increases the belief propagation threshold to prove this we establish some properties related to the dual witness for lp decoding more precisely we prove that the existence of a dual witness which was previously known to be sufficient for lp decoding success is also necessary and is equivalent to the existence of certain acyclic hyperflows we also derive a sublinear in the block length upper bound on the weight of any edge in such hyperflows both for regular low density parity check lpdc codes and spatially coupled codes and we prove that the bound is asymptotically tight for regular ldpc codes moreover we show how to trade crossover probability for lp excess on all the variable nodes for any binary linear code