design of capacity approaching irregular low density parity check codes we design low density parity check ldpc codes that perform at rates extremely close to the shannon capacity the codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides our theoretical analysis of the codes is based on the work of richardson and urbanke see ibid vol 47 no 2 p 599 618 2000 assuming that the underlying communication channel is symmetric we prove that the probability densities at the message nodes of the graph possess a certain symmetry using this symmetry property we then show that under the assumption of no cycles the message densities always converge as the number of iterations tends to infinity furthermore we prove a stability condition which implies an upper bound on the fraction of errors that a belief propagation decoder can correct when applied to a code induced from a bipartite graph with a given degree distribution our codes are found by optimizing the degree structure of the underlying graphs we develop several strategies to perform this optimization we also present some simulation results for the codes found which show that the performance of the codes is very close to the asymptotic theoretical bounds

the generalized singular value decomposition and the method of particular solutions a powerful method for solving planar eigenvalue problems is the method of particular solutions mps which is also well known under the name point matching method the implementation of this method usually depends on the solution of one of three types of linear algebra problems singular value decomposition generalized eigenvalue decomposition or generalized singular value decomposition we compare and give geometric interpretations of these different variants of the mps it turns out that the most stable and accurate of them is based on the generalized singular value decomposition we present results to this effect and demonstrate the behavior of the generalized singular value decomposition in the presence of a highly ill conditioned basis of particular solutions

refined iterative algorithms based on arnoldi s process for large unsymmetric eigenproblems abstract arnoldi s method has been popular for computing the small number of selected eigenvalues and the associated eigenvectors of large unsymmetric matrices however the approximate eigenvectors or ritz vectors obtained by arnoldi s method cannot be guaranteed to converge in theory even if the approximate eigenvalues or ritz values do in order to circumvent this potential danger a new strategy is proposed that computes refined approximate eigenvectors by small sized singular value decompositions it is shown that refined approximate eigenvectors converge to eigenvectors if ritz values do moreover the resulting refined algorithms converge more rapidly we report some numerical experiments and compare the refined algorithms with their counterparts the iterative arnoldi and arnoldi chebyshev algorithms the results show that the refined algorithms are considerably more efficient than their counterparts

rate compatible puncturing of low density parity check codes in this correspondence we consider puncturing of low density parity check ldpc codes for additive white gaussian noise awgn channels we show that good puncturing patterns exist and that the puncturing can be performed in a rate compatible fashion furthermore rate compatible puncturing results in a small loss of performance with respect to threshold namely the punctured code is good in terms of threshold across a range of rates when compared with the optimal codes for each rate this allows one to implement a single mother encoder and decoder that is good across a wide range of rates

a class of quasi cyclic ldpc codes over gf 2 m low density parity check ldpc codes over gf 2m are an extension of binary ldpc codes performances of gf 2m ldpc codes have been shown to be higher than binary ldpc codes but the complexity of the encoders decoders increases hence there is a substantial lack of implementations for ldpc codes over gf 2m codes this paper presents a class of quasi cyclic ldpc codes over gf 2m these codes can alleviate the encoding decoding complexity without excessive loss of performances it is shown how from a performance point of view such codes are better than m times bigger binary codes and as good as 2m longer binary codes

self compensation technique for simplified belief propagation algorithm the min sum algorithm is the most common method to simplify the belief propagation algorithm for decoding low density parity check ldpc codes however there exists a performance gap between the min sum and belief propagation algorithms due to nonlinear approximation in this paper a self compensation technique using dynamic normalization is thus proposed to improve the approximation accuracy the proposed scheme scales the min sum algorithm by a dynamic factor that can be derived theoretically from order statistics moreover applying the proposed technique to several ldpc codes for dvb s2 system the average signal to noise ratio degradation which results from approximation inaccuracy and quantization error is reduced to 0 2 db not only does it enhance the error correcting capability of the min sum algorithm but the proposed self compensation technique also preserves a modest hardware cost after realized with 0 13 mum standard cell library the dynamic normalization requires about 100 additional gates for each check node unit in the min sum algorithm

towards a generalized singular value decomposition we suggest a form for and give a constructive derivation of the generalized singular value decomposition of any two matrices having the same number of columns we outline its desirable characteristics and compare it to an earlier suggestion by van loan siam j numer anal 13 1976 pp 76 83 the present form largely follows from the work of van loan but is slightly more general and computationally more amenable than that in the paper cited we also prove a useful extension of a theorem of stewart siam rev 19 1977 pp 634 662 on unitary decompositions of submatrices of a unitary matrix

efficient erasure correcting codes we introduce a simple erasure recovery algorithm for codes derived from cascades of sparse bipartite graphs and analyze the algorithm by analyzing a corresponding discrete time random process as a result we obtain a simple criterion involving the fractions of nodes of different degrees on both sides of the graph which is necessary and sufficient for the decoding process to finish successfully with high probability by carefully designing these graphs we can construct for any given rate r and any given real number spl epsiv a family of linear codes of rate r which can be encoded in time proportional to ln 1 spl epsiv times their block length n furthermore a codeword can be recovered with high probability from a portion of its entries of length 1 spl epsiv rn or more the recovery algorithm also runs in time proportional to n ln 1 spl epsiv our algorithms have been implemented and work well in practice various implementation issues are discussed

anytime reliable transmission of real valued information through digital noisy channels the problem of reliably transmitting a real valued random vector through a digital noisy channel is relevant for the design of distributed estimation and control techniques over networked systems one important example consists in the remote state estimation under communication constraints in this case an anytime transmission scheme consists of an encoder which maps the real vector into a sequence of channel inputs and a decoder which sequentially updates its estimate of the vector as more and more channel outputs are observed the encoder performs both source and channel coding of the data assuming that no channel feedback is available at the transmitter this paper studies the rates of convergence to zero of the mean squared error two coding strategies are analyzed the first one has exponential convergence rate but is expensive in terms of its encoder decoder computational complexity while the second one has a convenient computational complexity but subexponential convergence rate general bounds are obtained describing the convergence properties of these classes of methods

construction of nonbinary cyclic quasi cyclic and regular ldpc codes a finite geometry approach this paper presents five methods for constructing nonbinary ldpc codes based on finite geometries these methods result in five classes of nonbinary ldpc codes one class of cyclic ldpc codes three classes of quasi cyclic ldpc codes and one class of structured regular ldpc codes experimental results show that constructed codes in these classes decoded with iterative decoding based on belief propagation perform very well over the awgn channel and they achieve significant coding gains over reed solomon codes of the same lengths and rates with either algebraic hard decision decoding or kotter vardy algebraic soft decision decoding at the expense of a larger decoding computational complexity

singular value and generalized singular value decompositions and the solution of linear matrix equations abstract the solution of the linear matrix equations i axb cyd e and ii axb fxg e h are considered new necessary and sufficient conditions for the consistency of the equations are derived some using the generalized singular value decomposition special cases iii ax yd e and iv axb e are treated using the singular value decomposition numerical algorithms for the solutions are also suggested

asymptotic results on modular representations of symmetric groups and almost simple modular group algebras in fact this is a question about infinite simple groups because it is easy for g finite and because a non trivial normal subgroup gives rise to a non trivial ideal different from the augmentation ideal also note that the problem reduces easily to the question on when the augmentation ideal is simple as a ring the first interesting class of groups with almost simple fg was discovered in 3 this class is rather exotic and contains groups like the universal hall group and algebraically closed groups for locally finite groups g a substantial progress was achieved recently by using representation theory of finite groups see 18 19 16 and others the representation theory approach transforms the problems on ideals to certain problems on asymptotic behavior of representations of finite groups which are often of independent interest the method is more effective over fields of characteristic zero as the theory of ordinary representations is much better elaborated than the modular theory this paper is devoted to the modular aspect of the theory one of our main results deals with the case where g is a direct limit of alternating groups and f is any field of characteristic p 2 we note that the case where charf 0 was considered in 18 let n be the set of natural numbers denote by alt and sym or simply an and n the alternating and symmetric groups respectively on a set with n let alt 1 alt 2 alt i 1

systematic ldpc convolutional codes asymptotic and finite length anytime properties here we propose an ensemble of non terminated systematic ldpc convolutional codes with increasing memory and show that over the binary erasure channel bec these codes achieve anytime reliability asymptotically when decoded with an expanding window message passing decoder the corresponding anytime exponents are determined through protograph based extrinsic information transfer charts fundamental complications arising when transmitting with finite block lengths are identified and a combinatorial performance analysis when transmitting over a static bec with a fixed number of erasures per codeword block is developed based on the performance analysis we explore the use of feedback for achieving anytime behavior with constraints on block length to meet complexity constraints with or without feedback the code memory can be limited at the cost of an error floor emerging with a delay proportional to the memory constraint although the analysis is developed for a static bec we show numerically that we can design efficient low complexity finite length codes with anytime properties even for the conventional bec

windowed decoding of spatially coupled codes spatially coupled codes have been of interest recently owing to their superior performance over memoryless binary input channels the performance is good both asymptotically since the belief propagation thresholds approach the shannon limit as well as for finite lengths since degree 2 variable nodes that result in high error floors can be completely avoided however to realize the promised good performance one needs large blocklengths this in turn implies a large latency and decoding complexity for the memoryless binary erasure channel we consider the decoding of spatially coupled codes through a windowed decoder that aims to retain many of the attractive features of belief propagation while trying to reduce complexity further we characterize the performance of this scheme by defining thresholds on channel erasure rates that guarantee a target erasure rate we give analytical lower bounds on these thresholds and show that the performance approaches that of belief propagation exponentially fast in the window size we give numerical results including the thresholds computed using density evolution and the erasure rate curves for finite length spatially coupled codes

expander codes using expander graphs we construct a new family of asymptotically good linear error correcting codes these codes have linear time sequential decoding algorithms and logarithmic time parallel decoding algorithms that use a linear number of processors we present both randomized and explicit constructions of these codes experimental results demonstrate the good performance of the randomly chosen codes

approximating the llr distribution for a class of soft output mimo detectors we present approximations of the llr distribution for a class of fixed complexity soft output mimo detectors such as the optimal soft detector and the soft output via partial marginalization detector more specifically in a mimo awgn setting we approximate the llr distribution conditioned on the transmitted signal and the channel matrix with a gaussian mixture model gmm our main results consist of an analytical expression of the gmm model including the number of modes and their corresponding parameters and a proof that in the limit of high snr this llr distribution converges in probability towards a unique gaussian distribution

approaching the rate distortion limit with spatial coupling belief propagation and decimation we investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled low density generator matrix codes the degree of the check nodes is regular and the one of code bits is poisson distributed with an average depending on the compression rate the performance of a low complexity belief propagation guided decimation algorithm is excellent the algorithmic rate distortion curve approaches the optimal curve of the ensemble as the width of the coupling window grows moreover as the check degree grows both curves approach the ultimate shannon rate distortion limit the belief propagation guided decimation encoder is based on the posterior measure of a binary symmetric test channel this measure can be interpreted as a random gibbs measure at a temperature directly related to the noise level of the test channel we investigate the links between the algorithmic performance of the belief propagation guided decimation encoder and the phase diagram of this gibbs measure the phase diagram is investigated thanks to the cavity method of spin glass theory which predicts a number of phase transition thresholds in particular the dynamical and condensation phase transition temperatures equivalently test channel noise thresholds are computed we observe that 1 the dynamical temperature of the spatially coupled construction saturates toward the condensation temperature and 2 for large degrees the condensation temperature approaches the temperature i e noise level related to the information theoretic shannon test channel noise parameter of rate distortion theory this provides heuristic insight into the excellent performance of the belief propagation guided decimation algorithm this paper contains an introduction to the cavity method

maximizing the spectral efficiency of coded cdma under successive decoding we investigate the spectral efficiency achievable by random synchronous code division multiple access cdma with quaternary phase shift keying qpsk modulation and binary error control codes in the large system limit where the number of users the spreading factor and the code block length go to infinity for given codes we maximize spectral efficiency assuming a minimum mean square error mmse successive stripping decoder for the cases of equal rate and equal power users in both cases the maximization of spectral efficiency can be formulated as a linear program and admits a simple closed form solution that can be readily interpreted in terms of power and rate control we provide examples of the proposed optimization methods based on off the shelf low density parity check ldpc codes and we investigate by simulation the performance of practical systems with finite code block length

analysis of sum product decoding of low density parity check codes using a gaussian approximation density evolution is an algorithm for computing the capacity of low density parity check ldpc codes under message passing decoding for memoryless binary input continuous output additive white gaussian noise awgn channels and sum product decoders we use a gaussian approximation for message densities under density evolution to simplify the analysis of the decoding algorithm we convert the infinite dimensional problem of iteratively calculating message densities which is needed to find the exact threshold to a one dimensional problem of updating the means of the gaussian densities this simplification not only allows us to calculate the threshold quickly and to understand the behavior of the decoder better but also makes it easier to design good irregular ldpc codes for awgn channels for various regular ldpc codes we have examined thresholds can be estimated within 0 1 db of the exact value for rates between 0 5 and 0 9 codes designed using the gaussian approximation perform within 0 02 db of the best performing codes found so far by using density evolution when the maximum variable degree is 10 we show that by using the gaussian approximation we can visualize the sum product decoding algorithm we also show that the optimization of degree distributions can be understood and done graphically using the visualization

anytime characteristics of protograph based ldpc convolutional codes anytime transmission has been shown to be an effective means of stabilizing an unstable system over noisy channels a crucial issue in implementing anytime transmission strategies is the design of anytime codes that offer good finite length performance and asymptotic anytime properties this paper introduces an efficient anytime code derived from protograph based low density parity check convolutional codes known as extended s ldpcc xs ldpcc codes the asymptotic anytime properties of xs ldpcc codes over both the binary erasure channel bec and the binary input additive white gaussian noise biawgn channel are demonstrated through density evolution by means of certain observations verified by simulations we provide accurate estimates of the delay exponents of xs ldpcc codes over the bec and the biawgn channel through simulations the finite length performance of xs ldpcc codes and the effects of different code design parameters are quantified

analysis of the iterative decoding of ldpc and product codes using the gaussian approximation we propose a novel density evolution approach to analyze the iterative decoding algorithms of low density parity check ldpc codes and product codes based on gaussian densities namely for these classes of codes we derive a one dimensional 1d map whose iterates directly represent the error probability both for the additive white gaussian noise awgn and the rayleigh fading channel these simple models allow a qualitative analysis of the nonlinear dynamics of the decoding algorithm as an application we compute the decoding thresholds and show that they are consistent with the simulation results available in the literature

anytime reliability of spatially coupled codes anytime transmission was previously shown to be necessary and sufficient for tracking and controlling an unstable plant over a noisy channel in this paper we propose an efficient channel coding scheme for anytime transmission we design this coding scheme based on spatially coupled codes and investigate its performance over binary erasure channels bec and binary input additive white gaussian noise biawgn channels through density evolution analysis we asymptotically show the desired anytime properties of the proposed code for the bec we derive the corresponding delay exponent and numerically predict the finite length performance of the proposed anytime code we also propose adaptive window decoding techniques to ensure low complexity anytime receivers

design of regular 2 d sub c ldpc codes over gf q using their binary images in this paper a method to design regular 2 dc ldpc codes over gf q with both good waterfall and error floor properties is presented based on the algebraic properties of their binary image first the algebraic properties of rows of the parity check matrix h associated with a code are characterized and optimized to improve the waterfall then the algebraic properties of cycles and stopping sets associated with the underlying tanner graph are studied and linked to the global binary minimum distance of the code finally simulations are presented to illustrate the excellent performance of the designed codes

scalar estimation and control with noisy binary observations we consider a simple one dimensional system whose observations are sent to a state estimator over a noisy binary communication link the interesting thing about the system is that it is unstable the problem is to design an encoding scheme and a decoder such that the estimation error is stable we explicitly construct a simple but efficient estimator for the binary symmetric channel bsc we are not aware of any such previous codes for the bsc we compare our results to the nonconstructive bounds of sahai

ldpc block and convolutional codes based on circulant matrices a class of algebraically structured quasi cyclic qc low density parity check ldpc codes and their convolutional counterparts is presented the qc codes are described by sparse parity check matrices comprised of blocks of circulant matrices the sparse parity check representation allows for practical graph based iterative message passing decoding based on the algebraic structure bounds on the girth and minimum distance of the codes are found and several possible encoding techniques are described the performance of the qc ldpc block codes compares favorably with that of randomly constructed ldpc codes for short to moderate block lengths the performance of the ldpc convolutional codes is superior to that of the qc codes on which they are based this performance is the limiting performance obtained by increasing the circulant size of the base qc code finally a continuous decoding procedure for the ldpc convolutional codes is described

recursive identification of fir systems with binary valued outputs and communication channels this paper investigates the recursive identification of the finite impulse response systems whose outputs are subject to both the binary valued quantization and the communication unreliability the discrete memoryless channel with packet losses and the additive white gaussian noise channel with time delay are considered a stochastic gradient type algorithm and a stochastic approximation type algorithm are respectively proposed to estimate the unknown system parameters under some mild conditions the algorithms can be proved to be convergent in the almost sure and mean square senses also the convergence rates are given numerical examples are included to illustrate the performance of the algorithms

distributed averaging on digital erasure networks iterative distributed algorithms are studied for computing arithmetic averages over networks of agents connected through memoryless broadcast erasure channels these algorithms do not require the agents to have any knowledge about the global network structure or size almost sure convergence to state agreement is proved and the communication and computational complexities of the algorithms are analyzed both the number of transmissions and the number of computations performed by each agent of the network are shown to grow not faster than poly logarithmically in the desired precision the impact of the graph topology on the algorithms performance is analyzed as well moreover it is shown how in the presence of noiseless communication feedback one can modify the algorithms significantly improving their performance versus complexity trade off

block markov superposition transmission construction of big convolutional codes from short codes a construction of big convolutional codes from short codes called block markov superposition transmission bmst is proposed the bmst is very similar to superposition block markov encoding sbme which has been widely used to prove multiuser coding theorems the bmst codes can also be viewed as a class of spatially coupled codes where the generator matrices of the involved short codes referred to as basic codes are coupled the encoding process of bmst can be as fast as that of the basic code while the decoding process can be implemented as an iterative sliding window decoding algorithm with a tunable delay more importantly the performance of bmst can be simply lower bounded in terms of the transmission memory given that the performance of the short code is available numerical results show that 1 the lower bounds can be matched with a moderate decoding delay in the low bit error rate ber region implying that the iterative sliding window decoding algorithm is near optimal 2 bmst with repetition codes and single parity check codes can approach the shannon limit within 0 5 db at the ber of 10 5 for a wide range of code rates and 3 bmst can also be applied to nonlinear codes

ldpc coded mimo multiple access with iterative joint decoding an efficient scheme for the multiple access multiple input multiple output mimo channel is proposed which operates well also in the single user regime as well as in a direct sequence spread spectrum ds cdma setting the design features scalability and is of limited complexity the system employs optimized low density parity check ldpc codes and an efficient iterative belief propagation bp detection which combines linear minimum mean square error lmmse detection and iterative interference cancellation ic this combination is found to be necessary for efficient operation in high system loads spl alpha 1 an asymptotic density evolution de is used to optimize the degree polynomials of the underlining ldpc code and thresholds as close as 0 77 db to the channel capacity are evident for a system load of 2 replacing the lmmse with the complex individually optimal multiuser detector io mud further improves the performance up to 0 14 db from the capacity comparing the thresholds of a good single user ldpc code to the multiuser optimized ldpc code both over the above multiuser channel reveals a surprising 8 db difference emphasizing thus the necessity of optimizing the code the asymptotic analysis of the proposed scheme is verified by simulations of finite systems which reveal meaningful differences between the performances of mimo systems with single and multiple users and demonstrate performance similar to previously reported techniques but with higher system loads and significantly lower receiver complexity

waterfall performance analysis of finite length ldpc codes on symmetric channels an efficient method for analyzing the performance of finite length low density parity check ldpc codes in the waterfall region when transmission takes place on a memoryless binary input output symmetric channel is proposed this method is based on studying the variations of the channel quality around its expected value when observed during the transmission of a finite length codeword we model these variations with a single parameter this parameter is then viewed as a random variable and its probability distribution function is obtained assuming that a decoding failure is the result of an observed channel worse than the codeiquests decoding threshold the block error probability of finite length ldpc codes under different decoding algorithms is estimated using an extrinsic information transfer chart analysis the bit error probability is obtained from the block error probability different parameters can be used for modeling the channel variations in this work two of such parameters are studied through examples it is shown that this method can closely predict the performance of ldpc codes of a few thousand bits or longer in the waterfall region

optimized design of finite length separable circulant based spatially coupled codes an absorbing set based analysis in this paper we characterize the finite length performance of separable circulant based spatially coupled scb sc ldpc codes for transmission over the additive white gaussian noise channel for a general class of finite length graph based codes it is known that the existence of small absorbing sets causes a performance degradation in the error floor regime we first present the mathematical conditions for the existence of absorbing sets in binary scb sc codes this analysis enables us to find the exact number of absorbing sets as a function of the design parameters in particular our results show that the choice of the cutting vector affects the number of absorbing sets and therefore the error floor performance of the code for a fixed column weight we find provably optimal cutting vectors that result in the least number of absorbing sets furthermore we extend our analysis to nonbinary scb sc codes where we show that the choice of the cutting vector is not as critical as in the binary case we provide an algorithm which provably removes the problematic nonbinary absorbing sets from nonbinary scb sc codes by informed selection of edge labels our simulation results show the superior error floor performance of our designed binary and nonbinary scb sc codes compared with binary unstructured and nonbinary quasi cyclic sc codes available in the open literature

iterative decoding threshold analysis for ldpc convolutional codes an iterative decoding threshold analysis for terminated regular ldpc convolutional ldpcc codes is presented using density evolution techniques the convergence behavior of an iterative belief propagation decoder is analyzed for the binary erasure channel and the awgn channel with binary inputs it is shown that for a terminated ldpcc code ensemble the thresholds are better than for corresponding regular and irregular ldpc block codes

a jacobi davidson type svd method we discuss a new method for the iterative computation of a portion of the singular values and vectors of a large sparse matrix similar to the jacobi davidson method for the eigenvalue problem we compute in each step a correction by approximately solving a correction equation we give a few variants of this jacobi davidson svd jdsvd method with their theoretical properties it is shown that the jdsvd can be seen as an accelerated inexact newton scheme we experimentally compare the method with some other iterative svd methods

an analysis of the block error probability performance of iterative decoding asymptotic iterative decoding performance is analyzed for several classes of iteratively decodable codes when the block length of the codes n and the number of iterations i go to infinity three classes of codes are considered these are gallager s regular low density parity check ldpc codes tanner s generalized ldpc gldpc codes and the turbo codes due to berrou et al it is proved that there exist codes in these classes and iterative decoding algorithms for these codes for which not only the bit error probability p sub b but also the block frame error probability p sub b goes to zero as n and i go to infinity

on the decomposition method for linear programming decoding of ldpc codes in this paper we focus on solving the linear programming lp problem that arises in the decoding of low density parity check ldpc codes by means of the revised simplex method in order to take advantage of the structure of the lp problem we reformulate the dual lp and apply the idea of dantzig wolfe d w decomposition method to solve the problem each subproblem in the d w decomposition method is an lp over a convex polyhedral cone we define the convex polyhedral cone as local parity check cone and discuss its special structures then we enumerate its extreme rays and use these extreme rays to design an efficient method for the general lp decoding problem the proposed method exhibits advantages in reducing both the storage and computational requirements

selective avoidance of cycles in irregular ldpc code construction this letter explains the effect of graph connectivity on error floor performance of low density parity check ldpc codes under message passing decoding a new metric called extrinsic message degree emd measures cycle connectivity in bipartite graphs of ldpc codes using an easily computed estimate of emd we propose a viterbi like algorithm that selectively avoids small cycle clusters that are isolated from the rest of the graph this algorithm is different from conventional girth conditioning by emphasizing the connectivity as well as the length of cycles the algorithm yields codes with error floors that are orders of magnitude below those of random codes with very small degradation in capacity approaching capability

systematic design of low density parity check code ensembles for binary erasure channels we propose a systematic method to design irregular low density parity check ldpc codes for binary erasure channels bec compared to the existing methods which are based on the application of asymptotic analysis tools such as density evolution or extrinsic information transfer exit charts in an optimization process the proposed method is much simpler and faster through a number of examples we demonstrate that the codes designed by the proposed method perform very closely to the best codes designed by optimization an important property of the proposed designs is the flexibility to select the number of constituent variable node degrees p the proposed designs include existing systematic designs as a special case with p n 1 where n is the maximum variable node degree compared to the existing systematic designs for a given rate and a given 0 the designed ensembles can have a threshold in neighborhood of the capacity upper bound with smaller values of p and n they can also achieve the capacity of the bec as n and correspondingly p and the maximum check node degree tend to infinity

lp decoding of regular ldpc codes in memoryless channels we study error bounds for linear programming decoding of regular low density parity check ldpc codes for memoryless binary input output symmetric channels we prove bounds on the word error probability that are inverse doubly exponential in the girth of the factor graph for memoryless binary input awgn channel we prove lower bounds on the threshold for regular ldpc codes whose factor graphs have logarithmic girth under lp decoding specifically we prove a lower bound of 0 735 upper bound of eb n0 2 67 db on the threshold of 3 6 regular ldpc codes whose factor graphs have logarithmic girth our proof is an extension of a recent paper of arora daskalakis and steurer stoc 2009 who presented a novel probabilistic analysis of lp decoding over a binary symmetric channel their analysis is based on the primal lp representation and has an explicit connection to message passing algorithms we extend this analysis to any mbios channel

regular and irregular progressive edge growth tanner graphs we propose a general method for constructing tanner graphs having a large girth by establishing edges or connections between symbol and check nodes in an edge by edge manner called progressive edge growth peg algorithm lower bounds on the girth of peg tanner graphs and on the minimum distance of the resulting low density parity check ldpc codes are derived in terms of parameters of the graphs simple variations of the peg algorithm can also be applied to generate linear time encodeable ldpc codes regular and irregular ldpc codes using peg tanner graphs and allowing symbol nodes to take values over gf q q 2 are investigated simulation results show that the peg algorithm is a powerful algorithm to generate good short block length ldpc codes

hybrid optical rf channels characterization and performance study using low density parity check codes a hybrid channel consisting of a free space optical fso link and a parallel radio frequency rf link is considered the fso link carries most of the transmitted data while leaving only a small fraction of the data to be carried by the rf link it is first shown that the capacity of the proposed hybrid structure is much higher than the capacity of a single fso link next the performance of the hybrid channel is studied using a low density parity check code a density evolution strategy for the hybrid channel is derived and a gaussian approximation technique is presented using these techniques the conditions for the convergence of the message passing algorithm in terms of minimum data carrying rate through the rf link are derived finally numerical results showing the benefits of the hybrid channel under various channel conditions are presented

finite length scaling for iteratively decoded ldpc ensembles we investigate the behavior of iteratively decoded low density parity check ldpc codes over the binary erasure channel in the so called ldquowaterfall region rdquo we show that the performance curves in this region follow a simple scaling law we conjecture that essentially the same scaling behavior applies in a much more general setting and we provide some empirical evidence to support this conjecture the scaling law together with the error floor expressions developed previously can be used for a fast finite length optimization

iterative decoding of binary block and convolutional codes iterative decoding of two dimensional systematic convolutional codes has been termed turbo de coding using log likelihood algebra we show that any decoder can be used which accepts soft inputs including a priori values and delivers soft outputs that can be split into three terms the soft channel and a priori inputs and the extrinsic value the extrinsic value is used as an a priori value for the next iteration decoding algorithms in the log likelihood domain are given not only for convolutional codes but also for any linear binary systematic block code the iteration is controlled by a stop criterion derived from cross entropy which results in a minimal number of iterations optimal and suboptimal decoders with reduced complexity are presented simulation results show that very simple component codes are sufficient block codes are appropriate for high rates and convolutional codes for lower rates less than 2 3 any combination of block and convolutional component codes is possible several interleaving techniques are described at a bit error rate ber of 10 sup 4 the performance is slightly above or around the bounds given by the cutoff rate for reasonably simple block convolutional component codes interleaver sizes less than 1000 and for three to six iterations

polar coded modulation a framework is proposed that allows for a joint description and optimization of both binary polar coding and 2m ary digital pulse amplitude modulation pam schemes for the latter the multilevel coding mlc approach as well as bit interleaved coded modulation bicm are considered the conceptual equivalence of polar coding and multilevel coding is covered in detail based on an alternative characterization of the channel polarization phenomenon rules for the optimum choice of the labeling in coded modulation schemes employing polar codes are developed simulation results regarding the error performance of the proposed schemes on the awgn channel are included

iterative multiuser joint decoding optimal power allocation and low complexity implementation we consider a canonical model for coded code division multiple access cdma with random spreading where the receiver makes use of iterative belief propagation bp joint decoding we provide simple density evolution analysis in the large system limit large number of users of the performance of the bp decoder and of some suboptimal approximations based on interference cancellation ic based on this analysis we optimize the received user signal to noise ratio snr distribution in order to maximize the system spectral efficiency for given user channel codes channel load users per chip and target user bit error rate ber the optimization of the received snr distribution is obtained by solving a simple linear program and can be easily incorporated into practical power control algorithms remarkably under the optimized snr assignment the suboptimal minimum mean square error mmse ic based decoder performs almost as well as the more complex bp decoder moreover for a large class of commonly used convolutional codes we observe that the optimized snr distribution consists of a finite number of discrete snr levels based on this observation we provide a low complexity approximation of the mmse ic decoder that suffers from very small performance degradation while attaining considerable savings in complexity as by products of this work we obtain a closed form expression of the multiuser efficiency me of power mismatched mmse filters in the large system limit and we extend the analysis of the symbol by symbol maximum a posteriori probability map multiuser detector in the large system limit to the case of nonconstant user powers and nonuniform symbol prior probabilities

maximum likelihood laplacian correlation channel estimation in layered wyner ziv coding the distributed source coding theory teaches that efficient compression can be achieved by exploiting the correlation between encoded sources at the decoder only a critical issue in this coding paradigm is how to accurately estimate the correlation statistics at the decoder this paper proposes a novel correlation channel estimation method designed for generic layered wyner ziv wz coding driven by real life applications i e video coding the correlation noise is assumed to be memoryless zero mean laplacian with a block stationary behavior unlike existing methods the proposed technique derives a novel maximum likelihood estimate of the noise scaling parameter per stationarity block adhering to layered coding the derived estimate is successively refined as more stages are decoded the cramer rao lower bound crlb for our estimator is derived and the mean squared error performance of the proposed successive refinement algorithm is evaluated through simulations apart from achieving an accuracy close to the derived crlb the proposed algorithm is shown to yield a wz rate distortion performance approaching the ideal but unrealistic scenario where the decoder knows a priori the correlation statistics moreover the proposed technique has been integrated into our latest wz video codec for wireless capsule endoscopy experimentation reveals that the proposed method delivers improved compression performance compared to state of the art techniques while simultaneously reducing the overall decoding complexity

time invariant quasi cyclic spatially coupled ldpc codes based on packings this paper presents two packings derived from balanced incomplete block designs to construct quasi cyclic spatially coupled ldpc convolutional codes sc ldpc ccs the construction gives time invariant codes since the sub blocks corresponding to each time instant of the parity check matrix are identical moreover it provides flexible design rates and constraint lengths simulation results show that the proposed packing based sc ldpc ccs outperform the existing time invariant codes and perform closely to the time varying protograph based codes

successive refinement for the wyner ziv problem and layered code design we examine successive refinement for the wyner ziv problem described in a recent paper by steinberg and merhav where the authors showed that if the side information for all stages is identical then the jointly gaussian source with squared error distortion measure is successively refinable we first extend this result to the case where the difference between the source and the side information is gaussian and independent of the side information as a byproduct we give an alternative proof that the wyner ziv problem for these sources has no rate loss a result that was recently shown by pradhan et al through invoking the duality between the gaussian wyner ziv problem and the costa problem we then perform layered wyner ziv code design for this general type of source based on nested scalar quantization bit plane coding and low density parity check ldpc code based slepian wolf coding source coding with side information we show that density evolution can be used to analyze the slepian wolf code performance provided that certain symmetry conditions which have been dubbed dual symmetry are satisfied by the hypothetical channel between the source and the side information we also show that the dual symmetry condition is indeed satisfied by the hypothetical channel in our slepian wolf coding setup this justifies the use of density evolution in our ldpc code based slepian wolf code design for wyner ziv coding for the jointly gaussian source our layered coder performs 1 29 to 3 45 db from the wyner ziv bound for rates ranging from 0 47 to 4 68 bits per sample when the side information is laplacian and the source equals the side information plus an independent gaussian noise term our layered coder performs 1 33 to 3 90 db from the wyner ziv bound for rates ranging from 0 48 to 4 64 bits per sample

threshold saturation via spatial coupling why convolutional ldpc ensembles perform so well over the bec convolutional low density parity check ldpc ensembles introduced by felstrom and zigangirov have excellent thresholds and these thresholds are rapidly increasing functions of the average degree several variations on the basic theme have been proposed to date all of which share the good performance characteristics of convolutional ldpc ensembles we describe the fundamental mechanism that explains why convolutional like or spatially coupled codes perform so well in essence the spatial coupling of individual codes increases the belief propagation bp threshold of the new ensemble to its maximum possible value namely the maximum a posteriori map threshold of the underlying ensemble for this reason we call this phenomenon threshold saturation this gives an entirely new way of approaching capacity one significant advantage of this construction is that one can create capacity approaching ensembles with an error correcting radius that is increasing in the blocklength although we prove the threshold saturation only for a specific ensemble and for the binary erasure channel bec empirically the phenomenon occurs for a wide class of ensembles and channels more generally we conjecture that for a large range of graphical systems a similar saturation of the dynamical threshold occurs once individual components are coupled sufficiently strongly this might give rise to improved algorithms and new techniques for analysis

an mse based transfer chart for analyzing iterative decoding schemes using a gaussian approximation an alternative to extrinsic information transfer exit charts called mean square error mse charts that use a measure related to the mse instead of mutual information is proposed using the relationship between mutual information and minimum mean square error mmse for the additive white gaussian noise awgn channel a relationship between the rate of any code and the area under a plot of mmse versus signal to noise ratio snr is obtained when the a priori log likelihood ratio llr is from a binary input gaussian channel using this result a justification is provided for designing concatenated codes by matching the exit curves of the inner and outer decoder when the llrs are assumed to be gaussian which is also the typical assumption used for code design using exit charts even though the gaussian assumption is almost never true the results presented in this paper represent a step toward the analysis of iterative decoding schemes using a single parameter finally for the special case of awgn channel it is shown that any capacity achieving code has an exit curve that is a step function

a jacobi davidson type method for the generalized singular value problem we discuss a new method for the iterative computation of some of the generalized singular values and vectors of a large sparse matrix our starting point is the augmented matrix formulation of the gsvd the subspace expansion is performed by approximately solving a jacobi davidson type correction equation while we give several alternatives for the subspace extraction numerical experiments illustrate the performance of the method

computing the generalized singular values vectors of large sparse or structured matrix pairs we present a numerical algorithm for computing a few extreme generalized singular values and corresponding vectors of a sparse or structured matrix pair a b the algorithm is based on the cs decomposition and the lanczos bidiagonalization process at each iteration step of the lanczos process the solution to a linear least squares problem with a rm t b rm t rm t as the coefficient matrix is approximately computed and this consists the only interface of the algorithm with the matrix pair a b numerical results are also given to demonstrate the feasibility and efficiency of the algorithm

two sided and alternating jacobi davidson we discuss two variants of a two sided jacobi davidson jd method which have asymptotically cubic convergence for nonnormal matrices and aim to find both right and left eigenvectors these methods can be seen as jacobi davidson analogs of ostrowski s two sided rayleigh quotient iteration rqi some relations between exact and inexact two sided jacobi davidson and exact and inexact two sided rayleigh quotient iteration are given together with convergence rates furthermore we introduce an alternating jacobi davidson process that can be seen as the jacobi davidson analog of parlett s alternating rayleigh quotient iteration the methods are extended to the generalized and polynomial eigenproblem advantages of the methods are illustrated by numerical examples

low density parity check codes over gaussian channels with erasures we consider low density parity check code ldpcc design for additive white gaussian noise awgn channels with erasures this model for example represents a common situation in magnetic and optical recording where defects or thermal asperities in the system are detected and presented to the decoder as erasures we give thresholds of regular and irregular ldpccs and discuss practical code design over the mixed gaussian erasures channel the analysis is an extension of the gaussian approximation work of chung et al in the two limiting cases of no erasures and large signal to noise ratio snr the analysis tends to the results of chung et al see ibid vol 47 p 657 670 feb 2001 and luby et al 1997 respectively giving a general tool for a class of mixed channels we derive a steady state equation which gives a graphical interpretation of decoder convergence this allows one to estimate the maximum erasure capability on the mixture channel or conversely to estimate the additional signal power required to compensate for the loss due to erasures we see that a good capacity approaching ldpcc over an awgn channel is also good over the mixed channel up to a moderate erasure probability we also investigate practical issues such as the maximum number of iterations of message passing decoders the coded block length and types of erasure patterns random block erasures finally we design an optimized ldpcc for the mixed channel which shows better performance if the erasure probability is larger than a certain value 0 1 in our simulation at the expense of performance degradation at unerased awgn channel and lower erasure probability regions less than 0 1 in our simulation

an analysis of belief propagation on the turbo decoding graph with gaussian densities motivated by its success in decoding turbo codes we provide an analysis of the belief propagation algorithm on the turbo decoding graph with gaussian densities in this context we are able to show that under certain conditions the algorithm converges and that somewhat surprisingly though the density generated by belief propagation may differ significantly from the desired posterior density the means of these two densities coincide since computation of posterior distributions is tractable when densities are gaussian use of belief propagation in such a setting may appear unwarranted indeed our primary motivation for studying belief propagation in this context stems from a desire to enhance our understanding of the algorithm s dynamics in a non gaussian setting and to gain insights into its excellent performance in turbo codes nevertheless even when the densities are gaussian belief propagation may sometimes provide a more efficient alternative to traditional inference methods

semiprimitivity of group algebras of locally finite simple groups let g be a locally finite simple group which is not a linear group for any field k we prove that the group algebra k g is semiprimitive the argument is almost entirely group theoretic in nature and ultimately rests upon the classification of the finite simple groups

performance analysis of iterative decoding algorithms with memory over memoryless channels density evolution is often used to determine the performance of an ensemble of low density parity check ldpc codes under iterative message passing algorithms conventional density evolution techniques over memoryless channels are based on the assumption that messages at iteration l are only a function of the messages at iteration l 1 and possibly the channel output this assumption is valid for many algorithms such as standard belief propagation bp and min sum ms algorithms however there are other important iterative algorithms such as successive relaxation sr versions of bp and ms and differential decoding with binary message passing dd bmp algorithm of mobini et al for which this assumption is not valid the reason is the introduction of memory in these algorithms in this work we propose a model for iterative decoding algorithms with memory which covers sr and dd bmp algorithms as special cases based on this model we derive a bayesian network for iterative algorithms with memory over memoryless channels and use this representation to analyze the performance of the algorithms using density evolution the density evolution technique is developed based on truncating the memory of the decoding process and approximating it with a finite order markov process and can be implemented efficiently as an example we apply our technique to analyze the performance of dd bmp on regular ldpc code ensembles and make a number of interesting observations with regard to the performance complexity tradeoff of dd bmp in comparison with bp and ms algorithms the model presented in this paper is based on certain simplifying assumptions about the memory structure of iterative algorithms such as the existence of memory only at the output of variable nodes in the code s tanner graph rather than at both outputs of variable and check nodes the bayesian network framework introduced here however can still be used to analyze the more general scenarios