an efficient algorithm to find all small size stopping sets of low density parity check matrices in this work we introduce an efficient algorithm to find all stopping sets of size less than some threshold of a fixed low density parity check ldpc matrix the solution is inspired by the algorithm proposed by rosnes and ytrehus in 2005 to find an exhaustive list of all small size turbo stopping sets in a turbo code the efficiency of the proposed algorithm is demonstrated by several numerical examples for instance we have applied the algorithm to the well known 3 5 regular 155 64 tanner code and found all stopping sets of size at most 18 in about 1 min on a standard desktop computer also we have verified that the minimum stopping set size of the 4896 2474 ramanujan margulis code is indeed 24 and that the corresponding multiplicity is exactly 204 furthermore we have applied the algorithm to the ieee 802 16e ldpc codes and determined the minimum stopping set size and the corresponding multiplicity exactly for these codes finally as an application we present a greedy algorithm to find a small number of redundant parity checks to add to the original parity check matrix in order to remove all stopping sets in the corresponding tanner graph of size less than the minimum distance an extensive case study of the 155 64 tanner code illustrates the usefulness of the algorithm and we present a 110 times 155 redundant parity check matrix for this code with no stopping sets of size less than the minimum distance simulation results of iterative decoding on the binary erasure channel show performance improvements for low to medium erasure probabilities when this redundant parity check matrix is used for decoding

dynamics and performance analysis of analog iterative decoding for low density parity check ldpc codes conventional iterative decoding with flooding or parallel schedule can be formulated as a fixed point problem solved iteratively by a successive substitution ss method in this paper we investigate the dynamics of a continuous time asynchronous analog implementation of iterative decoding and show that it can be approximated as the application of the well known successive relaxation sr method for solving the fixed point problem we observe that sr with the optimal relaxation factor can considerably improve the error rate performance of iterative decoding for short low density parity check ldpc codes compared with ss our simulation results for the application of sr to belief propagation sum product and min sum algorithms demonstrate improvements of up to about 0 7 db over the standard ss for randomly constructed ldpc codes the improvement in performance increases with the maximum number of iterations and by accordingly reducing the relaxation factor the asymptotic result corresponding to an infinite maximum number of iterations and infinitesimal relaxation factor represents the steady state performance of analog iterative decoding this means that under ideal circumstances continuous time asynchronous analog decoders can outperform their discrete time synchronous digital counterparts by a large margin our results also indicate that with the assumption of a truncated gaussian distribution for the random delays among computational modules the error rate performance of the analog decoder particularly in steady state is rather independent of the variance of the distribution the proposed simple model for analog decoding and the associated performance curves can be used as an ideal analog decoder benchmark for performance evaluation of analog decoding circuits

superlinear convergence and implicit filtering in this paper we show how the implicit filtering algorithm can be coupled with the bfgs quasi newton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached in this way we give insight into the observations of good performance in practice of quasi newton methods when they are coupled with implicit filtering we also report on numerical experiments that show how an implementation of implicit filtering that exploits these new results can improve the performance of the algorithm

self compensation technique for simplified belief propagation algorithm the min sum algorithm is the most common method to simplify the belief propagation algorithm for decoding low density parity check ldpc codes however there exists a performance gap between the min sum and belief propagation algorithms due to nonlinear approximation in this paper a self compensation technique using dynamic normalization is thus proposed to improve the approximation accuracy the proposed scheme scales the min sum algorithm by a dynamic factor that can be derived theoretically from order statistics moreover applying the proposed technique to several ldpc codes for dvb s2 system the average signal to noise ratio degradation which results from approximation inaccuracy and quantization error is reduced to 0 2 db not only does it enhance the error correcting capability of the min sum algorithm but the proposed self compensation technique also preserves a modest hardware cost after realized with 0 13 mum standard cell library the dynamic normalization requires about 100 additional gates for each check node unit in the min sum algorithm

embedded image coding using zerotrees of wavelet coefficients the embedded zerotree wavelet algorithm ezw is a simple yet remarkably effective image compression algorithm having the property that the bits in the bit stream are generated in order of importance yielding a fully embedded code the embedded code represents a sequence of binary decisions that distinguish an image from the null image using an embedded coding algorithm an encoder can terminate the encoding at any point thereby allowing a target rate or target distortion metric to be met exactly also given a bit stream the decoder can cease decoding at any point in the bit stream and still produce exactly the same image that would have been encoded at the bit rate corresponding to the truncated bit stream in addition to producing a fully embedded bit stream the ezw consistently produces compression results that are competitive with virtually all known compression algorithms on standard test images yet this performance is achieved with a technique that requires absolutely no training no pre stored tables or codebooks and requires no prior knowledge of the image source the ezw algorithm is based on four key concepts 1 a discrete wavelet transform or hierarchical subband decomposition 2 prediction of the absence of significant information across scales by exploiting the self similarity inherent in images 3 entropy coded successive approximation quantization and 4 universal lossless data compression which is achieved via adaptive arithmetic coding

reversible low density parity check codes low density parity check ldpc codes may be decoded using a circuit implementation of the sum product algorithm which maps the factor graph of the code by reusing the decoder for encoding both tasks can be performed using the same circuit thus reducing area and verification requirements motivated by this iterative encoding techniques based upon the graphical representation of the code are proposed code design constraints which ensure encoder convergence are presented and then used to design iteratively encodable codes while also preventing 4 cycle creation we show how the jacobi method for iterative matrix inversion can be applied to finite field matrices viewed as message passing and employed as the core of an iterative encoder we present an algebraic construction of 4 cycle free iteratively encodable codes using circulant matrices analysis of these codes identifies a weakness in their structure due to a repetitive pattern in the factor graph the graph supports pseudo codewords of low pseudo weight in order to remove the repetitive pattern in the graph we propose a recursive technique for generating iteratively encodable codes the new codes offer flexibility in the choice of code length and rate and performance that compares well to randomly generated quasi cyclic and extended euclidean geometry codes

on the unequal error protection for progressive image transmission in this paper we consider the unequal error protection uep for progressive image transmission when the coded packet size is fixed first we prove that for the source code with convex rate distortion r d function and practically used channel codes the channel code rate for each packet in the optimal rate allocation is nondecreasing indeed then we give an upper bound for the channel code rate of the last packet so that the number of rate allocations in the exhaustive search can be predicted further we propose a heuristic optimization method which has low complexity and obtains performance approaching to the optimal solutions for various channel conditions and transmission rates

a new construction of structured binary regular ldpc codes based on steiner systems with parameter t 2 this paper presents a novel method for constructing structured regular low density parity check ldpc codes based on a special type of combinatorial designs known as steiner systems this code design approach can be considered as a generalization of the well known method which uses the point block incidence matrix of a steiner 2 design for the code construction though the given method can be applied on any steiner system s t k v in this paper we focus only on steiner systems with t 3 furthermore we show that not only a steiner system x b itself but also its residual design with respect to an arbitrary point x x can be employed for code construction we also present a technique for constructing binary and non binary qc ldpc codes from steiner systems the tanner graph of the constructed codes is free of 4 cycles and hence the codes have girth at least six simulation results show that the so constructed codes perform well over the awgn channel with iterative message passing decoding

partial singular value decomposition algorithm algorithm 32 abstract the partial singular value decomposition psvd subroutine computes a basis of the left and or right singular subspace of a matrix corresponding to its smallest singular values the dimension of the desired subspace may be fixed or depend on a given upper bound for those smallest singular values as only a basis of the desired singular subspace is needed the classical singular value decomposition svd algorithm is modified in three ways first the householder transformations of the bidiagonalization are only applied to the base vectors of the desired singular subspace second the bidiagonal is only partially diagonalized and third the convergence rate of the iterative diagonalization is improved by an appropriate choice between qr and ql iteration steps depending on the gap the desired numerical accuracy and the dimension of the desired subspace psvd can be three times faster than the classical svd algorithm while the same accuracy can be maintained the new algorithm can be successfully used in total least squares applications in the computation of the null space of a matrix an in solving non homogeneous linear equations

on the convergence theory of trust region based algorithms for equality constrained optimization in a recent paper dennis el alem and maciel proved global convergence to a stationary point for a general trust region based algorithm for equality constrained optimization this general algorithm is based on appropriate choices of trust region subproblems and seems particularly suitable for large problems this paper shows global convergence to a point satisfying the second order necessary optimality conditions for the same general trust region based algorithm the results given here can be seen as a generalization of the convergence results for trust regions methods for unconstrained optimization obtained by more and sorensen the behavior of the trust radius and the local rate of convergence are analyzed some interesting facts concerning the trust region subproblem for the linearized constraints the quasi normal component of the step and the hard case are presented it is shown how these results can be applied to a class of discretized optimal control problems

braided block codes a new class of binary iteratively decodable codes with good decoding performance is presented these codes called braided block codes bbcs operate on continuous data streams and are constructed by interconnection of two component block codes bbcs can be considered as convolutional or sliding version of either elias product codes or expander codes in this paper we define bbcs describe methods of their construction analyze code properties and study asymptotic iterative decoding performance

provably convergent multifidelity optimization algorithm not requiring high fidelity derivatives functions does not alter the con vergence proof of the optimization algorithm and is shown to be robust to poor low delity information the algorithm is compared with an unconstrained single delity quasi newton algorithm and two

construction of low density parity check codes based on balanced incomplete block designs this correspondence presents a method for constructing structured regular low density parity check ldpc codes based on a special type of combinatoric designs known as balance incomplete block designs codes constructed by this method have girths at least 6 and they perform well with iterative decoding furthermore several classes of these codes are quasi cyclic and hence their encoding can be implemented with simple feedback shift registers

evolutionary optimization of computationally expensive problems via surrogate modeling we present a parallel evolutionary optimization algorithm that leverages surrogate models for solving computationally expensive design problems with general constraints on a limited computational budget the essential backbone of our framework is an evolutionary algorithm coupled with a feasible sequential quadratic programming solver in the spirit of lamarckian learning we employ a trust region approach for interleaving use of exact modelsfortheobjectiveandconstraintfunctionswithcomputationallycheapsurrogatemodelsduringlocalsearch in contrastto earlier work we construct local surrogatemodels using radial basis functionsmotivated by theprinciple of transductive inference further the present approach retains the intrinsic parallelism of evolutionary algorithms and can hence be readily implemented on grid computing infrastructures experimental results are presented for some benchmark test functions and an aerodynamic wing design problem to demonstrate that our algorithm converges to good designs on a limited computational budget

a recursive approach to low complexity codes a method is described for constructing long error correcting codes from one or more shorter error correcting codes referred to as subcodes and a bipartite graph a graph is shown which specifies carefully chosen subsets of the digits of the new codes that must be codewords in one of the shorter subcodes lower bounds to the rate and the minimum distance of the new code are derived in terms of the parameters of the graph and the subeodes both the encoders and decoders proposed are shown to take advantage of the code s explicit decomposition into subcodes to decompose and simplify the associated computational processes bounds on the performance of two specific decoding algorithms are established and the asymptotic growth of the complexity of decoding for two types of codes and decoders is analyzed the proposed decoders are able to make effective use of probabilistic information supplied by the channel receiver e g reliability information without greatly increasing the number of computations required it is shown that choosing a transmission order for the digits that is appropriate for the graph and the subcodes can give the code excellent burst error correction abilities the construction principles

the partial total least squares algorithm abstract the partial total least squares ptls subroutine solves the total least squares tls problem ax b by using a partial singular value decomposition psvd hereby improving the computational efficiency of the classical tls algorithm a factor 2 approximately the computational improvement of ptls is based on the observation that the tls solution can be computed from any orthogonal basis of the right singular subspace corresponding to the smallest singular values of the augmented matrix a b the dimension of this subspace may be determined by the rank of the tls approximation of a b or depend on a given upper bound e g a perturbation level for those smallest singular values the routine ptls allows more than one vector in b it also solves determined and underdetermined sets of equations by computing the minimum norm solution

construction of nonbinary cyclic quasi cyclic and regular ldpc codes a finite geometry approach this paper presents five methods for constructing nonbinary ldpc codes based on finite geometries these methods result in five classes of nonbinary ldpc codes one class of cyclic ldpc codes three classes of quasi cyclic ldpc codes and one class of structured regular ldpc codes experimental results show that constructed codes in these classes decoded with iterative decoding based on belief propagation perform very well over the awgn channel and they achieve significant coding gains over reed solomon codes of the same lengths and rates with either algebraic hard decision decoding or kotter vardy algebraic soft decision decoding at the expense of a larger decoding computational complexity

product code optimization for determinate state ldpc decoding in robust image transmission we propose a novel scheme for error resilient image transmission the proposed scheme employs a product coder consisting of low density parity check ldpc codes and reed solomon codes in order to deal effectively with bit errors the efficiency of the proposed scheme is based on the exploitation of determinate symbols in tanner graph decoding of ldpc codes and a novel product code optimization technique based on error estimation experimental evaluation demonstrates the superiority of the proposed system in comparison to recent state of the art techniques for image transmission

efficient encoding of low density parity check codes low density parity check ldpc codes can be considered serious competitors to turbo codes in terms of performance and complexity and they are based on a similar philosophy constrained random code ensembles and iterative decoding algorithms we consider the encoding problem for ldpc codes more generally we consider the encoding problem for codes specified by sparse parity check matrices we show how to exploit the sparseness of the parity check matrix to obtain efficient encoders for the 3 6 regular ldpc code for example the complexity of encoding is essentially quadratic in the block length however we show that the associated coefficient can be made quite small so that encoding codes even of length n spl sime 100000 is still quite practical more importantly we show that optimized codes actually admit linear time encoding

crc assisted error correction in a convolutionally coded system in communication systems employing a serially concatenated cyclic redundancy check crc code along with a convolutional code cc erroneous packets after cc decoding are usually discarded the list viterbi algorithm lva and the iterative viterbi algorithm iva are two existing approaches capable of recovering erroneously decoded packets we here employ a soft decoding algorithm for cc decoding and introduce several schemes to identify error patterns using the posterior information from the cc soft decoding module the resultant iterative decoding detecting idd algorithm improves error performance by iteratively updating the extrinsic information based on the crc parity check matrix assuming errors only happen in unreliable bits characterized by small absolute values of the log likelihood ratio llr we also develop a partial idd p idd alternative which exhibits comparable performance to idd by updating only a subset of unreliable bits we further derive a soft decision syndrome decoding sdsd algorithm which identifies error patterns from a set of binary linear equations derived from crc syndrome equations being noniterative sdsd is able to estimate error patterns directly from the decoder output the packet error rate per performance of sdsd is analyzed following the union bound approach on pairwise errors simulations indicate that both idd and iva are better tailored for single parity check pc codes than for crc codes sdsd outperforms both idd and lva with weak cc and strong crc applicable to awgn and flat fading channels our algorithms can also be extended to turbo coded systems

tree expectation propagation for ml decoding of ldpc codes over the bec we propose a decoding algorithm for ldpc codes that achieves the maximum likelihood ml solution over the binary erasure channel bec in this channel the tree structured expectation propagation tep decoder improves the peeling decoder pd by processing check nodes of degree one and two however it does not achieve the ml solution as the tree structure of the tep allows only for approximate inference in this paper we provide the procedure to construct the structure needed for exact inference this algorithm denoted as generalized tree structured expectation propagation gtep modifies the code graph by recursively eliminating any check node and merging this information in the remaining graph the gtep decoder upon completion either provides the unique ml solution or a tree graph in which the number of parent nodes indicates the multiplicity of the ml solution we also explain the algorithm as a gaussian elimination method relating the gtep to other ml solutions compared to previous approaches it presents an equivalent complexity it exhibits a simpler graphical message passing procedure and most interesting the algorithm can be generalized to other channels

asymptotic enumeration methods for analyzing ldpc codes we show how asymptotic estimates of powers of polynomials with nonnegative coefficients can be used in the analysis of low density parity check ldpc codes in particular we show how these estimates can be used to derive the asymptotic distance spectrum of both regular and irregular ldpc code ensembles we then consider the binary erasure channel bec using these estimates we derive lower bounds on the error exponent under iterative decoding of ldpc codes used over the bec both regular and irregular code structures are considered these bounds are compared to the corresponding bounds when optimal maximum likelihood ml decoding is applied

high rate quasi cyclic low density parity check codes derived from finite affine planes this paper shows that several attractive classes of quasi cyclic qc low density parity check ldpc codes can be obtained from affine planes over finite fields one class of these consists of duals of one generator qc codes presented here for codes contained in this class are the exact minimum distance and a lower bound on the multiplicity of the minimum weight codewords further it is shown that the minimum hamming distance of a code in this class is equal to its minimum additive white gaussian noise awgn pseudoweight also discussed is a class consisting of codes from circulant permutation matrices and an explicit formula for the rank of the parity check matrix is presented for these codes additionally it is shown that each of these codes can be identified with a code constructed from a constacyclic maximum distance separable code of dimension 2 the construction is similar to the derivation of reed solomon rs based ldpc codes presented by chen and djurdjevic experimental results show that a number of high rate qc ldpc codes with excellent error performance are contained in these classes

mean field and mixed mean field iterative decoding of low density parity check codes in this paper the mean field mf and mixed mean field mmf algorithms for decoding low density parity check ldpc codes are considered the mf principle is well established in statistical physics and artificial intelligence instead of using a single completely factorized approximated distribution as in the mf approach the mixed mf algorithm forms a weighted average of several mf distributions as an approximation of the true posterior probability distribution the mf decoding algorithm for linear block codes is derived and shown to be an approximation of the a posteriori probability app decoding algorithm the mf approach is then developed in the context of iterative decoding and presented as an approximation of the popular belief propagation decoding method these results are extended to iterative decoding with the mmf algorithm simulation results show that the mf and mmf decoding algorithms yield a good performance complexity tradeoff especially when employed for decoding ldpc codes based on finite geometries

cyclic and quasi cyclic ldpc codes on constrained parity check matrices and their trapping sets this paper is concerned with construction and structural analysis of both cyclic and quasi cyclic codes particularly low density parity check ldpc codes it consists of three parts the first part shows that a cyclic code given by a parity check matrix in circulant form can be decomposed into descendant cyclic and quasi cyclic codes of various lengths and rates some fundamental structural properties of these descendant codes are developed including the characterization of the roots of the generator polynomial of a cyclic descendant code the second part of the paper shows that cyclic and quasi cyclic descendant ldpc codes can be derived from cyclic finite geometry ldpc codes using the results developed in the first part of the paper this enlarges the repertoire of cyclic ldpc codes the third part of the paper analyzes the trapping set structure of regular ldpc codes whose parity check matrices satisfy a certain constraint on their rows and columns several classes of finite geometry and finite field cyclic and quasi cyclic ldpc codes with large minimum distances are shown to have no harmful trapping sets of size smaller than their minimum distances consequently their error floor performances are dominated by their minimum distances

a trust region approach to nonlinear systems of equalities and inequalities in this paper two new trust region algorithms for the numerical solution of systems of nonlinear equalities and inequalities are introduced the formulation is free of arbitrary parameters and possesses sufficient smoothness to exploit the robustness of the trust region approach the proposed algorithms are one sided least squares trust region algorithms the first algorithm is a single model algorithm and the second one is a multimodel algorithm where the cauchy point computation is a model selection procedure global convergence analysis for the two algorithms is presented our analysis generalizes to nonlinear systems of equalities and inequalities the well developed theory for nonlinear least squares problems numerical experiments on the two algorithms are also presented the performance of the two algorithms is reported the numerical results validate the effectiveness of our approach

consistent approximations and approximate functions and gradients in optimal control because of the unavoidable use of numerical integration methods such as runge kutta or finite elements the numerical solution of optimal control problems with either ode or pde dynamics is governed by a discretization parameter such as the integration mesh size usually when explicit integration techniques are used function and derivative values can be computed exactly for the discretized problems recently we have come across some examples where function and derivative values of the explicitly discretized problems had to be approximated by the outcome of n iterations of a solver consequently the discretization of these problems is controlled by two parameters the mesh size and the number of iterations of the solver referring to e polak optimization algorithms and consistent approximations springer verlag 1997 we find a theory for solving optimization problems that require discretization it deals with two situations in the first which is referred to as that of consistent approximations it is assumed that an infinite dimensional optimization problem can be suitably approximated by a family of progressively higher dimensional optimization problems in this case strategies in the form of algorithm models are presented for diagonalizing the solution process in the second situation it is assumed that numerical solution of the dynamic equations does not result in a family of finite dimensional consistent approximations e g when implicit integration methods are used for this case the theory provides models for the implementation of conceptual algorithms unfortunately neither of these situations envisions the possibility of two discretization parameters in this paper we present new algorithm models that can be used with two discretization parameters the first one controls the mesh size of an explicit integration scheme and the second one controls the precision with which functions and gradients associated with a fixed mesh size are computed the result can be seen as a framework of quasi consistent approximations we implemented these new algorithm models using an approximate steepest descent method for the solution of two problems a two point boundary value problem in which the discretized linear ode dynamics are solved approximately using the gauss seidel method and a distributed control problem in which the discretized dynamics are solved using a domain decomposition algorithm which can be implemented on parallelized computers our numerical results show that these new algorithms perform quite well and are fairly insensitive to the selection of user set parameters also they appear to be superior to some alternative ad hoc schemes

bounds on the size of parity check matrices for quasi cyclic low density parity check codes in this paper we investigate the cycle properties of quasi cyclic low density parity check qc ldpc codes using the sequence representation of a parity check matrix for a qc ldpc code we analyze a necessary and sufficient condition for a cycle of a given length to exist we then derive bounds which are necessary conditions for a qc ldpc code to have a given girth in terms of its parameters we also give a bound which is a sufficient condition for a qc ldpc code of a given girth to be constructed by a greedy algorithm the bounds derived here are applicable to any regular or irregular qc ldpc codes as well as they improve the existing bounds in many classes of regular ldpc codes

a pde sensitivity equation method for optimal aerodynamic design the use of gradient based optimization algorithms in inverse design is well established as a practical approach to aerodynamic design a typical procedure uses a simulation scheme to evaluate the objective function from the approximate states and its gradient then passes this information to an optimization algorithm once the simulation scheme cfd flow solver has been selected and used to provide approximate function evaluations there are several possible approaches to the problem of computing gradients one popular method is to differentiate the simulation scheme and compute design sensitivities that are then used to obtain gradients although this black box approach has many advantages in shape optimization problems one must compute mesh sensitivities in order to compute the design sensitivity in this paper we present an alternative approach using the pde sensitivity equation to develop algorithms for computing gradients this approach has the advantage that mesh sensitivities need not be computed moreover when it is possible to use the cfd scheme for both the forward problem and the sensitivity equation then there are computational advantages an apparent disadvantage of this approach is that it does not always produce consistent derivatives however for a proper combination of discretization schemes one can showasymptotic consistencyunder mesh refinement which is often sufficient to guarantee convergence of the optimal design algorithm in particular we show that when asymptotically consistent schemes are combined with a trust region optimization algorithm the resulting optimal design method converges we denote this approach as thesensitivity equation method the sensitivity equation method is presented convergence results are given and the approach is illustrated on two optimal design problems involving shocks

ad 3 alternating directions dual decomposition for map inference in graphical models we present ad3 a new algorithm for approximate maximum a posteriori map inference on factor graphs based on the alternating directions method of multipliers like other dual decomposition algorithms ad3 has a modular architecture where local subproblems are solved independently and their solutions are gathered to compute a global update the key characteristic of ad3 is that each local subproblem has a quadratic regularizer leading to faster convergence both theoretically and in practice we provide closed form solutions for these ad3 subproblems for binary pairwise factors and factors imposing first order logic constraints for arbitrary factors large or combinatorial we introduce an active set method which requires only an oracle for computing a local map configuration making ad3 applicable to a wide range of problems experiments on synthetic and real world problems show that ad3 compares favorably with the state of the art

progressive transmission of images over fading channels using rate compatible ldpc codes in this paper we propose a combined source channel coding scheme for transmission of images over fading channels the proposed scheme employs rate compatible low density parity check codes along with embedded image coders such as jpeg2000 and set partitioning in hierarchical trees spiht the assignment of channel coding rates to source packets is performed by a fast trellis based algorithm we examine the performance of the proposed scheme over correlated and uncorrelated rayleigh flat fading channels with and without side information simulation results for the expected peak signal to noise ratio of reconstructed images which are within 1 db of the capacity upper bound over a wide range of channel signal to noise ratios show considerable improvement compared to existing results under similar conditions we also study the sensitivity of the proposed scheme in the presence of channel estimation error at the transmitter and demonstrate that under most conditions our scheme is more robust compared to existing schemes

algebraic construction of quasi cyclic ldpc codes for the awgn and erasure channels this paper is concerned with construction of quasi cyclic qc low density parity check ldpc codes for three different types of channels the additive white gaussian noise the binary random erasure and the binary burst erasure channels two algebraic methods for systematic construction of qc ldpc codes are presented codes constructed perform well over all three types of channels

new classes of partial geometries and their associated ldpc codes the use of partial geometries to construct parity check matrices for binary low density parity check ldpc codes has resulted in the design of successful codes with a probability of error on the awgn channel close to the shannon capacity at bit error rate down to 10 15 such considerations have motivated this further investigation a new and simple construction of a type of partial geometries with a quasi cyclic qc structure is given and their properties are investigated two new classes of this type of partial geometries one based on prime fields and the other based on cyclic subgroups of prime orders of finite fields are constructed qc ldpc codes with good error performances are constructed based on these two new classes of partial geometries the trapping sets of the partial geometry codes were previously considered using the geometric aspects of the underlying structure to derive information on the size of allowable trapping sets this topic is further considered here finally there is a natural relationship between partial geometries and strongly regular graphs the eigenvalues of the adjacency matrices of such graphs are well known and it is of interest to determine if any of the tanner graphs derived from the partial geometries are good expanders for certain parameter sets since it can be argued that codes with good geometric and expansion properties might perform well on the awgn channel under message passing decoding

on the suboptimality of iterative decoding for turbo like and ldpc codes with cycles in their graph representation in this paper we focus on the suboptimality of iterative decoding on graphs with cycles through examining the use of a reliability based decoding algorithm for some concatenated codes with an interleaver known as turbo like codes the a posteriori probabilities delivered by the iterative decoding are regarded as reliability information and an efficient algorithm for the overall linear block code is applied at certain iterations simulation results show that the suboptimality of iterative decoding due to cycles can be at least partially compensated by this approach some insights about the potential additional coding gains achievable are investigated based on the characteristics of the constituent decoders these characteristics are related to the nature of suboptimality in the overall iterative decoding the effects of some code parameters and channel conditions on the behavior of iterative decoding are also studied for a better understanding of its suboptimality

space mapping for optimal control of partial differential equations solving optimal control problems for nonlinear partial differential equations represents a significant numerical challenge due to the tremendous size and possible model difficulties e g nonlinearities of the discretized problems in this paper a novel space mapping technique for solving the aforementioned problem class is introduced analyzed and tested the advantage of the space mapping approach compared to classical multigrid techniques lies in the flexibility of not only using grid coarsening as a model reduction but also employing perhaps less nonlinear surrogates the space mapping is based on a regularization approach which in contrast to other space mapping techniques results in a smooth mapping and thus avoids certain irregular situations at kinks a new broyden update formula for the sensitivities of the space map is also introduced this quasi newton update is motivated by the usual secant condition combined with a secant condition resulting from differentiating the space mapping surrogate the overall algorithm employs a trust region framework for global convergence issues involved in the computations are highlighted and a report on a few illustrative numerical tests is given

computing the stopping distance of a tanner graph is np hard two decision problems related to the computation f stopping sets in tanner graphs are shown to be np complete it follows as a consequence that there exists no polynomial time algorithm for computing the stopping distance of a tanner graph unless p np

newton s method with a model trust region modification a modified newton method for unconstrained minimization is presented and analyzed the modification is based upon the model trust region approach this report contains a thorough analysis of the locally constrained quadratic minimizations that arise as subproblems in the modified newton iteration several promising alternatives are presented for solving these subproblems in ways that overcome certain theoretical difficulties exposed by this analysis very strong convergence results are presented concerning the minimization algorithm in particular the explicit use of second order information is justified by demonstrating that the iterates converge to a point which satisfies the second order necessary conditions for minimization with the exception of very pathological cases this occurs whenever the algorithm is applied to problems with continuous second partial derivatives

reduced complexity iterative decoding of low density parity check codes based on belief propagation two simplified versions of the belief propagation algorithm for fast iterative decoding of low density parity check codes on the additive white gaussian noise channel are proposed both versions are implemented with real additions only which greatly simplifies the decoding complexity of belief propagation in which products of probabilities have to be computed also these two algorithms do not require any knowledge about the channel characteristics both algorithms yield a good performance complexity trade off and can be efficiently implemented in software as well as in hardware with possibly quantized received values

an efficient and reliable algorithm for computing the singular subspace of a matrix associated with its smallest singular values abstract in this paper an improved algorithm psvd for computing the singular subspace of a matrix corresponding to its smallest singular values is presented as only a basis of the desired singular subspace is needed the classical singular value decomposition svd algorithm can be modified in three ways first the householder transformations of the bidiagonalization need only to be applied on the base vectors of the desired singular subspace second the bidiagonal must only be partially diagonalized and third the convergence rate of the iterative diagonalization can be improved by an appropriate choice between qr and ql iteration steps an analysis of the operation counts as well as computational results show the relative efficiency of psvd with respect to the classical svd algorithm depending on the gap the desired numerical accuracy and the dimension of the desired subspace psvd can be three times faster than the classical svd algorithm while the same accuracy can be maintained the new algorithm can be successfully used in total least squares applications in the computation of the null space of a matrix and in solving non homogeneous linear equations based on psvd a very efficient and reliable algorithm is also derived for solving nonhomogeneous equations

a trust region method for norm constrained problems a trust region method for the solution of nonlinear optimization problems with norm constraints is analyzed such optimization problems arise in parameter identification and nonlinear eigenvalue problems the characteristics of these applications motivate the formulation and analysis of the trust region method the algorithms studied here allow for inexact gradient information and the use of subspace methods for the approximate solution of subproblems characterizations and the descent properties of trust region steps are given criteria for the existence of successful iterations under inexact gradient information and under the use of subspace methods are established and global convergence of the method is proven

a class of trust region methods for parallel optimization we present a new class of optimization methods that incorporates a parallel direct search pds method within a trust region newton framework this approach combines the inherent parallelism of pds with the rapid and robust convergence properties of newton methods numerical tests have yielded favorable results for both standard test problems and engineering applications in addition the new method appears to be more robust in the presence of noisy functions which are inherent in many engineering simulations

a lattice based systematic recursive construction of quasi cyclic ldpc codes this paper presents a low complexity recursive and systematic method to construct good well structured low density parity check ldpc codes the method is based on a recursive application of a partial kronecker product operation on a given gamma x q q ges 3 a prime integer lattice l gamma x q the n 1 fold product of l gamma x q by itself denoted ln gamma x q represents a regular quasi cyclic qc ldpc code denoted see pdf of high rate and girth 6 the minimum distance of see pdf is equal to that of the core code see pdf introduced by l gamma x q the support of the minimum weight codewords in see pdf are characterized by the support of the same type of codewords in see pdf from performance perspective the constructed codes compete with the pseudorandom ldpc codes

an efficient instanton search algorithm for lp decoding of ldpc codes over the bsc we consider linear programming lp decoding of a fixed low density parity check ldpc code over the binary symmetric channel bsc the lp decoder fails when it outputs a pseudo codeword which is not equal to the transmitted codeword we design an efficient algorithm termed the instanton search algorithm isa which generates an error vector called the bsc instanton we prove that a the lp decoder fails for any error pattern with support that is a superset of the support of an instanton b for any input the isa outputs an instanton in the number of steps upper bounded by twice the number of errors in the input error vector we then find the number of unique instantons of different sizes for a given ldpc code by running the isa sufficient number of times

lp decoding of regular ldpc codes in memoryless channels we study error bounds for linear programming decoding of regular low density parity check ldpc codes for memoryless binary input output symmetric channels we prove bounds on the word error probability that are inverse doubly exponential in the girth of the factor graph for memoryless binary input awgn channel we prove lower bounds on the threshold for regular ldpc codes whose factor graphs have logarithmic girth under lp decoding specifically we prove a lower bound of 0 735 upper bound of eb n0 2 67 db on the threshold of 3 6 regular ldpc codes whose factor graphs have logarithmic girth our proof is an extension of a recent paper of arora daskalakis and steurer stoc 2009 who presented a novel probabilistic analysis of lp decoding over a binary symmetric channel their analysis is based on the primal lp representation and has an explicit connection to message passing algorithms we extend this analysis to any mbios channel

efficient encoding of quasi cyclic low density parity check codes efficient encoding of quasi cyclic low density parity check codes quasi cyclic qc low density parity check ldpc codes form an important subclass of ldpc codes these codes have encoding advantage over other types of ldpc codes this paper addresses the issue of efficient encoding of qc ldpc codes two methods are presented to find the generator matrices of qc ldpc codes in systematic circulant form from their parity check matrices given in circulant form based on the systematic circulation form of the generator matrix of a qc ldpc code various types of encoding circuits using simple shift registers are devised it is shown that the encoding complexity of a qc ldpc code is linearly proportional to the number of parity bits of the code for serial encoding and to the length of the code for high speed parallel encoding

decoding algorithms for nonbinary ldpc codes over gf q in this letter we address the problem of decoding nonbinary low density parity check ldpc codes over finite fields gf q with reasonable complexity and good performance in the first part of the letter we recall the original belief propagation bp decoding algorithm and its fourier domain implementation we show that the use of tensor notations for the messages is very convenient for the algorithm description and understanding in the second part of the letter we introduce a simplified decoder which is inspired by the min sum decoder for binary ldpc codes we called this decoder extended min sum ems we show that it is possible to greatly reduce the computational complexity of the check node processing by computing approximate reliability measures with a limited number of values in a message by choosing appropriate correction factors or offsets we show that the ems decoder performance is quite good and in some cases better than the regular bp decoder the optimal values of the factor and offset correction are obtained asymptotically with simulated density evolution our simulations on ultra sparse codes over very high order fields show that nonbinary ldpc codes are promising for applications which require low frame error rates for small or moderate codeword lengths the ems decoder is a good candidate for practical hardware implementations of such codes

spectral shape of doubly generalized ldpc codes efficient and exact evaluation this paper analyzes the asymptotic exponent of the weight spectrum for irregular doubly generalized ldpc d gldpc codes in the process an efficient numerical technique for its evaluation is presented involving the solution of a 4 4 system of polynomial equations the expression is consistent with previous results including the case where the normalized weight or stopping set size tends to zero the spectral shape is shown to admit a particularly simple form in the special case where all variable nodes are repetition codes of the same degree a case which includes tanner codes for this case it is also shown how certain symmetry properties of the local weight distribution at the cns induce a symmetry in the overall weight spectral shape function finally using these new results weight and stopping set size spectral shapes are evaluated for some example generalized and doubly generalized ldpc code ensembles

iterative algorithms for computing the singular subspace of a matrix associated with its smallest singular values abstract many problems in science require the computation of only one singular vector or more generally a singular subspace of a matrix instead of computing the complete singular value decomposition iterative methods can be used to improve the computational speed in particular when a priori information about the solution is available this paper deals with the computation of a basis of a singular subspace of a matrix associated with its smallest singular values three iterative methods inverse chebyshev and inverse chebyshev iteration are compared by analyzing their convergence properties based on the convergence rate and the operation counts per iteration step it is shown in which problems a particular iterative algorithm is most efficient if the gap between the singular values associated with the desired and undesired singular subspace is large inverse iteration is clearly the best choice if not convergence can be accelerated by applying inverse chebyshev iteration provided sufficiently tight bounds delineating the undesired singular value spectrum are known the smaller the gap the larger the gain in speed unless the matrix is structured or sparse this method is shown to be always more efficient than ordinary chebyshev iteration

about existence and uniqueness of the restricted total least squares estimation total least squares tls is an estimation method for the solutiona of the linear system tilde xa tilde y when both data sets tilde x and tilde y are subject to error the tls method minimizes the functional theta tls left x a right lambda left x tilde x right 2 left xa tilde y right 2 with weighting parameter in this paper the tls functional is analyzed by the technique of lagrangian multipliers the main part of the work deals with the case when the estimatea is restricted by an inequality of the formd a b 0 d a diagonal matrix it is shown that there exists a unique estimatea if the weighting parameter is chosen sufficiently large

encoding and decoding binary product lattices a binary product lattice is generated from two binary component lattices of lower dimensions by employing the kronecker product this work focuses on codes carved from binary product lattices defined as such an intriguing problem is that of effectively mapping independent data sequences onto a selected subset of lattice points a novel approach is disclosed yielding an explicit connection between source bits and lattice points decoding methods typically used for binary product codes do not apply for product lattices several alternative decoding approaches are discussed in particular a provably bounded distance decoder is presented it relies on the fact that a product lattice code point may be regarded as a two dimensional array whose rows and columns are points in the component lattices the obtained results are compared with classical lattices known in the art

near optimum universal belief propagation based decoding of low density parity check codes in this paper we propose a belief propagation bp based decoding algorithm which utilizes normalization to improve the accuracy of the soft values delivered by a previously proposed simplified bp based algorithm the normalization factors can be obtained not only by simulation but also importantly theoretically this new bp based algorithm is much simpler to implement than bp decoding as it requires only additions of the normalized received values and is universal i e the decoding is independent of the channel characteristics some simulation results are given which show this new decoding approach can achieve an error performance very close to that of bp on the additive white gaussian noise channel especially for low density parity check ldpc codes whose check sums have large weights the principle of normalization can also be used to improve the performance of the max log map algorithm in turbo decoding and some coding gain can be achieved if the code length is long enough

content based multiple bitstream image transmission over noisy channels we propose a novel combined source and channel coding scheme for image transmission over noisy channels the main feature of the proposed scheme is a systematic decomposition of image sources so that unequal error protection can be applied according to not only bit error sensitivity but also visual content importance the wavelet transform is adopted to hierarchically decompose the image the association between the wavelet coefficients and what they represent spatially in the original image is fully exploited so that wavelet blocks are classified based on their corresponding image content the classification produces wavelet blocks in each class with similar content and statistics therefore enables high performance source compression using the set partitioning in hierarchical trees spiht algorithm to combat the channel noise an unequal error protection strategy with rate compatible punctured convolutional cyclic redundancy check rcpc crc codes is implemented based on the bit contribution to both peak signal to noise ratio psnr and visual quality at the receiving end a postprocessing method making use of the spiht decoding structure and the classification map is developed to restore the degradation due to the residual error after channel decoding experimental results show that the proposed scheme is indeed able to provide protection both for the bits that are more sensitive to errors and for the more important visual content under a noisy transmission environment in particular the reconstructed images illustrate consistently better visual quality than using the single bitstream based schemes

convergence properties of minimization algorithms for convex constraints using a structured trust region in this paper we present a class of trust region algorithms for minimization problems within convex feasible regions in which the structure of the problem is explicitly used in the definition of the trust region this development is intended to reflect the possibility that some parts of the problem may be more accurately modelled than others a common occurrence in large scale nonlinear applications after describing the structured trust region mechanism we prove global convergence for all algorithms in our class

bounds on the expansion properties of tanner graphs this work focuses on the expansion properties of a tanner graph because they are known to be related to the performance of associated iterative message passing algorithms over various channels by analyzing the eigenvalues and corresponding eigenvectors of the normalized incidence matrix representing a tanner graph lower bounds on these expansion properties are derived specifically for the binary erasure channel these results lead to two lower bounds on stopping distance for any given binary linear code and an upper bound on stopping redundancy for the family of difference set codes type i 2 d projective geometry low density parity check ldpc codes

algebraic relations between the total least squares and least squares problems with more than one solution this paper completes our previous discussion on the total least squares tls and the least squares ls problems for the linear systemax b which may contain more than one solution 12 13 generalizes the work of golub and van loan 1 2 van huffel 8 van huffel and vandewalle 11 the tls problem is extended to the more general case the sets of the solutions and the squared residuals for the tls and ls problems are compared the concept of the weighted squares residuals is extended and the difference between the tls and the ls approaches is derived the connection between the approximate subspaces and the perturbation theories are studied it is proved that under moderate conditions all the corresponding quantities for the solution sets of the tls and the modified ls problems are close to each other while the quantities for the solution set of the ls problem are close to the corresponding ones of a subset of that of the tls problem

searching for binary and nonbinary block and convolutional ldpc codes a unified approach to search for and optimize codes determined by their sparse parity check matrices is presented replacing the nonzero elements of a binary parity check matrix the base or parent matrix either by circulants or by companion matrices of elements from a finite field gf 2 m we obtain quasi cyclic low density parity check ldpc block codes and binary images of nonbinary ldpc block codes respectively by substituting monomials of a formal variable d we obtain the polynomial description of an ldpc convolutional code a set of performance measures applicable to different classes of ldpc codes is considered and a greedy algorithm for code performance optimization is presented the heart of the new optimization algorithm is a fast procedure for searching for ldpc codes with large girth of their tanner graphs for a few classes of ldpc codes examples of codes combining good error correcting performance with compact representation are obtained in particular we present optimized convolutional ldpc codes and conclude that the ldpc block codes are still superior to their convolutional counterparts if both decoding complexity and coding delay are considered moreover a specific channel model can easily be embedded into the optimization loop thereby the code can be optimized for a specific channel the efficiency of such an optimization is demonstrated via an example of faster than nyquist ftn signaling using ldpc codes the ftn strategy combined with a rate r 1 2 ldpc code of length 64 800 optimized for effective data rate r 3 4 gains more than 0 5 db compared with the standard ldpc codes of the same rate and length the obtained gain corresponds to transmission at the capacity of the binary input additive white gaussian noise channel in most numerical examples we consider codes with bidiagonal structure of the parity check matrix this restriction preserves low encoding complexity and allows fair comparison with codes selected for communication standards

reduced complexity decoding of ldpc codes various log likelihood ratio based belief propagation llr bp decoding algorithms and their reduced complexity derivatives for low density parity check ldpc codes are presented numerically accurate representations of the check node update computation used in llr bp decoding are described furthermore approximate representations of the decoding computations are shown to achieve a reduction in complexity by simplifying the check node update or symbol node update or both in particular two main approaches for simplified check node updates are presented that are based on the so called min sum approximation coupled with either a normalization term or an additive offset term density evolution is used to analyze the performance of these decoding algorithms to determine the optimum values of the key parameters and to evaluate finite quantization effects simulation results show that these reduced complexity decoding algorithms for ldpc codes achieve a performance very close to that of the bp algorithm the unified treatment of decoding techniques for ldpc codes presented here provides flexibility in selecting the appropriate scheme from performance latency computational complexity and memory requirement perspectives

error protection for progressive image transmission over memoryless and fading channels product channel codes are proposed to protect progressively compressed and packetized images for noisy channels within packets the product code uses the concatenation of a rate compatible punctured convolutional code and an error detecting parity check code across packets reed solomon codes are used benefits include flexible choice of delay adaptability of error protection level i e unequal error protection and scalable decoding complexity the system outperforms the best known image coders for memoryless channels and performs well on fading channels

on the global convergence of trust region algorithms using inexact gradient information trust region algorithms are an important class of methods that can be used to solve unconstrained optimization problems strong global convergence results are demonstrated for a class of methods where the gradient values are approximated rather than computed exactly provided they obey a simple relative error condition no requirement is made that gradients be recomputed to successively greater accuracy after unsuccessful iterations

iterative decoding algorithms for a class of non binary two step majority logic decodable cyclic codes this paper presents two iterative decoding algorithms for a class of non binary two step majority logic nb ts mlg decodable cyclic codes a partial parallel decoding scheme is also introduced to provide a balanced trade off between decoding speed and storage requirements unlike non binary one step mlg decodable cyclic codes the tanner graphs of which are 4 cycle free nb ts mlg decodable cyclic codes contain a large number of short cycles of length 4 which tend to degrade decoding performance the proposed algorithms utilize the orthogonal structure of the parity check matrices of the codes to resolve the degrading effects of the short cycles of length 4 simulation results demonstrate that the nb ts mlg decodable cyclic codes decoded with the proposed algorithms offer coding gains as much as 2 5 db over reed solomon codes of the same lengths and rates decoded with either hard decision or algebraic soft decision decoding

numerical experience with a class of algorithms for nonlinear optimization using inexact function and gradient information for optimization problems associated with engineering design parameter estimation image reconstruction and other optimization simulation applications low accuracy function and gradient values are frequently much less expensive to obtain than high accuracy values the computational performance of trust region methods for nonlinear optimization is investigated for cases when high accuracy evaluations are unavailable or prohibitively expensive and earlier theoretical predictions that such methods are convergent even with relative gradient errors of 0 5 or more is confirmed the proper choice of the amount of accuracy to use in function and gradient evaluations can result in orders of magnitude savings in computational cost

inexact objective function evaluations in a trust region algorithm for pde constrained optimization under uncertainty this paper improves the trust region algorithm with adaptive sparse grids introduced in siam j sci comput 35 2013 pp a1847 a1879 for the solution of optimization problems governed by partial differential equations pdes with uncertain coefficients the previous algorithm used adaptive sparse grid discretizations to generate models that are applied in a trust region framework to generate a trial step the decision whether to accept this trial step as the new iterate however required relatively high fidelity adaptive discretizations of the objective function in this paper we extend the algorithm and convergence theory to allow the use of low fidelity adaptive sparse grid models in objective function evaluations this is accomplished by extending conditions on inexact function evaluations used in previous trust region frameworks our algorithm adaptively builds two separate sparse grids one to generate optimization models for the step computation and one to approximate the objective function

hierarchical and high girth qc ldpc codes we present an approach to designing capacity approaching high girth low density parity check ldpc codes that are friendly to hardware implementation and compatible with some desired input code structure defined using a protograph the approach is based on a mapping of any class of codes defined using a protograph into a family of hierarchical quasi cyclic hqc ldpc codes whereas the parity check matrices of standard quasi cyclic qc ldpc codes are composed of circulant submatrices those of hqc ldpc codes are composed of a hierarchy of circulant submatrices that are in turn constructed from circulant submatrices and so on through some number of levels next we present a girth maximizing algorithm that optimizes the degrees of freedom within the family of codes to yield a high girth hqc ldpc code subject to bounds imposed by the fact that hqc codes are still quasi cyclic finally we discuss how certain characteristics of a code protograph will lead to inevitable short cycles and show that these short cycles can be eliminated using a squashing procedure that results in a high girth qc ldpc code although not a hierarchical one we illustrate our approach with three design examples of qc ldpc codes two girth 10 codes of rates 1 3 and 0 45 and one girth 8 code of rate 0 7 all of which are obtained from protographs of one sided spatially coupled codes

design and performance of turbo gallager codes the most powerful channel coding schemes namely those based on turbo codes and low density parity check ldpc gallager codes have in common the principle of iterative decoding however the relative coding structures and decoding algorithms are substantially different this paper shows that recently proposed novel coding structures bridge the gap between these two schemes in fact with properly chosen component convolutional codes a turbo code can be successfully decoded by means of the decoding algorithm used for ldpc codes i e the belief propagation algorithm working on the code tanner graph these new turbo codes are here nicknamed turbo gallager codes besides being interesting from a conceptual viewpoint these schemes are important on the practical side because they can be decoded in a fully parallel manner in addition to the encoding complexity advantage of turbo codes the low decoding complexity allows the design of very efficient channel coding schemes

distance properties of expander codes the minimum distance of some families of expander codes is studied as well as some related families of codes defined on bipartite graphs the weight spectrum and the minimum distance of a random ensemble of such codes are computed and it is shown that it sometimes meets the gilbert varshamov gv bound a lower bound on the minimum distances of constructive families of expander codes is derived the relative minimum distance of the expander code is shown to exceed the product bound i e the quantity spl delta sub 0 spl delta sub 1 where spl delta sub 0 and spl delta sub 1 are the minimum relative distances of the constituent codes as a consequence of this a polynomially constructible family of expander codes is obtained whose relative distance exceeds the zyablov bound on the distance of serial concatenations

performance analysis of iterative decoding algorithms with memory over memoryless channels density evolution is often used to determine the performance of an ensemble of low density parity check ldpc codes under iterative message passing algorithms conventional density evolution techniques over memoryless channels are based on the assumption that messages at iteration l are only a function of the messages at iteration l 1 and possibly the channel output this assumption is valid for many algorithms such as standard belief propagation bp and min sum ms algorithms however there are other important iterative algorithms such as successive relaxation sr versions of bp and ms and differential decoding with binary message passing dd bmp algorithm of mobini et al for which this assumption is not valid the reason is the introduction of memory in these algorithms in this work we propose a model for iterative decoding algorithms with memory which covers sr and dd bmp algorithms as special cases based on this model we derive a bayesian network for iterative algorithms with memory over memoryless channels and use this representation to analyze the performance of the algorithms using density evolution the density evolution technique is developed based on truncating the memory of the decoding process and approximating it with a finite order markov process and can be implemented efficiently as an example we apply our technique to analyze the performance of dd bmp on regular ldpc code ensembles and make a number of interesting observations with regard to the performance complexity tradeoff of dd bmp in comparison with bp and ms algorithms the model presented in this paper is based on certain simplifying assumptions about the memory structure of iterative algorithms such as the existence of memory only at the output of variable nodes in the code s tanner graph rather than at both outputs of variable and check nodes the bayesian network framework introduced here however can still be used to analyze the more general scenarios