  In this paper, we study codes with sparse generator matrices. More
specifically, low-density generator matrix (LDGM) codes with a certain
constraint on the weight of all the columns in the generator matrix are
considered. In this paper, it is first shown that when a binary-input
memoryless symmetric (BMS) channel $W$ and a constant $s>0$ are given, there
exists a polarization kernel such that the corresponding polar code is
capacity-achieving and the column weights of the generator matrices are bounded
from above by $N^s$.
  Then, a general construction based on a concatenation of polar codes and a
rate-$1$ code, and a new column-splitting algorithm that guarantees a much
sparser generator matrix is given. More specifically, for any BMS channel and
any $\epsilon > 2\epsilon^*$, where $\epsilon^* \approx 0.085$, an existence of
sequence of capacity-achieving codes with all the column wights of the
generator matrix upper bounded by $(\log N)^{1+\epsilon}$ is shown.
Furthermore, coding schemes for BEC and BMS channels, based on a second
column-splitting algorithm are devised with low-complexity decoding that uses
successive-cancellation. The second splitting algorithm allows for the use of a
low-complexity decoder by preserving the reliability of the bit-channels
observed by the source bits, and by increasing the code block length. In
particular, for any BEC and any $\lambda >\lambda^* = 0.5+\epsilon^*$, the
existence of a sequence of capacity-achieving codes where all the column wights
of the generator matrix are bounded from above by $(\log N)^{2\lambda}$ and
with decoding complexity $O(N\log \log N)$ is shown. The existence of similar
capacity-achieving LDGM codes with low-complexity decoding is shown for any BMS
channel, and for any $\lambda >\lambda^{\dagger} \approx 0.631$.
