## 3 Information Bottleneck Decoders for Irregular LDPC Codes Using Message Alignment
### Information Bottleneck Decoders for Regular LDPC Codes  
As mentioned in the Introduction, the need for quasi-continuous LLRs and computationally complex node operations makes belief propagation decoding challenging. Information bottleneck decoders have been shown to overcome this impairment. That is, they pair low implementation complexity and near-optimal performance [10; 12; 13]. The fundamental idea of these decoders is to propagate compressed but highly informative integer-valued messages along the edges of a Tanner graph. In a second step, node operations optimized for discrete input alphabets are designed. These operations are look-up operations mapping a set of discrete incoming messages onto a discrete outgoing message, thereby neglecting the original arithmetic operations. The vital task of this mapping is to ensure that as much _relevant_ information as possible gained by the processing of the incoming messages is contained in the compactly represented output. A suitable information-theoretic framework for such problems is the information bottleneck method [21].  
The information bottleneck method constructs relevant-information-maximizing clusterings given a joint probability distribution of the observed random variable \(Y\) and relevant random variable \(X\). In the context of LDPC decoder design, the observed random variables are the \(M\) incoming discrete messages \(\mathbf{y}=[y_{1},\ldots,y_{M}]^{\mathrm{T}}\), and the relevant random variable \(X\) is a codeword bit. For a variable node, \(X\) represents the underlying code bit \(b_{i}\) of a particular node, whereas if the mapping is designed for a check node, \(X\) represents the (mod 2)-sum of the connected code bits \(b_{1},\ldots,b_{M}\) (cf. Figure 1). The joint distribution \(p(x,\mathbf{y})\) serving as input for the information bottleneck algorithms is determined using discrete density evolution [13]. Although classical density evolution was originally intended to find the decoding thresholds of an LDPC code ensemble, previous works have shown that the joint distributions exchanged in discrete density evolution equal exactly the necessary input distributions for the information bottleneck method [10; 11; 12; 13]. The term _discrete_ in discrete density evolution implies that instead of processing continuous joint distributions, the event space of the observed random variable \(Y\) (i.e., \(\mathcal{Y}=\{0,1,\ldots,|\mathcal{Y}|-1\}\)) is also discrete, meaning that the realizations \(y\) are from a finite alphabet  
Figure 4: (**a**) Discrete memoryless channel (DMC) with subsequent quantizer; (**b**) Information bottleneck graph of the respective information optimum quantizer.  
with cardinality \(|\mathcal{Y}|\). Hence, given an observation vector \(\mathbf{y}\), \(|\mathcal{Y}|^{M}\) possible input combinations exist. To prevent an exponential growth in the number of input combinations while passing \(p(x,\mathbf{y})\) over the Tanner graph during discrete density evolution, \(p(x,\mathbf{y})\) is squeezed through a compact information bottleneck. That is, a relevant-information preserving clustering \(p(t|\mathbf{y})\) is introduced such that the outgoing message \(t\in\mathcal{T}=\{0,1,\ldots,|\mathcal{T}|-1\}\) is from a finite alphabet with cardinality \(|\mathcal{T}|\ll|\mathcal{Y}|^{M}\). Once this clustering is found, the actual decoding simplifies to simple look-ups in offline generated tables, which map the sequence of incoming integers \(\mathbf{y}\) onto an outgoing integer-valued message \(t\). For a more mathematical analysis and a detailed description of information bottleneck decoders for regular codes, we refer the reader to [10; 11; 12; 13].