## 3 Information Bottleneck Decoders for Irregular LDPC Codes Using Message Alignment
### Message Alignment Algorithm  
To the best of our knowledge, due to the structure of (8) and the restriction to deterministic mappings \(p_{d}(z|t)\), deriving an optimal implicit solution for (8) is not possible. In the literature, iterative algorithms have been proposed to minimize problems involving the Kullback-Leibler divergence [29]. Thus, we propose an iterative algorithm as depicted in Figure 6.  
First, only the independently generated mappings for each node degree and the corresponding degree-specific cluster meanings \(p_{d}(x|t)\) exist. The black squares in Figure 6 depict the actual message alignment steps. According to (8), in each step, a particular \(p_{d}(x|t)\) is aligned with respect to the beliefs of the average distribution \(p_{\bar{d}}(x|z)\) (cf. Figure 6). To start the algorithm, \(p_{\bar{d}}(x|z)\) is initialized with an  
Figure 6: Flow graph of iterative message alignment.  
arbitrary mapping. To find a reordering \(p_{d}(z|t)\) minimizing (8), a search over all candidates \(t\) for the best new cluster index \(z\) is performed. That is,  
\[z=\arg\min_{z^{*}}D_{\text{KL}}\left\{p_{d}(x|T=t)||p_{\bar{d}}(x|Z=z^{*})\right\},\,\forall t, \tag{9}\]  
which yields \(p_{d}(z|t)\). Afterwards, \(p_{d}(x|z)\) is computed using the determined \(p_{d}(z|t)\). Then, \(p_{\bar{d}}(x,z)\) is updated similar to (4), but considering only the weights \(\omega_{d}\) of the already-aligned nodes. As visualized in Figure 6, this averaged distribution \(p_{\bar{d}}(x|z)\) is forwarded and serves as new input in the next alignment step, together with \(p_{\bar{d}^{\prime}}(x|t)\)--that is, the not-aligned cluster meanings of a different node with degree \(d^{\prime}\). Again, the search according to (9) is performed. Once \(p_{d^{\prime}}(z|t)\) is found, the averaged meaning is updated and forwarded. This processes, as shown in Figure 6, is performed for all node degrees and is repeated iteratively.  
Once stable solutions for all message alignment mappings \(p_{d}(z|t)\) are determined, the final average joint distribution \(p_{\bar{d}}(x,z)\) is computed:  
\[p_{d}(x,z)=\sum_{d=2}^{\omega_{\text{max}}}\omega_{d}\sum_{t\in\mathcal{T}} \sum_{\mathbf{y}\in\mathcal{Y}_{d}^{\text{vec}}}p_{d}(z|t)p_{d}(t|\mathbf{y} )p_{d}(x,\mathbf{y}). \tag{10}\]  
This distribution is passed in discrete density evolution and leveraged to design the relevant-information-preserving mappings.  
Figure 5b visualizes the LLRs \(L_{2}(x|z)\) and \(L_{4}(x|z)\), that is, \(L_{2}(x|t)\) and \(L_{4}(x|t)\) after applying the deterministic mapping \(p_{2}(z|t)\) and \(p_{4}(z|t)\) which where found using the algorithm described above. The LLRs are now aligned (i.e., \(L_{2}(x|z)\approx L_{4}(x|z)\)). Consequently, the average LLR \(L_{\bar{d}}(x|z)\) is quite similar to \(L_{2}(x|z)\) and \(L_{4}(x|z)\) (i.e., the original beliefs can propagate through the Tanner graph without significant distortion). The next section investigates how this reduced distortion improves the decoding performance compared to information bottleneck decoders designed without message alignment. The look-up tables used while decoding are \(p_{d}(z|\mathbf{y})=\sum_{t\in\mathcal{T}}p_{d}(z|t)p_{d}(t|\mathbf{y})\).  
We advocate that message alignment does not affect the implementation complexity of an information bottleneck decoder. The look-up tables \(p_{d}(z|\mathbf{y})\) have the same size as \(p_{d}(t|\mathbf{y})\), and \(z\) instead of \(t\) is exchanged over the edges of the Tanner graph. Thus, introducing message alignment neither increases the number of needed lookup operations, nor does it increase the number of required lookup tables. However, when constructing the lookup tables offline, the message alignment algorithm has to be incorporated in the construction step. This results in slightly increased computational complexity of the construction process compared to the construction of information-bottleneck decoders without message alignment.