## 3 Parameter Learning of Trainable Functions to Maximize the Relevant Information
### Computational Domain Technique  
The authors of [24; 25] proposed an interesting idea to implement check and variable node operations of LDPC decoders that, just as with the information bottleneck approaches based on lookup tables described above, aims at maximizing the preserved relevant information under compression. Their approach is called mutual information-maximizing quantized belief propagation decoding. We briefly recall this approach that is illustrated in Figure 13 in the following. The authors of [24; 25] distinguished between variable and check node operations in an LDPC decoder. Here, however, we study a general input/output relation that maps an incoming vector \(\mathbf{y}=[y_{0},y_{1},\ldots,y_{N-1}]\) onto an outgoing \(t\). For simplicity, we assume the most practical approach of all \(y_{n}\) and \(t\) to be from the same set \(\{0,1,\ldots,2^{q}-1\}\) of \(q\)-bit unsigned integers. The computational domain approach consists of three subsequent steps [24; 25]:  
1. Use a predefined reconstruction function \(\phi(. )\) to transfer the incoming messages \(y_{n}\) to numbers \(\phi(y_{n})\) in a computational domain \(\mathbb{D}\);
2. Use a function \(\Phi:\mathbb{D}^{N}\longrightarrow\mathcal{A}\) to process the numbers in the computational domain and to map them onto a single number \(a\in\mathcal{A}\);
3. Apply a scalar quantizer \(Q_{\theta}(. )\) with \(2^{q}-1\) ordered thresholds \(\boldsymbol{\theta}=[\theta_{0},\theta_{1},\ldots,\theta_{2^{q}-2}]\) on \(a\) that quantizes \(a\in\mathcal{A}\) back to the set \(\{0,1,\ldots,2^{q}-1\}\). In [24; 25], certain reconstruction functions for the check and variable nodes of LDPC decoders and also reasonable functions \(\Phi(. )\) for these applications, which we do not recall here in detail for brevity, are proposed. The idea of maximizing the preserved relevant mutual information in the information bottleneck sense is applied in the design of the scalar quantizer \(Q_{\theta}(. )\) in the computational domain approach from [24; 25]. The thresholds \(\boldsymbol{\theta}=[\theta_{0},\theta_{1},\ldots,\theta_{2^{q}-2}]\) of this quantizer are chosen such that \(\mathrm{I}(\mathsf{X};\mathsf{T})\) is maximized. Of course, one can understand the quantization thresholds stored in vector \(\boldsymbol{\theta}=[\theta_{0},\theta_{1},\ldots,\theta_{2^{q}-2}]\) as parameters of a function implementing the node operation  
\[t=f_{\boldsymbol{\theta}}(\mathbf{y})=Q_{\theta}(\Phi(\phi(y_{0}),\phi(y_{1}), \ldots,\phi(y_{N-1}))), \tag{21}\]  
Figure 13: Illustration of the computational domain approach for information bottleneck-like LDPC decoding from [24; 25]. Figure adapted from [24; 25] with minor adaptions of the notation. such that the design of the node operation ends up in the exact same problem formulation as that given in Equation (20) (that is, the learning parameters \(\mathbf{\theta}\) of a trainable function \(f_{\mathbf{\theta}}(\mathbf{y})\) such that \(\mathrm{I}(\mathrm{X};f_{\mathbf{\theta}}(\mathrm{Y}))\longrightarrow\mathrm{max}\)). The major difference is that the number of parameters that are tuned to maximize \(\mathrm{I}(\mathrm{X};f_{\mathbf{\theta}}(\mathrm{Y}))\) can be drastically reduced in the computational domain approach with respect to the lookup table. Essentially, only \(2^{q}-1\) quantization thresholds instead of \(2^{qN}\) entries of a lookup table need to be learned and optimized to maximize \(\mathrm{I}(\mathrm{X};\mathrm{T})\). Interestingly, the computational domain approach employs arithmetical operations in the function \(\Phi(.)\). In return, it needs much fewer parameters to preserve significant amounts of \(\mathrm{I}(\mathrm{X};\mathrm{T})\) than the lookup table approach. We note, however, that the choice of the reconstruction function \(\phi(. )\) and the function \(\Phi(. )\) as well as their abilities to preserve significant amounts of \(\mathrm{I}(\mathrm{X};\mathrm{T})\) are problem-specific and depend on \(p(x,\mathbf{y})\).