## 3 Parameter Learning of Trainable Functions to Maximize the Relevant Information
### Lookup Tables  
As mentioned in Section 2.1, it is intuitive that any deterministic compression mapping \(p(t|y)\) for discrete \(t\) and \(y\) can be implemented in a lookup table. For that purpose, one just has to store the respective \(t\in\mathcal{T}\) for all possible \(y\in\mathcal{Y}\). Of course, this also holds if the realizations of \(\mathcal{Y}\) are random vectors \(\mathbf{y}=[y_{0},y_{1},\ldots,y_{N-1}]\). For simplicity, we assume that \(y_{n}\in\{0,1,\ldots,|\mathcal{Y}|_{n}-1\}\) in this section (i.e., the \(y_{n}\) in vector \(\mathbf{y}\) are unsigned integers). Please note that the elements of the event space are totally irrelevant for the mutual information \(\mathrm{I}(\mathcal{X};\mathcal{Y})\), as this mutual information only depends on the probabilities implied by \(p(x,\mathbf{y})\), \(p(x)\) and \(p(\mathbf{y})\). We can easily understand a lookup table holding the \(t\) for each possible \(\mathbf{y}\) as a trainable function \(f_{\theta}(\mathbf{y})\) with parameters \(\theta\). Consider a software implementation of a lookup table consisting of a row vector \(\boldsymbol{\theta}=[\theta_{0},\theta_{1},\ldots,\theta_{|\mathcal{Y}|-1}]\). This vector holds the outcome \(t\) for every possible input vector \(\mathbf{y}\) at a certain position that is termed the address. To obtain the output \(t\), the input \(\mathbf{y}\) has to be mapped onto the address of the corresponding output \(t\) in the vector. Let the function implementing this address transformation be denoted by \(\xi(\mathbf{y})\). It is not important which address transformation \(\xi(\mathbf{y})\) is used as long as it maps the input vector \(\mathbf{y}\) onto the address of the corresponding \(t\in\mathcal{T}\) uniquely. Then, the lookup table delivers \(t=f_{\theta}(\mathbf{y})=\theta_{\xi(\mathbf{y})}\) by just accessing the vector \(\boldsymbol{\theta}\) at the calculated address. With this quite formal description of a simple lookup table, it is clear that the problem of maximizing the preserved relevant information \(\mathrm{I}(\mathcal{X};\mathcal{T})\) with \(t\in\mathcal{T}\) and \(\mathbf{y}\in\mathcal{Y}\) is formally equivalent to determining a vector of the optimum discrete parameters:  
\[\theta_{\beta\to\infty}^{\mathrm{IB}}=\arg\max_{\boldsymbol{\theta}}\mathrm{ I}(\mathcal{X};f_{\theta}(\mathcal{Y})). \tag{20}\]  
Therefore, we can summarize that any deterministic mapping of \(\mathbf{y}\) onto \(t\) that is implemented in a lookup table and determined to maximize the preserved relevant information with an information bottleneck algorithm can equivalently be interpreted as a trainable function \(f_{\theta}(\mathbf{y})\) with a maximum of \(|\mathcal{Y}|\) discrete parameters \(\theta_{\xi(\mathbf{y})}\in\mathcal{T}\). The number of parameters \(|\mathcal{Y}|\) is identical to all possible input configurations \(\mathbf{y}\in\mathcal{Y}\). This allows representing arbitrary mappings in this setup and has the consequence that a lookup table can implement every possible input/output relation for discrete \(\mathbf{y}\) and \(t\), including one that maximizes \(\mathrm{I}(\mathcal{X};\mathcal{T})\) globally. This holds independent of the probability distribution \(p(x,\mathbf{y})\). Yet, we note that finding an optimum mapping with the available information bottleneck algorithms is not guaranteed. However, for discrete \(\mathbf{y}\in\mathcal{Y}\) and discrete \(t\in\mathcal{T}\), we can conclude that no other mapping can offer more flexibility than a lookup table. Therefore, in theory, a lookup table can be seen as an optimum choice in terms of providing the possibility to maximize \(\mathrm{I}(\mathcal{X};f_{\theta}(\mathcal{Y}))\) with properly tuned parameters \(\boldsymbol{\theta}\). The way to learn these parameters in the receiver presented above was feeding the joint probability distributions \(p(x,\mathbf{y})\) to the information bottleneck algorithms.