## 2 The Information Bottleneck Method and Coarsely Quantized Information Bottleneck Signal Processing
### General View on Information Bottleneck Signal Processing for Receiver Design  
Conceptually, the purpose of the information bottleneck method is intuitively related to the most famous fundamental problem of communications cited at the beginning. In  
Figure 3: Examples of information bottleneck graphs. The trapezoid nodes correspond to the compression mappings \(p(t|y)\) and \(p(t|y_{0},y_{1},y_{2},y_{3},y_{4})\), respectively. Both are designed to preseve \(\mathsf{I}(\mathsf{X};\mathsf{T})\). order to explain this connection, Figure 4 compares the information bottleneck method at the top with a model view of a communication system at the bottom. As shown in the figure and discussed above, the transmitted user data \(\mathsf{X}\) can be considered to be relevant, and the task is to estimate the realization \(x\) after the transmitter has transformed it into a transmit signal \(s(x)\) to prepare it for the transmission over a physical transmission channel. The transmission of \(s(x)\) then results in the received realization \(\hat{y}\), which is typically continuous. As shown in Figure 4, we assume that \(\hat{y}\) is quantized to a discrete representation \(y\) by an analog-to-digital converter with the quantization function \(f_{\mathsf{Q}}(. )\) in the receiver (i.e., \(y=f_{\mathsf{Q}}(\hat{y})\)). The quantized representation \(y\) is then used for further digital signal processing steps to estimate \(x\). Digital communication receivers often have to process huge blocks of the received samples at once due to the transmitter-side channel coding and modulation techniques that spread information on a transmitted bit \(x\) over a huge block of transmitted symbols. To obtain a simple notation that is coherent with the information bottleneck notation used above, we do not explicitly highlight the fact the variables \(\mathsf{X},\mathsf{Y}\) and \(\mathsf{T}\) could be random vectors in the communication system from Figure 4. However, one should keep in mind that a communication receiver typically estimates a vector \(\mathbf{x}\) of transmitted bits from a long vector \(\mathbf{\hat{y}}\) of channel observations. In this case, the application of \(f_{\mathsf{Q}}(. )\) has to be understood as an element-wise application (i.e., \(y_{k}=f_{\mathsf{Q}}(\hat{y}_{k})\), with the time index \(k\) enumerating subsequent samples). The amount of information present about \(\mathsf{X}\) in the quantized received signal \(\mathsf{Y}\) is bounded by the capacity of the quantized output transmission channel \(p(y|x)\) at the upper bound, which is given by  
\[C=\max_{p(x)}\mathrm{I}(\mathsf{X};\mathsf{Y})\leq\max_{p(x)}\mathrm{I}( \mathsf{X};\tilde{\mathsf{Y}}). \tag{6}\]  
Hence, the task of the communication receiver is to extract the information on the relevant \(\mathsf{X}\) and to provide the best possible estimate \(\hat{x}\) at its output. Clearly, such an estimate can be obtained from a receiver output variable \(\mathsf{T}\) that fulfils \(\mathrm{I}(\mathsf{X};\mathsf{T})\to\max\). As a result, the further receiver-side digital signal processing modeled by \(p(t|y)\) should maximize  
Figure 4: Illustration of the information bottleneck design idea of a communication system. The digital communication receiver shall be designed such that the relevant information \(\mathrm{I}(\mathsf{X};\mathsf{T})\) is maximized from end to end. I(X;T). Then, \(p(x|t)\) can provide an estimate of the transmitted data \(x\), for example, using the maximum a-posteriori criterion:  
\[\hat{x}=\arg\max_{x}p(x|t). \tag{7}\]  
Please note that \(p(x|t)\) is inherently obtained when the information bottleneck method is applied according to Equation (4). This already indicates that it is possible to estimate \(x\) based on a compressed representation \(t\) of the quantized received signal \(y\).