## 2 Prerequisites
### Information-Bottleneck Signal Processing and Information Bottleneck Graphs  
Despite having its origin in machine learning and numerous classification problems, the information bottleneck has recently been used to design various communication systems. Application fields pairing the information bottleneck and signal processing include LDPC decoding, channel estimation, relaying, C-RAN, polar code construction, sensor networks, etc. [10; 18; 19; 24; 25; 26].  
Figure 1: Illustration of a Tanner graph and message passing between variable nodes (circles) and check nodes (squares).  
Figure 2: Information bottleneck setup, where \(I(X;T)\) is the relevant information, \(I(X;Y)\) is the original mutual information, and \(I(Y;T)\) is the compression information.  
Leveraging information theoretical concepts to design practical systems is termed _information-bottleneck_ signal processing [15]. Information-bottleneck signal processing using the information bottleneck method differs from conventional approaches in mainly three points. The first fundamental difference is the focus on the preservation of the so-called _relevant_ information as a distortion measure in contrast to conventional measures like the Euclidean distance or the mean-squared error. As a result, it is possible to obtain compact but very informative representations of the processed samples. In a second step, these compression mappings or clusterings replace the actual arithmetic operations. Thus, in addition to a reduction of memory due to the compression, the computational complexity is also reduced by replacing arithmetic operations with look-up operations. Finally, information-bottleneck signal processing is a system-oriented design technique rather than a function-oriented technique. That is, instead of finding local approximations of certain functions, the overall aim of information-bottleneck signal processing is to optimize the flow of relevant information in the system from the data aggregation to the decision unit. For a more detailed review of information-bottleneck signal processing, we refer the reader to [15].  
Due to the system-oriented design, finding the most efficient structure which optimizes the flow of relevant information can be quite cumbersome [15]. Therefore, information bottleneck graphs were introduced in [27] which extend factor graphs [28] by a compact description of the _flow of relevant information_ to provide a vivid graphical representation of the system. Information bottleneck graphs indicate where the use of information-bottleneck clusterings is beneficial, and thus help to simplify the design [27]. For this purpose, a modified node symbol is used for all factor nodes representing compression mappings \(p(t|y)\) that were designed using the information bottleneck method. In an information bottleneck graph, the typical square notation of factor nodes is replaced by a trapezoid symbol for all compression mappings. An example is shown in Figure 3. The relevant variable labels are written in the center of the trapezoid symbol to illustrate the information bottleneck with respect to this variable. The compression variable is connected to the shortest side of the trapezoid. All other connected variables correspond to a (possibly multivariate) observed variable. The example in Figure 3 illustrates the flow of relevant information on \(x\) from random vector \(\mathbf{y}=[y_{1},\dots,y_{M}]^{\mathrm{T}}\) to \(t\). That is, \(t=f(\mathbf{y})\) compresses \(\mathbf{y}\) while preserving relevant information on \(x\).