# Genetic Algorithms to Maximize the Relevant Mutual Information in Communication Receivers +
## 2 Preliminaries
### Distance Metrics  
In this section, we want to briefly recall two elementary distance metrics for vectors \(\mathbf{y}=[y_{0},y_{1},\dots,y_{N-1}]\) and \(\mathbf{\theta}_{t}=[\theta_{t,0},\theta_{t,1},\dots,\theta_{t,N-1}]\) that will be used frequently in the  
Figure 1: Illustration of a two point crossover in the processing of a genetic algorithm. The genotypes of the children are formed by combining the genotypes of both parents. remainder of the article. A well-known distance measure is the Euclidean distance between \(\mathbf{y}\) and \(\boldsymbol{\theta}_{t}\), i.e.,  
\[d_{\mathrm{E}}(\mathbf{y},\boldsymbol{\theta}_{t})=\sqrt{\sum_{n=0}^{N-1}(y_{n} -\theta_{t,n})^{2}}. \tag{2}\]  
When it comes to implementation, the Euclidean distance has some disadvantages. In particular, taking the square under the root in Equation (2) requires costly multiplications in digital hardware. In addition, the square root is also costly on some signal processing platforms. As a result, in some applications a more favorable distance is the Manhattan distance [31] given by  
\[d_{\mathrm{M}}(\mathbf{y},\boldsymbol{\theta}_{t})=\sum_{n=0}^{N-1}|y_{n}- \theta_{t,n}|. \tag{3}\]  
This distance measure only requires sign inversions and additions which are fairly low-cost operations. Design of Parametrized Compression Mappings That Maximize Relevant Information for Communication Receivers  
The general system setup that we consider in this article is sketched in Figure 2.  
As shown there, we consider a generic receiver-sided signal processing scheme that inputs an observed random variable \(\mathsf{Y}\). The observed random variable \(\mathsf{Y}\) is a random vector with realizations \(\mathbf{y}=[y_{0},y_{1},\ldots,y_{N-1}]\) because many signal processing components in communications process more than one scalar input variable. The system has \(M\) tunable parameters \(\theta_{m}\in\mathbb{R}\), \(m\in\{0,1,\ldots,M-1\}\). The design idea for tuning the parameters \(\theta_{m}\) is choosing them, such that the mutual information \(\mathrm{I}(\mathsf{X};\mathsf{T})\) is maximized. Like this, the system output \(\mathsf{T}\) shares a desired huge amount of information with the relevant random variable \(\mathsf{X}\). We consider the system output \(t\in\mathcal{T}\) to be from some finite set \(\mathcal{T}\) with cardinality \(|\mathcal{T}|\). Only this cardinality \(|\mathcal{T}|\), the mapping rule of \(\mathbf{y}\) onto \(t\) implied by \(t=f_{\boldsymbol{\theta}}(\mathbf{y})\) and the joint probability distribution \(p(x,\mathbf{y})\) determine \(\mathrm{I}(\mathsf{X};\mathsf{T})\). In contrast, \(\mathrm{I}(\mathsf{X};\mathsf{T})\) does not depend on the particular elements of \(\mathcal{T}\). The reason is that \(\mathrm{I}(\mathsf{X};\mathsf{T})\) is determined only by the probability distributions \(p(x,t)\), \(p(x)\) and \(p(t)\), as this mutual information is given by  
\[\mathrm{I}(\mathsf{X};\mathsf{T})=\sum_{x\in\mathcal{X}}\sum_{t\in\mathcal{T} }p(x,t)\,\log\frac{p(x,t)}{p(x)\,p(t)}. \tag{4}\]  
After all, the considered system design can be understood as an instance of the information bottleneck method described in Section 2.1. In contrast to the classical information bottleneck approach from [1], however, a parametrized system design for the mapping of realizations \(\mathbf{y}\) onto \(t\) by \(t=f_{\boldsymbol{\theta}}(\mathbf{y})\) is considered here. In addition, the choice of the output cardinality \(|\mathcal{T}|\) allows us to adjust an inherent compression level achieved by the system, as this cardinality determines the number of bits required to represent the system output. Figure 2: Optimizing parameters in a parametrized information bottleneck like setup. The parameters \(\boldsymbol{\theta}\) shall be tuned to maximize \(\mathrm{I}(\mathsf{X};\mathsf{T})\) for a given parametrized function \(f_{\boldsymbol{\theta}}(\mathbf{y})\). The system design approach introduced in Figure 2 has very intuitive applications in the communications context.