# Semi-Supervised GNSS Scintillations Detection Based on DeepInfomax
## 3 The DeepInfomax Approach  
In this section, we give a brief overview of the DeepInfomax approach [21]. Appendix A provides mode details about the technique. Figure 2 schematically depicts the idea.  
The main idea is to build a two stage compression, using a cascade of two neural networks \(e_{\eta}^{1}(\cdot)\) and \(e_{\beta}^{2}(\cdot)\), of an input \(X\), respectively generating a first equivalent feature \(H=e_{\eta}^{1}(X)\) and a second one \(F=e_{\beta}^{2}(H)\), such that the Mutual Information (MI) between \(H,F\) is maximized. In practice, as also explained in [21], a proxy to the mutual information that worked better in practice, the Jensen Shannon Divergence, was used. The authors in [21] introduce, and also maximize, a new metric: the Local Mutual Information. It is defined as the sum of all mutual information between each element \(H^{(z)},\quad z=1,\cdots|H|\) of the variable \(H\), that is, a multidimensional random vector and the whole feature vector \(F\). In equations, \(MI_{loc}=\frac{1}{|H|}\sum\limits_{z=1}^{|H|}\mathrm{I}\left(H^{(z)};F\right)\), where \(I(A,B)\) is the mutual information (MI) between \(A\) and \(B\).  
The sketch of the architecture we consider in our work is depicted in Figure 3. We found that, for this work, considering only the local Mutual Information was the best choice in terms of performance. Moreover, a _Uniform Prior regularizer_, similarly to the one described in [21], was included to ensure that the features \(F\) cover evenly the latent space.  
It is possible to show ([21; 22], and also Appendix A) that a Local MI estimator can be built as depicted in Figure 4.  
Figure 4: Detailed representation of high level scheme depicted in Figure 3.  
Figure 3: High level scheme of architecture used in this work.  
Figure 2: Deep Infomax–high level scheme.  
Couples extracted from the joint and marginal distributions of \(H,F\) are built (respectively \(p(H,F)\), \(p(H)p(F)\)). These couples are fed to two copies of the same network \(g_{\theta}^{loc}(\cdot)\) whose output is the input of a cost function described by Equation (A7). Notice that, to build couples sampled from the joint distribution \(p(H,F)\), it is sufficient to observe joint realizations of variables \(H,F\), while, to build samples whose distribution is the product of the marginals \(p(H)p(F)\), we can instead randomly shuffle one of the two sets of variables \(H\) or \(F\) and build new couples with the scrambled versions.  
To perform regularization instead, always Figure 4, a discriminator network is fed with both the random variables uniformly generated and the features \(F\), and trained to distinguish among the two. The rest of the system is trained adversarially [27] to fool the discriminator, and it is possible to show that, in an equilibrium regime, the distribution of \(F\) is the multivariate uniform distribution.  
The whole process serves the purpose of extracting high quality and informative feature vectors \(F\) in an unsupervised way (i.e., no external labels are needed), which can be used for other downstream tasks such as classification or clustering.