# Semi-Supervised GNSS Scintillations Detection Based on DeepInfomax
## Appendix A Variational Representation of Divergence and Deep Infomax  
In this appendix, we briefly review the concept of convex divergence between probability distributions, the particular case of mutual information (MI), and its usage for the construction of a feature extractor.  
The class of f-convex divergences between distributions of two random variables \(X_{1}\sim p(x)\), \(X_{2}\sim q(x)\) is defined as  
\[\mathrm{D}(p||q)=\int p(x)f\left(\frac{q(x)}{p(x)}\right)dx. \tag{1}\]  
Besides from very special cases, such as multivariate Gaussian distributions, and simple \(f\) functions, no closed form exists for the computation. Moreover, in general, not even a parametric model about the distributions \(p,q\) is known, but all the information we have about the density functions is related to same empirical samples \(\{x_{1}^{i}\}_{i=1}^{N_{1}},\{x_{2}^{j}\}_{j=1}^{N_{2}}\) drawn from the corresponding distributions.  
The main idea of variational representation is that, roughly speaking, thanks to convex analysis, we can write the divergence as  
Figure 8: Feature importance analysis using random permutation method. The lower the accuracy with respect to the baseline, the higher the feature importance.  
\[\mathrm{D}(p||q)=\sup_{g(x)\in\mathcal{G}}\left\{\int p(x)g(x)dx-h\left(\int q(x)t( g(x))dx\right)\right\}, \tag{10}\]  
where \(h,t\) are analytically known functions that can be computed once \(f\) is fixed. Thanks to this representation, we can estimate the divergence without knowing the actual distributions \(p,q\) using Monte Carlo methods:  
\[\mathrm{D}(p||q)\simeq\sup_{g(x)\in\mathcal{G}}\left\{\sum_{i=1}^{N_{1}}g(x_{1 }^{i})-h\left(\sum_{j=1}^{N_{2}}t(g(x_{2}^{j}))\right)\right\}. \tag{11}\]  
An attentive reader might notice that since there is the problem of extremization \(\sup_{g(x)\in\mathcal{G}}\), we just moved the complexity of the problem to another point. The main idea discussed in [22], is that, however, what holds however is that  
\[\mathrm{D}(p||q)=\sup_{g(x)\in\mathcal{G}}\left\{\cdot\right\}\geq\sup_{g(x) \in\mathcal{N}}\left\{\cdot\right\},\]  
where the space \(\mathcal{N}\subset\mathcal{G}\) is chosen such that we can perform \"easily\" optimization (such as the space of Neural Networks, i.e., \(g_{\theta}(x)\) is a neural network with parameters \(\theta\)). We can finally estimate the divergence with the following approximation by optimizing (i.e., training) the network and using Monte Carlo techniques  
\[\mathrm{D}(p||q)\simeq\sup_{\theta}\left\{\sum_{i=1}^{N_{1}}g_{\theta}(x_{1}^ {i})-h\left(\sum_{j=1  
_Infomax._ The (oversimplified) description of the system, schematically depicted in Figure 11 is that, using two neural network encoders \(e_{\eta}^{1}(\cdot),e_{\beta}^{2}(\cdot)\) and two discriminator networks \(g_{\theta}^{glob}(\cdot),g_{\theta}^{loc}(\cdot)\), it is possible to produce two latent representations \(H,F\), respectively computed as \(H=e_{\eta}^{1}(X),F=e_{\beta}^{2}(H)\), such that both the MI between \(H,F\) and the Local Mutual Information \(MI_{loc}=\frac{1}{|H|}\sum\limits_{z=1}^{|H|}\mathrm{I}\left(H^{(z)};F\right)\) (a new metric introduced by the authors of [21]) are maximized. We moreover clarify the role of \(r_{\zeta}(\cdot)\) that is a discriminator network that tries to distinguish between samples drawn from \(F\) and samples drawn from a multivariate uniform distributions (\(\tilde{F}\sim U(-1,1)\)). The system is trained adversarially [27], where \(r_{\zeta}(\cdot)\) and the rest of the network are the two competing agents (at equilibrium \(F\) has uniform distribution and this property is used to ensure that the latent variable covers evenly the latent space as a form of regularization). Notice that, in our implementation (see Section 5 and Appendix B), we only considered local mutual information and did not implement the portion of the system linked to the global, i.e., classical, mutual information.