# Semi-Supervised GNSS Scintillations Detection Based on DeepInfomax
## 7 A Qualitative View on Features Importance  
Neural networks are known for being black box models, in which obtaining human understandable information from the inner functioning is extremely complex, if not impossible. Several techniques have been proposed to determine and inspect feature importance on networks decisions, usually with specific methods tailored to particular experiments and particular network architectures.  
In this subsection, we present results on features importance based on the simple but powerful features random permutation technique (similarly to the method introduced in [31]). The idea is to consider a trained classifier, randomly shuffle the variables one by one, and record the decrease in terms of performance. The measured decrease for all the shuffling experiments with respect to the baseline (the unshuffled dataset) is used as a proxy to determine the shuffled feature importance. Notice that this investigation strategy somehow lacks the capability of measuring joint higher order importance of features. In fact, two or more features can be extremely correlated and important for the considered task, but shuffling just one of them possibly does not impact performance in a substantial way, producing thus an underestimation of the feature importance.  
In Figure 8, we report the results of permutation tests on the classifier trained with percentage of labeled examples equal to 10%. Despite the possible limitations of the considered measurement technique, we obtain results that are consistent with the physical knowledge of the problem as well as previous results [11].  
The features in order of importance (computed as the decrease in terms of accuracy with respect to the baseline) are: \(\left[S_{I},\theta_{el},S_{4},C/N_{0},\zeta,\theta_{az},I,Q,\sigma_{\varphi}, \phi\right]\). In the considered data collections, amplitude scintillations' events are observed. It is thus reasonable to assess that Signal Intensity, \(S_{I}\), is the most important feature, with a drop in accuracy of more than 40% when the feature is randomly permuted. The three features \(\theta_{el},S_{4},C/N_{0}\) when permuted induces basically the same drop in terms of accuracy. Elevation angle \(\theta_{el}\) is informative for classification: signal statistics for low elevation satellites greatly vary from the high in sky ones. The amplitude scintillation index \(S_{4}\) is clearly an important feature since amplitude scintillations events are considered. Finally, the \(C/N_{0}\) is yet another indirect measurement of signal intensity. Notice, moreover, that \(\theta_{el},S_{4},C/N_{0}\) are the three features used for the so-called semihard detection rule [11]. The raw phase and the azimuth angle appear to contain less information about the scintillation events and the features \(I,Q,\sigma_{\phi},\phi\) have the lowest impact when randomized. It is possible that the relevant information contained in \(I,Q\) is already summarized in \(S_{I}\), while being the scintillation events linked to amplitude fluctuations the scintillation phase information is not that important.  
A complete analysis of feature importance would require to retrain from scratch different versions of the classifier by completely excluding some of the features from the beginning of training, but, since this is computationally expensive and outside the scope of this paper, we leave this analysis for future works.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**\% Labeled** & **Accuracy** & **Precision** & **Recall** & **F-Score** & \(\mathbf{T_{n}}\) & \(\mathbf{F_{p}}\) & \(\mathbf{F_{n}}\) & \(\mathbf{T_{p}}\) \\ \hline
10 & 0.9633 & 0.9254 & 0.9267 & 0.9260 & 0.7339 & 0.0185 & 0.0182 & 0.2294 \\
20 & 0.9798 & 0.9757 & 0.9444 & 0.9598 & 0.7389 & 0.0060 & 0.0142 & 0.2409 \\
30 & 0.9929 & 0.9858 & 0.9860 & 0.9859 & 0.746