## 5 Results
### Direct Comparison for Simple Detection  
A first meaningful test to perform is an exact comparison between classic threshold-based detection and innovative techniques. Since variance-based detection is able to emit three flags plus a simple binary \"out-of-range\" check, classifiers are trained on a balanced dataset, whose entries exhibit nominal, anomalous, stuck, and loss-of-data behaviors: anomalous behavior can mean any of the faults among bias, erratic, drift, and spikes. Standard threshold-based detection is fed 20 s snippets of signals, as a moving window observing the live evolution of parameters: Machine Learning algorithms are trained on a feature pool extracted by the same dataset, which splices up dataset entries to mimic the same scanning window, as seen in Figure 9.  
By looking at the accuracy results obtained in Table 5, some considerations can already be made. First off, the consistency of the column dedicated to classic detection, except for one outlier in the last row, reflects how fine-tuning of both faults and the detection thresholds makes it extremely easy to reach high detection rates. The accuracy percentage is not 100% only because classic methods are particularly in trouble when trying to detect drifting signals: if the slope is not steep, variance-based detection is not triggered unless it rapidly comes back to nominal values, therefore generating a spike in the moving variance (see Figure 6b). However, if the drift does not end before the end of the simulation, and no instrument range is crossed, classic detection might not be triggered at all. In this testing campaign, classic detection is considered successful only if the correct flag is triggered in \begin{table}
\begin{tabular}{l c c} \hline
**Name** & **Kernel (SVM)** & **Hidden Layer Nodes (ANN)** \\ \hline Linear SVM & \(x_{1}^{T}x_{2}\) & - \\ Quadratic SVM & \((x_{1}^{T}x_{2}+1)^{2}\) & - \\ Cubic SVM & \((x_{1}^{T}x_{2}+1)^{3}\) & - \\ Gaussian SVM & \(\exp(-||x_{1}-x_{2}||^{2}/(2\sigma^{2}))\) & - \\ Narrow Neural Network & - & [10] \\ Medium Neural Network & - & [25] \\ Large Neural Network & - & [100] \\ Bilayered Neural Network & - & [10/10] \\ Trilayered Neural Network & - & [10/10/10] \\ \hline \end{tabular}
\end{table} 
Table 4: Classifiers employed during the testing campaign. the first 5 s from the start of a fault: given that faults are programmed to last at least 15% of the time span of the full dataset entry, it is safe to assume that drifts might be almost never detected by variance-based methods. By considering that drifting signals are one of the possible four behaviors labeled under \"Anomaly\", and that there are four labels in total, we obtain a probability of a drifting trend happening in any observed signal of \(0.25\times 0.25=0.0625\): classic detection misses exactly 6.25% to reach complete successful detection of the dataset, which are all instances of drifts. Shifting the focus on classifiers, and by looking at the results mode by mode, some small increase in accuracy can be perceived on those signals whose trend is notoriously smoother (for example, Sun position sensors during a quite steady Earth-pointing mode); apart from that, Gaussian SVMs are reportedly the most unstable and less predictable in terms of behavior. No particular suggestion on the use of SVMs rather than ANNs can be confidently given: this will come down to ease of implementation, simplicity in design, and computational resources used during their activity, aspects which will be tackled later. The main takeaway from this first batch of tests is that Machine Learning classifiers, when tasked with the same exact duty as the benchmark detection method, reach medium-high accuracy, but are not robust enough to let them deal on their own with this task with no external checks.