## 3 Results
### Maintenance Operations Detection Results  
In this section, the classification results, using both median and mean filtered data, will be presented. Since this methodology will be used to validate the execution of maintenance operations in an FB, it must avoid the false positives. However, there are few true positive examples in the available data, and this is the reason why there is a thin balance between the recall and precision of each class. If the ANN is trained to detect all operations, it will decrease the precision of the classifier. For the validation dataset, 10 ANN were generated with the previous specifications, to assess the sensitivity of the ANN classifier. The results are presented in Tables 6 and 7. The identification of maintenance operations is not possible in all cases. With the median filtered data, the detection results are worse in the validation dataset. Although, it is important to remember that in the validation dataset only six examples correspond to maintenance operations, so an error in just one example has a huge impact in these metrics. The precision of 57% for the detection of the operations is low, but is needed to guarantee that false positives rarely occur. Additionally, the recall value of 87% shows that most of the operations were detected. A more detailed analysis of the results for the validation dataset showed that the errors usually occur in continuous operations, just one of the months being identified. Evaluating the use of the mean filter, it can be verified that with the validation dataset, there is more precision in the detection (64%), i.e., there are less false detections of maintenance operations. The drawback is that the recall \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Median Filter Data**} & \multicolumn{2}{c}{**Mean Filter Data**} \\ \cline{2-5}
**Detection** & **Yes** & **No** & **Yes** & **No** \\ \hline Recall & 9\% & 98\% & 97\% & 99\% \\ Precision & 94\% & 98\% & 89\% & 97\% \\ F1-Score & 93\% & 98\% & 93\% & 98\% \\ Relative Error & & 3.1\% & & 3.3\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Classification results for the training dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Median Filter Data**} & \multicolumn{2}{c}{**Mean Filter Data**} \\ \cline{2-5}
**Detection** & **Yes** & **No** & **Yes** & **No** \\ \hline Recall & 87\% & 97\% & 77\% & 98\% \\ Precision & 57\% & 99\% & 64\% & 99\% \\ F1-Score & 68\% & 98\% & 70\% & 99\% \\ Relative Error & & 2.9\% & & 2.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Classification results for the validation dataset (average of all generated classifiers). Figure 13: Artificial Neural Network (ANN) cross validation results for different groups of features: (a) error; (b) detection recall. decreases to 77%, being more difficult to detect the real operations. So, the median filter allows more operations detections, but due to its increased sensibility to transitions in the data, it produces more false detections than using the mean filter. Both classifiers were applied to the real case scenario (Marisol) and correctly detected one maintenance operation in March 2018. Unfortunately, there were also two false detections in May and June of 2017. These can be explained with the fact that there was not any clear observation in May 2017 in Marisol, which led to more significant changes in the data in this period. The observations that confirm the results are in Figure 14.  
Finally, relative to the incomplete operations test dataset, the nine cases were divided in three groups of operation completeness. The results by group are presented in Table 8. (they were the same for both noise filters). Only one case was wrongly detected and it was in the 50%-75% group, where completeness of the operation on the FB was 73%. Additionally, in the tested FBs with a partial intervention of less than 50%, no operations were detected.