# Neural Network Based Approach to Recognition of Meteor Tracks in the Mini-EUSO Telescope Data
## 4 Results
### Active Pixel Selection  
At the second step, we want to separate pixels of 3-dimensional meteor chunks selected by the CNN into two groups: those containing the signal of a meteor (active pixels) and all the rest. Since meteor signals have a typical shape resembling the bell curve, as shown in the top left panels of Figures 1 and 2, it is straightforward to train a multilayer perceptron (MLP) to solve the task. The input dataset consists of vectors of length \(T=48\) now. We followed the same way of training and testing the neural network as was used at the first stage. Namely, we trained an MLP on data extracted from seven sessions and tested it on the remaining one with all possible combinations of sessions. In order to avoid duplicate entries in the training dataset, we extracted data vectors from chunks of the size \(48\times 48\times 48\). Due to a comparatively small shift \(dt=8\) GTUs, we obtained samples with meteor signal peaks located at almost all possible positions along the time axis. The number of vectors (pixels) containing meteor signals in training data sets was up to 30 thousand, with non-meteor samples ten times greater. The input data was standardized similar to the CNN case. Twenty percent of training samples were utilized for validation. Binary crossentropy was used as the loss function and validation loss was employed to adjust the learning rate and to avoid overfitting. Testing was performed on all vectors extracted from meteor chunks of the size \(8\times 8\times 48\) selected by the CNN. The number of chunks used for testing of each particular session data can be found in Table 1.  
We compared a number of possible configurations of simple MLPs with one, two, and three hidden layers and different number of neurons. An optimizer, activation functions and a performance metric used for training were the same as for the CNN described above. Table 3 presents results obtained with a two-layer MLP with 96 and 64 neurons in the two layers, respectively. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline
**Test Session** & **5** & **6** & **7** & **8** & **11** & **12** & **13** & **14** \\ \hline Training meteor chunks & 91,060 & 72,124 & 95,924 & 81,188 & 80,724 & 88,456 & 89,228 & 86,820 \\ \hline Testing meteor chunks & 1712 & 6446 & 474 & 4180 & 4274 & 2341 & 2148 & 2772 \\ Testing meteors & 65 & 280 & 18 & 186 & 193 & 106 & 90 & 130 \\ \hline \end{tabular}
\end{table} 
Table 1: The top row: sessions of observations used for testing CNNs trained on data of all other sessions. The second row: the total number of \(8\times 8\times 48\) chunks (after augmentation) with signals of meteors used for training the respective CNNs. The number of original chunks is four times less. The last two rows: the number of chunks with meteor signals and the real number of meteors, respectively, in test sessions. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline
**Test Session** & **5** & **6** & **7** & **8** & **11** & **12** & **13** & **14** \\ \hline ROC AUC & 0.992 & 0.994 & 0.999 & 0.993 & 0.994 & 0.998 & 0.993 & 0.996 \\ PR AUC & 0.937 & 0.955 & 0.876 & 0.933 & 0.946 & 0.973 & 0.931 & 0.956 \\ MCC & 0.872 & 0.894 & 0.732 & 0.857 & 0.888 & 0.921 & 0.782 & 0.901 \\ F1 & 0.873 & 0.901 & 0.718 & 0.863 & 0.892 & 0.922 & 0.776 & 0.904 \\ \hline FNR (met) & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline \end{tabular}
\end{table} 
Table 2: Performance of the CNN on different sessions of observations. See the text for details. Similar to Table 2, Table 3 presents results illustrating performance of the MLP trained and tested on different sessions of data collection. Values of one more performance metric are shown here, namely, the mean values of the intersection-over-union (IoU) score. This function and its versions are often used in tasks of labeling pixels of images. It can be written as  
\[\text{IoU}=\frac{\text{TP}}{\text{TP}+\text{FP}+\text{FN}}.\]  
It can be seen that values of the mean IoU metric are slightly above those of the \(F_{1}\) score.