# Deep Learning Methods for Space Situational Awareness
## 4 Results and Discussions  
In order to carry out our experiments, we have built a dataset that has been structured as follows: range-Doppler maps have been generated as the position and speed varied, and by assigning to each spectrum one of four possible labels, where each represented the number of targets detected (zero if there are no objects, one if there is only one target and so on). Once the images were generated, they were given as input to neural networks for classification. The networks, therefore, classify whether the image, when the position of the object and the speed at which it moves varies, belongs to the first, second, third, or fourth case mentioned. Figures 5-7 show the range-Doppler maps in presence of no object, and one, two, and three objects moving. The positions of each target change from a range from 1 to 3000 m, and the maximum detectable speed is 225 m/s. The entire dataset consists of 800 images, and it was exploited as follows: 640 images (80% of the dataset) were used for the training phase, while 160 images (20% of the dataset) were used for testing the networks. In addition, we used the stochastic gradient descent with momentum (SGDM) optimizer, and the learning rate is 0.0003 with 50 epochs and 50 batch sizes. Hyperparameters are chosen by looking at the literature experiences and best practices. In this sense, hyperparameter optimization is out of the scope of this study. Nevertheless, it should be explored in the future and extended works. As an example, Figure 8 illustrates the training curves of the SqueezeNet network (the learning curves of the other approaches are not reported here for the sake of space). Figure 5: Example of range-Doppler maps generated with (respectively, from the upper left image to the lower right image) 0, 1, 2, 3 targets. Figure 6: Example of range-Doppler maps generated with fixed positions at different speeds of (respectively, from the left to right) 1, 2, 3 targets. Figure 7: Example of range-Doppler maps generated with speeds of 225 m/s and different positions of (respectively, from the left to right) 1, 2, 3 targets. To understand the performance of the considered deep learning classifiers, we have exploited the following metrics:  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{ FN}} \tag{1}\]  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{2}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{3}\]  
\[\text{F}-\text{Measure}=\frac{2\text{Precision Recall}}{\text{Precision}+ \text{Recall}} \tag{4}\]  
True positives (TP) are the number of samples correctly classified as belonging to its true class, while true negatives (TN) are the number of samples correctly classified as not belonging to a given class. False positives (FP) are the number of samples misclassified as coming from a given class, but they belong to another class. On the contrary, false negatives (FN) are the number of samples classified as belonging to a class that does not correspond to the true class. Accuracy, as depicted in (1), is a function of these quantities and it provides an intuitive way to determine which model is best at classifying input data. Furthermore, the better a model can generalize the 'unseen' data, the better predictions and insights it can produce. Precision, (2), quantifies the number of positive class predictions that belong to the positive class, while recall, as noted in (3), is the number of positive class predictions obtained from all positive examples in the dataset. The advantage of using precision and recall is that they can fairly describe classification performance also in presence of unbalanced datasets. Finally, the F-Measure in (4) provides a single score that balances both the concerns of precision and recall in one number. In Figure 9, we can see the results in terms of the overall accuracy of the SqueezeNet, VGG-16, GoogLeNet, and AlexNet networks. The obtained results are very good, and all oscillate around 85-95%.