# Deep Learning Methods for Space Situational Awareness
## 3 Case Study
### Neural Network Frameworks  
The range-Doppler maps are then used as inputs for the deep learning frameworks that we use for comparisons. We test convolutional neural networks to work correctly and efficiently with 2D image inputs. The chosen networks were SqueezeNet, VGG-16, AlexNet, and GoogLeNet, adjusting their parameters and levels for this specific use case:  
* SqueezeNet is a very light architecture which, nevertheless, achieves outstanding performance in computer vision tasks. The basic idea of the SqueezeNet network is to create a small neural network with few parameters, which can easily adapt to portable devices, thus having a lower computational burden, lower memory demand, and reduced inference time. It is made up of 18 layers. The compression layer consists of 1-by-1 convolutions, which combine all input data channels into a single channel. This procedure reduces the number of inputs for the next level. Data reduction is obtained also by using max-pooling layers, which perform a pooling operation that calculates and retains the maximum value of each patch inside each feature map. In a SqueezeNet, the last learnable layer is the final convolutional layer, unlike in most networks, where the last layer with learnable weights is generally a fully connected layer. * VGG-16 is a sixteen-layer deep neural network with about 138 million parameters. This implies that it takes a long time to be trained and to make an inference. It also occupies a significant amount of memory (roughly 533 MB). Despite this, it has been used in several image classification problems, and in terms of performance, it provided the best result with a test error of about 7.0%. [114; 115]. The main design idea was to increase the depth by using smaller (\(3\times 3\)) multiple convolutional filters than those of previous networks. VGG-16 is composed of a stack of 13 convolutional layers followed by three fully connected layers. Each convolutional block consists of multiple convolutional layers, each with different \(3\times 3\) filter kernels. As the depth increases, the number of filters in the levels grows, from 64 up to 512, in order to extract increasingly detailed feature maps from each block. The convolutional layers are followed by a rectified linear unit activation layer, and each block ends with a maximum pooling layer, with a \(2\times 2\) sliding filter with step 2. At the end of the network, there are three fully connected layers: the first two layers have 4096 nodes each; the third performs the 1000-way ILSVRC classification and, therefore, contains 1000 output nodes. This final layer comes with a soft-max activation function for classification. Since VGG-16 is computationally demanding and the output layer does not match our case study, wecustomized it by modifying the last layer. We set the number of output nodes equal to the number of classes that we have defined in our use case (4, as we will describe later in the next section). As a final step, we froze the first ten layers, and we performed new training to learn new network weights for the last layers. * The AlexNet architecture consists of five layers with a combination of max pooling, followed by three fully connected layers [116]. It uses rectified linear units instead of a hyperbolic tangent function. The advantage is twofold: it is faster to compute, especially during network training, and it mitigates the problem of vanishing gradients [117]. An interesting property of this network is that it allows parallel-GPU training by placing half of the model neurons on one GPU and the other half on the second one. This allows us to train larger models or to reduce training time. Moreover, dropout layers are used to avoid overfitting. This technique works by randomly firing a set of neurons within the first two fully connected layers during the whole training. The price to pay is that it increases the training time required for model convergence.