# Deep-Feature-Based Approach to Marine Debris Classification
## 3 Experiments
### Extraction of Features  
In conventional CNNs, the feature maps from the last convolutional layer are flattened into a feature vector and forwarded to fully connected layers and final softmax classification layer [21, 22]. [PERSON] et al. [62] offer an alternative with global average pooling layers, which spatially compresses the information contained in feature maps into one vector. Dense feature vectors for marine debris images are extracted using the state-of-the-art convolutional model architectures described in Section 2.2, as illustrated in Figure 15. For each model, the last layer with three-dimensional output is selected as the feature extraction layer. Global average pooling is applied to the output of that layer to obtain a feature vector from a three-dimensional output volume. Let \(w\times h\times n\) be the shape of the feature extraction layer's output and let \(f_{1},f_{2},\ldots,f_{n}\) denote \(n\) feature maps in the output volume. By applying a global average pooling on the given volume, we obtain a vector \((\overline{f}_{1},\overline{f}_{2},\ldots,\overline{f}_{n})\) where \(\overline{f}_{i}\), represents the mean value of \(w\cdot h\) values in the feature map \(f_{i}\), \(i=1,\ldots,n\). Thus, the size of the feature vector matches the depth of the output volume of the feature extraction layer. All deep models used for feature extraction are loaded with weights pre-trained on the _ImageNet_[34] dataset. In each deep CNN architecture, all layers that follow the feature extraction layer are dropped and replaced by a global average pooling for feature extraction. Table 2 lists all deep architectures employed in this work for feature extraction together with the information about the total number of their parameters (after dropping the layers following the feature extraction layer) and extracted vectors' size. A new neural network (NN) classifier, which has two fully connected (FC) layers with 256 and 128 neurons followed by the Softmax layer with six neurons, is then added on top of the pooling layer, as illustrated in Figure 16. The batch normalization [63] technique is applied on newly added FC layers. Figure 15: Extraction of dense feature vectors. In this paper, we use deep CNN feature extractor in three different ways: (1) we freeze all its layers and train only the newly added NN classifier; (2) we fine-tune all weights of the deep CNN and train it together with the NN classifier, i.e., we use loaded Image-Net weights as the starting point for the training of the model; (3) first, we freeze deep CNN and train the NN classifier on top, and afterward unfreeze the weights of deep CNN to fine-tune them. In following sections, latter cases are denoted as (1) fixed feature extractor (FFE); (2) fine-tuning (FT); (3) FFE+FT. For the training of all models, we use Adam [64] optimizer with learning rates as in Table 3, \(\beta_{1}=0.9\), \(\beta_{2}=0.999\) and \(\epsilon=10^{-7}\). The learning rate of each model is chosen from the set of predefined values on a logarithmic scale using the 5-fold cross-validation. All models were trained for 100 epochs. In the FFE+FT case, for the first 25 epochs, we train only the NN classifier on top and keep the layers of the deep CNN frozen. In the remaining 75 epochs, we fine-tune the weights of the deep CNN together with the NN classifier. For all models, we used small mini-batches of size 16, which require a smaller memory footprint than larger ones. Moreover, small batch sizes provide better generalization performance and optimization convergence [65; 66]. Since we have a limited amount of data for the training at hand, we use data augmentation to expand the training set artificially. During the training, we augment images using random rotations, width and height shifts, shearing, and horizontal flipping.