# Deep-Feature-Based Approach to Marine Debris Classification
## 2 Materials and Methods
### Deep Convolutional Architectures
#### 2.2.2 InceptionV3  
Although compellingly architecturally simple and with good generalization performance, VGGNets come with high computational costs. To obtain an efficient deep model with reduced computational costs and high generalization ability, [PERSON] et al. stack Inception modules in a 22 weight layer deep GoogLeNet architecture [50]. In contrast to conventional architectures where either convolution (with a single filter size) or pooling operation is performed, Inception modules perform convolution with different filter sizes (\(1\times 1\), \(3\times 3\) and \(5\times 5\)) in parallel along with the max-pooling operation and pass the concatenated results forward throughout the network. Varying filter sizes enable the model to capture spatial information at different scales at the same level in the network. The computational efficiency is preserved by adding an extra \(1\times 1\) convolution before expensive \(3\times 3\) and \(5\times 5\) convolutions and after the pooling layer, as illustrated in Figure 3. Two auxiliary classifiers are connected to intermediate Inception modules to propagate gradients through all network layers effectively.  
A later variant of the GoogLeNet architecture employed in this study, _InceptionV3_ architecture [51], modifies the original one in the following ways: (1) larger convolutions in the Inception modules are factorized into smaller ones, i.e., \(5\times 5\) convolution is replaced by two \(3\times 3\) convolutions in the Inception module A; (2) Inception module B factorizes symmetric \(7\times 7\) convolutions into asymetric \(1\times 7\) and \(7\times 1\) convolutions; (3) the Inception module C, which is introduced for promotion of high dimensional representations, replaces \(3\times 3\) convolution with parallel asymmetric \(1\times 3\) and \(3\times 1\) convolutions; (4) one auxiliary classifier with employed batch normalization is used as a regularizer together with the label smoothing technique; (5) efficient size reduction with parallel convolutional and pooling  
Figure 3: GoogLeNet Inception module. The concatenation layer concatenates outputs \(\mathbf{x}_{1},\dots,\mathbf{x}_{4}\) of the four preceding convolutional layers of the shape \((w,h,d_{i})\), \(i=1,\dots,4\), into one volume \(\mathbf{x}^{\prime}=[\mathbf{x}_{1},\dots,\mathbf{x}_{4}]\) of the shape \((w,h,\sum_{i=1}^{4}d_{i})\), which is forwarded to the following network’s layer.  
Figure 2: VGG19 network architecture. Input to the VGG19 network is \(224\times 224\) RGB image. Convolutional layers are labeled as “conv \(s\), \(n^{\prime}\), where \(s\) denotes the filter size and \(n\) number of filters in a given layer. All layers, besides the last Softmax layer, use the ReLU activation function. MaxPooling is applied on \(2\times 2\) patches with a stride of 2. The feature maps produced by the max-pooling layer in Block 5 are flattened into a single 25088-dimensional vector and fed into fully connected layers.  
blocks with a stride two is employed in the reduction modules. Figures 4 and 5 show modules and schematic representation of the InceptionV3 architecture. All convolutional layers employ batch normalization and use the ReLU activation function.