# An Empirical Study on Ensemble of Segmentation Approaches
## 2 Materials and Methods
### Deep Learning for Semantic Image Segmentation  
In the literature, different deep learning models are proposed to address semantic segmentation problems. Semantic segmentation intends to identify objects in an image, with relative boundaries. The main purpose is therefore to assign a class at the pixel level; a task achieved thanks to FCNs (fully convolutional networks). FCNs have very high performance, and unlike CNNs they use a fully convolutional last layer instead of a fully connected one [18]. In order to obtain deconvolutional networks, such as U-Net, FCNs and autoencoders are combined together. U-Net represents the first attempt to use autoencoders in an image segmentation task. Through the autoencoder, it is possible to downsample the input and simultaneously increment the number of features used to describe the input space. We can find another symbolic example in SegNet [19]: here, the max pool indices of the relative encoder level feed the decoders, while VGG is used for encoding. This allows it to reduce memory usage by having also a more promising segmentation. DeepLab [20] consists of a series of autoencoder models introduced by Google, and has shown excellent results also in semantic segmentation applications [21]. These are some of the main features introduced to guarantee good performance:  
1. A dilated convolution reduces the effects of pooling and stride, thereby greatly increasing the resolution. 2. Through an Atrous Spatial Pyramid Pooling, information is obtained at various scales. 3. A union of CNNs and probabilistic graphic models make it possible to detect the boundaries of objects. We find in DeelLabV3, two most important innovations: a \(1\times 1\) convolution in Atrous Spatial Pyramid Pooling and a batch normalization; a set of modules placed in parallel and in cascade for convolutional dilation. DeepLabV3+ [5], an expansion of the family developed by Google, is adopted in this work. This expansion includes, among the most important features, a decoder with depth-wise convolutions and point-wise convolutions. The depth-wise works in the same location but with various channels, while the point-wise uses the same channel in various locations. In order to obtain different designs for a framework, we can consider other characteristics of the structure of a model, in fact the architecture model itself that is used is only one choice. In this paper, we will investigate ResNet101 [22], a very famous CNN that acquires a residual function by referring to the block input (we recommend [23] for an exhaustive list of CNN structures). We adopt the ResNet101 network, pre-trained on the VOC segmentation dataset and then modulated through the parameters suggested, ([[https://github.com/matlab-deep-learning/pretrained-deeplabv3](https://github.com/matlab-deep-learning/pretrained-deeplabv3) plus]([https://github.com/matlab-deep-learning/pretrained-deeplabv3](https://github.com/matlab-deep-learning/pretrained-deeplabv3) plus), (accessed on 20 April 2022)). These parameters have not been modified in order to prevent overfitting; thus, they are the same in all the tested datasets:  
* initial learning rate = 0.01;
* number of epoch = 10 (using the simple data augmentation approach) or 15 (the latter more complex data augmentation approach due to the slower convergence using this larger augmented training set);
* momentum = 0.9;
* L2 Regularization = 0.005;
* Learning Rate Drop Period = 5;
* Learning Rate Drop Factor = 0.2;
* Shuffle training images every-epoch;
* Optimizer = SGD (stochastic gradient descent). Firstly, we propose an ensemble of the DeepLabV3+ models obtained by applying various loss and data augmentation methods, and then we combine the ensemble with HarDNet-MSEG [6], and Pyramid Vision Transformers (PVT) [7]. The HarD-Net-MSEG (harmonic densely connected network), a model influenced by densely connected networks, allows the reduction of memory consumption in this way: it decreases most of the connection layers at the DenseNet level, in order to reduce the costs of concatenation.