# Deep Ensembles Based on Stochastic Activations for Semantic Segmentation
## 3 Results on Colorectal Cancer Segmentation
### Experiments  
The first experiment (Table 2) is aimed at comparing the different backbone networks listed in Section 2.1. Since the size of images in the Kvasir dataset was quite large, we also evaluated versions of the ResNet with a larger input size, i.e., 299 \(\times\) 299 (resnet18-299/resnet50-299) and 352 \(\times\) 352 (resnet18-352/resnet50-352).  
The second experiment (Table 3) aimed at designing effective ensembles by varying the activation functions. Each ensemble is the fusion by the average rule of 14 models (since we used 14 activation functions). The ensemble name is the concatenation of the name of the backbone network and a string to identify the creation approach:  
* _act: each network was obtained by deterministically substituting each activation layer with one of the activation functions of Section 2.2 (the same function for all the layers, but a different function for each network)
* _sto: ensembles of stochastic models, whose activation layers were replaced by a randomly selected activation function (which may be different for each layer)
* _sel: ensembles of \"selected\" stochastic models. The network selection was performed using cross-validation on the training set among 100 resnet50 stochastic models. The selection procedure was based on the idea of sequential forward floating selection (SFFS) [49], a selection method originally proposed for feature selection and used here for selecting the most performing/independent classifiers to be added to the ensemble. SFFS is an iterative method that, at each step, adds to the final ensemble the model that provides the highest incremental of performance to the existing subset of models. Then, a backtracking step is performed to exclude the worst model from the actual ensemble. Since SFFS requires a training phase, we performed threefold cross-validation on the training set. For a fair comparison with other ensembles, we selected a set of 14 networks, which were finally fine-tuned on the whole augmented training set at a larger resolution.
* _relu: an ensemble of original models that differ only for the random initialization before training. It means that all the starting models in the ensemble are the same, except for the initialization. This ensemble is the baseline for comparisons with the approaches above.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Backbone** & **IoU** & **Dice** & **F2** & **Prec.** & **Rec.** & **Acc.** \\ \hline Mobilenetv2 & 0.734 & 0.823 & 0.827 & 0.863 & 0.841 & 0.947 \\ resnet18 & 0.759 & 0.844 & 0.845 & 0.882 & 0.856 & 0.952 \\ resnet50 & 0.751 & 0.837 & 0.836 & 0.883 & 0.845 & 0.952 \\ xception & 0.699 & 0.799 & 0.792 & 0.870 & 0.800 & 0.943 \\ IncR & 0.793 & 0.871 & 0.878 & 0.889 & 0.892 & 0.961 \\ efficientnetb0 & 0.705 & 0.800 & 0.801 & 0.860 & 0.814 & 0.944 \\ resnet18-299 & 0.782 & 0.863 & 0.870 & 0.881 & 0.883 & 0.959 \\ resnet50-299 & 0.798 & 0.872 & 0.876 & **0.898** & 0.886 & 0.962 \\ resnet18-352 & 0.787 & 0.865 & 0.871 & 0.891 & 0.884 & 0.960 \\ resnet50-352 & **0.801** & **0.872** & **0.884** & 0.881 & **0.900** & **0.964** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experiments with different backbones in the Kvasir dataset (best results in bold).  
Finally, in Table 4, a comparison with some state-of-the-art results is reported.