# Smart(Sampling)Augment: Optimal and Efficient Data Augmentation for Semantic Segmentation
## 4 Experiments and Results
### Ablation Studies
#### 4.4.2 SmartSamplingAugment Ablation Studies  
In these ablation studies, we analyze the impact of annealing the probability hyperparameter over epochs and weighting the augmentation operations. For the experiments without annealing, we set the probability of augmentation \(P\) to 1.  
The results in Table 7 show that for three out of four datasets, annealing as well as weighting the augmentations are good design choices. Additionally, Table 7 shows that the combination of annealing the probability of augmentation and weighting the augmentations for RaVeNNa and ErFASst datasets improves the performance. Overall, SmartSamplingAugment does comparatively well and outperforms DefaultAugment and TrivialAugment across all datasets (see Table 1).  
In the following, we give some possible explanations as to why annealing the augmentations for the EM dataset and weighting the augmentations for the KITTI dataset might not perform well. According to the hyperparameter importance study in Table 5, which quantifies the effect of how the values of these parameters affects the overall performance, the probability of augmenting data is the most important one for the EM dataset. Figure 7b, indicates that the EM dataset benefits from a high percentage of data augmentation; and therefore setting the probability of augmentation to 1 over the whole training yields better results rather than slowly increasing the probability of applied augmentations over the total number of epochs can be suboptimal. In contrast to the RaVeNNa dataset, where the probability of augmenting data is also an important hyperparameter (see Table 5), always augmenting data (\(P=1\)) hurts performance. Figure 7a shows that annealing or progressively increasing the probability of augmentation for this particular dataset does seem to be a better alternative since we do early stopping.  
Furthermore, for the KITTI dataset, the geometric magnitude is the most important hyperparameter and Figure 9 shows that sampling a high geometric magnitude can hurt performance. In SmartSamplingAugment, rotation is strongly weighted, and there is a considerable probability that a higher magnitude for this operation is sampled. Figure 10 visualizes three KITTI and EM images, each rotated with a different magnitude, and gives an intuition why augmenting the KITTI dataset with high geometric operations can have a detrimental effect on performance. We note that we select the weights based on a study performed on a classification dataset, which probably is sub-optimal for semantic segmentation. However, this gives insight that studies focusing on optimizing the weights for augmentation operations can be a next step for further research.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**Weighting**} & \multicolumn{2}{c}{**Without Weighting**} \\  & **Annealing** & **No-Annealing** & **Annealing** & **No-Annealing** \\ \hline KITTI & 66.53 & 67.15 & **67.49** & 67.13 \\ RaVeNNa & **90.72** & 87.07 & 85.65 & 85.68 \\ EM & 78.52 & **79.26** & 77.94 & 78.47 \\ ErFASst & **70.24** & 68.27 & 64.99 & 64.51 \\ \hline \hline \end{tabular}
\end{table}
Table 7: SmartSamplingAugment ablation study analyzing the impact of weighting the augmentations and annealing the probability hyperparameter over the whole epochs for different datasets. We evaluated each experiment three times using different seeds to obtain the mean IoU. For the experiments without annealing, we set the probability \(P\) of applying augmentations to 1.  
Figure 10: Visualization of three images from the KITTI (left) and EM (right) datasets, each rotated with a different magnitude.  
Figure 9: Results from SmartAugment on the KITTI dataset indicate that the geometric magnitude, which is the most important hyperparameter for this particular dataset, should be low. Taking this into account gives a possible explanation for why weighting the data augmentation with weights that focus on geometric augmentations might hurt performance (mean IoU) for the KITTI dataset.