# Smart(Sampling)Augment: Optimal and Efficient Data Augmentation for Semantic Segmentation
## 5 Conclusions  
In this work, we provide a first and extensive study of data augmentation for segmentation and introduce two novel approaches: SmartAugment, a new state-of-the-art method that finds the best configuration for data augmentation with hyperparameter optimization, and SmartSamplingAugment, a parameter-free, resource-efficient approach that performs competitively with the previous state-of-the-art approaches. Both methods achieve excellent results on different and diverse datasets.  
With SmartAugment, we show that Bayesian Optimization can effectively find an optimal augmentation strategy from a search space where the number of color and geometric augmentations and their magnitudes are optimized separately, along with a probability hyperparameter for applying data augmentations. Our results show that the type of applied augmentation is essential in making good decisions for improved performance. Furthermore, a hyperparameter importance study indicates that the probability of applying a data augmentation strategy could have considerable responsibility for the mean IoU's variability across the configuration space.  
With SmartSamplingAugment, we develop a simple and cheap-to-evaluate algorithm that weighs the augmentations and anneals augmentations to increase the percentage of augmented images systematically. The results show that this is a powerful and efficient approach that is competitive to the more resource-intensive approaches and outperforms TrivialAugment, a comparably cheap-to-evaluate method. Furthermore, SmartSamplingAugment opens the gate for more research on weighting and annealing data augmentation. A possible future work will study an extension of our methods to image classification, detection and 3D segmentation, particularly for biomedical applications.  
Conceptualization, M.N., D.W.; methodology, M.N., D.W.; software, M.N., D.W.; validation, M.N., D.W. and A.R.; investigation, M.N., D.W.; resources, A.R.; data curation, M.N.; writing--original draft preparation, M.N., D.W.; writing--review and editing, M.N., D.W. and A.R.; visualization, M.N., D.W.; supervision, A.R.; project administration, M.N., A.R.; funding acquisition, A.R. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the German Federal Ministry of Education and Research (13 GW0203A) and approved by the local Ethical Committee of the University of Freiburg, Germany.  
Not applicable.  
The data used are referenced in the article.  
The authors would like to thank the Department of Urology of the Faculty of Medicine in University of Freiburg for annotation of cystoscopic images that were used to build the RaVeNNA (Ravenna 4 pi) dataset. Furthermore, thanks to [PERSON] for providing us with the ErfAsst dataset.  
The authors declare no conflict of interest.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l} BO & Bayesian Optimization \\ IoU & Intersection over Union \\ SGD & Stochastic Gradient Descent \\ \(N_{C}\) & Number of color augmentations \\ \(N_{G}\) & Number of geometric augmentations \\ \(M_{C}\) & Color magnitude \\ \(M_{G}\) & Geometric magnitude \\ \(P\) & Probability of applying augmentations \\ \end{tabular}