Title: Functional Form For Segmentation Accuracy Prediction for Comparing Architectures and Estimating Cost-Optimal Training Dataset
Primary area: interpretability and explainable AI
Keywords: Machine Learning, Asymptotic Estimation, Performance Prediction, Image Segmentation, Entropy

Abstract: Medical image segmentation plays a significant role in pre-treatment diagnosis, treatment, and post-treatment assessment of various medical conditions. To improve image segmentation accuracy, novel architectures are proposed, and their performance is compared with existing architectures to demonstrate superiority; benchmark datasets are used for these comparisons. For conciseness, only accuracies using the full benchmark datasets are reported, although results for subsets of varying sizes could also be readily obtained. However, there is currently no established method for predicting the expected accuracy of machine learning (ML) models as the training dataset size increases. In this work, we propose a procedure for developing functional forms that predict performance as a function of training dataset size and model architecture (with a fixed structure). We empirically demonstrate that these models can estimate performance for any dataset size, compute asymptotic performance, and compare the relative effectiveness of different machine learning (ML) models. Additionally, they can be used to determine the cost-optimal training dataset size. The scope of this study is limited to medical image segmentation models.

TL;DR: We introduce a procedure for developing functional forms that predict segmentation accuracy from dataset size, enabling architecture comparison, asymptotic performance estimation and cost-optimal training dataset size selection.