# From Classic to Cutting-Edge: A Near-Perfect Global Thresholding Approach with Machine Learning
## 2 Materials and Methods
### Proposed Solution  
In order to achieve nested cross-validation, the dataset was split into 11 folds, one fold was set apart as the outer test set, and the remaining 10 folds were set as the outer training set. An MLNET AutoML regression experiment was set up with cross-validationon the outer training set, with a time limit of 60 min, using MSE as the optimizing metric, exploring all the model types available for regression, searching for hyperparameters using an implementation of CFT for hierarchical search spaces. Lower time limits were also tested but produced slightly worse results. Higher time limits were not tested due to increased computational cost. Each of the 10 folds in the outer training set formed an inner test set, with the remaining 9 forming an inner training set. The model instances were fitted on the inner train sets and evaluated on the inner test sets, thus producing 10 different values for the evaluation metrics. The model instances were then refitted on the outer training set and evaluated on the outer test set. Each of the 10 folds was then switched with the one in the outer test set and the model instances were refit on the inner training sets and evaluated on the inner test sets, resulting in a total of 110 different values for the evaluation metrics for models trained on 9 folds and evaluated on one fold and a total of 11 different values for the evaluation metrics for models trained on 10 folds and evaluated on one fold. We denote \(FM_{r}^{in}\) the values for \(FM_{r}\) obtained on images from the inner test sets and \(FM_{r}^{out}\) the values for \(FM_{r}\) obtained on images from the outer test sets. The nested cross-validation estimand of \(FM_{r}\) is denoted as \(F\hat{M}_{r}\) and is the average over all the inner test sets of \(FM_{r}^{in}\). For each outer test set, two values are computed:  
\[ \begin{array}{c}MSE_{in-out}=\left(\overline{FM_{r}^{in}}-\overline{FM_{r}^ {out}}\right)^{2}\\ \sigma_{out}^{2}=variance\big{(}FM_{r}^{out}\big{)},\end{array}  \tag{34}\]  
where the \(\overline{x}\) denotes the average value of the measure \(x\), and \(FM_{r}^{in}\) refers to all the inner test sets corresponding to the outer test set. The nested cross-validation estimand of the mean squared error of \(F\hat{M}_{r}\) is denoted as \(MS\hat{\Sigma}_{FM}\) and is the difference between \(\overline{MSE_{in-out}}\) and \(\overline{\sigma_{out}^{2}}\). Analogous estimands were computed for the PSNR and the MSE of the binarized image but were not used in the ranking of the models. A score was defined as  
\[F\hat{M}_{r}-3\sqrt{MS\hat{\Sigma}_{FM}}, \tag{35}\]  
and assigned to each model instance and used to determine the best one (a higher score is better). The best instance was then refit on the whole dataset, and the average metrics were computed over the entire dataset to compare the model's results with the input algorithms. Algorithm 1 illustrates, using pseudocode, how the best model was selected using NCV. ``` Input: dataset Output: ML model
1. partitions = split the dataset into 11 equally sized partitions
2. for i from 1 to 11
3. outerFolds[i].testSet = partitions[i]
4. outerFolds[i].trainSet = all partitions except partitions[i]
5. innerFolds = split outerFolds [1].trainSet into 10 folds
6. models = run AutoML 10-fold cross-validation experiment on innerFolds
7. bestScore = \(-\infty\), bestModel = null
8. for each model in models
9. esfm = empty array
10. for oi from 1 to 11
11. innerFolds = split outerFolds[oi].trainSet into 10 folds
12. einfm = empty array

## 3 Results

Table 1 shows the resulting best model instances of several runs of the same nested cross-validation (NCV) experiment, as described in Section 2.4, using all features. The procedure produces consistent results, with a LightGbm regression model always taking first place.