## 7 Discussion
### Test Dataset Considerations  
In this section, we describe how the methods mentioned in this review are evaluated using a test dataset. In this section, the test dataset, also known as the holdout dataset, refers to the dataset used to calculate evaluation metrics at the end of the model development process. In a machine-learning-based method, this dataset should be completely independent of the training and validation datasets, used to train models and to set the model's hyperparameters, respectively [59].  
Figure 13: An example of a Bland-Altman plot for EAT volume from [44].  
The best test dataset would be a sufficiently large set of data collected independently and after the model has been developed in real-world applications. In practice, this is often infeasible so researchers rely on splitting their originally available dataset into training, validation, and testing datasets, or using cross-validation to train and evaluate the model on several independent subsets of the original dataset.  
In addition, when using learned models such as machine- or deep learning-based methods, it is vital that the test dataset is independent of the training dataset, i.e., that the researchers avoid data leakage. Data leakage happens when the training dataset contains examples from the test dataset [60]. If a learned model had access to the testing data during training, its performance on the test dataset will no longer be representative of any real-world performance.  
When training machine learning-based models for EAT and PAT segmentation on 3D data like CT scans, another more subtle form of data leakage is possible. Some researchers perform a train, validation, and test split slice-wise, by randomly sampling a percentage of all slices from all patients into the train, validation, and test datasets. By doing so, different slices of the same patient end up in different datasets. Two neighboring slices of the same patient will be very similar, so a model trained on one or more slices of a patient will perform misleadingly well on their neighboring slices in the test dataset. This means that a slice-based test dataset is not independent of the training set and is not representative of any real segmentation or quantification performance. In real-world uses, the model would get slices from completely new patients which the model has not seen before.  
Several papers mentioned in this work use slice-based dataset splitting including [45] and, while it is not explicitly stated, potentially also [20] (describes a very similar method with similar results to [45]), and [46] (uses \(10\)-fold cross-validation which would not be feasible with only \(20\) patients). Ref. [47] does not provide sufficient information to determine how the validation dataset was obtained but has results similar to the above-mentioned methods. Note that these papers are also the ones that report much higher performance in terms of Dice coefficients and accuracy than any other paper in this review, suggesting that data leakage has occurred and that the results are not representative of real-world performance. The results of methods that split their datasets based on slices should not be directly compared to the results of other methods which perform their dataset split per patient.  
We recommend that dataset splits should always be made per patient and not per slice since that better represents how a method would be used in a real setting. In addition, authors should consider using cross-validation, especially when the number of patients is relatively low like in the publicly available dataset from [20]. Finally, authors should clearly describe how they perform the validation and training dataset split so that their results are reproducible for future researchers.