## 2 Materials and Methods
### Evaluation of Contours  
For our evaluation of contours, we performed Automatic vs. Manual and Manual vs. Manual contour comparisons. For each of these comparisons, 3 distinct types of comparisons were performed: same-expert same-session (SESS) match, same-expert different-session (SEDS) match, and different-experts (DE) match. For Automatic vs. Manual comparisons, the autocontours generated by our algorithm (PES-specific optimized and trained) were compared against manual contours from each of the SESS, SEDS, and DE. As previously mentioned, each PES dataset consisted of 300 images, which were divided into 160 for training, 70 for validation, and 70 for testing. For the Automatic vs. Manual SESS match, the algorithm was trained using manual contours drawn by an expert from a single session (utilizing 160 training and 70 validation images). The trained algorithm then generated autocontours for the 70 unseen testing images. For evaluation, these autocontours were compared against the expert's manual contours from the same  
Figure 1: A potential clinical workflow for implementing the in-house tumor-autocontouring algorithm. session. Therefore, the training, testing, and evaluation of the algorithm were all based on contours drawn during one session by an expert. On the other hand, for the Automatic vs. Manual SEDS match, the algorithm was similarly trained on manual contours from a single session (160 training and 70 validation images) and generated autoconours on 70 testing images. However, for evaluation, the resulting autocontours were compared against the expert's manual contours drawn during a different session. Thus, while training and testing were based on contours drawn during one session by an expert, the evaluation was performed using contours drawn in a different session. This setup enabled the assessment of the algorithm's performance in the context of intra-observer variability. For the Automatic vs. Manual DE match, the algorithm was trained using an expert's manual contours and the autocontours were compared to a different expert's contours. For both Automatic vs. Manual and Manual vs. Manual comparisons, the SEDS match represents intra-observer variabilities, while the DE match represents inter-observer variabilities. For quantitative comparisons, three evaluation metrics were chosen based on the metric selection guidelines by [PERSON] et al. (2014). First, DC ([PERSON] et al., 2015) was used, defined as:  
\[DC=2\frac{Area(ROI_{A}\cap ROI_{B})}{Area(ROI_{A})+Area(ROI_{B})} \tag{1}\]  
where \(ROI_{A}\) and \(ROI_{B}\) are contours \(A\) and \(B\), respectively. DC measures the agreement between two contours based on their overlap. This makes DC robust to outlier contours, such as small islands that are irrelevant to the tumor contour ([PERSON] et al., 2015). However, DC has a limitation: when comparing small contours, even a slight change in the intersecting area (e.g., a few pixels) can cause a significant change in the overall DC value. Conversely, for larger contours, the same slight change in the intersecting area results in a relatively smaller change in the DC value. Therefore, for evaluating small contours, which are those with at least one dimension being significantly smaller (e.g., <5%) than the corresponding image dimension, distance-based metrics are recommended over overlap-based metrics ([PERSON] et al., 2015). As indicated in Table 1, tumors of varying sizes were contoured in this study, and tumors with sizes of <5.1 cm\({}^{2}\) (i.e., tumors with one dimension < 5% of the image dimension 40 cm) were classified as small contours based on the study by [PERSON] et al. (2014). Hence, centroid displacement (CD) and the Hausdorff distance (HD) ([PERSON] et al., 2015) were also employed to address the limitations of DC.