Predicting DNA Content Abnormalities in Barrett’s Esophagus: A Weakly Supervised Learning Paradigm

Authors: caner  ercan
thomas g paulson
matthew d stachler
carlo c maley
william m grady
yinyin  yuan
Venue: MIDL_io_2024_Conference
Published: 2024-01-29 19:15:46

Abstract: Barrett’s esophagus (BE) is the sole precursor to esophageal adenocarcinoma (EAC), and is an opportunity for developing biomarkers for cancer risk assessment. DNA content abnormalities, including aneuploidy, have been implicated in the progression to EAC in BE patients, but molecular assays require valuable tissue for its detection. We propose utilizing images from routine histology to detect ploidy status using deep learning.
Employing a weakly supervised deep learning approach, multi-instance learning (MIL), we trained a model to predict ploidy using hematoxylin and eosin-stained whole slide images of endoscopic biopsies and flow cytometry results. The study introduces a novel data augmentation method for MIL, sequentially altering features from original and augmented images during training loops. This method improved the average area under curve (AUC) from 0.43, 0.64 and 0.81 for ResNet50, DenseNet121 and REMEDIS foundation model, respectively (training without any augmentation), to 0.61, 0.87 and 0.91 with the proposed augmentation strategy.
The top-performing model, using REMEDIS foundation model as the backbone, achieved 0.93 AUC and 0.83 balanced accuracy to predict aneuploidy in the test cohort biopsies (n=279). Across all the patients (n=123), predicted aneuploidy status was correlated with progression to EAC (p=6.55e-06), similar to correlation with ploidy status based on flow cytometry results (p=2.84e-7). Supporting the findings, histologic nuclear features typically associated with dysplasia and DNA content abnormalities such as enlarged, hyperchromatic nuclei and loss of nuclear polarity, were seen in the samples called abnormal compared to the control diploid samples.
In conclusion, our deep learning model efficiently predicts aneuploidy, a mechanism that has been shown to underpin BE progression to EAC. This method, preserving precious biopsy tissues, complements routine histology, offering potential for identifying individuals at high risk of progression through molecular-based advancements.

Main paper: 
 introduction: 
 Due to chronic gastroesophageal reflux and the resultant inflammation, the lower esophagus can go through a metaplastic change where the normal squamous lining is replaced with columnar cells that develop intestinal-like differentiation, Barrett's esophagus (BE). BE is the sole detectable precursor lesion for esophageal adenocarcinoma (EAC), a deadly cancer with an increasing incidence (Desai et al., 2012) . The progression to EAC is characterized by progressive histologic and genetic alterations of the BE (Killcoyne and Fitzgerald, 2021) . DNA content abnormalities, defined as the presence of aneuploidy (hypo-, hyperand tetraploidy) or an increased percent (>6%) of cells in the G2/M phase of the cell cycle, have been implicated in the development of EAC in BE patients (Reid et al., 1992; Rabinovitch et al., 2001; Hadjinicolaou et al., 2020) . However, the existing molecular assays for DNA content abnormality detection (e.g., flow cytometry) necessitate the use of precious tissue samples, leading to difficulties in validation studies. Our objective is to develop a deep learning-based image analysis tool to predict the DNA content status from the whole slide images (WSI) of standard haematoxylin and eosin (H&E)-stained endoscopic biopsies.Fully-supervised approaches have achieved remarkable success in detecting a wide range of objects (e.g. cell nucleus, tumor), relying on pathologists' annotations as the ground truth (Montezuma et al., 2023) . However, when dealing with tasks that only have WSIlevel annotations and lack local features identifiable by pathologists, such as molecular alterations, this approach faces limitations. To address this problem, multi-instance learning (MIL) emerges as a viable solution (Ilse et al., 2018) . In this framework, the dataset is composed of bags (in here WSIs) with classification labels, where each bag contains multiple instances (patches of a WSI) without labels. If a bag contains at least one positive instance, it is labeled as positive; otherwise, it is labeled as negative.In recent years, many researchers have implemented MIL methods in computational pathology using convolutional neural networks (Wang et al., 2019) . Lu et al. (2021) proposed Clustering-constrained Attention Multiple Instance Learning (CLAM), which derives from the attention-based MIL framework introduced by Ilse et al. (2018) . This pipeline uses an attention mechanism to identify critical regions associated with diagnosis automatically, and then generates global features based on this region. However, there is currently a scarcity of research applied to BE biopsies using this approach.Data augmentation is effective in tackling a lack of generalisation, and data memorization. In computational pathology, augmentation can be done at the image level and data level. Image-level augmentations, such as bluring, zoom, and color brightness change, can serve dual purposes, increasing resilience of model performance to image quality alterations and increasing training data. (Krizhevsky et al., 2012) However, this traditional approach is not directly applicable to MIL training. On the other hand, the data-level augmentation approaches working on increasing the training data on feature vector (embedding) or bag (WSI) level data mixtures and pseudo-data generations. (Liu et al., 2023; Gadermayr et al., 2023; Yang et al., 2022) Contrary to most training approaches that involve encoding images to low-level features and training the model on-the-fly, MIL takes a different path by aggregating patch features and using them sequentially for training the network. This difference makes the traditional image augmentation method, application of random transformations during each epoch, impractical. To address this challenge, we introduce an innovative method. Multiple sets of low-level patch features are generated, with the initial set derived from original images and subsequent sets incorporating random image transformations before feature extraction. During MIL training, a set of augmented features is employed every other epoch. This approach ensures that the model is primarily trained with original images while harnessing the advantages of image augmentation to improve overall performance and generality. Additionally, we integrated Pseudo-Bag Mixup Augmentation (PseMix) into our pipeline to examine the impact of feature-level data augmentation (Liu et al., 2023) . PseMix is an effective Mixup variant for WSI classification. In this method, instances from WSI bags are first clustered into different phenotypes, and then, each bag undergoes phenotype-stratified sampling. Comparative experiments and ablation studies confirm that PseMix is an effective Mixup variant for WSI classification (Liu et al., 2023) .In this study, our objective was to develop a deep learning pipeline utilizing the MIL paradigm for predicting aneuploidy in BE. Our key contributions include: (1) To the best of our knowledge, this is the first study applying deep learning for predicting DNA content status based on histology images of BE. For this aim, we utilized a unique BE biopsy dataset having known DNA content results. (2) We utilized the MIL framework in the context of BE to predict cancer outcome-related features. Additionally, we examined the changes in BE epithelial cell features within regions exhibiting high aneuploidy probability to elucidate the connection between MIL model outputs and histology. (3) The study introduces a novel augmentation method for histology images in BE prediction. 
dataset: 
 H&E-stained WSIs of endoscopic biopsies from BE were utilized in this study. The biopsy slides, sourced from the Seattle Barrett's Esophagus Annotated Resource (BEAR), constituted the training and validation sets. All research participants contributing clinical data and biospecimens provided written informed consent, governed by oversight from the Fred Hutchinson Cancer Center IRB Committee D (reg. ID 5619). Biopsies obtained from patients were divided into two halves. One half underwent processing for flow cytometric DNA content assessment, as described previously (Reid et al., 1992) . The other half was subjected to standard fixation and staining with H&E. The H&Estained slides in the first cohort were scanned at 20x (0.4548 µm/pixel) using a Hamamatsu NanoZoomer. An independent cohort, composed of patients from a different subset of BEAR, served as the test set, with H&E-stained slides scanned on an Leica Aperio AT2 slide scanner at 20x (0.5013 µm/pixel). (Figure 1A ) Slides with significant out-of-focus issues, air bubbles, and tissues with aneuploid results lacking BE regions were excluded from the study. The flow-analyzed tissue pieces were manually selected using the rectangular regions of interest, generating two sets for each slide when possible. The first cohort consisted of 388 slides (51 aneuploid, 337 diploid) from 34 patients, while the second cohort had 279 slides (36 aneuploid, 243 diploid) from 89 patients. The first cohort (training) underwent Monte Carlo cross-validation, being split into training (85%) and validation (15%) datasets constrained to the patients, while the second cohort served as the test dataset.We adapted the CLAM framework to train a deep neural network classifier for WSI-level, predicting DNA content status as abnormal or normal (Stachler et al., 2015) Preprocessing and Feature extraction: The tissue segmentation and patching tasks were achieved as described previously (Lu et al., 2021) . After segmentation, each tissue region was split into patches with a size of 256 x 256 pixels without downsampling to fit the input capacity of deep learning models. (Figure 1B ) Following patching, convolutional neural networks were employed to extract feature vectors from each image patch in every WSI. To assess performance differences, differences, we utilized ResNet50 (He et al., 2016) and Densenet121 (Huang et al., 2017 ) models pre-trained on ImageNet, along with the REMEDIS 152x2 small foundation model (Azizi et al., 2023 (Azizi et al., , 2022)) , pre-trained on The Cancer Genome Atlas. Each network underwent modification to convert its output into a 1,024-dimensional feature vector for every patch. Augmentations and Training: To implement image augmentation, four combinations of image transformations, randomly applied with different probabilities, were employed on all patches. (Appendix A) This ensured diversity in the transformations applied across the dataset. Each transformation was assigned a probability and factor, resulting in four additional sets of features for analysis. Additionally, a consistent seed was used for each set of augmented features to maintain uniformity across different backbones. (Figure 1C )The training iterations began with using the feature vectors which were computed using the original image patches. (Figure 1D ) For every other epoch, we used one of the feature vector sets which were extracted from the images after image augmentations. This strategy uses raw and augmented features simultaneously, expanding the feature pool and making the learning process effective. For the data augmentation experiment, PseMix data augmentation (Liu et al., 2023) initiated with an initial learning rate of 1e-3 and a scheduler with a 0.5 factor, 5 epochs patience, and an 8-epoch cooldown. We employed the Adam optimizer (Kingma and Ba, 2017), with a 1e-5 weight decay, and dropout with a 0.25 probability. Focal cross entropy was used as loss function. Model size was set to small and bag weight was set as 0.7. The training sessions were conducted with a minimum of 60 epochs, and an early stopping mechanism with 20 epochs patience. For the WSI-level classification, the optimum threshold for probability was calculated on the training dataset using the ROC curve based on the lowest TPR -(1 -FPR) level.For the investigation of aneuploidy histological features, the cell features were analysed and compared between top-attention scored patches. For computation of cell nucleus features, we trained a nucleus instance segmentation and classification model with our own data using StarDist algorithm. (Weigert et al., 2020 ) Fifteen WSI's from the test cohort slides were selected randomly and the cell nuclei in selected regions were annotated by one pathologist (C.E.). For the training, 14,576 cells (7464 immune cells, and 7112 epithelial cells) were annotated. After the inference, QuPath (v.0.4.4) (Bankhead et al., 2017) was used for calculation of nuclei intensity (n=5), texture (n=13) and shape (n=6) features.For statistical comparisons for the cell feature analysis, Student's t-Test was used. For cancer outcome analysis, Pearson's Chi-squared test was used. Confusion matrices were obtained using cvms (Olsen et al., 2023) . All statistical tests were two-sided, and P < 0.05 was considered statistically significant. To adjust P values for multiple comparisons, the Benjamini-Hochberg method was used. Plotting was done using ggplot2 v.3.4.4 and ggpubr v.0.6.0 packages. All statistical analyses were conducted in R v.4.3.1, except for ROC analysis, which was performed using scikit-learn v.1.3.2 in Python v.3.11.6. Symbols used in figures represent following; ns : not significant, p-value <0.05:*, <0.01:**, <0.001:*** 
effective aneuploidy prediction with dlo/dla alternation: 
 The MIL model was trained using the flow-cytometry results (DNA content abnormal or not) as the slide level annotation and BE biopsy slide images using CLAM framework. Feature maps for each WSI were generated using three different backbones: ResNet50, DenseNet121, and the Foundation Model (REMEDIS). Alongside the feature maps generated from original images, four additional sets were created after applying various combinations of image augmentation functions and factors. (Figure 1C )Training involved the combination of a dataloader based on original images (DLO) and four dataloaders based on different settings of augmented images (DLAs). (Figure 1D ) 1 ). In comparison, ResNet50 and DenseNet121 achieved AUCs of 0.72 and 0.88, respectively (Figure 2A ). The balanced accuracy of the foundation modelbased DLO/DLA alternating model reached 0.83 (Figure 2B ). Without augmentation, the average AUC from original images was 0.80, 0.43, and 0.81 for ResNet50, DenseNet121, and REMEDIS, respectively. The new augmentation strategy improved the performance of DenseNet121 and REMEDIS, with average AUCs of 0.61 and 0.91, respectively (Figure 2A ). In the ablation study, the proposed method was found to be superior to other settings (Appendix B-C).In the experiment evaluating PseMix feature-level data augmentation in a single iteration, the augmentation method produced similar performance to training without augmentation for DenseNet121 and the foundation model. However, when combined with proposed image augmentation, the AUC improved from 0.53 and 0.48 to 0.64 and 0.76, respectively (Appendix D). In contrast, training sessions using only image-based augmentation showed superior performance, with AUCs of 0.88 and 0.93, respectively. Training sessions using features from ResNet50 yielded the best performance in the no augmentation setting. Interestingly, training sessions with PseMix resulted in significantly inferior performance.To assess the predictive relevance of our model, we compared cancer outcomes (CO) between patients with and without DNA content abnormalities. Based on both flow results and our model's predictions, patients with abnormalities have significant higher rates of CO (p-value 2.84e-07 and 6.55e-06, respectively) (Figure 3A ). 55.6% (10/18) of patients with CO had aneuploidy based on flow results while 66.7% (12/18) based on predictions. This result shows the model predictions can be informative for patient clinical outcome as well.Finally, we aimed to investigate aneuploidy-specific histological features using the MIL model predictions. This investigation serves two purposes: understanding relevant features for aneuploidy predictions by the MIL model and validating these features with known aneuploidy-specific features. In analyzing histological features, we compared cell features between top-attention scored patches from each WSI. This analysis provided valuable insights into the relevant features for DNA content classifications. Notably, patches categorized as abnormal exhibited enlarged, hyperchromatic nuclei with a loss of cell nucleus orientation and frequent mitotic figures (Figure 3B ). In contrast, patches from diploid samples often displayed no signs of atypia, with rare reactive changes.Furthermore, we calculated and compared cell nuclei features in the top attention-scored patches between the two groups. For this purpose, we trained a cell nucleus instance segmentation and an immune cell-epithelial cell classification model and achieved 76.7% accuracy and 86.8% F1-score (Table 1 , Figure 3C ). Cell feature comparison analysis revealed significant differences between diploid and abnormal epithelial cells. Abnormal epithelial cells exhibited notably increased cell nucleus size compared to diploid ones. Additionally, we conducted a texture analysis, uncovering significant differences in mean nucleus entropy, standard deviation of inverse difference moment, and difference variance values between the top-attention-scored/accurately predicted diploid and abnormal groups. (Figure 3D )  
discussion: 
 In this study, we developed an image-analysis model capable of predicting DNA content abnormalities on H&E stained BE biopsy WSIs, using MIL as a weakly supervised training framework. Our findings revealed a correlation between DNA content predictions and cancer progression, a well-known aspect of BE. The nature of image analysis, which allows preservation of precious small tissue samples, positions it as a promising candidate for application in routine clinical practice. Despite limited efforts in this direction (Yu et al., 2024) , to the best of our knowledge, this study marks the first application of deep learning to predict DNA content status based on histology images of BE.MIL approaches allow us to stratify patients using slide-level labels in a weakly supervised setting with exceptional clinical-grade performance (Campanella et al., 2019; Lu et al., 2021) . However, an efficient image-based data augmentation technique for MIL is still an unmet need. We applied an inovative image augmentation strategy based on generating multiple dataloaders through different image transformations. The method showed increased performance compared to approaches without data augmentation. while the foundation model with proposed image augmentation consistently performed well, while ResNet and DenseNet showed variability, likely due to the inherent challenges of predicting aneuploidy from slide images and dataset imbalance. Combining augmentation strategies with the foundation model may ensure more robust performance across challenging tasks. Additionally, we implemented PseMix as a feature-level data augmentation method in our training. From the results of Appendix C, we found that PseMix augmentation improves the performance of MIL models in the presence of basic image augmentation. This experiment could further verify the adaptability of combinations of feature-based and image-based augmentations.The top attention-scored patches provide crucial insights into the model's predictive features. Notably, DNA content abnormal patches showed enlarged, hyperchromatic nuclei with a loss of cell nucleus orientation and frequent mitotic figures. Traditional cell feature calculations for these patches revealed significant differences between abnormal and diploid samples in multiple features, including nucleus size and texture differences. The observed nuclear volume and changes in chromatin distribution align with known manifestations of increased chromosome content (Fischer, 2020; Chow et al., 2012) . These findings emphasize the model's ability to explain relevant morphological disparities.Despite the our method's inspiring performance, some limitations remain. Firstly, utilizing flow cytometry for the DNA content ground truth requires tissue disaggregation and destruction, restricting image analysis to adjacent tissue. Consequently, focal DNA abnormalities may lead to differing ploidy states in biopsy halves. Secondly, we argue non-BE regions may hinder the performance of abnormality prediction by weighting irrelevant regions, like dense inflammation. Future research will involve a Barrett's epithelium segmentation model to focus solely on relevant image patches.To conclude, we showed the utility of a weakly supervised approach on feature prediction in a precancerous lesion, Barrett's esophagus. The novel image augmentation strategy along with utilization of a foundation model backbone showed high performance. Our model is efficient, capable of processing hundreds of samples in a tissue preserving setting, and thus an ideal adjunct to standard histologic evaluation. This classifier may facilitate molecularbased improvements in identifying both individuals at high risk for progression as well as those that may have already developed EAC. 
 
 Appendix:  



Reviews: review: paper_summary: paper_summary: The study proposes a weakly supervised multi-instance learning (MIL) method to predict ploidy status in Barrett's esophagus (BE) using routine histology images. The top-performing model achieved a high accuracy of 0.93 AUC and 0.75 balanced accuracy in predicting aneuploidy in test cohort biopsies, showing correlation with BE progression to esophageal adenocarcinoma (EAC) and suggesting potential for identifying high-risk individuals.
main_review: main_review: My final recommendation is borderline accept because the authors clarified multiple points, improved the manuscript (based on all reviewers comments). They also added a comparison with PseMix, and a combination of feature- and image-level augmentation. The sentences added to the manuscript must, however, be checked for grammar and typos.
strength_weakness: strength_weakness: The motivation for the augmentation method is a bit weak (primarily training with original images while leveraging the benefits of image augmentation to enhance overall performance and generality). It is shown empirically to work on this dataset, but it would be good to validate on other sets and compare with other approaches.

The method is compared only with a baseline without any augmentation. It is not the best comparison. It should be compared with existing augmentation methods, e.g. Liu et al., 2023; Gadermayr et al., 2023; Yang et al., 2022.

There are some incoherencies I do not understand (see detailed comments).
questions: questions: Please fix or explain the mentioned incoherencies. 
A comparison with other augmentation methods is also needed.
paper_summary: paper_summary: This work is devoted to detecting DNA abnormalities in Barrett's esophagus. These abnormalities are implicated in the progression to cancer. For that matter, the authors are using histology images. Using the MIL paradigm, detection is using bag of tiles with annotated slides. These tiles are encoded through a feature extractor network. In this work, the authors propose a novel technique alternating at training between features from original slides and features extracted after augmentation of these tiles. A cohort from Seattle is used from which an independent test set is extracted and uses a different slide scanner. Satisfactory results are obtained on the binary classification (diploid vs aneuploid). To investigate the model's predictive features, top attention-scored patches and their cell morphological features are analysed.
main_review: main_review: The paper has improved but many typos need to be corrected. I still have questions about the robustness of the approach considering the large difference in results between backbones/augmentation strategies. It should be discussed a bit in the document.
strength_weakness: strength_weakness: Besides the limitation on ground truth that is identified in the paper, the main weakness is the statistical power of the results. Although the number of patients is significant, there seems to be a significant variability (e.g. on AUC as shown in Fig 2) between folds (not for the combination of the foundation model and the strategy). 

One could also have investigated different augmentation strategy to really show that the proposed approach is optimal. As each patch has 5 versions of its features (original + 4 extractions after augmentation), the dataloader could have taken one of its version randomly on the fly.
questions: questions: The main questions to address in the rebuttal would be on the evaluation of the generalization of the approach (e.g. with a discussion of the large error bars in AUC on Fig 2) and the validation of the alternation strategy.
paper_summary: paper_summary: This paper describes some experiments on the creation and validation of models for detecting aneuploidy (an important cancer biomarker) from standard H&E stained whole slide images (WSIs) taken from biopsies of Barretts oesophagus. The authors adapt the popular CLAM multiple instance learning model architecture in which pretrained feature extractors (three are tested in this paper) are run on each patch, and then a "bag" of patches is aggregated by learnable attention-based mechanisms. Furthermore the authors introduce an augmentation process by which augmented patch-level features are introduced during the training process.
main_review: main_review: The inclusion of the PSEMix augmentation results as a baseline is a good addition and does much to address my comment about comparison to other established augmentation methods for MIL WSI models.

My comment about the feature vector size was unfortunately not addressed.

However, the new results are also highly erratic. For example, PSEMix reduced the performance of the ResNet50 model from 0.788 to 0.301 (an enormous drop in performance) while for the other models (which we would expect to behave quite similarly) it increased performance dramatically. This leads me to suspect that these models are trained and validated on datasets that are too small to have any confidence in the results presented in this paper. I therefore lean towards rejection.
strength_weakness: strength_weakness: The methods part of this paper is claimed as a key contribution but is considerably weaker than the application. The authors introduce a method for introducing augmentations into the training of a multiple instance learning approach. This is not necessarily straightforward for these models since the feature vectors are pre-computed as it is computationally too expensive to perform feature extraction again on each iteration of the MIL model training. In this paper, the approach is to precompute features from a limited number (4) of augmented versions of each tile, and then incorporate these into the training process. This appraoch has very limited novelty (being esentially the standard way to do augmentation in other types of model) and unfortunately is somewhat limited as it does not scale well to a greater number of augmetnations without taking up considerable storage to save the precomputed features. However, the authors show that an improvement can be seen with 4 versions, which is reasonable. The biggest problem however is that, while augmentation within MIL approaches is a problem ripe for further investigation, there are some other papers describing successful methods to do this (including xUP-MIL and ReMix, both cited in this manuscript), but the proposed method is not compared to these existing methods. This means it is impossible to evaluate the importance of this contribution.

A further weakness is that there are some aspects of the results that reduce my confidence in them. It seems very strange to me that the features extracted from the original images using a DenseNet model would give essentially a useless classifier (AUROC = 0.43) but features extracted in the same way from ResNet would give a *much* better classifier (AUROC=0.80), even though the two models were trained on the same dataset (ImageNet). This is counter to all my experience with these models, in which their performance is very similar. Similarly, I find the huge range between the AUROCs of the five cross-validation folds for the ResNet model (from < 0.5 to > 0.8) very troubling and suggestive of a significantly undersized dataset for this task or a deeper problem with the training process.
questions: questions: The claims of novelty of the proposed augmentation approach should be toned down, as should the conclusions about the effectiveness of the method as other published approaches have not been evaluated.

Comments: title: Thank you for the constructive feedback and suggestions provided by the reviewer. We have carefully addressed each point raised and updated the manuscript accordingly. Below are our responses to each of the reviewer's comments:
comment: We appreciate the reviewer's feedback and their acknowledgment of the improvements made in this revision.
We acknowledge the concerns of the reviewer. Predicting DNA content abnormality from WSIs indeed presents a challenging task. Our study aimed to address this challenge by exploring various backbone models and augmentation strategies. The main output model of the study, foundation model as the feature extraction backbone combined with image-based augmentation, demonstrates a robust performance at decent consistency.
While DenseNet and ResNet have demonstrated success in tasks such as tumor classification, achieving robust and consistent performance in a challenging task like DNA content abnormality prediction may require alternative solutions. We have duly noted the limitations of our study regarding the DenseNet and ResNet experiments in the limitations section of the manuscript.
We apologize for any oversights in grammar and typos. The manuscript has been thoroughly checked and these errors have been corrected. Additionally, we have amended the figure legends to enhance the overall clarity of the document.
We apologize for any oversights in grammar and typos. The manuscript has been thoroughly checked and these errors have been corrected. Additionally, we have amended the figure legends to enhance the overall clarity of the document.
Thank you for your response and modifications. 
The sentences added to the manuscript must, however, be checked for grammar and typos.
The paper has clearly been improved in this revision. Yet I am not completely convinced by the results. One may question the robustness of the approach esp when looking at Table 2 in appendix B and the difference between backbones/strategy combinations. 
There are still many typos in the paper: Model size small on page 4, "stuy", "ReNet" on page 6, "transfromations" in appendix...
R: Other Augmentation methods
We appreciate the suggestion from the Reviewer. Due to time constraints, we evaluated our method against PseMix (Liu et al., IEEE TMI 2024), which is well-established in the literature. While we recognize the importance of feature-based augmentation methods, we believe image-level augmentations offer unique advantages, such as addressing staining and quality variations. Combining PseMix with image augmentation improved performance compared to no augmentation, although image-based augmentation alone showed superior results. Further fine-tuning may be needed for PseMix to be effective in challenging tasks like aneuploidy prediction. This experiment underscores the potential of combining feature-based and image-based augmentation methods in MIL models. We have updated the results and discussion sections accordingly.

R: Why is ResNet50 better than DenseNet121? Why is the range huge?
We appreciate the reviewer's feedback and concerns regarding the variability in performance between ResNet50 and DenseNet121 models.
Upon investigation, we discovered an oversight in the training process of the DenseNet121 model related with initialisation of final layer during the initial submission. This adjustment improved the performance of the DenseNet-based models, resulting in performance slightly better than ResNet50.
We appreciate the acknowledgment of the concern regarding the variability in AUROC values observed in the cross-validation folds for the ResNet model. Upon reflection, we recognize that this variability may indeed be attributed to potential issues with the dataset size, particularly the imbalance between the positive and negative classes.

R: were the same augmentations applied to all patches in a WSI, or was a different random augmentation applied to each patch?
We apologize for any confusion. Each tile in every slide underwent different augmentations with varying factors. The application of image transformations (such as blur, brightness, and zoom) was randomly determined for each image patch. Additionally, the factor for each image transformation was randomly selected within a specified range. Detailed probabilities and ranges for each transformation can be found in Appendix A.
In addition to the original patch features, we generated four sets of features after applying image transformations. To ensure consistency across different backbones, we assigned predefined seeds for the generation of each set of feature vectors. By using different seeds for each set of augmented features, we ensured the utilization of different views of each patch for feature extraction, resulting in a variety of training data. This approach maintained diversity between sets within a training session while ensuring uniformity between backbones.

R: Section 2.3 is presented without any context
We are sorry for the oversight and appreciate the reviewer's suggestion. The cell feature classification analysis serves multiple purposes: (1) Understanding and explaining the histological features important for MIL model predictions, and (2) validating these features with known DNA content abnormality-related cell nucleus changes, such as enlarged and hyperchromatic nuclei. We have included a sentence in the introduction related to this part, and the relevant methods and results sections have been elaborated accordingly.

R: What are the four categories on the x-axis of Figure 3D? E.g. what is meant by "diploid-diploid"? I would expect two categories, just "diploid" and "aneuploid".	
We are sorry for the confusion. The X-axis labels are showing the ground truth and predictions together. The part before underscore is the reference (flow analysis result) and the second part is the prediction. The figure was updated for clarity.

R: How many texture features were analyzed, and was a multiple hypothesis correction applied.
We thank to reviewer to draw our attention to the statistical calculation point. 
We calculated in total 24 features ( 6 shape, 5 intensity and 13 texture features). 
Formerly we did not apply any multiple hypothesis correction. After the reviewer’s comment, we applied Benjamini & Hochberg (1995) ("BH") p adjustment and the fig 3D was updated based on adjusted p values. 

R: simple "classical" classification approach based on cell features
A: We thank to the reviewer for the suggestion. Although the cell features show significant difference on aneuploid samples, the analysis is depended on MIL classification. The features are calculated in the epithelial cells in top-attention scored patches. Given that aneuploidy is often a local event, it is un expected that analysis of whole slides and training a basic classifier provides similar performance.

R:Fig2A
The figure was updated and moved to appendix.

R: There are lots of missing spaces in the text before parentheses and citations.
We apologise for the grammar mistakes. The errors were corrected.
One could also have investigated different augmentation strategy
We thank the reviewer for the suggestion. We implemented PseMix (Liu et al., IEEE TMI 2024) data-augmentation method to the study. The reason for selecting PseMix was its comprehensive demonstration of robustness in published literature.
However, we propose image-level augmentation as an alternative to feature-level data augmentation (e.g., PseMix) because they offer different strengths and weaknesses. Image-level augmentations provide variation at the image level for possible staining, scanner, or slide quality-related variations, such as color, brightness, noise, blur, saturation, etc., which differs from Mixup and pseudobag-based feature-level data augmentation methods.
Combining PseMix and image augmentation resulted in improved performance compared to the no augmentation setting. Yet, image-based augmentation alone showed superior performance compared to PseMix. While the authors demonstrated the robustness of the PseMix method in tumor classification tasks (Liu et al., IEEE TMI 2024), fine-tuning may be required for the clustering, masking, phenotype-stratified sampling, and mixing steps to work effectively in a more challenging setting such as DNA content abnormality (aneuploidy) prediction in small esophagus endoscopical biopsies. This experiment highlights the effectiveness and potential of combining feature-based and image-based augmentation methods in MIL models in computational pathology.
We have updated the results and discussion sections accordingly.

Could the threshold be adjusted to improve this score
We apologize for the oversight regarding the threshold value for WSI-level classification. The default threshold of 0.5 was indeed used, but it was not explicitly stated in the document. Following your recommendation, we calculated the optimal threshold using the ROC curve and the TPR-(1-FPR) formula based on probabilities of the training dataset samples. As a result, the balanced accuracies for DenseNet and ResNet were noticeably improved. We have updated the manuscript text and tables to reflect this methodology and the results. The thresholds for the test set were adjusted from 0.41, 0.13, and 0.85 to 0.67, 0.50, and 0.83 for ResNet, DenseNet, and the foundation model, respectively. Table 1 has been updated accordingly. We appreciate the insightful recommendation from the reviewer.

I do not understand why on page 4, the first cohort is split into training (85%) / validation (15%) while on page 6, the authors write that "all training sessions utilized 5-folds of training/validation splits"? It is a leftover from a previous version.
We apologize for the confusion. we applied 5 iterations of Monte Carlo cross-validation for the training. Therefore, there is no leftovers. We have updated the relevant section in the methods to clarify this discrepancy. Thank you for bringing this to our attention.
A comparison with other augmentation methods is also needed.

We appreciate the reviewer's suggestion to compare our image augmentation method with other data augmentation methods developed for MIL training. Due to time constraints for revision, we opted to select one approach and test it against our results. Among the available options, we chose PseMix (Liu et al., IEEE TMI 2024) due to its comprehensive demonstration of robustness in the published literature.
However, we propose image-level augmentation as an alternative to feature-level data augmentation (e.g., PseMix) because they offer different strengths and weaknesses. Image-level augmentations provide variation at the image level for possible staining, scanner, or slide quality-related variations, such as color, brightness, noise, blur, saturation, etc., which differs from Mixup and pseudobag-based feature-level data augmentation methods.
Combining PseMix and image augmentation resulted in improved performance compared to the no augmentation setting. Yet, image-based augmentation alone showed superior performance compared to PseMix. While the authors demonstrated the robustness of the PseMix method in tumor classification tasks (Liu et al., IEEE TMI 2024), fine-tuning may be required to work effectively in a more challenging setting such as DNA content abnormality (aneuploidy) prediction in small esophagus endoscopical biopsies. This experiment highlights the effectiveness and potential of combining feature-based and image-based augmentation methods in MIL models in computational pathology.
We have updated the results and discussion sections accordingly.


I do not understand the results in the abstract and rest of the paper later. 0.91 AUC is mentioned (again in Appendix B), then 0.93 (again e.g. in Table 1).
We apologize for any confusion regarding our performance metrics. The performance metrics were reported in two forms:
1- The average of multiple Monte Carlo cross-validation training/validation splits (n=5), as indicated in Table Appendix B, and the second paragraph of the abstract.
2- The top AUC score from a specific training/validation split, as depicted in Figure 2A-B, Table 1, and the third paragraph of the abstract. The model utilizing this split and the foundation for feature extraction was selected for further analysis, including cell morphology correlation analysis.


In the abstract, it reads all patients n=117. But I read 34+89 patients in 2.1.
We apologise for the mistake. The correct patient count is 123. The number in the abstract was wrong. It is corrected. 
In the abstract, you mention 2 backbone models (DenseNet and REMEDIS), but ResNet is also used in the Results.
Thank you for noticing the missing part. We added the average AUC of features based ResNet50 in the abstract. 
Annotations are at the WSI level. If a correlation is found between aneuploidy and EAC progression, why not also trying to predict directly EAC?
We are grateful to the reviewer for raising an important point regarding the direct prediction of esophageal adenocarcinoma (EAC) and its potential clinical implications. While our current focus centers on unraveling the biological significance of aneuploidy in Barrett Esophagus, we acknowledge the merit of exploring direct prediction models for EAC. Additionally, aneuploidy is the most strongly correlated biological change in BE which can be predictive for EAC.
In our study, our primary objective is to delve into the underlying biological mechanisms of molecular alterations, specifically aneuploidy, within Barrett Esophagus and its association with cancer progression. While predicting EAC directly is certainly an intriguing prospect, our focus lies in elucidating the significance of aneuploidy in this context. By training our model to predict aneuploidy, we not only provide valuable molecular and clinical prognostic insights but also pave the way for subsequent analyses. We think, investigation of the provenly correlating middle steps in the cancer progression can provide valuable insight and soil for deeper understanding of the progression. 


I am not sure all the mentioned packages and versions are necessary. It would be better to provide the code and requirements on github.
We thank the reviewer for the suggestion to simplify the manuscript. We have made the necessary changes in the methods section accordingly. The list of used packages and the code will be shared upon publication.


Some typos to fix, e.g. "as the presence aneuploidy", "necessitate use of precious", "provides the model is primarily" etc.
We thank to the reviewer for bringing those typos to our attention. We have corrected them accordingly.