RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Supplemental Information
(A). Workflow of sparse stain matrix decomposition constraint Cycle GAN color normalization model (SDCC-GAN). (B). Representative tiles of the color normalization network. The original tiles are in the first and second rows; The corresponding color-normalized tiles are in the third and fourth rows.
Figure 1: Development and interpretability of a deep learning algorithm for detecting BRCA gene status.
(A) The H&E images underwent preprocessing, which included scanning, delineating and segmenting tumor regions, and normalizing the color of tiles. The normalized tiles were then input into the BiAMIL model, which comprises ResNet 34 feature extraction and an attention mechanism. The model output the predicted BRCA status for each WSI. (B) The model’s high-attention regions were visualized in a heatmap, where red indicates morphological features significantly contributing to the model’s predictions. (C) The cell morphological features within the model’s high-attention regions were quantified, comparing the differences between the BRCA PV group and the BRCA WT group. BiAMIL, Bi-directional Self-Attention Multiple Instance Learning; PV, pathogenic variants; WT, wild type; WSI, whole slide image.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-1
Figure 2: Performance of the segmentation network.
(A) Receiver operating characteristic (ROC) curve comparing the performance of the segmentation network with three pathologists (senior, medium, and junior) in identifying tumor areas. (B) Confusion matrix for the segmentation network. (C) Original WSIs and annotation maps identified by the segmentation network. Left: original WSI images; right: corresponding annotation maps.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-2
Figure 3: Architecture of the BiAMIL model.
First, color-normalized tiles are input into the pre-trained ResNet 34 model to extract 1,000-dimensional features. Next, these features are aggregated using a bi-directional self-attention mechanism approach, where the attention mechanism assesses the relevance and importance among the tiles. Finally, the aggregated feature matrix is fed into a MLP classifier that outputs the probability of BRCA gene mutation status for each WSI. MLP, Multilayer perceptron.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-3
Figure 4: Predictive performance of the BiAMIL and ResNet34 models.
(A and B) ROC curves for the BiAMIL model on the internal and external test sets. (C and D) ROC curves for the ResNet 34 model on the internal and external test sets. (E and F) Comparison of ACC, Sen, Spe, PPV, NPV, and F1 score for the BiAMIL and ResNet 34 models on the internal and external test sets. ROC, receiver operating characteristic; ACC, accuracy; Sen, sensitivity; Spe, specificity; PPV, positive predictive value; NPV, negative predictive value.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-4
Figure 5: Visualization of important regions in H&E images for predicting BRCA status.
(A and B) Tiles with high attention score and CAM heatmaps from the BiAMIL model, highlighting regions of high contribution in red. (C) Representative H&E pathological image and cell segmentation map, with segmentation performed using Qupath software. Red represents tumor cells, and green represents TILs. (D) Comparison of the proportion of TILs between the BRCA PV group and the WT group. Differences between the groups in the box plot were calculated using the two-tailed Wilcoxon rank-sum test. The black horizontal line represents the sample median, the box extends from the first quartile to the third quartile, and the whiskers extend to 1.5 times the interquartile range. (E) Visualization of the prediction results using the t-SNE algorithm, with red indicating predictions of BRCA PV, and blue indicating predictions of BRCA WT. TILs, Tumor infiltrating lymphocytes; PV, pathogenic variants; WT, wild type.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-5
Figure 6: Analysis of tumor cell morphological features in high-attention regions.
(A) Visualization of tumor cell feature clustering in high-attention regions identified by the BiAMIL model using the k-means algorithm. Each color represents a different cluster (Cluster 0 to Cluster 5). (B) Heatmap displaying the correlation between clusters and cell morphological features. (C-E) Comparison of tumor cell area, nuclear area, and nuclear cell area ratio between the BRCA PV and BRCA WT groups. The differences between the two groups were calculated using the two-tailed Wilcoxon rank-sum test.

  Download full-size image
DOI: 10.7717/peerj.18098/fig-6
Characteristics of patients in the training, internal test, and external test sets.
The performance of the BiAMIL in predictingBRCAPV in hormone receptor + and triple-negative breast cancer.