Knowledge-guided adaptation of pathology foundation models effectively improves cross-domain generalization and demographic fairness

Domain: physics
Published: 2025-12-12
Editorial outcome: minor_revision

Abstract
Foundation models in computational pathology suffer from site-specific and demographic biases, which compromise their generalizability and fairness. We introduce FLEX, a framework that employs a task-specific information bottleneck, guided by visual and textual domain knowledge, to disentangle robust pathological features from these artifacts. Using three large cohorts (The Cancer Genome Atlas, Clinical Proteomic Tumor Analysis Consortium, and an in-house dataset) across 16 clinical tasks, totaling over 9,900 slides, we demonstrate that FLEX achieves superior zero-shot generalization to unseen external cohorts, significantly outperforming baselines and narrowing the performance gap between seen and unseen domains. A comprehensive fairness analysis confirms that FLEX also effectively mitigates disparities across demographic groups. Furthermore, its versatility and scalability are proven through compatibility with various foundation models and multiple-instance learning architectures. Our work establishes FLEX as a promising solution for developing more generalizable and equitable pathology AI for diverse clinical settings.

Introduction
Histopathology remains the gold standard for cancer diagnosis and treatment planning, yet its traditional manual analysis faces challenges of subjectivity and limited scalability1–3. Recent advances in artificial intelligence (AI) have revolutionized computational pathology, enabling the automated extraction of complex patterns from histopathology images with high accuracy and efficiency4–8. These advancements have paved the way for clinical applications such as automated cancer diagnosis, molecular biomarker prediction, and treatment response assessment9–11. A pivotal development in this field is the emergence of pathology foundation models12–19, which are pre-trained on large-scale pathology datasets via self-supervised learning (SSL)12,13,14,15,16,18,20–26. These models act as powerful image feature extractors, acquiring transferable representations from whole slide image (WSI) patches and enabling accurate predictions for various clinical tasks by integrating multiple-instance learning (MIL) models5,27–31.

Despite these advancements, a critical limitation persists: features extracted by pathology foundation models exhibit significant site-specific signatures. These signatures, which stem from variations in tissue preparation, staining protocols, and scanning equipment across different clinical centers (e.g., hospitals or research institutions) where samples are processed (Fig. 1a), directly impede cross-domain generalization. Specifically, they foster shortcut learning32–34 (Fig. 1b), where models learn superficial site-specific patterns rather than the underlying pathological features essential for robust diagnosis. As a result, these models often perform well on in-domain (IND) testing data (i.e., data from the same submitting sites as the training data), but exhibit drastically reduced accuracy and unreliable predictions on out-of-domain (OOD) testing data (i.e., data from different sites with distinct site-specific patterns), compromising diagnostic reliability and posing a serious risk of misdiagnosis and incorrect treatment decisions in clinical practice. Although stain normalization10,35,36 has been proposed as a potential solution37, our findings demonstrate its ineffectiveness in eliminating complex site-specific artifacts. Adding to these challenges of cross-site generalization, recent studies have highlighted a concerning lack of demographic fairness in pathology foundation models35, arising from variations in tissue appearance across demographic groups and imbalances in training data. Therefore, effectively mitigating site-specific biases and demographic disparities is paramount for the responsible and reliable deployment of pathology foundation models in diverse clinical settings.

In this work, we introduce FLEX (Feature-Level Enhancement for Cross-domain generalization), an approach that leverages domain knowledge to address the intertwined challenges of cross-site generalization and demographic fairness in pathology foundation models. FLEX enhances the model’s focus on generic pathological patterns by employing an information bottleneck. This bottleneck is guided by visual and textual concepts enriched with domain-specific prior knowledge, and strategically aligns patch image features with task-specific textual concepts. These textual concepts are generated by the text encoder of pathology foundation models, and thus, originating from the text domain, they are inherently free from site-specific signatures and demographic biases present in the images. As a result, pathology images from diverse source sites and demographic groups, which exhibit distinct feature distributions, are effectively aligned to a unified feature space. This alignment enables the model to generalize across heterogeneous sites and demographic groups, while enhancing the discriminativeness and expressiveness of encoded patch features for specific tasks. To validate our approach, we first confirm the presence of confounding site-specific signatures and demographic biases in existing foundation models. We then assess FLEX across 16 diverse clinical tasks using a rigorous two-stage evaluation. This involves a Site-Preserved Monte Carlo Cross-Validation (SP-MCCV) within The Cancer Genome Atlas (TCGA) cohort and a zero-shot external validation on two independent cohorts: the public Clinical Proteomic Tumor Analysis Consortium (CPTAC; n=1738) and our in-house Nanfang Hospital dataset (NFH; n=5318). This comprehensive evaluation demonstrates three key strengths of FLEX. First, it yields an improvement in cross-domain generalization, with consistent performance gains on both held-out TCGA sites and the external CPTAC and NFH cohorts. Second, it promotes demographic fairness by reducing performance disparities across demographic groups. Third, FLEX shows practical versatility through its compatibility with various pathology vision-language models (VLMs) and downstream MIL architectures, and maintains effectiveness across different data scales. These findings establish FLEX as a promising solution for improving the generalizability and fairness of pathology AI. By addressing the critical barriers of domain shift and bias, it offers a robust pathway for the responsible, and equitable deployment of foundation models in clinical settings.

Results
We first systematically evaluated the persistence of site-specific signatures and demographic biases in representations from pathology foundation models, and their negative impact on cross-domain generalization and fairness. We then demonstrate that our proposed FLEX framework mitigates these issues. As outlined in Fig. 1d, FLEX leverages visual and textual prior knowledge to enhance the discriminability and robustness of patch features. Our results are presented in three parts. First, we show that FLEX improves cross-domain generalization within the TCGA cohort and in zero-shot validation on external CPTAC and in-house cohorts. Second, we assess its impact on demographic fairness, revealing a marked reduction in performance disparities across racial and ancestry groups. Finally, we establish the practical versatility of FLEX by confirming its compatibility with various foundation models and MIL architectures across different data scales. These findings highlight FLEX as a robust solution for developing more generalizable and equitable pathology AI models.

FLEX improves cross-domain generalization and facilitates accurate clinical usage of pathology foundation models
This section evaluates how site-specific patterns in feature representations extracted by pathology foundation models affect cross-site generalization, potentially limiting their reliability for clinical applications. TCGA is a cornerstone resource for cancer research, which includes WSIs from multiple contributing sites and introducing site-specific signatures. For instance, the non-small cell lung carcinoma (NSCLC) cohort comprises WSIs from 66 sites, breast carcinoma (BRCA) from 36 sites, stomach adenocarcinoma (STAD) from 22 sites, and colorectal adenocarcinoma (CRC) from 37 sites (Fig. 1c). To investigate this, we utilized UMAP38 to visualize patch features extracted by the CONCH18 image encoder from the BRCA cohort (Fig. 2a, middle). Patches were color-coded by the submitting site (left panel) and diagnostic class (right panel). The clear clustering by site revealed the presence of pervasive site-specific signatures, a finding that is robust to UMAP hyperparameter variations (Supplementary Fig. 1). These patterns serve as problematic shortcuts, enabling models to predict clinical labels without learning meaningful task-relevant features and thereby compromising cross-domain generalization, as illustrated in Fig. 1b.

We first quantified this issue across 16 pathology tasks using an SP-MCCV strategy (see ‘Site-preserved monte carlo cross-validation’ in “Methods”) within TCGA, where test data were split into IND and OOD subsets based on submitting sites (Fig. 2b). Patch features were extracted using CONCH18 and aggregated with an ABMIL model27. The results showed a significant drop in area under the receiver operating characteristic curve (AUROC) performance for OOD data compared to IND data across all task categories (Fig. 2c). Specifically, the average AUROC for IND was 0.918, 0.754, and 0.651 for morphology, molecular biomarker, and gene mutation tasks, respectively. In contrast, the OOD AUROC dropped to 0.853, 0.721, and 0.628. This performance decline illustrates the substantial impact of domain shift, a challenge that persists even for state-of-the-art pathology foundation models.

A common method for addressing domain shift is stain normalization, such as the Reinhard method39–41, which aligns the color distribution of WSIs to a reference template. While this approach can be effective when domain shift is primarily due to color variation, it is often insufficient for more complex biases. As shown in Fig. 2a (right panel), features after Reinhard normalization remain strongly clustered by site. Consequently, this method yielded minimal and inconsistent performance gains, and in some cases, even reduced performance (Fig. 2d, Supplementary Table 1).

In contrast, FLEX is designed to address a broader spectrum of site-specific signatures, including not only staining variations but also subtle variations from tissue preparation and scanner artifacts, through its knowledge-guided information bottleneck. When applied within the TCGA cross-validation, FLEX significantly improved model generalizability for both IND and OOD testing (Fig. 2c). Specifically, the average OOD AUROC increased by 6.45%, 7.21%, and 4.94% for the three task categories, substantially narrowing the IND-OOD performance gap. UMAP visualizations for the BRCA-TYPE task (Fig. 2a, left panel) further confirmed that FLEX effectively suppressed site-specific signatures, forcing features to align based on their biological class rather than their site of origin. Detailed results across all 16 tasks (see ‘Dataset and clinical tasks description’ in “Methods” and Supplementary Table 2) show consistent performance improvements in both IND and OOD settings (Fig. 2d). Notably, FLEX achieved a 15.13% increase in OOD AUROC for the STAD-LAUREN task, reducing the IND-OOD gap from 20.59% to 8.64%. This highlights FLEX’s potential to address key challenges in cross-site generalization.

Furthermore, we conducted a zero-shot evaluation on two independent external cohorts: the public CPTAC dataset and our in-house NFH dataset (see Supplementary Tables 3, 4). Models trained exclusively on TCGA were applied directly to these unseen cohorts without any fine-tuning. These datasets introduce unique site-specific signatures distinct from TCGA, representing a true out-of-distribution challenge. Across all tasks and both cohorts, the baseline model’s performance was modest. However, applying FLEX resulted in a consistent and significant performance uplift, demonstrating its ability to generalize to new clinical environments. The average AUROC for the CPTAC cohort increased from 0.679 to 0.714 (P = 9.432 × 10−16) and for the NFH cohort from 0.733 to 0.771 (P = 8.900 × 10−12), as shown in Fig. 2c. The radar plot in Fig. 2e provides a comprehensive overview, showing that FLEX (pink line) consistently improves performance. Detailed results for each task are provided in Supplementary Fig. 2 and Supplementary Tables 5, 6. These analyses indicate that FLEX learns transferable biological representations rather than memorizing source-domain shortcuts.

To provide qualitative insights into FLEX’s mechanism, we visualized attention maps from the ABMIL models (Fig. 2f). Here, noisy regions are defined as visual information irrelevant to the diagnostic task, such as processing artifacts or task-agnostic biological structures (e.g., non-neoplastic stroma). For the BRCA-TYPE task (Fig. 2f, top panel), the baseline model’s attention is diffuse, scattered across tumor cells and distracting noisy elements. In contrast, the model with FLEX consistently focuses on critical, task-relevant features like the irregular glandular and ductal formations characteristic of Invasive Ductal Carcinoma (IDC) without being distracted by noisy foreground elements. This focused attention provides an intuitive explanation for the quantitative performance gains. Similarly, for the STAD-MSI task, the FLEX-enhanced model successfully identified potential Microsatellite Instability-High (MSI-H) regions, where tumor cells exhibit greater heterogeneity, whereas the baseline model failed to do so, leading to incorrect predictions. This suggests that FLEX enhances the model’s focus on salient pathological patterns while suppressing task-agnostic information.

FLEX effectively improves demographic fairness
Demographic biases within large-scale datasets like the TCGA dataset exacerbate fairness concerns, leading to significant performance disparities in deep learning models, particularly disadvantaging underrepresented populations from minority demographics35. Because self-reported race may not fully capture complex biological heritage, we also stratified cohorts by genetic ancestry42, which provides a more biologically grounded measure. Our fairness analysis was conducted across both stratification methods in the TCGA dataset to ensure a comprehensive evaluation. Additionally, we performed a zero-shot fairness evaluation on the CPTAC dataset, which contains only self-reported race. As baseline experiments confirm (Fig. 3b, c), models trained on original features exhibit notable performance disparities across both stratification methods. Simple stain normalization, while useful for reducing inter-site staining variations, does not effectively address these demographic biases.

By addressing this critical limitation through feature disentanglement, FLEX offers a significant improvement in both demographic fairness and model reliability. We evaluated this using a multi-faceted approach, assessing performance across both self-reported race and ancestry cohorts (Fig. 3). This improvement is also observed within individual clinical sites (Supplementary Fig. 3).

First, we examine the Fairness Gap (AUROC gap ratio)43 in Fig. 3a, which measures the absolute performance difference between the best- and worst-performing racial groups relative to the overall AUROC. On average, FLEX consistently reduces the fairness gap across tasks. For instance, in the NSCLC-TYPE task, FLEX reduces the mean fairness gap for self-reported race (from 0.052 to 0.041) and ancestry (from 0.062 to 0.040). For the external CPTAC cohort, the fairness gaps for the C-BRCA-TYPE and C-LUAD-EGFR tasks decreased from 0.287 and 0.494 to 0.245 and 0.246, respectively. The scatter plots and marginal distributions in Fig. 3a show that FLEX’s results (pink dots) are more tightly clustered at lower gap values, indicating improved fairness. Even in cases like CRC-BRAF where an outlier widens the confidence interval, the central tendency of FLEX’s performance remains superior or comparable to the baselines.

For a more robust assessment of model reliability, we analyzed the True Positive Rate (TPR) disparity35 (Fig. 3c), which quantifies the difference between the TPR of a specific demographic group and the overall TPR. The violin plots for FLEX are consistently narrower and more centered around the zero-disparity line for both self-reported race and ancestry stratifications, demonstrating that the model applies its diagnostic capability more equitably. This is quantitatively supported by the TPR disparity RMSE values (Fig. 3d). In the challenging C-LUAD-EGFR task, for example, FLEX reduces the average RMSE from 0.140 to 0.085, indicating a substantial improvement in fairness. Similarly, the race-wise AUROC distributions in Fig. 3b show that FLEX yields tighter and more equitable performance clusters.

FLEX is versatile across different pathology VLMs
To further evaluate the effectiveness and adaptability of FLEX across different pathology VLMs, we integrated it with three state-of-the-art pathology VLMs (CONCH18, PathGen-CLIP19, and QuiltNet17), which served as patch-level feature extractors. Using ABMIL27 as the downstream MIL framework, we tested performance on 16 pathology tasks, focusing on the OOD generalization setting.

The results show that FLEX consistently and significantly enhances performance across all three foundational VLMs (Fig. 4a). While the baseline performance varies between VLMs, applying FLEX delivers a robust improvement in nearly all scenarios. For CONCH, FLEX improved OOD performance in all 16 tasks, with significant improvement (P < 0.05) observed in 14 out of 16 tasks. For PathGen-CLIP, FLEX improved performance in all tasks, with significant improvement in 9 out of 16 tasks. For QuiltNet, performance improved in 14 out of 16 tasks, with significant improvement in 9 tasks. This broad compatibility underscores that FLEX is a model-agnostic solution for improving generalization. The varying magnitude of improvement may be attributed to differences in the expressivity and discriminativeness of the original patch features extracted by different pathology VLMs. Since FLEX relies on visual and textual knowledge as guidance to enhance the patch features, the quality of this guidance depends on the features extracted by the VLMs. Visual knowledge and textual knowledge are generated by encoding pilot patch images and text prompts with the image and text encoders of the pathology VLMs. If the VLMs lack expressivity and discriminability, the resulting visual and textual knowledge may be less informative and discriminative, limiting the performance improvement achieved by FLEX.

We further analyzed the mechanism behind this improvement by visualizing the patch feature space for the STAD-EBV task using UMAP38 (Fig. 4b). For all three VLMs, the original features show strong clustering by site, with poor separation between EBV-positive and EBV-negative classes. This is confirmed by low Local Inverse Simpson’s Index (LISI) scores, which measure local data mixing by site, and Silhouette Scores, which measure class separability. After applying FLEX, the feature distributions become more uniform, with site-specific clusters dissolving. This is validated by a significant increase in the LISI score for all three VLMs (e.g., from 2.22 to 3.22 for CONCH), indicating better integration of features from different sites. Concurrently, the separation between EBV classes improves, reflected by a notable increase in the Silhouette Score (e.g., from 0.09 to 0.24 for CONCH). These results demonstrate that FLEX mitigates site-specific biases while enhancing the discriminability of task-relevant biological features, a dual benefit that holds across different foundational models.

FLEX is effective with different sizes of training data and flexible with different MIL models
To evaluate the scalability of FLEX with varying training data scales, we conducted additional experiments on the BRCA cohort. The experimental design is illustrated in Fig. 5a. The dataset was partitioned into five site-preserved outer folds, with one fold held out as the OOD test set in each of five outer loop iterations. Training sets of increasing size (1, 2, 3, and 4 folds) were constructed from the remaining four folds to create a learning curve. Three inner MCCV runs were performed for each scale to ensure stability. For this analysis, three tasks spanning different domains were selected: BRCA-TYPE, BRCA-HER2, and BRCA-PIK3CA. The corresponding training and OOD test set sizes are in Supplementary Table 7. The results shown in Fig. 5b demonstrate that FLEX consistently and significantly (P < 0.05) improves OOD performance across all tasks and data scales, underlining its ability to generalize effectively even with larger training datasets.

To further assess the flexibility of FLEX across MIL models, we integrated it with five state-of-the-art MIL architectures: ABMIL27, CLAM-SB5, ACMIL28, DTFD-MIL29, and ILRA-MIL30. As shown in Fig. 5c, FLEX consistently improved OOD performance across all models. The average OOD performance gains were 6.01%, 5.55%, 3.82%, 3.53%, and 6.06%, respectively. Additionally, FLEX significantly reduced the performance gap between IND and OOD testing. Specifically, the gap decreased from 0.034, 0.040, 0.041, 0.038, and 0.035 to 0.022, 0.024, 0.021, 0.025, and 0.020 for the corresponding MIL models. Detailed per-task performance metrics for all combinations of VLM and MIL models evaluated are provided in Supplementary Tables 8–23.

The performance improvement seen in Fig. 5c varies across the 16 pathology tasks due to three main factors: intrinsic task difficulty, the utility of feature refinement for a given task, and the complexity of the MIL model. First, inherent task difficulty sets the upper bound on performance; for example, morphology classification tasks consistently yield higher AUROCs than most gene mutation prediction tasks. Second, the magnitude of improvement from FLEX depends on task-specific conditions. In the STAD-LAUREN task, for instance, all MIL models show a substantial performance gain. This may be because the TCGA-STAD cohort is relatively small, causing baseline models to overfit to noisy features. FLEX’s core mechanism—refining features by filtering task-irrelevant information—improves the signal-to-noise ratio, which is particularly beneficial when training data is limited. Third, performance gains are more pronounced with simpler MIL models like ABMIL and CLAM-SB compared to more complex architectures. We hypothesize this is related to the joint optimization of FLEX and the MIL model. A simpler MIL backbone may create a more stable learning environment, allowing gradients to effectively guide the feature refinement process within FLEX. In contrast, with more intricate MIL models, the increased number of parameters and interacting components can complicate convergence to an optimal state, potentially limiting the observable performance gain from FLEX.

Methods
This retrospective study was conducted in compliance with all relevant ethical regulations. The research utilized publicly accessible data from the TCGA and the CPTAC cohorts, which were originally collected under approved protocols with informed consent from all participants. Information regarding participant compensation for these public cohorts was not available to the authors. The in-house NFH dataset was collected and used under the approval of the Medical Ethics Committee of NanFang Hospital of Southern Medical University (approval number: NFEC-2025-419), which granted a waiver of informed consent due to the retrospective nature of the study and the use of de-identified data, posing minimal risk to participants. No compensation was provided for the retrospective collection of NFH data. All data were handled with a strict commitment to patient privacy, fairness, and transparency.

The demographic characteristics of the patient cohorts, including sex and age, are detailed in the subsequent dataset descriptions. For all cohorts, participant sex and race were determined from self-reported information available in the associated clinical metadata, while genetic ancestry was derived from genomic data as established by the TCGA Pan-Cancer Atlas project42. Patient sex was not considered in the study design. Our fairness analysis focused primarily on self-reported race and genetic ancestry to directly address significant disparities previously reported in the literature for these groups in computational pathology.

Dataset and clinical tasks description
This study utilized datasets from the TCGA cohort, including data from breast invasive carcinoma (BRCA), non-small cell lung cancer (NSCLC, encompassing lung adenocarcinoma (LUAD) and lung squamous cell carcinoma (LUSC)), stomach adenocarcinoma (STAD), and colorectal carcinoma (CRC, encompassing colon adenocarcinoma (COAD) and rectum adenocarcinoma (READ)). The original WSIs were collected from the publicly available TCGA program (https://www.cancer.gov/ccg/research/genome-sequencing/tcga), and corresponding clinical, molecular, and gene mutation information was obtained from the cBioPortal database (https://www.cbioportal.org). In this study, we conducted 16 WSI analysis tasks to comprehensively evaluate the effectiveness of FLEX across different task types. These tasks include: three morphology classification tasks across three organs, five molecular biomarker status prediction tasks across two organs, and eight gene mutation prediction tasks across four organs. This diverse range of tasks ensures a robust assessment of FLEX in various computational pathology applications. A detailed breakdown of patient and slide counts for each task in the TCGA cohort is provided in Supplementary Table 2.

The demographic and clinical characteristics of the primary TCGA cohorts are as follows: the TCGA-BRCA cohort included 876 female (98.8%) and 11 male (1.2%) patients, with a median age of 58 years (range: 26–90). The TCGA-NSCLC cohort comprised 341 female (40.4%) and 503 male (59.6%) patients, with a median age of 67 years (range: 38–90). The TCGA-STAD cohort included 132 female (34.0%) and 256 male (66.0%) patients, with a median age of 67 years (range: 30–90). The TCGA-CRC cohort consisted of 275 female (48.2%) and 296 male (51.8%) patients, with a median age of 67 years (range: 31–90).

Morphology classification. For the morphology classification tasks, the objective was to classify histopathology images into distinct subtypes of the same cancer type:
• BRCA-TYPE: This binary classification task involves classifying histopathology images from the TCGA-BRCA cohort into invasive ductal carcinoma (IDC) and invasive lobular carcinoma (ILC)44.
• NSCLC-TYPE: This two-class classification task categorizes histopathology images from the TCGA-NSCLC cohort into lung adenocarcinoma (LUAD) and lung squamous cell carcinoma (LUSC)45.
• STAD-LAUREN: This task focuses on the Lauren classification of histopathology images from the TCGA-STAD cohort, distinguishing between the intestinal type and diffuse type46.

Molecular biomarker status prediction. The molecular biomarker status prediction tasks aim to identify clinically relevant biomarker statuses of cancer based on histopathology images, leveraging specific morphological correlations:
• BRCA-HER2, BRCA-ER, and BRCA-PR: These binary classification tasks predict the human epidermal growth factor receptor 2 (HER2), estrogen receptor (ER), and progesterone receptor (PR) statuses of breast cancer using histopathology images from the TCGA-BRCA cohort47–49.
• STAD-EBV and STAD-MSI: These binary classification tasks predict the Epstein-Barr virus (EBV) and Microsatellite Instability (MSI) statuses of stomach cancer using histopathology images from the TCGA-STAD cohort50,51.

Gene mutation prediction. Gene mutation prediction tasks aim to predict the mutation status of specific genes in cancer using histopathology images. In this study, the following binary classification tasks were conducted:
• BRCA-PIK3CA and BRCA-CDH1: These tasks predict the PIK3CA and CDH1 gene mutation statuses in breast cancer using histopathology images from the TCGA-BRCA cohort52,53.
• LUAD-EGFR and LUAD-STK11: These tasks predict the EGFR and STK11 gene mutation statuses in lung adenocarcinoma using histopathology images from the TCGA-LUAD cohort54,55.
• STAD-TP53 and STAD-MUC16: These tasks predict the TP53 and MUC16 gene mutation statuses in stomach adenocarcinoma using histopathology images from the TCGA-STAD cohort56,57.
• CRC-BRAF and CRC-TP53: These tasks predict the BRAF and TP53 gene mutation statuses in colorectal carcinoma using histopathology images from the TCGA-CRC cohort58,59.

By targeting tasks of varying complexity and clinical significance, we aimed to comprehensively assess the effectiveness of FLEX across a diverse range of WSI analysis applications.

External Validation Datasets
To evaluate the zero-shot generalization capability of models trained on TCGA, we used two independent external cohorts for testing: the public CPTAC dataset and a large in-house dataset from NFH. These datasets introduce significant domain shifts, including different patient populations, slide preparation protocols, and scanning equipment, providing a stringent test of real-world applicability. Detailed statistics for the tasks performed on these external cohorts are available in Supplementary Table 3 for CPTAC and Supplementary Table 4 for NFH. The specific tasks for external validation were selected based on data availability and their overlap with the TCGA tasks.

CPTAC Dataset. The CPTAC collections are a publicly available resource (https://portal.gdc.cancer.gov/projects/CPTAC-2, https://portal.gdc.cancer.gov/projects/CPTAC-3) that includes genomic and proteomic data along with corresponding WSIs. For our external validation, we selected a subset of tasks that mirror those in our primary TCGA analysis. The CPTAC patient demographics for the tasks used in this study were as follows: the CPTAC-BRCA cohort included 116 female patients with a median age of 59 years (range: 30–89). The CPTAC-NSCLC cohort included 82 female (30.9%) and 183 male (69.1%) patients with a median age of 65 years (range: 36–88). The CPTAC-COAD cohort consisted of 61 female (58.7%) and 43 male (41.4%) patients with a median age of 65 years (range: 35–93). The specific tasks covered three cancer types:
• BRCA (C-BRCA-TYPE, C-BRCA-PIK3CA, C-BRCA-CDH1): Including subtype classification (IDC vs. ILC) and prediction of PIK3CA and CDH1 gene mutations using histopathology images from the CPTAC-BRCA cohort.
• NSCLC (C-NSCLC-TYPE, C-LUAD-EGFR, C-LUAD-STK11): Including subtype classification between LUAD and LUSC, and prediction of EGFR and STK11 gene mutations in LUAD using histopathology images from the CPTAC-NSCLC cohort.
• COAD (C-COAD-BRAF, C-COAD-TP53): Including prediction of BRAF and TP53 gene mutations in COAD using histopathology images from the CPTAC-COAD cohort.

Nanfang Hospital of Southern Medical University (NFH) In-house Dataset. The NFH dataset is a large-scale, retrospectively collected in-house cohort from Nanfang Hospital of Southern Medical University, Guangzhou, China. This dataset provides a challenging test case due to its distinct technical and demographic characteristics compared to the Western-centric TCGA cohort. The demographic breakdown for the cohorts used in this study is as follows: the NFH-BRCA cohort consisted of 761 female (99.9%) and 1 male (0.1%) patient, with a median age of 52 years (range: 17–89). The NFH-NSCLC cohort included 910 female (46.7%) and 1037 male (53.3%) patients with a median age of 59 years (range: 15–87). The NFH-STAD cohort included 146 female (37.0%) and 249 male (63.0%) patients with a median age of 60 years (range: 20–89). The NFH-CRC cohort included 482 female (41.6%) and 678 male (58.4%) patients with a median age of 62 years (range: 21–93). The validation tasks on the NFH dataset cover four major cancer types:
• BRCA (N-BRCA-TYPE, N-BRCA-HER2, N-BRCA-ER, N-BRCA-PR): Including subtype classification (IDC vs. ILC) and prediction of HER2, ER, and PR biomarker statuses using histopathology images from the NFH-BRCA cohort.
• NSCLC (N-NSCLC-TYPE, N-LUAD-EGFR): Subtype classification between LUAD and LUSC and prediction of EGFR gene mutation status in LUAD using histopathology images from the NFH-NSCLC cohort.
• STAD (N-STAD-LAUREN): Lauren classification between Intestinal and Diffuse type in STAD using histopathology images from the NFH-STAD cohort.
• CRC (N-CRC-BRAF): Prediction of BRAF gene mutation status in CRC using histopathology images from the NFH-CRC cohort.

By evaluating model performance on these diverse external datasets without any fine-tuning, we assess the effectiveness of FLEX in overcoming dataset-specific biases and generalizing to unseen clinical environments.

Preprocessing of histopathology images
Segmentation and patching. We adopted the preprocessing steps outlined in CLAM5, which involve segmenting and cropping tissue regions from each WSI and extracting instance features from each patch using a feature extractor. Specifically, each WSI is first loaded into memory at a downsampled resolution (i.e., 32× downscale) and converted from RGB to HSV color space to facilitate tissue segmentation. A binary mask for the foreground is then computed by thresholding the saturation channel after applying median blurring to smooth edges. Morphological closing operations are subsequently performed to fill small gaps and holes within the tissue regions. The segmented foreground tissue regions are further cropped into 512×512 patches from within the foreground contours at 20× magnification for each slide.

Feature extraction. Due to the potentially vast size of the patch bag for each WSI (e.g., exceeding 10,000 patches), each patch was transformed into a compact feature representation using the pretrained feature extractor neural network. In this study, the feature extractor was implemented using the frozen image encoder of pathology VLMs, including CONCH18, PathGen-CLIP19, and QuiltNet17.

Stain normalization. For stain normalization, we employed the Reinhard method60 and the Macenko method39 to standardize the color distribution of the histopathology images. For implementation, we utilized the torch-staintools package for efficient normalization (https://github.com/CielAl/torch-staintools).

Pathology vision-language foundation models
With the advancement of large-scale pre-trained VLMs in computer vision and natural language processing, several pathology-specific VLMs have been developed to integrate histopathology images and textual concepts. Compared to vision-only foundation models, pathology VLMs better capture the intricate relationships between histopathology images and textual data. This alignment enhances the generalizability and robustness of image features22. The proposed FLEX is designed to seamlessly integrate with various pathology VLMs to improve the performance of WSI analysis tasks. In this study, we employed three state-of-the-art pathology VLMs:

CONCH18. CONCH is a visual-language foundation model pre-trained on over 1.17 million image-caption pairs, leveraging diverse histopathology images and biomedical text. It supports versatile applications in histopathology with minimal or no supervised fine-tuning, addressing label scarcity and enhancing adaptability across diseases and tasks. The CONCH model is publicly available on GitHub: https://GitHub.com/mahmoodlab/CONCH.

PathGen-CLIP19. PathGen-CLIP is a pathology-specific VLM trained using PathGen-1.6M19, a dataset containing 1.6 million high-quality image-caption pairs generated from large-scale WSI datasets such as TCGA. The PathGen-CLIP model is publicly available on GitHub: https://GitHub.com/PathGen-1-6M/PathGen-1-6M.

QuiltNet17. QuiltNet is a visual-language model for histopathology trained on Quilt-1M17, a dataset of 1 million image-text pairs curated from sources such as YouTube, Twitter, and research papers. QuiltNet is publicly available on GitHub: https://GitHub.com/wisdomikezogwo/quilt1m.

Multiple instance learning methods
MIL61 is the predominant weakly supervised learning paradigm in computational pathology. In MIL, each WSI is treated as a bag of patch instances, and the model predicts the bag-level label based on aggregated instance features. To demonstrate the adaptability and generalizability of FLEX, we integrated it with five state-of-the-art MIL methods in our experimental comparisons. The selected MIL methods are as follows:

ABMIL27. ABMIL models the Bernoulli distribution of a bag and parameterizes the aggregation operator using an attention mechanism. We used the original implementation available on GitHub: https://GitHub.com/AMLab-Amsterdam/AttentionDeepMIL.

ACMIL28. ACMIL mitigates overfitting by introducing multi-branch attention and stochastic Top-k instance masking, addressing attention value concentration, and enhancing the capture of diverse discriminative instances. The original implementation was used from GitHub: https://GitHub.com/dazhangyu123/ACMIL.

CLAM-SB5. CLAM-SB identifies diagnostically relevant sub-regions and refines the feature space through instance-level clustering of these regions. The implementation from the original GitHub repository was followed: https://GitHub.com/mahmoodlab/CLAM.

DTFD-MIL29. DTFD-MIL addresses WSI classification challenges with small sample sizes by using pseudo-bags to expand training data and a double-tier MIL framework for better feature utilization. The original implementation was used from GitHub: https://GitHub.com/hrzhang1123/DTFD-MIL.

ILRA-MIL30. ILRA-MIL employs a low-rank constraint to group similar patches and separates different ones, utilizing low-rank latent vectors for efficient global interaction modeling. The original GitHub implementation was used: https://GitHub.com/jinxixiang/low_rank_wsi.

Site-preserved Monte Carlo cross-validation
Histopathology images within the TCGA cohort originate from diverse source sites (e.g., hospitals, research institutions), often exhibiting site-specific signatures. These signatures can induce models to learn spurious correlations, hindering their ability to generalize to new, unseen sites. To rigorously evaluate the generalizability and effectiveness of FLEX and specifically assess performance on data from novel source sites, we implemented an SP-MCCV strategy. This approach is specifically designed to ensure models are tested on completely independent source sites not encountered during training. As illustrated in Fig. 6d, this nested SP-MCCV strategy is conceptually inspired by the Preserved-Site Cross-Validation method37 and includes outer and inner cross-validation loops:

Outer loop: site-preserved partitioning. Our strategy treats each Tissue Source Site (TSS) as an independent entity, representing a unique source of protocols and potential biases. We do not group sites based on known factors (e.g., scanner type), as this would fail to capture the full complexity of site-specific artifacts and would represent a less stringent test of generalization. Instead, we partition the entire set of TSS codes (see https://gdc.cancer.gov/resources-tcga-users/tcga-code-tables/tissue-source-site-codes for site-TSS mapping) into Ks mutually exclusive site-folds using convex optimization to ensure that each site-fold contains a balanced proportion of patients for the diagnostic labels of interest37. In each outer loop of the cross-validation, one of these site-folds is designated as the IND dataset, while the remaining Ks − 1 site-folds are aggregated to form the OOD test set, simulating a deployment to unseen medical centers.

Inner fold: Monte Carlo cross-validation within IND data. Within each site-preserved outer loop, the IND dataset (comprised of seen sites) was further subjected to Monte Carlo Cross-Validation (MCCV). Specifically, the IND data was randomly divided into training and IND testing subsets at a 7:3 ratio. The OOD dataset from the held-out site-fold was kept entirely separate and used exclusively for the final OOD performance evaluation.

This nested SP-MCCV procedure guarantees that model training and validation occur on data originating from distinct, site-specific distributions. For the BRCA and NSCLC datasets, we employed Ks = 5 site-preserved splits, with each IND fold undergoing 3 MCCV iterations, resulting in a total of 15 evaluation folds. For datasets with fewer source sites and WSIs, we reduced Ks to 3 to maintain sufficient training data per fold, while increasing MCCV iterations to 5 per IND fold, also yielding a total of 15 folds. This adjustment ensures robust statistical evaluation across datasets of varying sizes and site diversity. In summary, this SP-MCCV strategy provides a rigorous and comprehensive evaluation of model performance, specifically across unseen source sites. By strictly separating training and testing data at the source site level, SP-MCCV allows for a robust and reliable assessment of FLEX’s generalizability and effectiveness in mitigating site-specific biases.

FLEX architecture
As illustrated in Fig. 1d, the proposed FLEX is a flexible and adaptable module that can be seamlessly integrated with existing workflows for WSI analysis tasks. For clarity, we denote the input histopathology dataset as D = {(P, y)}, where P = {p1, p2, …, pN} is a set of N patches cropped from the WSI and y is the corresponding label. The traditional WSI analysis pipeline comprises two stages: feature extraction and MIL-based classification. In the feature extraction stage, the image encoder Ei of pathology VLMs is employed to extract patch-level features from the patches: X = Ei(P), where X = {x1, x2, …, xN} is the extracted original patch-level features with dimension of D. In the MIL-based classification stage, MIL models aggregate the patch-level features into a comprehensive representation, upon which a classifier is applied to predict the bag-level label y^: y^ = CLS(MIL(X)), where CLS and MIL are the classifier and the MIL model, respectively. The objective of FLEX is to enhance the original patch-level features X by leveraging the knowledge from visual concepts V and textual concepts T. This process filters out site-specific signatures and task-agnostic information, resulting in enhanced features that are subsequently fed into the MIL model for generalizable and robust WSI analysis: Z = FLEX(X, V, T), y^ = CLS(MIL(Z)), where Z = {z1, z2, …, zN} represents the enhanced patch-level features, each with a dimension of D. Specifically, the visual concepts V consist of representative patch images for the target class, which are used to retrieve important patches from the original WSI that may contain task-related information. The textual concepts T, comprising representative text prompts with learnable contexts, guide the feature enhancement process by filtering out site-specific signatures and task-agnostic information while preserving task-related information. In the following sections, we detail the design of FLEX and its integration with pathology VLMs and MIL models.

Visual concept generation. The process of visual concept generation is illustrated in Fig. 6a. First, an ABMIL model was trained on the training data, and patches with high attention scores were randomly selected from the training set. However, due to potential overfitting, these high-attention patches may not be representative of the target class. To mitigate this issue, a multimodal large language model (e.g., GPT-4o) was employed to filter out task-irrelevant and noisy patches. This was achieved by prompting GPT-4o with queries such as whether the input patch could reliably indicate its belonging to the target class. This step served as a preliminary screening to identify representative patches. To further ensure task relevance, a human expert manually verified the selected patch candidates. Finally, the visual concept for each target class was generated by averaging the features of the selected patches, which were extracted using the image encoder of the pathology VLMs.

Textual concept generation. The process of textual concept generation is illustrated in Fig. 6b. It involves creating task-specific and task-agnostic prompts that serve as positive and negative supervision signals. Recognizing the need for accuracy in medical applications, our methodology incorporates an expert validation protocol. First, the prompt generation is constrained to simple, descriptive labels grounded in established oncologic terminology (e.g., “invasive ductal carcinoma”, “connective tissue”), which minimizes the risk of factual inaccuracies. Second, and most critically, we implemented a human-in-the-loop expert validation process. All textual concepts, whether initially derived from established class names or drafted with assistance from GPT-4o, were rigorously reviewed, edited for clinical precision, and formally approved by a board-certified pathologist. This expert oversight ensures that every prompt used is medically accurate and relevant to the prediction task. The specific task-specific and task-agnostic prompts used for each cancer type are detailed in Supplementary Tables 24–29. With these controls, prompts were defined as follows: for tasks with clear class boundaries, such as BRCA subtyping, task-specific prompts are the class names themselves, following previous studies18,62. However, for more complex tasks with vague class boundaries, high complexity, or limited data, class names alone may be insufficient. To address this, we propose using the same class names as task-specific prompts attached to learnable contexts. These contexts adaptively capture task-specific information from the images during training, enhancing the prompts’ effectiveness. Task-agnostic prompts consist of general descriptions irrelevant to the task, such as the names of other tissue types (e.g., “connective tissue” or “normal tissue”). These prompts guide the feature enhancement process to filter out task-agnostic information. These are also attached to learnable contexts to improve their effectiveness in filtering out irrelevant features. Finally, textual concepts are generated by averaging the embeddings of these prompts (with their learnable contexts) produced by the VLM’s text encoder.

Knowledge-guided feature enhancement. The feature enhancement process, illustrated in Fig. 6c, is guided by visual and textual knowledge and comprises three main steps: visual concept-guided pilot patch selection, textual concept-guided feature calibration, and feature enhancement using a variational information bottleneck. To enhance the original patch-level features, we leverage the information bottleneck principle63,64, which filters out site-specific signatures and task-agnostic information while preserving task-related information. The information bottleneck principle seeks a compressed representation of the input data that minimizes the mutual information between the input and the compressed representation and maximizes the mutual information between the compressed representation and the target output.

First, cropped patches from WSIs are processed through the frozen image encoders of the pathology VLMs to extract the original patch-level features X. For each original feature x and its corresponding enhanced feature z, we define the mutual information between them as:
I(x, z) = ∫ p(x, z) log p(z|x) / p(z) dx dz, (1)
= ∫ p(x, z) log p(z|x) dx dz − ∫ p(z) log p(z) dz, (2)
where p(x, z) represents the joint distribution of the original and enhanced features, p(x) and p(z) are the marginal distributions, and p(z|x) is the conditional distribution of the enhanced features given the original features. However, the computing of p(z) = ∫ p(z|x) p(x) dx is intractable. To address this issue, we let the Gaussian distribution of the enhanced features z, r(z), be a variational approximation of the true marginal distribution p(z). Since the KL divergence between the variational posterior r(z) and the true posterior p(z) is non-negative, we have: ∫ p(z) log p(z) dz ≥ ∫ p(z) log r(z) dz. Therefore, we derive an upper bound for the mutual information: I(x, z) ≤ ∫ p(x) p(z|x) log p(z|x) / r(z) dx dz.

We further utilize a variational distribution qθ(z|x) to approximate the true posterior p(z|x), which was implemented by a neural network with the original features x as input and the mean and variance of the Gaussian distribution of the enhanced features z as output. After that, we can sample the enhanced features z from the variational distribution: z ~ N(NNμ(x), NNΣ(x)), where NNμ and NNΣ are the neural networks for the mean and variance of the Gaussian distribution of the enhanced features, respectively. To enable efficient optimization, we apply the reparameterization trick65, which provides an unbiased gradient estimate to optimize the objective66. Using this approach, the upper bound of the mutual information I(x, z) can be computed as: ∫ p(x) p(z|x) log qθ(z|x) / r(z) dx dz ≈ Ex [qθ(z|x) log qθ(z|x) / r(z)], which can be optimized by minimizing the Kullback-Leibler (KL) divergence between the variational posterior qθ(z|x) and the Gaussian distribution r(z): LIB = Ex [DKL(qθ(z|x) || r(z))]. Traditionally, the maximization of mutual information between the enhanced features and the target output is achieved by minimizing the cross-entropy loss between the bag-level predictions and the ground-truth labels. However, weakly supervised MIL models often fail to provide accurate supervision signals for optimizing the variational information bottleneck, which operates at the patch level. To address this challenge, we propose leveraging visual and textual concepts to guide the feature enhancement process, offering more accurate patch-level supervision signals.

Specifically, as only a subset of patches contains task-related information, we select pilot patches zpilot to guide the feature enhancement process. These pilot patches are identified by calculating the cosine similarity between the original patch features and the visual concept of the target class. Patches with high similarity scores are designated as pilot patches, ensuring they contain relevant task-related information to effectively drive feature refinement.

Following the selection of pilot patches, textual concepts are employed to guide the feature enhancement process. Given that the text encoder of pathology VLMs is inherently general due to pre-training on large-scale, diverse pathology datasets with domain-agnostic learning objectives, we posit that textual concepts are robust and generalizable. These concepts can effectively filter out site-specific signatures, while task-specific textual concepts provide relevant information for the target class. This enables the enhancement process to preserve task-related information and eliminate task-agnostic noise. Consequently, the task-specific textual concept for the target class is treated as the positive supervision signal cp_pos, whereas other task-specific and task-agnostic textual concepts are treated as negative supervision signals cp_neg. To implement this guidance, we utilize the InfoNCE loss67, which minimizes the distance between the enhanced features and the positive supervision signal while maximizing the distance between the enhanced features and the negative supervision signals:
Lp_NCEz = − 1/M log [exp(zp_pilot · cp_pos / τ) / (exp(zp_pilot · cp_pos / τ) + ΣK i=1 exp(zp_pilot · cp_negi / τ))], (3)
where M represents the number of pilot patches, K denotes the number of negative supervision signals, and τ is the temperature parameter. To further enhance the distinction between task-specific textual concepts and task-agnostic concepts, and to ensure task-specific textual concepts from different classes are more distinguishable, we propose an additional optimization strategy. In this approach, the corresponding task-specific textual concept of the target class serves as the optimization target, the pilot patches are treated as positive samples, and the other task-specific textual concepts and task-agnostic textual concepts are treated as negative samples:
Lp_NCEc = − log [exp(cp_pos · zpilot / τ) / (exp(cp_pos · zpilot / τ) + ΣK i=1 exp(cp_pos · cp_negi / τ))]. (4)

The enhanced patch-level features are obtained by inputting the original patch-level features into the information bottleneck. These features are then aggregated by the MIL model to produce the bag-level features, denoted as zbag = MIL(Z). Subsequently, we propose further enhancing the aggregated bag-level features through a contrastive learning process, leveraging the positive supervision signals cs_pos and negative supervision signals cs_neg, similar to the feature enhancement process applied to the patch-level features. It is important to note that the textual concepts used for patch-level (cp_pos and cp_neg) and slide-level (cs_pos and cs_neg) feature enhancements share the same initialization but are optimized independently. The resulting InfoNCE losses for the slide-level feature enhancement are defined as:
Ls_NCEz = − log [exp(zbag · cs_pos / τ) / (exp(zbag · cs_pos / τ) + ΣK i=1 exp(zbag · cs_negi / τ))], (5)
Ls_NCEc = − log [exp(cs_pos · zbag / τ) / (exp(cs_pos · zbag / τ) + ΣK i=1 exp(cs_pos · cs_negi / τ))]. (6)

The final prediction of the bag-level label is obtained by feeding the enhanced bag-level features into the classifier y^ = CLS(zbag). The prediction loss is then optimized with the ground-truth label by minimizing the cross-entropy loss LCE.

The total loss for FLEX is a combination of the cross-entropy loss for the MIL model, the variational information bottleneck loss, and the InfoNCE losses:
LFLEX = LCE + λ1 LIB + λ2 (Lp_NCEz + Lp_NCEc) + λ3 (Ls_NCEz + Ls_NCEc), (7)
where λ1, λ2, and λ3 are the hyperparameters to balance the different components of the loss function.

Training details
All experiments were conducted on a server with 8 NVIDIA L40S GPUs. The length of the learnable context was set to 4 tokens for morphology classification and molecular biomarker status prediction tasks, and 2 tokens for gene mutation prediction tasks. An ablation study was performed to evaluate the impact of the learnable context length by tuning it within 0, 2, 4, 6, 8, 10, 12 for each task. The average performance for each task type is reported in the supplementary material. During training, all weights of the pathology VLMs were frozen, and the weights of the MIL model and the FLEX module were optimized. For ABMIL, ACMIL, and CLAM-SB, the learning rate was set to 2e-4, and for DTFD-MIL and ILRA-MIL, the learning rate was set to 1e-4. The batch size was set to 1, and all tasks were trained for 20 epochs. For evaluation, the AUROC and the F1 score were adopted as metrics. Results for all models are reported as the mean and standard deviation over 15 folds of SP-MCCV for both OOD and IND testing data.

Evaluation metrics
To comprehensively assess the efficacy of FLEX, we evaluated its performance across two key dimensions: standard performance metrics and demographic fairness metrics.

Performance metrics. Given the inherent class imbalance often encountered in WSI analysis tasks, we employed the AUROC and the F1 score for evaluating primary model performance. These metrics are well-suited for imbalanced classification scenarios. The Receiver Operating Characteristic (ROC) curve graphically depicts the trade-off between the True Positive Rate (TPR) and the False Positive Rate (FPR) across varying classification thresholds. AUROC, representing the area under this curve, provides a threshold-agnostic measure of a model’s ability to discriminate between positive and negative instances. Its robustness to class imbalance makes AUROC particularly appropriate for our evaluation. Complementing AUROC, the F1 score offers a balanced measure of precision and recall, providing a more granular perspective on classification accuracy. Defined as the harmonic mean of precision and recall, the F1 score is calculated as follows:
F1 = 2 × TP / (2 × TP + FP + FN), (8)
where True Positives (TP), False Positives (FP), and False Negatives (FN) are defined, respectively. The F1 score is particularly useful for evaluating model performance in scenarios where the cost of false positives and false negatives is not equivalent. The F1 score does not proportionately favor either precision or recall, rewarding those with balanced performance.

Fairness metrics. To evaluate FLEX’s impact on demographic fairness, we used three metrics: AUROC gap ratio43, TPR disparity35, and race-wise AUROC35. These metrics collectively quantify and characterize potential performance disparities across different demographic groups, offering a multi-faceted assessment of fairness.

The AUROC gap ratio, as defined in 43, quantifies the relative AUROC performance disparity between the best-performing and worst-performing demographic groups, normalized by the overall AUROC. A smaller AUROC gap ratio indicates improved fairness. TPR disparity, as introduced in 35, measures the difference between the overall TPR and the TPR for each demographic group. TPR disparity values closer to zero suggest greater fairness, indicating that the model’s true positive detection rate is more consistent across demographic groups and less biased towards any particular group. Furthermore, race-wise AUROC provides a granular view of performance by calculating the AUROC separately for each demographic group. Consistent AUROC values across different demographic groups are indicative of a fairer model, demonstrating that the model’s discriminatory power is not significantly influenced by demographic factors.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_2": "This manuscript discusses the impacts of foundation models on computational pathology approaches, highlighting biases that are appearing in many of these models. Of note, there are out-of-domain data issues, where models overfit to a subset of the data on which they are trained, and do not generalize well to new tasks, which suggests that these foundation models are not learning the true underlying biology in the pathology slides. The authors perform a comprehensive analysis of some of these biases and proposed a method, FLEX, to correct for them. FLEX is applied to 16 predictive tasks in 4 different TCGA cancer data sets. FLEX shows statistically significant improvement in reducing non-clinically-relevant biases in pathology slide data, notably clinical site biases that are not relevant to the scientific questions the model is being trained to answer but which often confound AI and ML models. This improved method is compared to current standard approaches.\nThe literature is well cited and the manuscript clearly written and easy to follow. Similarly, the figures are beautiful. The work supports the conclusions however additional evidence, outlined in my comments below, would make the claims stronger. As the manuscript is written now, it is unclear if these results will translate to other datasets or if they are specific to TCGA data. There are correlations between clinical sites and other demographic factors that may impact the results, creating relationships between the different sites discussed in the manuscript. The authors do not take these into account when stratifying the data to create IND and OOD subsets of the data. Overall, it is a well written manuscript addressing a significant concern in the field.\nComments:\nThe sites are not defined until the supplemental text. Please make this clearer earlier in the text, that sites reference the literal clinical sites that processing the samples.\nBased on the results, it looks more like stain normalization works for some but not all sites - what site is out of the norm? Why? Please discuss.\nOn page 6, please reference the supplemental text and tables describing the 16 tasks.\nThe racial differences in performance are interesting. TCGA also has information about genetic ancestry (see https://doi.org/10.1016/j.ccell.2020.04.012). It would be interesting to see performance differences across genetic ancestry groups, as well as racial stratification. Doing this avoids misclassifying patients with ancestry from multiple human populations (common in African Americans in the TCGA data). This would be a better way to stratify the samples. There are also site-specific race and other demographic factor correlations. How is this handled in the analysis? Are training dataset sizes held constant?\nSeveral of the biomarker prediction tasks (e.g. BRCA HER2, ER, and PR) have known racial biases. Others are not clearly known. Please discuss the variations in performance given known biases and performance improvements relative to the tasks - is there a larger performance gain in tasks where there are known racial biases?\nIt's unclear how much of the model performance is dependent on dataset biases beyond demographic data. Ideally, the trained model would be applied to non-TCGA and racial bias and other biases analyzed in this way. Other datasets exist with pathology slide images, and models trained in this manuscript could be applied to those.\nFigure 2:\n- Why is figure 2 referenced before figure 1?\n- Figure 2 plots should have axis labels.\n- In Figure 2, UMAP is a visualization technique that is affected by parameter tuning. How much of that is causing the biases here?\n- How are the site stratifications actually done, as visualized in Fig 2b? This visualization doesn't explain it well. How were sites grouped? What about site-site relationships? For example, is there a relationship between the clinical equipment, the location, or something else?\nFigure 3:\n- Please move figure legends under the matching figures.\n- It's unclear exactly what Figure 3a is plotting. Please provide more clarity in the caption description.\n- Please add a horizontal line for the avg performance in the data overall, in Figure 3b.\n- Plots used in Figure 3c are not appropriate for the data being modeled. Please update these to not use dynamite plots.\n- For all of figure 3, please define the error bars and other data annotations for all subfigures.\nFigure 4:\n- Please do not use dynamite plots for Figure 4a. Boxplots, violin plots, or another visualization are a more representative way of visualizing these results.\n- Please update the y-axis ranges to be consistent across each row in Figure 4a so it is easier to compare model performance across tasks.\n- It's difficult to see the EBV +/- shapes in the Figure 4b plots. Please adjust to make these easier to distinguish.\nFigure 5:\n- Figure 5a missing a legend and the statistical tests is not defined. Are these adjusted for multiple hypotheses? Also, are the data repeatedly stratified then models tested? How much overlap is there in data between each test, and how is the significance of performance differences calculated? What size is the training data for each of these splits?\n- Figure 5b would be more impactful if visualized differently, to show the change in performance across original, improvement, and FLEX. Perhaps using a bipartite graph.\n- Additionally, there seems to be a large amount of variability in performance in Figure 5b, suggesting that there are other factors in the data splits driving performance differences. Please discuss this in the text.", "reviewer_3": "The authors developed a machine learning framework aimed at improving the generalizability of computational pathology diagnostic models. It focuses on the issue of site-specific tissue patterns and demographic profiles that hinder model performance across locations for data collection. Their approach uses visual and textual domain knowledge to guide an information bottleneck that filters out certain features and focuses on task-relevant ones. The framework is evaluated on binary classification tasks using data from The Cancer Genome Atlas (TCGA). Computational results show modest gains in the area under receiver operating characteristic curves (AUROC), particularly for performance in different participatory sites of the TCGA and reductions in performance gaps across demographic groups. Below are my comments.\nGeneral Comments:\nThe proposed method presents an interesting model design. There are a few potential issues regarding its scalability, interpretability, and practical utility. Notable limitations include reliance on human experts for patch selection, dependence on GPT-4o for concept generation without verification, and modest improvements in out-of-domain performance. In addition, the absence of statistical testing across tasks may undermine the method’s robustness and its real-world applicability.\nSpecific Comments:\n1. In the visual concept generation module, the selection of representative patches requires both GPT-4o filtering and manual validation by human experts. This dependency on expert input may limit the scalability and automation potential of the proposed method, particularly in clinical settings where expert time is costly and throughput demands are high.\n2. The use of GPT-4o for generating textual concepts is not accompanied by a validation or correction mechanism. Given potential issues with hallucination and inconsistency in large language models, relying solely on GPT-4o outputs for guiding feature enhancement may introduce potential inaccuracies, especially in high-stakes medical applications.\n3. Although the method is described as a cross-domain generalization method, the actual performance improvements on out-of-domain (OOD) datasets are modest. In some cases, the proposed method (Feature-Level Enhancement for Cross-domain generalization; FLEX) reduces the performance gap from only 4% to 3%, which may have limited clinical relevance or operational impact.\n4. In Figure 2a, the UMAP visualizations show overlapping clusters even after applying the proposed method, indicating that site-specific signatures may not be fully eliminated. A quantitative measure of intra- and inter-class distances or silhouette scores would strengthen the claims of feature disentanglement and clustering quality.\n5. Interestingly, standard stain normalization via the Reinhard method achieves better results than their methods in a few tasks. The authors could further discuss reasons leading to this discrepancy and address questions about the added value of their method.\n6. In Figure 2d and related visualizations, the authors mentioned their methods reduced attention paid to noisy regions. However, what constitutes noise was not clearly defined or objectively measured.\n7. In Figure 3, the fairness gap (measured in AUROC differences across racial groups) ranges widely, sometimes as high as 0.7. Surprisingly, the proposed methods exhibit the widest performance spread in certain tasks. The authors could discuss the implications of this observation and the potential impact on the reliability of the proposed methods.\n8. While the proposed method is compared against baseline and the standard Reinhard stain normalization method in Figure 3, no statistical tests (e.g., paired t-tests or Wilcoxon signed-rank tests) are reported to determine whether the observed performance differences are significant. The observed differences in results between methods appear minimal at times.\n9. In Figure 4b, UMAP plots for the features obtained from the proposed methods still show substantial overlap between class labels across sites. The visual evidence does not compellingly support the claim of improved separability of diagnostic categories.\n10. In Figure 5, performance improvements across different data scales and MIL model variants are relatively small, sometimes between 0.000 and 0.005 in AUROC. The authors could discuss the practical benefits of integrating the proposed methods into real-world pathology diagnosis."}, "author_response": "Responses to Reviewers’ Comments for Manuscript NCOMMS-25-16314\nKnowledge-Guided Adaptation of Pathology\nFoundation Models Effectively Improves Cross-domain\nGeneralization and Demographic Fairness\nAddressed Comments for Publication to\nNature Communications\nDear Reviewers,\nWe are pleased to resubmit a revised version of our manuscript entitled “Knowledge-Guided Adaptation of\nPathology Foundation Models Effectively Improves Cross-domain Generalization and Demographic Fairness”\n(ManuscriptID:NCOMMS-25-16314)inresponsetothedecisionformajorrevision. Wesincerelyappreciate\nyoureffortsandthetimededicatedtoreviewingourpaper. Wehavecomprehensivelyaddressedallcomments\nand believe the manuscript has been substantially strengthened as a result.\nTo facilitate your review, we have adopted the following structure:\n• The original comments from reviewers are presented in blue boxes.\n• Our point-by-point responses follow each comment directly.\n• Key sentences revised in the manuscript are highlighted within our responses in gray boxes.\n• All changes within the revised manuscript file itself have been marked with red text.\nThe main revisions to the manuscript are summarized as follows:\n• External validation on two independent cohorts: To address concerns about generalizability of\nourfindingsbeyondtheTCGAdataset(Reviewer#2,Reviewer#3),wehaveperformedanewzero-shot\nevaluation on two large-scale, independent external cohorts: the public Clinical Proteomic Tumor\nAnalysis Consortium collections (CPTAC; n=1,738) and our in-house dataset from Nanfang Hospital\nof Southern Medical University (NFH; n=5,318), totaling 7,056 new slides for external validation. The\nresults (Fig. R6) demonstrate that FLEX achieves consistent and statistically significant performance\nimprovements in real-world, out-of-distribution scenarios.\n• Expanded and more comprehensive fairness analysis: To address concerns about demographic\nandgeneticbiases(Reviewer#2),wehavesubstantiallyexpandedourfairnessanalysis. Wenowstratify\nperformancebybothself-reportedraceandgeneticancestryforamorebiologically-groundedevaluation\nof fairness (Fig. R3). Furthermore, we conducted a fine-grained intra-site analysis to disentangle site\nand race confounding, demonstrating that FLEX consistently mitigates bias in both settings (Fig. R4).\n• Strengthened quantitative/statistical evaluation: In response to feedback on the need for\nmore objective evidence (Reviewer #3), we supplemented UMAP visualizations with objective met-\nrics (LISI and Silhouette Score) and integrated comprehensive statistical testing (paired t-tests and\nWilcoxonsigned-ranktestswithBenjamini-Hochbergcorrection)acrossrelevantcomparisonstoconfirm\nsignificance.\n• Clarified scalability and reliability of the workflow: To address concerns about the reliance\non human experts and GPT-4o (Reviewer #3), we clarified in Methods that expert-guided concept\ngeneration is a one-time offline procedure performed during training and is not required at inference,\nand that the LLM (GPT-4o) is used within a constrained, human-in-the-loop validation pipeline with\nexpert pathologist review.\n• Figure and manuscript presentation improvements: Based on detailed feedback, all major\nfigures have been revised for clarity (e.g., replaced less informative plot types with boxplots and\ndumbbell plots where appropriate; improved legends and captions).\nWe believe that these extensive revisions have significantly improved the clarity, robustness, and impact of\nour work. We sincerely hope that our responses have fully addressed the your concerns and that the revised\nmanuscript is now suitable for publication in Nature Communications.\nThank you again for your time and consideration.\nSincerely,\nAuthors\nGeneral Comments. Thismanuscriptdiscussestheimpactsoffoundationmodelsoncomputational\npathology approaches, highlighting biases that are appearing in many of these models. Of note,\nthere are out-of-domain data issues, where models overfit to a subset of the data on which they are\ntrained, and do not generalize well to new tasks, which suggests that these foundation models are not\nlearning the true underlying biology in the pathology slides. The authors perform a comprehensive\nanalysis of some of these biases and proposed a method, FLEX, to correct for them. FLEX is applied\nto 16 predictive tasks in 4 different TCGA cancer data sets. FLEX shows statistically significant\nimprovement in reducing non-clinically-relevant biases in pathology slide data, notably clinical site\nbiases that are not relevant to the scientific questions the model is being trained to answer but\nwhich often confound AI and ML models. This improved method is compared to current standard\napproaches. The literature is well cited and the manuscript clearly written and easy to follow.\nSimilarly, the figures are beautiful. The work supports the conclusions however additional evidence,\noutlined in my comments below, would make the claims stronger. As the manuscript is written now,\nit is unclear if these results will translate to other datasets or if they are specific to TCGA data.\nThere are correlations between clinical sites and other demographic factors that may impact the\nresults, creating relationships between the different sites discussed in the manuscript. The authors do\nnot take these into account when stratifying the data to create IND and OOD subsets of the data.\nOverall, it is a well written manuscript addressing a significant concern in the field.\nResponse:\nThank you for your constructive feedback and for acknowledging the clarity of our manuscript and the\nimportance of our work. We have performed new experiments and analyses to address your primary concerns\nregarding generalizability and the disentanglement of confounding factors.\n1. Generalizability Beyond TCGA: To address your concern about the TCGA-specificity of our findings,\nwe conducted zero-shot external validation. We trained our model exclusively on TCGA data and then\nevaluated its performance on two large-scale, entirely independent external cohorts: the public Clinical\nProteomic Tumor Analysis Consortium (CPTAC; n=1,738 slides) and our in-house dataset from Nanfang\nHospital of Southern Medical University (NFH; n=5,318 slides). As detailed in our response to Comment 6,\nFLEX demonstrates significant and consistent performance improvements on both external cohorts (Fig. R6).\nThis provides strong evidence that the model learns fundamental, transferable biological features rather than\noverfitting to dataset-specific artifacts.\n2. Disentangling Site and Demographic Correlations: To address your point about correlations\nbetweenclinicalsitesanddemographicfactors,wehaveimplementedafine-grainedintra-sitefairnessanalysis.\nAs detailed in our response to Comment 4, this analysis evaluates model performance and fairness within\nindividual clinical sites, thereby isolating site-specific effects. The results (Fig. R4) demonstrate that FLEX\neffectively mitigates demographic biases when disentangling site and demographic factors.\nWe believe these comprehensive revisions have substantially strengthened the manuscript. Detailed point-by-\npoint responses to each of your comments are provided below.\nComment 1\nThe sites are not defined until the supplemental text. Please make this clearer earlier in the text, that\nsites reference the literal clinical sites that processing the samples.\nResponse:\nThank you for this suggestion. We have now explicitly defined this term in the Introduction (Page 2, Line\n72) to ensure clarity from the outset.\n3\n———————————————– Revision in Page 2, Line 72 ———————————————–\n...These signatures, which stem from variations in tissue preparation, staining protocols, and scanning\nequipment across different clinical centers (e.g., hospitals or research institutions) where samples are\nprocessed (Fig. 1a), directly impede cross-domain generalization...\nComment 2\nBased on the results, it looks more like stain normalization works for some but not all sites - what site\nis out of the norm? Why? Please discuss.\nResponse:\nThank you for this insightful suggestion. We performed a site-level analysis to investigate the differential\neffectiveness of stain normalization. Our analysis reveals that the degree of performance improvement\ncorrelates with how effectively stain normalization aligns a site’s feature distribution with the global average.\nTo investigate this, we quantified the change in image feature deviation for each site relative to the overall\nfeatures of all sites, both before and after normalization. We then correlated this change with the per-site\nAUC improvement on two tasks. To measure this “feature deviation”, we employed several metrics, including\ncosine, Euclidean, L2 norm, and Manhattan distances between a site’s mean feature vector and the global\nmean feature vector. A negative change in these metrics signifies that a site’s features became more aligned\nwith the global average post-normalization.\nFor the BRCA-TYPE task (Fig. R1), we observed a statistically significant negative correlation between\nthe change in feature deviation and AUC improvement (e.g., for the ‘overall_mean_rel_dev’ metric:\nr =−0.560,p=0.037). This indicates that sites whose features moved closer to the global mean experienced\ngreater performance gains. This finding was replicated in the CRC-TP53 task (Fig. R2), where we again\nfound significant negative correlations (e.g., r =−0.525,p=0.030).\nIn conclusion, a site is “out of the norm” when stain normalization is less effective at correcting its specific\n its specific\nfeature-level variations and aligning them with the global data distribution. This analysis quantitatively\nexplains the inconsistent benefits of stain normalization and underscores the need for more robust methods.\nComment 3\nOn page 6, please reference the supplemental text and tables describing the 16 tasks.\nResponse:\nThank you for this suggestion. We have added a reference to the supplemental text and tables on page 6 to\ndirect the reader to the detailed descriptions of the 16 predictive tasks.\n——————————————— Revision in Page 6, Line 305 ———————————————\n...Detailed results across all 16 tasks (see ‘Dataset and clinical tasks description’ in Methods and\nSupplementary Table 2; Fig. 2d) show consistent performance improvements in both IND and OOD\nsettings...\nComment 4\nTheracialdifferencesinperformanceareinteresting. TCGAalsohasinformationaboutgeneticancestry\n(seehttps://doi.org/10.1016/j.ccell.2020.04.012). It would be interesting to see performance differences\nacross genetic ancestry groups, as well as racial stratification. Doing this avoids misclassifying patients\nwith ancestry from multiple human populations (common in African Americans in the TCGA data).\nThis would be a better way to stratify the samples. There are also site-specific race and other\ndemographic factor correlations. How is this handled in the analysis? Are training dataset sizes held\nconstant?\nsedilSforebmuN\nAUC Before Stain Normalization\nnoitazilamroN\nniatS\nretfA\nCUA\nAUC Improvement\nedoC\netiS\nPearson Correlation with AUC Improvement\ntnemevorpmI\nCUA\ntnemevorpmI\nCUA\na b\nc d\ne f\n𝑟=−0.560 𝑟=−0.467\n𝑝=0.037 𝑝=0.081\noverall_mean_rel_dev overall_mean_abs_dev\nFigure R1: The impact of stain normalization on model performance and its correlation with\nstain deviation metrics across different acquisition sites in the BRCA-TYPE task. a,\nDistribution of the number of slides for each site included in the analysis (sites with 20 slides). b, A scatter\nplot comparing the site-level AUC before and after stain normalization. Points above the dashed identity\nline indicate performance improvement. c, Bar chart showing the AUC improvement for each site after\nnormalization. Green bars represent an increase in AUC, while red bars represent a decrease. d, A heatmap\nof Pearson correlation coefficients between various stain deviation metrics and the AUC improvement.\nDarker blue indicates a stronger negative correlation. e, f, Scatter plots visualizing the relationship between\nAUC improvement and the two metrics with the strongest negative correlations: (e) overall_mean_rel_dev\n(r =−0.560,p=0.037) and (f) overall_mean_abs_dev (r =−0.467,p=0.081).\nsedilSforebmuN\nAUC Before Stain Normalization\nnoitazilamroN\nniatS\nretfA\nCUA\nAUC Improvement\nedoC\netiS\nPearson Correlation with AUC Improvement\ntnemevorpmI\nCUA\ntnemevorpmI\nCUA\na b\nc d\ne f\n𝑟=−0.525 𝑟=−0.517\n𝑝=0.030 𝑝=0.033\nL2_norm_rel_dev L2_norm_abs_dev\nFigure R2: The impact of stain normalization on model performance and its correlation with\nstain deviation metrics across different acquisition sites in the CRC-TP53 task. a, Distribution\nof the number of slides for each site included in the analysis (sites with 20 slides). b, A scatter plot\ncomparing the site-level AUC before and after stain normalization. Points above the dashed identity\nline indicate performance improvement. c, Bar chart showing the AUC improvement for each site after\nnormalization. Green bars represent an increase in AUC, while red bars represent a decrease. d, A heatmap\nof Pearson correlation coefficients between various stain deviation metrics and the AUC improvement.\nDarker blue indicates a stronger negative correlation. e, f, Scatter plots visualizing the relationship between\nAUC improvement and the two metrics with the strongest negative correlations: (e) L2_norm_rel_dev\n(r =−0.525,p=0.030) and (f) L2_norm_abs_dev (r =−0.517,p=0.033).\nThank you for thisinsightfulcommentand for thevaluablesuggestions. Weagree thatusing genetic ancestry\nfor fairness evaluation is a more robust approach and that disentangling site-demographic correlations is\ncrucial. In response, we have conducted two new analyses, with results presented in Fig. R3 and R4.\n1. Fairness Analysis Using Genetic Ancestry\nTheracialdifferencesinperformanceareinteresting. TCGAalsohasinformationaboutgeneticancestry\n(see https://doi.org/10.1016/j.ccell.2020.04.012). It would be interesting to see performance differences\nacross genetic ancestry groups, as well as racial stratification. Doing this avoids misclassifying patients\nwith ancestry from multiple human populations (common in African Americans in the TCGA data).\nThis would be a better way to stratify the samples.\nActingonyoursuggestion,wehaveexpandedourfairnessanalysistoincludestratificationbygeneticancestry,\nproviding a more biologically rigorous evaluation alongside self-reported race. As presented in Fig. R3,\nour analysis confirms that performance disparities exist across genetic ancestry groups with the baseline\nmodel. Crucially, FLEX enhances fairness across all ancestry groups. This is demonstrated by the reduced\nAUC gap ratios (Fig. R3a), more uniform AUC scores across groups (Fig. R3b), and TPR disparity values\nclustered closer to zero (Fig. R3c-d). This new analysis provides more biologically grounded evidence for\nFLEX’s effectiveness in promoting demographic fairness. We have also updated the manuscript to reflect\nthis expanded analysis.\n——————————————— Revision in Page 8, Line 395 ———————————————\n...Because self-reported race may not fully capture complex biological heritage, we also stratified\ncohortsbygeneticancestry,whichprovidesamorebiologicallygroundedmeasure. Ourfairnessanalysis\nwas conducted across both stratification methods in the TCGA dataset to ensure a comprehensive\nevaluation....\n2. Disentangling Site-Specific and Demographic Correlations\nThere are also site-specific race and other demographic factor correlations. How is this handled in the\nanalysis? Are training dataset sizes held constant?\nWe agree that the correlation between clinical sites and demographic factors is a significant confounding\nvariable. To address this, we conducted a multi-faceted analysis to demonstrate that FLEX mitigates\ndemographic bias independently of site-level effects.\nFirst, we performed a fine-grained intra-site fairness analysis to disentangle these variables (Fig. R4). This\nanalysis provides evidence from the following two perspectives:\n• Consistent Performance Across Sites: We first confirmed that FLEX improves performance across\nall major sites, irrespective of their demographic compositions (Fig. R4b).\n• Intra-Site Fairness: Crucially,whenevaluatedwithinindividualsites,FLEXstillsignificantlyreduces\nthe performance gap between White and Black patient groups (Fig. R4c-f).\nThis intra-site analysis demonstrates that FLEX’s fairness improvements are not merely a byproduct of\ncorrecting site-level batch effects but stem from a more fundamental mitigation of demographic bias.\nSecond, we confirmed these findings generalize beyond TCGA’s unique confounding structure. Our zero-shot\nvalidation on two independent external cohorts (CPTAC and NFH, Fig. R3 and Fig. R6) showed that a\nmodel trained only on TCGA maintains superior performance and fairness on these OOD datasets. This\nfurther validates that FLEX learns generalizable biological features rather than relying on dataset-specific\nartifacts or confounds.\nFinally, this empirical success is rooted in FLEX’s core design. FLEX uses an information bottleneck\nguided by textual pathology concepts (e.g., “invasive ductal carcinoma”) that are inherently free of site\nor demographic information. This architecture constrains the model to learn features aligned with these\nunbiased concepts and to discard spurious correlations, including entangled site and demographic signatures.\nWe have updated the manuscript to clarify this mechanism.\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nE P Y e c a R\noitar\npag C U A C U A C O R\nytirapsid\nR PT R R I I D a a LC c c C e e 0 0 . .\nO\n0 0 3 4 5 0 0 0 . . 0 0\nR\n3 39 3 0 0 . . 0 0\nF\n3 5 3 3 b\nreriaF\nc Race-wise AUROC\nT\nA C R\nB y rts e\nc n A\noitar\npag\nC U A\nC U A C\nO R\nytirapsid\nR PT\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 A\nA\nA\nn\nn\nv\nI c\nc\ne\nI\nD\nL\ne\ne\nr\nC\nC\na\ns\ns\ng\nt\nt\nr\nr\ne\ny\ny\n0\n0\n0\n.\n.\n.\n1\n2\n1\n5\n0\n0\n0\n7\n3 0\n0\n0\n.\n.\n.\n1\n2\n1\n3\n0\n03\n9\n2\n0\n0\n0\n.\n.\n.\n0\n0\n1\n8\n3\n7\n7\n6\na b c d\nFairness gap Race-wise AUROC TPR disparity TPR disparity RMSE 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nBetter Separation\nreriaF\nC\nE P Y T-\ne\nc a R\noitar\npag C U A C U A\nytirapsid\nR PT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001\nL L R R U U a a A S c c C D e e 0 0 . .\nO\n0 0 9 8 1 1 0 0 . . 0 1\nR\n7 0 0 5 0 0 . . 0 0\nF\n7 6 5 2\nL C S Ny rts e\nc\nn A\noitar pag\nC U A ∗\nC U\nA\nytirapsid\nR PT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 A A\nA\nn n L L\nv\nc c U U\ne\ne e\nr\nA S\na\ns s C D\ng\nt t r r\ne\ny y 0 0\n0\n.\n.\n.\n0 0\n0\n9 7\n8\n7 4\n6\n0 0\n0\n.\n.\n.\n0 1\n0\n7 0\n8\n9 2\n9 0\n0 0\n.\n.\n.\n0\n0 0 7 5\n6\n9 7\n8\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nROC AUC Method LUAD LUSC\np < 0.001\nR P\ne\nc a R\noitar\npag C U A ∗ C U A\nytirapsid\nR PT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001\nR R N P a a o e c c s g e e 0 0 . .\nO\n0 0 9 9 0 5 0 0 . . 1 0\nR\n0 8 5 2 0 0 . . 1 0\nF\n0 8 5 0\nA-\nC R By\nrts e c n A\noitar\npag C U A C U A\nytirapsid\nR PT\nA\nA A\nn\nn v\nN\nP\nc\nc e\ne\ne o r\ne\na\ns\ns s\ng\ng\nt\nt\nr\nr e\ny\ny\n0\n0 0\n.\n.\n.\n1\n1 1\n1\n3 0 5\n1\n8\n0\n0 0\n.\n.\n.\n1\n1 1\n2\n0 2 7\n3\n0 0\n0\n0 .\n.\n.\n1\n1\n0\n2\n9 0\n7\n7 2 TPR O R D r M ig is i S n p E a a l rity 0 R . I 0 D a 3 c C 5 e 3 0 R . I 0 a L 3 C c 9 e 9 A 0 n . I c 1 D e 5 C s 2 t 8 ry A 0 n . c I 2 L e 0 C s 0 t 0 ry\nReinhard 0.0327 0.0390 0.1388 0.2017\nROC AUC Method Neg Pos FLEX 0.0331 0.0532 0.0873 0.1318\nF A R\ne\nc a R\noitar\npag C\nU A\nC U A\nytirapsid\nR PT\nR\nR M\nW a\na U\nc\nc\nT\nT\ne\ne\n0\n0.\n.\nO\n1\n28\n1\n7\n9 0\n0.\n.0\n0\n8\n0\n9 0\n0.\n.1\n2\n20\nC R C - y\nrts e c n A\noitar\npag C U A C U A\nytirapsid\nR PT\nA\nA A\nn\nn v M\nc\nc e\nW e\ne U ra\nT s\ns T g\nt\nt\nr\nr e\ny\ny\n0\n0 0\n.\n.\n.\n1\n2 2\n2\n7 0 7 2\n5 0\n0 0\n.\n.\n.\n0\n1 1\n9\n9 4\n8\n7 6\n0\n0 0\n.\n.\n.\n1\n2 1\n1\n4 7\n8\n6 7\nROC AUC Method WT MUT\nA\nE\nC\nP\nR\nY\nC B T - -e c a R\noitar\npag C U A ∗ C U A\nytirapsid\nR PT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001\nR R I I D a a LC c c C e e 0 0 . .\nO\n0 3 5 0 8 2 0 0. .0 3\nR\n6 0 2 2 0 0. . 2 0\nF\n5 9 7 7\nAverage 0.180 0.182 0.177\nROC AUC Method IDC ILC\nD\nR F\nA\nG\nU C E L - -e c a R\noitar\npag C U A ∗∗ C U A\nytirapsid\nR PT\np < 0.001 p < 0.001 p < 0.001 p < 0.001\nR R M W a a U c c T T e e 0 0 . .\nO\n1 1 5 2 0 9 0 0 . . 1 1\nR\n7 1 1 6 0 0 . . 0 1\nF\n6 0 8 1\nAverage 0.140 0.144 0.085\nROC AUC Method WT MUT\nO R F\nOriginal Reinhard FLEX White / EUR Black / AFR Asian / EAS White / EUR mean Black / AFR mean Asian / EAS mean Original Reinhard FLEX\nFigure R3: (Figure 3 in manuscript) Evaluation of FLEX’s effectiveness in improving\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nA C 3 K IP A C R B y rts e e c c n a R A o o i i t t a a r r p p a a g g C C U U A A C C U U A A C C O O R R y y t t i i r r a a p p s s i i d d R R P P T T 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 A A A n n v R R M M c c e W W a a e e U U r c c a T T s s T T e e g t t r r e y y 0 0 0 0 0 . . . . . O 1 1 1 1 1 0 3 1 7 3 5 9 4 4 9 0 0 0 0 0 . . . . . 0 1 1 1 1 R 9 2 2 8 3 6 1 8 6 3 0 0 0 0 0 . . . . . 1 1 1 1 1 F 1 3 1 6 3 4 5 1 7 2 p d c b S i S n r m t e o e d o a r o m s a i f t t c o s i l s s a o - l r t e v t t m g r r i a e c a r l s a d a p i a n d o l b p p c a t s e i h e s t n i t g i i t g r o d n c e e n i r ( i p ff fi f 5 r f e f a c 0 o a e r i a 0 s l i e r d r n e n n n r n s c c e . e e e t e s s s F a i ( s s b n * m . o . d e p r M t p i F w v e < l a a e a i e d i s r c r e 0 ) u g h n n . . i a 0 e n f l 5 a t s o a h s , , f l l o d e * T i l d s , * d b h i p m s e s e e ; t v s < e r a l t f i t a a - l b r u r 0 i i u a r g a . c n 0 n t e s t i 1 e d e o d f s ) d n o s o w r f s t a r g o s o c e s a r r h a m w s p o c o t i s h - w t f a s p o h d e r r p t r e e a h a fi f r m c o e i r v e r r o o e e d m r a g d i r n s r i b e n a t W d p a r p g i r r a h b i s e l s n i u c s u c s e o c t h b - n e x i l o o g a s t o w n r t a b n r o t e y o u i t l s v h f p i g g e g e A s r r n t o o m U r e a u u e d s C e l p p k - a a s r s t n g w a . i u v a n e a T s e p k r n i e n h t d r t g o e a e e 9 s t s A r t i t t 5 e h o i % U s m d e s u C e a a o l C n t t c v s o g I e r e . t a d o r f e r p A s a s o v s ll m r i t l f a a o h A o t w l e 1 U i d o e 5 s C r . is . ra t t h io e\nimprovement of FLEX over the Original method. b, The AUROC distribution across subgroups. Boxes\nROC AUC WT MUT\nindicate the interquartile range of the AUROC distribution (n=15 folds), with the center being the 50th\npercentile and the whiskers extending to data points within 1.5× the interquartile range. Tighter\ndistributions indicate more equitable performance. c, Violin plots of True Positive Rate (TPR) disparity,\ndefined as the difference between a subgroup’s TPR and the overall TPR. Distributions centered closer to\nthe zero-disparity line indicate higher fairness. P-values from the paired t-test are shown. The dashed line\nrepresents the overall TPR across three methods. d, Quantitative summary of TPR disparity, showing the\nRoot Mean Square Error (RMSE) for Original (O), Reinhard (R), and FLEX (F). Lower values indicate\nbetter fairness.\nc\nO R F\nBH IDC ILC R ID ac C e 0.030 0.045 0.037\nWhite 107 7 R I a LC ce 0.213 0.163 0.116\nBlack 10 1 Average 0.122 0.104 0.077\nO R F\nE2 IDC ILC R ID ac C e 0.035 0.034 0.037\nWhite 54 7 R I a LC ce 0.177 0.206 0.135\nAverage 0.106 0.120 0.086\nBlack 9 3\nO R F\nR ID ac C e 0.022 0.034 0.039 B6 IDC ILC\nWhite 28 4 R I a LC ce 0.211 0.097 0.000\nAverage 0.117 0.066 0.020\nBlack 11 2\nO R F\nEW IDC ILC R ID ac C e 0.063 0.048 0.062\nWhite 18 6 R I a LC ce 0.163 0.200 0.091\nBlack 8 2 Average 0.113 0.124 0.077\noitar pag\nCUA\nCUA ytirapsid\nRPT\nd\ne\nf\noitar pag\nCUA\noitar pag\nCUA\noitar pag\nCUA\nCUA\nCUA\nCUA\nytirapsid\nRPT\nytirapsid\nRPT\nytirapsid\nRPT\nb\nSite code\nSite:BH\nUniversity of Pittsburgh\nROC AUC IDC ILC\nSite:E2\nRoswell Park\nROC AUC IDC ILC\nSite:B6\nDuke\\ROC AUC IDC ILC\nSite:EW\nUniversity of Miami\nROC AUC IDC ILC\nO R F\nOriginal Reinhard FLEX White Black White mean Black mean Original Reinhard FLEX\nsedils\nfo\nrebmuN\n∗∗\n∗∗\n∗∗\na\n∗∗∗ ∗∗ ∗∗ ∗∗∗ ∗\nMethod\nMethod\nMethod\nMethod\nCUA\nSite code\nFigure R4: (Supplementary Figure 5 in manuscript) Disentangling site and demographic\nbiases: FLEX mitigates racial performance gaps within individual clinical sites. a, Distribution\nof slides by self-reported race across all TCGA sites, highlighting demographic imbalances and site-specific\ncorrelations. b, Overall performance (AUROC) comparison on the BRCA-TYPE task across eight major\nclinical sites (slides > 5 per class). FLEX consistently outperforms baseline methods. Statistical significance\nis denoted by asterisks (*p<0.05, **p<0.01, ***p<0.001) based on a paired t-test between FLEX and\nthe second-best method. c-f, Fine-grained fairness analysis within four of the major individual clinical sites\n(BH, E2, B6, EW). For each site, the analysis includes: Sample distribution table: Detailed patient\ncounts stratified by race (White, Black) and tumor subtype (IDC, ILC). Fairness gap plot: Visualizes the\nAUC gap (absolute performance difference between White and Black patients) against overall site-level\nAUROC. Small dots represent individual cross-validation folds, while large dots with error bars show the\nmean and 95% CI. A lower ratio (closer to the y-axis) indicates better fairness. Statistical significance\n(p<0.05) from a paired Wilcoxon signed-rank test denotes the improvement of FLEX over the Original\nmethod. Race-wise AUROC plot: Box plots showing the AUROC distribution for White and Black\npatient groups. Tighter and more aligned distributions indicate more equitable performance. Boxes show\nQ1-Q3 interquartile ranges, with the center line indicating the median and whiskers extending to 1.5× the\ninterquartile range. TPR disparity plot: Violin plots of True Positive Rate (TPR) disparity, defined as\nthe difference between a subgroup’s TPR and the site’s overall TPR. Distributions centered closer to the\nzero-disparity line indicate higher fairness. TPR disparity RMSE table: A quantitative summary of\nTPR disparity, showing the Root Mean Square Error (RMSE) for Original (O), Reinhard (R), and FLEX\n(F). Lower values indicate better fairness.\n9\n——————————————— Revision in Page 12, Line 633 ———————————————\n...FLEX represents a significant step in adapting foundation models for computational pathology. It\neffectively mitigates the negative impacts of site-specific signatures and demographic biases while\nimproving feature discriminability for downstream tasks. The core mechanism of FLEX uses an\ninformation bottleneck guided by textual pathological concepts to disentangle task-relevant features\nfrom spurious variations. Because these textual concepts are inherently free of site or demographic\ninformation, this process forces the model to discard extraneous signals, including complex site and\ndemographic signatures. The efficacy of FLEX was validated through extensive evaluations across\n16 pathology tasks, using various pathology VLMs and MIL frameworks. These consistent results,\nobtained via a site-preserved cross-validation strategy and supported by fairness analyses, underscore\nthe method’s robustness. Beyond performance improvements, FLEX offers insights into the strategic\nuse of multimodal prior knowledge for task-specific adaptation of foundation models, contributing to\nthe development of more effective and equitable diagnostic tools....\nRegarding training data sizes, they were not held constant. We used all available training data in each fold\nof our site-preserved cross-validation. However, the partitioning strategy uses convex optimization to ensure\nthat each site-fold contains a balanced proportion of patients for the diagnostic labels [1], resulting in roughly\ncomparable training set sizes across folds (Table R1). Furthermore, our ablation study (Fig. 5 in manuscript)\ndemonstrates that FLEX consistently improves performance across different training data scales.\nComment 5\nSeveral of the biomarker prediction tasks (e.g., BRCA HER2, ER, and PR) have known racial biases.\nOthers are not clearly known. Please discuss the variations in performance given known biases and\nperformance improvements relative to the tasks - is there a larger performance gain in tasks where\nthere are known racial biases?\nResponse:\nThank you for this valuable suggestion. We performed a quantitative comparison across biomarker prediction\ntasks with well-documented racial disparities (e.g., BRCA-HER2, BRCA-ER, BRCA-PR) and tasks where\nsuch disparities are less established (e.g., STAD-EBV, STAD-MSI). As shown in Fig. R5, our analysis did\nnot reveal a clear correlation between the magnitude of improvement from FLEX and the presence of known\nclinical racial disparities.\nThis finding, while potentially counter-intuitive, suggests several potential insights into the nature of bias in\ncomputational pathology and the mechanism of FLEX.\nFirst, in datasets like TCGA, biases from data acquisition (“site-specific signatures”) are heavily confounded\nwith demographic factors. Our results suggest that these procedural, data-driven artifacts may be a more\ndominant source of model error than the currently understood biological differences. FLEX excels at\ncorrecting these pervasive data-driven biases.\nSecond, FLEX operates by aligning visual features with a bias-free textual concept space. This mechanism\nacts as a filter for any spurious visual correlation not explicitly related to the core biological concept. Its\ndebiasingeffectisthereforeinherentlygeneralandagnostictothesourceofbias,ratherthanbeingspecifically\ntuned to one type.\nThird, clinically reported racial disparities often reflect macroscopic phenomena. It is plausible that these\nmacro-level differences do not always manifest as distinct, learnable microscopic morphological features on a\npathology slide, which could weaken the expected correlation.\nWhile our current analysis does not establish a clear link between known clinical biases and the extent of\nperformance gain, the interplay between biological and data-driven biases is undoubtedly complex. We have\nmarked this as an important area for future investigation.\n10\na\nFairness gap\nFairness gap Fairness gap\necaR\nyrtsecnA\necaR\nyrtsecnA\nRace-wise AUROC\nOriginal Reinhard FLEX White / EUR Black / AFR Asian / EAS\noitar\npag\nCUA\noitar\npag\nCUA\noitar\npag\nCUA\noitar\npag\nCUA\nCUA\nCUA\nCUA\nCUA\nb\nROC AUC\n∗\n∗\n∗\nBRCA-HER2\necaR\nyrtsecnA\necaR\nyrtsecnA\noitar\npag\nCUA\noitar\npag\nCUA\noitar\npag\nCUA\noitar\npag\nCUA\nCUA\nCUA\nCUA\nCUA\nOOD Performance IND Performance\nOriginal ∗∗\n∗∗ Reinhard\nFLEX\n∗∗∗ ∗ ∗∗∗ ∗\nBRCA-ER\nROC AUC ROC AUC\nFairness gap Race-wise AUROC\nBRCA-PR STAD-MSI\nROC AUC\nRace-wise AUROC Race-wise AUROC\nCUA\nTask Task\nMethod Method\nFigure R5: (Supplementary Figure 6 in manuscript) Analysis of FLEX’s performance and\nfairness improvements on biomarker prediction tasks, stratified by self-reported race and\ngenetic ancestry. a, Bar plots comparing Out-of-Domain (OOD) and In-Domain (IND) AUROC\nperformance of the original features, features after Reinhard stain normalization, and features enhanced by\nFLEX across five biomarker prediction tasks. FLEX consistently improves diagnostic performance, with\nasterisks indicating statistical significance of the improvement compared to the original model (*p<0.05,\n**p<0.01, ***p<0.001). Error bars represent 95% CI. b, Detailed fairness analysis for four representative\ntasks, stratified by self-reported race (top row for each task) and genetic ancestry (bottom row for each\ntask). Fairness gap plots (left): The y-axis represents the fairness gap (performance difference between\nthe best- and worst-performing groups), and the x-axis shows the overall AUROC. The goal is to move\ntowards the bottom-right corner (lower gap, higher performance). Race/Ancestry-wise AUROC plots\n(right): Box plots showing the AUROC distribution for White/European (EUR), Black/African (AFR), and\nAsian/East Asian (EAS) groups. The results demonstrate that FLEX consistently reduces the fairness gap\nand mitigates performance disparities across demographic groups, for both self-reported race and the more\ngranular genetic ancestry stratification. Boxes show Q1-Q3 interquartile ranges, with the center line\nindicating the median and the whiskers extending to data points within 1.5× the interquartile range.\nIt’s unclear how much of the model performance is dependent on dataset biases beyond demographic\ndata. Ideally, the trained model would be applied to non-TCGA and racial bias and other biases\nanalyzed in this way. Other datasets exist with pathology slide images, and models trained in this\nmanuscript could be applied to those.\nResponse:\nThank you for this crucial point. To address this, we conducted a zero-shot external validation. We trained\nFLEX exclusively on TCGA data and then evaluated its performance on two large, independent external\ncohorts: thepublicCPTACdataset(n=1,738,threecancertypes)andalargein-housedatasetfromNanfang\nHospital of Southern Medical University (NFH, n =5,318, four cancer types). These datasets introduce\nnew sources of variation, including different patient populations, slide preparation protocols, and scanners,\nproviding a stringent test of out-of-distribution generalization.\nThe results (Fig. R6) demonstratethat FLEX significantly enhancesthe model’s ability to generalizeto these\nunseen datasets. As summarized in Fig. R6a, the TCGA-trained model with FLEX achieves a higher average\nAUC across all tasks on both CPTAC (0.714 vs. 0.679 for baseline, p<0.001) and NFH (0.771 vs. 0.733,\np<0.001) compared to the original model. This indicates a consistent improvement in out-of-distribution\ngeneralization. The radar plot (Fig. R6b) and detailed box plots (Fig. R6c, R6d) demonstrate that FLEX\nconsistently expands the performance frontier across most tasks in both external cohorts.\nThese findings provide strong evidence that the performance gains from FLEX are not an artifact of\novercoming biases specific to TCGA. Instead, FLEX enables the model to learn more fundamental and\nrobust biological features, allowing it to generalize effectively across the significant domain shifts inherent\nin new clinical datasets. This demonstrates that FLEX is a valuable tool for building more reliable and\ngeneralizable models for real-world application.\nComment 7\nComments on Figure 2:\n1. Why is figure 2 referenced before figure 1?\n2. Figure 2 plots should have axis labels.\n3. In Figure 2, UMAP is a visualization technique that is affected by parameter tuning. How much\nof that is causing the biases here?\n4. How are the site stratifications actually done, as visualized in Fig 2b? This visualization doesn’t\nexplain it well. How were sites grouped? What about site-site relationships? For example, is\nthere a relationship between the clinical equipment, the location, or something else?\nResponse:\nThank you for the detailed feedback on Figure 2. We have revised the figure and manuscript to address\nthese points.\n1. Why is figure 2 referenced before figure 1?\nWe have corrected the text to ensure figures are referenced in order.\n2. Figure 2 plots should have axis labels.\nWe have added axis labels to all plots in Figure 2.\n3. In Figure 2, UMAP is a visualization technique that is affected by parameter tuning. How much of\nthat is causing the biases here?\nC-NSCLC-TYPE IDC vs ILC N-CRC-BRAF\nLUAD vs LUSC WT vs MUT\nTCGA Training FLEX E T x e t s e t r i n n a g l C N P F TA H C C-B W RC T v A s - P M I U K T 3CA N-L W U T A v D s M -E U G T FR\nC-BRCA-CDH1 N-BRCA-PR WT vs MUT Pos vs Neg\nC-LUAD-EGFR N-BRCA-ER\nWT vs MUT Pos vs Neg\nC-LUAD-STK11 N-BRCA-HER2\nWT vs MUT Pos vs Neg\nC-COAD-BRAF N-STAD-LAUREN\nWT vs MUT Intestinal vs Diffuse\nq < 0.05\nq q < < 0 0 . . 0 0 1 01 C- W CO T v A s D M -T U P T 53 N- I B D R C C v A s - I T L Y C PE N L - U N A S D C v L s C L - U T S Y C PE\nOriginal Reinhard FLEX CPTAC NFH\nc\nCPTAC\nOriginal\nReinhard\nFLEX\nq < 0.05\nq < 0.01\nq < 0.001\nC-BRCA-TYPE C-NSCLC-TYPE C-BRCA-PIK3CA C-BRCA-CDH1 C-LUAD-EGFR C-LUAD-STK11 C-COAD-BRAF C-COAD-TP53\nd\nNFH\nOriginal\nReinhard\nFLEX\nq < 0.05\nq < 0.01\nq < 0.001\nN-BRCA-TYPE N-NSCLC-TYPE N-STAD-LAUREN N-BRCA-HER2 N-BRCA-ER N-BRCA-PR N-LUAD-EGFR N-CRC-BRAF\n976.0 976.0 417.0\n0.72\n0.70\n0.68\n0.66\nO R F\n337.0 037.0 177.0\n0.78\n0.76\n0.74\n0.72\n0.70\nO R F\nCUA CUA\na b\nCPTAC Public\nCPTAC-NSCLC\nn=1,198\nCPTAC-BRCA\nn=323\nCPTAC-COAD CPTAC NFH\nn=217\np<0.001 p<0.001\nNFH\nIn-house\nNFH-NSCLC\nn=2,493\nNFH-BRCA\nn=882\nNFH-STAD\nn=395\nNFH-CRC\nn=1,548\nO Original R Reinhard F FLEX\nFigure R6: Zero-shot generalization of TCGA-trained models to external CPTAC and\nin-house NFH cohorts. Models were trained exclusively on TCGA data and evaluated directly on two\nindependent external datasets without fine-tuning. a, Overall experimental design and results. Bar plots\ndisplay the average AUROC across all tested tasks on the CPTAC (left) and in-house NFH (right) datasets.\nFLEX significantly improves the average performance of the TCGA-trained model on both external cohorts\n(p<0.001, paired t-test). “O”, “R”, and “F” denote Original, Reinhard, and FLEX, respectively. b, Radar\nplot comparing task-specific AUROC across models on CPTAC and NFH data. FLEX consistently expands\nthe performance radius across tasks. Asterisks indicate statistical significance of the improvement by FLEX\ncompared to the original model (*q <0.05, **q <0.01, ***q <0.001, Benjamini-Hochberg FDR correction).\nc, d, Detailed box plots showing task-by-task performance on the (c) CPTAC and (d) NFH cohorts. FLEX\nconsistently outperforms or matches the baseline and Reinhard-normalized models, demonstrating its robust\nability to enhance zero-shot transfer learning by mitigating dataset-specific biases. Boxes indicate quartile\nvalues of the AUROC distribution (n=15 folds), with the center line indicating the median and whiskers\nextending to 1.5× the interquartile range.\nand ‘min_dist=0.1’ based on common practice and preliminary experiments to balance local and global\nstructure. To ensure that our conclusions are not artifacts of specific UMAP settings, we conducted a\nsensitivity analysis by varying these hyperparameters. As shown in Fig. R7, the results for both Original and\nReinhard features show strong site-specific clusters across all tested parameter settings, demonstrating that\nthe observed bias is an inherent property of these feature spaces. In contrast, features enhanced by FLEX\nconsistently show a well-mixed distribution regardless of the UMAP parameters, confirming the robustness\nof our method’s effect.\n4. How are the site stratifications actually done, as visualized in Fig 2b? This visualization doesn’t\nexplain it well. How were sites grouped? What about site-site relationships? For example, is\nthere a relationship between the clinical equipment, the location, or something else?\nThank you for this essential question. We appreciate the opportunity to provide a more detailed clarification.\nOur site splitting is based on the established Preserved-Site Cross-Validation technique [1], [2], which is\nspecifically designed for a stringent evaluation of out-of-distribution (OOD) generalization. The principle is\nto test if a model can perform well on data from a source it has never seen.\nTherefore,wedonotgroupsitesbasedonpartialinformationlikeequipmentorlocation. Thetrue“site-specific\nsignature” is a complex and often undocumented amalgamation of numerous factors (scanner calibration,\nstaining reagents, pre-analytic procedures). Attempting to group sites based on a few known variables would\nfail to capture this full spectrum of potential bias.\nInstead, our method treats each site as an independent entity. We partition the entire set of TCGA sites into\ndistinct folds for cross-validation. To ensure a stable evaluation, we use the convex optimization strategy\nfrom Howard et al. [1] to balance the patient-level diagnostic labels across each fold.\nThis process guarantees that each OOD test set is composed of sites entirely unseen during training while\nhaving a label distribution comparable to the training data, providing a robust assessment of a model’s\nability to generalize to new data sources.\nComment 8\nComments on Figure 3:\n1. Please move figure legends under the matching figures.\n2. It’s unclear exactly what Figure 3a is plotting. Please provide more clarity in the caption\ndescription.\n3. Please add a horizontal line for the avg performance in the data overall, in Figure 3b.\n4. Plots used in Figure 3c are not appropriate for the data being modeled. Please update these to\nnot use dynamite plots.\n5. For all of figure 3, please define the error bars and other data annotations for all subfigures.\nResponse:\nWe appreciate the specific suggestions for improving Figure 3 and have made the following revisions:\n1. Please move figure legends under the matching figures.\nThe legends have been moved to be placed directly under their corresponding subfigures.\n2. It’s unclear exactly what Figure 3a is plotting. Please provide more clarity in the caption description.\nWe have rewritten the caption for Figure 3a to provide a more precise description.\n3. Please add a horizontal line for the avg performance in the data overall, in Figure 3b.\n14\nn_neighbors = 5 n_neighbors = 30 n_neighbors = 100\nFLEX\nmin_dist\n=\n0.2\nmin_dist\n=\n0.4\nOriginal\nReinhard\nOriginal\nReinhard\nFLEX\nIDC ILC\nFigure R7: (Supplementary Figure 4 in manuscript) Robustness analysis of UMAP\nvisualizations to hyperparameter tuning. Parameters ‘n_neighbors’ and ‘min_dist’ were varied to\nassess their impact on the UMAP representation of different feature types. The original and\nReinhard-normalized features show strong site-specific clustering (color-coded by site) across all parameter\nsettings. In contrast, features enhanced by FLEX consistently exhibit a well-mixed distribution, indicating\neffective mitigation of site-specific signatures.\nin Fig. R3c. We have also added a quantitative summary of TPR disparity in Fig. R3d, showing the Root\nMean Square Error (RMSE) of TPR disparity values across demographic groups for each method. This\nprovides an objective measure of fairness, demonstrating that FLEX achieves the lowest average RMSE\nacross most tasks.\n4. Plots used in Figure 3c are not appropriate for the data being modeled. Please update these to not\nuse dynamite plots.\nWe have replaced them with box plots to better represent the data distribution.\n5. For all of figure 3, please define the error bars and other data annotations for all subfigures.\nWe have now clearly defined all error bars and other annotations in the figure captions.\nComment 9\nComments on Figure 4:\n1. Please do not use dynamite plots for Figure 4a. Box plots, violin plots, or another visualization\nare a more representative way of visualizing these results.\n2. Please update the y-axis ranges to be consistent across each row in Figure 4a so it is easier to\ncompare model performance across tasks.\n3. It’s difficult to see the EBV +/- shapes in the Figure 4b plots. Please adjust to make these\neasier to distinguish.\nResponse:\nThank you for the constructive feedback on Figure 4. We have updated the figure as follows:\n1. Please do not use dynamite plots for Figure 4a. Box plots, violin plots, or another visualization are a\nmore representative way of visualizing these results.\nWe have replaced the plots with box plots for a more representative visualization.\n2. Pleaseupdatethey-axisrangestobeconsistentacrosseachrowinFigure4asoitiseasiertocompare\nmodel performance across tasks.\nWe have adjusted the y-axis ranges to be consistent across each row in Figure 4a.\n3. It’s difficult to see the EBV +/- shapes in the Figure 4b plots. Please adjust to make these easier to\ndistinguish.\nThank you for this constructive feedback. We have revised the figure to improve readability and provide\nstronger, quantitative evidence for our claims, as shown in Fig. R8. We now present two parallel UMAP\nsubplots for each method: one colored by clinical site (to assess batch effect removal) and one colored by\nEBV status (to evaluate class separability). This faceted presentation is much clearer.\nMore importantly, to provide objective proof, we have included two key metrics calculated on the high-\ndimensional features: the Local Inverse Simpson’s Index (LISI) and the Silhouette Score. A higher LISI score\nindicatesbettermixingofsites(weakerbatcheffects), whileahigherSilhouetteScoreindicatesbetter-defined\nclass clusters. These metrics provide quantitative support for the visual improvements.\nLISI:3.22 (2.95-3.49) Silhouette Score :0.24 (0.19-0.29) LISI:2.94 (2.65-3.23) Silhouette Score :0.01 (-0.04-0.06) LISI:2.81 (2.56-3.04) Silhouette Score :0.02 (-0.02-0.05)\nlanigirO\nXELF\nUMAPvisualization\nCONCH PathGen-CLIP QuiltNet\nVQ HU FP R5 CG EQ D7 CD 3M IN MX F1 HF ZA KB ZQ B7 HJ BR RD SW EBVNegative EBV Positive\nFigure R8: (Figure 4b in manuscript) UMAP visualizations illustrating the effect of FLEX on the\npatch feature space for the STAD EBV task. For each VLM, parallel subplots are colored by site (left) to\nvisualize batch effect mitigation, and by EBV status (right) to visualize class separability. The Local Inverse\nSimpson’s Index (LISI) for site integration (higher is better) and the Silhouette Score for class separation\n(higher is better) provide quantitative evidence. The visualizations and scores demonstrate that FLEX\nreduces site-specific clustering while improving the discriminability of task-relevant classes.\nComment 10\nComments on Figure 5:\n1. Figure5amissingalegendandthestatisticaltestsisnotdefined. Aretheseadjustedformultiple\nhypotheses? Also, are the data repeatedly stratified then models tested? How much overlap is\nthereindatabetweeneachtest,andhowisthesignificanceofperformancedifferencescalculated?\nWhat size is the training data for each of these splits?\n2. Figure 5b would be more impactful if visualized differently, to show the change in performance\nacross original, improvement, and FLEX. Perhaps using a bipartite graph.\n3. Additionally, there seems to be a large amount of variability in performance in Figure 5b,\nsuggesting that there are other factors in the data splits driving performance differences. Please\ndiscuss this in the text.\nResponse:\nThank you for this detailed feedback on Figure 5. We have made substantial revisions to the figure and its\ncaption to clarify the experimental design and results.\n1. We have revised the figure and caption to address your points, as shown in the new Fig. R9.\n1.1: Figure 5a missing a legend and the statistical tests is not defined. Are these adjusted for multiple\nhypotheses?\nWe have added a legend and clarified the statistical tests in the caption. We performed paired t-tests for\nperformance comparisons and adjusted p-values for multiple hypotheses using the Benjamini-Hochberg FDR\nprocedure. Asterisks in the figure indicate significance levels.\n1.2: Also, are the data repeatedly stratified then models tested?\n17\nb\nNumber of IND Splits Number of IND Splits\nCORUA\nDifferentscalesoftrainingdata\nBRCA-TYPE BRCA-HER2 BRCA-PIK3CA\nq < 0.05\nq < 0.01\nq < 0.001\nFLEX\nOriginal\nNumber of IND Splits\nc\n1.0\nABMIL\nACMIL\nCLAM-SB\n0.9\nDTFD-MIL\nILRA-MIL\n0.8\n0.7\n0.6\nBRCA-TYP\nN\nE SCLC-TY\nS\nP\nT\nE AD-LAUREN BRCA-HER2 BRCA-ER BRCA-PR STAD-EBV STAD-M\nB\nS\nR\nI CA-PIK3CA BRCA-CDH1 LUAD-EGFR LUAD-STK11 STAD-TP5\nS\nCORUA\na\nOuterFold1 OuterFold2 OuterFold3\nSite 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 Site 8\nSite-preserved\nSplit IND IND\nSplits Splits …\n1IND Split 1OOD Split\n2IND Splits\nLabel A 3IND Splits OOD OOD\nSplit Split\nLabel B 4IND Splits\nDifferentMILmodels\nOriginal\nFLEX\nq<0.05\nq<0.01\nq<0.001\nFigure R9: (Figure 5 in manuscript) Performance of FLEX across training data scales and\nMIL models. a, Schematic of the experimental design for evaluating performance at different training data\nscales. The procedure follows our main Site-Preserved Monte Carlo Cross-Validation (SP-MCCV) strategy,\nwhich involves 5 outer folds and 3 inner Monte Carlo Cross-Validation (MCCV) folds, for a total of 15\nevaluation runs. Specifically for this experiment, the dataset is first partitioned into 5 site-preserved outer\nfolds. In each of the 5 outer loop iterations, one fold is held out as the OOD test set. Training sets of\nincreasing size (1, 2, 3, and 4 folds) are constructed from the remaining 4 folds to create a learning curve.\nFor each scale condition, 3 inner Monte Carlo Cross-Validation (MCCV) runs are performed to ensure stable\nresults, for a total of 15 evaluation runs per condition. b, OOD AUROC comparison between FLEX and\nbaseline (Original). Box plots show aggregated results from 15 runs for each training data scale.\nAsterisks indicate statistically significant improvement (Benjamini-Hochberg adjusted q-value: *< 0.05, **< 0.01,\n***< 0.001). Box plots show Q1–Q3 interval with the median; whiskers are 1.5× interquartile range. c,\nDumbbell plot showing OOD AUROC after integrating FLEX with five state-of-the-art MIL models across\n16 tasks. FLEX consistently improves performance across different MIL architectures.\nSite-Preserved Monte Carlo Cross-Validation (SP-MCCV) strategy with 5 outer folds and 3 inner Monte\nCarlo Cross-Validation (MCCV) folds, for a total of 15 evaluation runs. Specifically, we partition the data\ninto 5 mutually exclusive, site-preserved folds. In each of the 5 iterations of the outer loop, one fold is held\nout as the out-of-domain (OOD) test set. For each training data scale (1, 2, 3, or 4 IND splits), we perform 3\nrepeated runs (inner MCCV folds) to ensure the stability and robustness of the results. This process results\nin a total of 15 evaluation runs (5 outer folds × 3 inner runs) for each data scale condition. The box plots in\nFig. R9b show the aggregated distribution of these 15 results.\n1.3: How much overlap is there in data between each test...?\nThe overlap between each test can be summarized as follows:\n• Between each outer fold: There is no overlap between the OOD test sets used in each of the 5\nouter loop iterations. They are, by design, mutually exclusive partitions. This ensures that our overall\nperformance metric is an average over multiple, independent unseen test sets.\n• Within an outer fold (across different data scales): Within a single outer loop iteration, the\nOOD test set is fixed, and the training sets are intentionally nested and overlapping. For example,\nthe training set for “2 IND splits” is a superset of the training set for “1 IND split”. Because of this\ndependency, our primary statistical claim focuses on the comparison between FLEX and the baseline\nat each data scale, rather than comparing performance across different scales.\n1.4: ...and how is the significance of performance differences calculated?\nThe significance was calculated using a paired t-test on the vector of 15 AUROC scores from our 5×3 nested\ncross-validation for each method. The resulting p-values were then adjusted using the Benjamini-Hochberg\nFDR procedure.\n1.5: What size is the training data for each of these splits?\nWe have included a new table (Table R1) in the Supplementary Materials detailing the training and OOD\ntest set sizes for each task and data scale condition, which provides a clear overview of how many samples\nwere used for training and testing at each stage of our evaluation.\nTable R1: Training and OOD Test Set Sizes for Each Task and Data Scale Condition. The table\nshows the number of samples in the training set for each task at different data scales (1 to 4 IND splits) and\nthe corresponding OOD test set sizes. O1 to O5 represent the five outer folds in the Site-Preserved Monte\nCarlo Cross-Validation (SP-MCCV) strategy.\nTraining Set Size (Samples) OOD Test Set Size (Samples)\nTask Number of IND Splits\nO1 O2 O3 O4 O5 O1 O2 O3 O4 O5\n1 117 117 118 135 123 186 174 178 190 209\n2 235 237 254 261 242 186 174 178 190 209\nBRCA-TYPE\n3 353 370 376 376 357 186 174 178 190 209\n4 491 501 493 496 480 186 174 178 190 209\n1 125 140 123 140 125 178 179 196 176 198\n2 262 259 259 261 250 178 179 196 176 198\nBRCA-HER2\n3 384 399 389 390 386 178 179 196 176 198\n4 527 527 515 530 512 178 179 196 176 198\n1 123 123 136 124 135 197 179 177 196 178\n2 249 262 264 258 258 197 179 177 196 178\nBRCA-PIK3CA\n3 387 387 399 391 389 197 179 177 196 178\n4 511 530 528 510 526 197 179 177 196 178\n19\n2. Figure 5b would be more impactful if visualized differently, to show the change in performance across\noriginal, improvement, and FLEX. Perhaps using a bipartite graph.\nThank you for this suggestion. Following your advice, we have redesigned the figure as a Dumbbell Plot\n(Fig. R9c). For each task and MIL model, a connecting line explicitly shows the performance change\nfrom baseline to FLEX. The length and direction of each line intuitively represent the magnitude of the\nperformance gain. Asterisks are used to indicate the statistical significance of the performance improvement,\nwith all p-values adjusted for multiple hypotheses using the Benjamini-Hochberg FDR procedure. We believe\nthis format is more direct and impactful in demonstrating the consistent improvements our method delivers\nacross different models and tasks.\n3. Additionally, there seems to be a large amount of variability in performance in Figure 5b, suggesting\nthat there are other factors in the data splits driving performance differences. Please discuss this in the\ntext.\nThank you for this observation. We have now added a discussion of this phenomenon to the manuscript:\n——————————————— Revision in Page 10, Line 530 ———————————————\n...TheperformanceimprovementseeninFig.5cvariesacrossthe16pathologytasksduetothreemain\nfactors: intrinsic task difficulty, the utility of feature refinement for a given task, and the complexity\nof the MIL model. First, inherent task difficulty sets the upper bound on performance; for example,\nmorphologyclassificationtasksconsistentlyyieldhigherAUROCsthanmostgenemutationprediction\ntasks. Second, the magnitude of improvement from FLEX depends on task-specific conditions. In the\nSTAD-LAURENtask, forinstance, allMILmodelsshowasubstantialperformancegain. Thismaybe\nbecausetheTCGA-STADcohortisrelativelysmall,causingbaselinemodelstooverfittonoisyfeatures.\nFLEX’s core mechanism—refining features by filtering task-irrelevant information—improves the\nsignal-to-noiseratio,whichisparticularlybeneficialwhentrainingdataislimited. Third,performance\ngains are more pronounced with simpler MIL models like ABMIL and CLAM-SB compared to more\ncomplex architectures. We hypothesize this is related to the joint optimization of FLEX and the MIL\nmodel. A simpler MIL backbone may create a more stable learning environment, allowing gradients\nto effectively guide the feature refinement process within FLEX. In contrast, with more intricate MIL\nmodels, the increased number of parameters and interacting components can complicate convergence\nto an optimal state, potentially limiting the observable performance gain from FLEX....\nReferences:\n[1] F. M. Howard, J. Dolezal, S. Kochanny, et al., “The impact of site-specific digital histology signatures\non deep learning model accuracy and bias,” Nature communications, vol. 12, no. 1, p. 4423, 2021.\n[2] A. Vaidya, R. J. Chen, D. F. Williamson, et al., “Demographic bias in misdiagnosis by computational\npathology models,” Nature Medicine, vol. 30, no. 4, pp. 1174–1190, 2024.\nGeneral Comments. The proposed method presents an interesting model design. There are a few\npotential issues regarding its scalability, interpretability, and practical utility. Notable limitations\ninclude reliance on human experts for patch selection, dependence on GPT-4o for concept generation\nwithout verification, and modest improvements in out-of-domain performance. In addition, the\nabsence of statistical testing across tasks may undermine the method’s robustness and its real-world\napplicability.\nResponse:\nThank you for your comprehensive feedback. We have substantially revised the manuscript to address your\nconcerns regarding scalability, reliance on external knowledge, interpretability, performance, and statistical\nrobustness.\n1. Scalability and Reliance on External Knowledge: We have clarified in the manuscript that\nknowledge integration is a one-time, offline procedure conducted solely during model training. The patch\nselection process is efficient, requiring expert validation on only a small number of patches and thereby\nminimizing the demand on expert time. This setup cost does not impact the scalability or speed of the final\nmodel at inference. Toaddress the concern about unverified AI outputs, we have detailed our human-in-the-\nloop validation protocol and constrained fact-based concept generation strategy to ensure clinical reliability\n(as detailed in our responses to Comments 1 and 2).\n2. Interpretability: To improve clarity, we have provided a more precise definition for “noisy regions” in\nour attention map visualizations, defining them as areas containing processing artifacts or task-irrelevant\nbiological structures (as detailed in our response to Comment 6).\n3. OOD Performance and Practical Utility: To address concerns about the magnitude of OOD\nperformance improvements and practical utility, we conducted an extensive zero-shot external validation\non two large, independent cohorts: the public Clinical Proteomic Tumor Analysis Consortium collections\n(CPTAC; n=1,738) and our in-house dataset from Nanfang Hospital of Southern Medical University (NFH;\nn=5,318). The results (Fig. R11c, R15) show that FLEX yields statistically significant performance\ngains in these challenging OOD scenarios, providing strong evidence of the method’s robust and generalizable\nbenefits for real-world applications (as detailed in our responses to Comments 3 & 10).\n4. Statistical Testing: We have now integrated comprehensive statistical analyses throughout the\nmanuscript. This includes paired statistical tests with Benjamini-Hochberg correction for all performance\ncomparisons(Figs.R11c,R12,R14,R15)andtheuseofobjectivequantitativemetrics(LISI,SilhouetteScore)\nto support our UMAP visualizations (Figs. R11a, R13). This addition ensures the statistical robustness of\nour claims (as detailed in our responses to Comments 4, 8 & 9).\nWe believe these comprehensive revisions significantly enhance the manuscript’s clarity, robustness, and\npractical applicability. We provide detailed responses to each specific comment below.\nComment 1\nIn the visual concept generation module, the selection of representative patches requires both GPT-4o\nfiltering and manual validation by human experts. This dependency on expert input may limit the\nscalability and automation potential of the proposed method, particularly in clinical settings where\nexpert time is costly and throughput demands are high.\nResponse:\nThank you for raising this important point. We have revised the manuscript to clarify this aspect. The\nexpert-guided selection of representative patches is a one-time offline procedure required only during the\ninitial model training phase. This process is efficient, as it requires expert validation on only a small set of\nrepresentative patches (10 per class), thereby minimizing the demand on expert time. Once trained, the\nmodel is fully automated and highly scalable at inference. It processes new whole-slide images and generates\n21\npredictions without needing the original patches or any further human intervention, as illustrated in our\ninference pipeline (Fig. R10c). The expert knowledge is encoded within the model’s parameters, ensuring\nhigh throughput suitable for clinical workflows. Furthermore, this initial investment provides lasting value.\nWhenadaptingthemodeltonewdata, suchasfine-tuningondatafromsimilardomains, re-selectingpatches\nis not required, as the initial expert-validated concepts provide a strong, generalizable foundation.\nIn summary, our approach uses a minimal, one-time expert investment to build a robust foundation, which\nin turn enables a fully automated and scalable model for clinical deployment. In the future, we wil develop\nmore automatic strategy to further reduce human intervetion. We have added a statement to the Methods\nsection to describe this one-time offline procedure and its implications for scalability.\nComment 2\nThe use of GPT-4o for generating textual concepts is not accompanied by a validation or correction\nmechanism. Given potential issues with hallucination and inconsistency in large language models,\nrelyingsolelyonGPT-4ooutputsforguidingfeatureenhancementmayintroducepotentialinaccuracies,\nespecially in high-stakes medical applications.\nResponse:\nThank you for raising this critical point. Our methodology incorporates a two-layer safeguard to ensure the\naccuracy and reliability of the textual concepts used to guide FLEX.\nFirst, our methodology includes a Human-in-the-Loop Expert Validation step. As an integral part of\nour workflow, all textual prompts generated by GPT-4o were reviewed and validated by a board-certified\npathologist (Fig. R10b). This verification step ensures that all prompts are medically accurate, clinically\nrelevant, and free from factual errors or hallucinations.\nSecond, we employ a Constrained and Fact-Based Generation strategy. The risk of hallucination is\ninherently minimal because the model is not tasked with creating open-ended narratives but with generating\nsimple descriptive labels grounded in established oncologic terminology (e.g., “invasive ductal carcinoma”,\n“connective tissue”). This fact-based approach minimizes the scope for LLM-induced inaccuracies, making\nthe subsequent expert validation both efficient and effective.\nFurthermore, like the visual concept selection, this concept generation and validation process is a one-time\noffline procedure that places no burden on the model at inference.\nOverall, this two-stage protocol provides a robust defense against potential inaccuracies. We have updated\nthe Methods section to detail this validation process.\n——————————————— Revision in Page 17, Line 935 ———————————————\nTextual concept generation. The process of textual concept generation is illustrated in Fig. 6b.\nIt involves creating task-specific and task-agnostic prompts that serve as positive and negative\nsupervision signals. Recognizing the need for accuracy in medical applications, our methodology\nincorporates an expert validation protocol. First, the prompt generation is constrained to simple,\ndescriptive labels grounded in established oncologic terminology (e.g., “invasive ductal carcinoma”,\n“connective tissue”), which minimizes the risk of factual inaccuracies. Second, and most critically, we\nimplemented a human-in-the-loop expert validation process. All textual concepts, whether initially\nderivedfromestablishedclassnamesordraftedwithassistancefromGPT-4o,wererigorouslyreviewed,\nedited for clinical precision, and formally approved by a board-certified pathologist. This expert\noversight ensures that every prompt used is medically accurate and relevant to the prediction task.\nWith these controls, prompts were defined as follows: for tasks with clear class boundaries, such as\nBRCA subtyping, task-specific prompts are the class names themselves, following previous studies.\nHowever, for more complex tasks with vague class boundaries, high complexity, or limited data,\nclass names alone may be insufficient. To address this, we propose using the same class names as\ntask-specific prompts attached to learnable contexts. These contexts adaptively capture task-specific\ninformation from the images during training, enhancing the prompts’ effectiveness. Task-agnostic\nprompts consist of general descriptions irrelevant to the task, such as the names of other tissue types\n22\na\nQ: Can it be\ndeduced from this\nimage whether it Representative Patch Representative\nbelongs to Invasive Image Candidates Patch Images VLM\nDuctal Carcinoma? Image\nEncoder\nA: This image demonstrates Verification by certain histological features … Human Expert …\nthat could be indicative of\nInvasive Ductal Carcinoma. GPT-4o\n…\n…\n…\n…\nClass 1\nVLM\nImage Encoder\nEncoder\n𝜇 𝛴\nOriginal Features\n…\n…\n𝒙 𝑞!(𝒛|𝒙)\n…\n…\nQ:\nWhat types of tissue or cells Representative a im re a p ge re s s o e f n b t r i e n a w s h t c o a le n s c l e id r? e Text Prompts VLM [invasive ductal carcinoma] Text\n[invasive lobular carcinoma] Encoder\nA 1. : Tumor Tissue [ [ c n o o n rm ne a c l t b iv re e a t s is t s t u is e s ] ue] … … 2. Normal Breast Tissue\n…\n𝒛\n~\n~\n~\n~\n~\n~\n…\nVisual Concepts\nClass 1 Class 2\nb\nTextual Concepts\nText Promptswith Class 1 Learnable Context Class 2 𝑉𝑉[invasive ductal carcinoma]\n𝑉𝑉[invasive lobular carcinoma]\n𝑉 𝑉 𝑉 𝑉 [ [ c n o o n rm ne a c l t b iv re e a t s is t s t u is e s ] ue] Task- … agnostic GPT-4o\nc\nTask-related Site-specific Task-agnostic Task-related\nInformation Signature Information Information Textual Concept Guided\nFeature Calibration\nMinimize Distance\nMaximize Distance\nMIL\nModel\nVisual Concept Guided\nEnhanced Features Visual Pilot Patch Selection\nConcept\nAggregated Features Select\nTextual\nVariational Distribution Class 1 … … Top-k Concepts\nLabel A\nLabel B\n…\nTraining IND Testing OOD Testing\n…\n…\n…\n…\n…\n…\n…\n…\n…\n…\n…\n…\n…\n…\nVisual Concept Generation\nTextual Concept Generation\nVerification by Human Expert\nArchitecture of FLEX\nTrainingPhase\nInference\nPhase\nd\nSite-preserved Monte Carlo Cross-Validation\nOuter Inner Fold Fold\nSite 1 Site 2 Site 3 Site 4 Site 5\nOuter Fold: 1\nSite-preserved Split INDSplit OODSplit\n1\nRandom Split\nFLEX\nFigure R10: (Figure 6 in manuscript) Overview of the proposed framework and experimental\nsetup. a, Visual knowledge generation pipeline. This is a one-time offline procedure during training.\nABMIL identifies high-attention patches, followed by GPT-4o-based filtering and human expert verification\nto select representative patches. Class-specific visual concepts are generated by averaging features from\nrepresentative patches using the VLM image encoder. b, Pipeline for textual knowledge generation, also a\none-time offline procedure. GPT-4o is prompted to create representative text prompts (both task-specific\nand task-agnostic). After expert verification, these prompts are enriched with learnable, data-driven\ncontexts and encoded by the VLM text encoder to create the final textual concepts. c, The architecture of\nFLEX. During inference, original patch features are extracted by the VLM image encoder; a variational\nencoder then generates parameters for a Gaussian distribution for each patch, from which enhanced features\nare sampled and aggregated by the MIL model. During training, a Visual Concept Guided Pilot Patch\nSelection module uses the pre-computed visual concept to select the top-k most relevant enhanced patches.\nThese selected patches are then used in the Textual Concept Guided Feature Calibration process, where an\nInfoNCE loss aligns the features with the textual concepts by minimizing the distance to the corresponding\nclass concept while maximizing their distance from other concepts. This process helps to optimize the\nvariational encoder of FLEX. d, Schematic of the SP-MCCV strategy. The dataset is partitioned into outer\nfolds based on the clinical site to create distinct training and OOD test sets. Inner folds are then used to\nrandomly split the training data for IND evaluation.\n23\n(e.g., “connective tissue” or “normal tissue”). These prompts guide the feature enhancement process\nto filter out task-agnostic information. These are also attached to learnable contexts to improve their\neffectiveness in filtering out irrelevant features. Finally, textual concepts are generated by averaging\nthe embeddings of these prompts (with their learnable contexts) produced by the VLM’s text encoder.\nComment 3\nAlthough the method is described as a cross-domain generalization method, the actual performance\nimprovements on out-of-domain (OOD) datasets are modest. In some cases, the proposed method\n(Feature-Level Enhancement for Cross-domain generalization; FLEX) reduces the performance gap\nfrom only 4% to 3%, which may have limited clinical relevance or operational impact.\nResponse:\nThank you for this insightful comment. We will address this from the following two perspectives.\nFirst, while the reduction in the performance gap between IND and OOD settings is one metric, we argue\nthe more critical metric is the absolute performance improvement brought by FLEX on the unseen OOD\ndata. For example, after applying FLEX, the average OOD AUROC was increased by 6.45% (0.853 to\n0.908), 7.21% (0.721 to 0.773), and 4.94% (0.628 to 0.659) for morphology classification, molecular biomarker\nprediction, and gene mutation prediction tasks, respectively, with all results being statistically significant\n(p<0.001) (Fig. R11c). This represents a substantial and statistically significant improvement on data from\nunseen clinical sites.\nSecond, to provide definitive evidence of clinical relevance, we evaluated FLEX on two independent external\ncohorts: CPTAC and our private in-house NFH dataset. This setup simulates real-world deployment in a\nnew clinical environment. The results (Fig. R11c and Fig. R15) confirm that the performance gains translate\ndirectly to these new settings. FLEX achieved significant AUROC increases of 3.5% on CPTAC (0.679 to\n0.714, p<0.001) and 3.8% on the in-house NFH dataset (0.733 to 0.771, p<0.001).\nThese findings demonstrate that the improvements delivered by FLEX are significant and robust, offering\ntangible operational impact for real-world clinical deployment.\nComment 4\nIn Figure 2a, the UMAP visualizations show overlapping clusters even after applying the proposed\nmethod, indicating that site-specific signatures may not be fully eliminated. A quantitative mea-\nsure of intra- and inter-class distances or silhouette scores would strengthen the claims of feature\ndisentanglement and clustering quality.\nResponse:\nThank you for this valuable feedback. Following your recommendation, we have performed a comprehensive\nquantitative analysis.\nTo measure the integration of features from different clinical sites, we used the Local Inverse Simpson’s Index\n(LISI). Our method achieves a LISI score of 4.18, a significant 38% improvement over the original features\n(3.03) and the Reinhard baseline (3.29). This quantitatively confirms that FLEX substantially mitigates\nsite-specific batch effects.\nCrucially, mitigating site effects must not come at the cost of biological signal. To measure class separability,\nweusedtheSilhouetteScore. Here,FLEXachievesascoreof0.38,a153%improvementoverboththeoriginal\n(0.15) and Reinhard-normalized features (0.14). This demonstrates that as FLEX removes confounding site\ninformation, it simultaneously makes the underlying biological classes (IDC vs. ILC) significantly more\ndistinct.\nWe have revised Figure 2a (now Fig. R11a) to include these quantitative scores and to present the UMAPs\ncolored by both site and biological class separately. This updated figure, backed by LISI and Silhouette\nmetrics, provides clear and robust evidence for our central claim: FLEX successfully disentangles site-specific\nnoise while enhancing the separation of true biological signals.\n24\na\nb c\nSite 1 Site 2 Site 3\nTraining IND TestingOOD Testing\nSamesite-specificsignatures\nDifferentsite-specificsignatures\nIND OOD\nd\nTCGA OOD Testing\nTCGA IND Testing\nWSI Original FLEX\n976.0 976.0 417.0\n0.72\n0.70\n0.68\n0.66\nO R F\n337.0 037.0 177.0\nFLEX Original Reinhard\nSite-specific Biological variance\nsignature removal preservation ① ① ② ②\n② ② ① ①\nLISI:4.18 (3.88-4.47) Silhouette Score:0.38 (0.34-0.43) LISI:3.03 (2.87-3.18) Silhouette Score:0.15 (0.12-0.17) LISI:3.29 (3.12-3.47) Silhouette Score:0.14 (0.12-0.16)\n① ② ① ②\nIDC\nILC\n0.78\n0.76\n0.74\n0.72\n0.70\nO R F\n819.0 358.0 519.0 368.0 939.0 809.0\n0.96\n0.92 7.08% 5.68%\n0.88\n0.84\n0.80\nO R F\n457.0 127.0 947.0 717.0 197.0 377.0\n0.80\n0.76 4.38% 4.27%\n0.72\n0.68\nO R F\n156.0 826.0 256.0 826.0 086.0 956.0\n0.70\n0.68 3.09%\n0.66 3.53% 3.68%\n0.64\n0.62\n0.60\nO R F\nCUA CUA CUA CUA CUA\nC M la o s r s p i h fi o c l a o t g io y n M S ol t e a c tu u s la P r r B e i d o i m ct a io rk n er Ge P n r e e d M ic u t t i a o t n ion CPTAC NFH\nO Original R Reinhard F FLEX\nC-BRCA-TYPE BRCA-TYPE C-NSCLC-TYPE IDC vs ILC N-CRC-BRAF\nNSCLC-TYPE IDC vs ILC CRC-TP53 LUAD vs LUSC WT vs MUT\nLUAD vs LUSC WT vs MUT\nC-BRCA-PIK3CA N-LUAD-EGFR\nWT vs MUT WT vs MUT\nSTAD-LAUREN CRC-BRAF\nIntestinal vs WT vs MUT\nDiffuse\nC-BRCA-CDH1 N-BRCA-PR\nWT vs MUT Pos vs Neg\nBRCA-HER2 STAD-MUC16\nPos vs Neg WT vs MUT\nC-LUAD-EGFR N-BRCA-ER\nWT vs MUT Pos vs Neg\nBRCA-ER STAD-TP53 Pos vs Neg WT vs MUT\nC-LUAD-STK11 N-BRCA-HER2\nWT vs MUT Pos vs Neg BRCA-PR LUAD-STK11\nPos vs Neg WT vs MUT\nC-COAD-BRAF N-STAD-LAUREN\nWT vs MUT Intestinal vs Diffuse\nSTAD-EBV LUAD-EGFR Pos vs Neg WT vs MUT\nC-COAD-TP53 N-NSCLC-TYPE\nNon S M T S A I- D H - v M s M SI SI-H BRCA-PIK3CA BR W C T A vs -C M D UT H1 WT vs MUT N-B ID R C C v A s - IL T C YPE LUAD vs LUSC\nWT vs MUT\nBRCA-TYPE\nNSCLC-TYPE IDC vs ILC CRC-TP53\nLUAD vs LUSC WT vs MUT\nSTAD-LAUREN CRC-BRAF\nIntestinal vs WT vs MUT\nDiffuse\nBRCA-HER2 STAD-MUC16\nPos vs Neg WT vs MUT\nBRCA-ER STAD-TP53\nPos vs Neg WT vs MUT\n500μm 500μm 500μm\nBRCA-PR LUAD-STK11\nPos vs Neg WT vs MUT\nSTAD-EBV LUAD-EGFR Pos vs Neg WT vs MUT\nSTAD-MSI BRCA-CDH1\nNon MSI-H vs MSI-H BRCA-PIK3CA WT vs MUT\nWT vs MUT\nOriginal\nReinhard\nFLEX 500μm 500μm 500μm\nEPYT-ACRB\nISM-DATS\n)CDI(2XD-00-Z10-JLAA-C3-AGCT\n)H-ISM(\n1XD-00-Z10-JM4A-DC-AGCT\n1.0\nAttention\nWeight\n0.0\np<0.001 p<0.001 p<0.001 p<0.001 p<0.001\np=0.003 3.30% p=0.001 2.23% p=0.001\ne\nExternalTesting\nTCGA\nTraining\nFLEX\nExternal\nTesting\nCPTAC\nNFH\nq < 0.05\nq < 0.01\nq < 0.001\nOriginal Reinhard FLEX CPTAC NFH\nf\nMorphologyClassification q < 0.05\nMolecular Biomarker Status Prediction q < 0.01\nGene Mutation Prediction q < 0.001\nperformance by mitigating site-specific signatures. a, UMAP visualization of patch features from the\nTCGA-BRCA dataset (BRCA-TYPE task), encoded by a CONCH model. Features are colored by\nsubmitting site (left panel) or diagnostic class (right panel). Original features and those after Reinhard stain\nnormalization show strong clustering by site and poor separation of diagnostic classes, confirmed by low site\nintegration (LISI: 3.03 and 3.29) and class separability (Silhouette Score: 0.15 and 0.14) scores. Callouts\nhighlight patches from the same site with different diagnoses clustering together, indicating that site-specific\nsignals overpower biological ones. In contrast, FLEX-processed features show effective removal of site\nsignatures (LISI: 4.18) and improved separation of diagnostic classes (Silhouette Score: 0.38). b, Schematic\nof the site-preserved cross-validation setup, defining in-domain (IND) test sets (from sites seen during\ntraining) and out-of-domain (OOD) test sets (from unseen sites). c, Aggregate AUROC performance across\ntask categories. On both IND and OOD splits of TCGA and on two external cohorts (CPTAC and NFH),\nFLEX significantly outperforms baselines (paired t-test, p<0.01). d, Radar plots of per-task OOD (top)\nand IND (bottom) AUROC on 16 TCGA tasks. FLEX (pink) consistently outperforms the Original (blue)\nand Reinhard (orange) methods across nearly all tasks. Statistical significance (paired t-test with\nBenjamini-Hochberg adjustment; *q <0.05, **q <0.01, ***q <0.001) is shown. e, Zero-shot generalization\nperformance on external CPTAC and NFH cohorts using models trained only on TCGA. f, Attention maps\nfrom an ABMIL model for BRCA-TYPE and STAD-MSI tasks. The Original model’s attention is often\nunfocused or distracted by artifacts. The FLEX-enhanced model correctly localizes its attention to relevant\ntumor regions, leading to more accurate and interpretable predictions.\nComment 5\nInterestingly, standard stain normalization via the Reinhard method achieves better results than their\nmethods in a few tasks. The authors could further discuss reasons leading to this discrepancy and\naddress questions about the added value of their method.\nResponse:\nThank you for this insightful comment. We believe this observation highlights the distinction between\nspecialized pre-processing and comprehensive feature-level generalization.\nThe Reinhard method is designed to correct one specific source of variation: stain color. In tasks where\nstain color is the dominant domain shift, this targeted approach can be effective. However, domain shifts in\npathology are rarely this simple; they often involve a complex interplay of variations from tissue preparation,\nscanner artifacts, and other latent biases. As our UMAP analysis shows (Fig. R11a), complex site-specific\nfeature clusters persist even after Reinhard normalization, demonstrating that stain correction alone is\ninsufficient. While Reinhard performs well in some IND testing scenarios, it does not generalize as well to\nthe broader challenges of OOD testing, where FLEX excels (Fig. R11d).\nThe added value of FLEX is its more holistic solution, offering advantages in three key areas:\n• Superior Overall Generalization: FLEX delivers consistently superior performance across the vast\nmajority of tasks, particularly in the more challenging OOD setting (Fig. R11d). Its ability to address\nthe full spectrum of site-specific biases results in stronger and more reliable generalization.\n• Enhanced Fairness: FLEX addresses critical challenges that simple pre-processing cannot. As shown\nin Fig. R12, it significantly reduces performance disparities across demographic groups, an essential\ncapability for the equitable deployment of medical AI that is beyond the scope of stain normalization.\nIn summary, while Reinhard is a valuable tool for a single source of variation, FLEX is a comprehensive\nframework for domain generalization. Its ability to build more robust, fair, and broadly applicable models\nprovides a clear advantage. We have revised the manuscript to include this discussion.\nComment 6\nIn Figure 2d and related visualizations, the authors mentioned their methods reduced attention paid\nto noisy regions. However, what constitutes noise was not clearly defined or objectively measured.\nThank you for this constructive feedback. We define “noisy regions” as any visual information within a slide\nthat is irrelevant to the specific diagnostic task. This includes two main categories: (1) processing artifacts,\nsuch as tissue folds, air bubbles, out-of-focus areas, and other technical imperfections; and (2) task-agnostic\nbiological structures, such as adipose tissue, non-neoplastic stroma, or necrosis when the goal is to classify\ntumor subtypes. We have now added this definition to the manuscript for clarity.\n——————————————— Revision in Page 6, Line 323 ———————————————\n...Here, “noisy regions” are defined as visual information irrelevant to the diagnostic task, such as\nprocessing artifacts or task-agnostic biological structures (e.g., non-neoplastic stroma)...\nWhile the gold standard for objective measurement would be to quantify attention against fine-grained\nregions annotated by expert pathologists, such an effort is beyond the scope of this study. We therefore\npresent the attention maps in Figure 2f as qualitative visualizations that provide an intuitive explanation\nfor our quantitative results. The most objective evidence of our method’s success lies in the significant and\nconsistent improvements in AUROC and fairness metrics across tasks. These robust performance gains\nserve as definitive proof that FLEX successfully guides the model to focus on task-relevant patterns while\nsuppressing the influence of these distracting signals.\nWe have also updated our Discussion section to acknowledge this limitation and propose the quantitative\nvalidation of attention against pathologist annotations as an important direction for future research.\n——————————————— Revision in Page 12, Line 651 ———————————————\n...Future research will focus on expanding FLEX’s versatility. Key directions include broadening\ncompatibility to vision-only foundationmodels andextending itsapplicationto awider rangeof tasks,\nincluding regression and prognosis. We will also explore its integration with pathology vision large\nlanguage models and work to enhance the interpretability and adaptability of these models. This\neffort includes the quantitative validation of attention mechanisms against expert-annotated regions\nby pathologists, which would provide an objective confirmation of the model’s focus. Such work will\nbe crucial for translating these advancements into clinically impactful and trustworthy tools.\nComment 7\nIn Figure 3, the fairness gap (measured in AUROC differences across racial groups) ranges widely,\nsometimes as high as 0.7. Surprisingly, the proposed methods exhibit the widest performance spread\nin certain tasks. The authors could discuss the implications of this observation and the potential\nimpact on the reliability of the proposed methods.\nResponse:\nThank you for this crucial observation. The wide performance spread in certain tasks is a statistical artifact\nof confidence interval calculation, which can be highly sensitive to single outliers across the 15 validation\nfolds, rather than an indicator of unreliability. We have provided more detailed analysis to demonstrate that\nFLEX is consistently more reliable at reducing fairness gaps.\nAs shown in the granular scatter plots (Fig. R12a), even in tasks like the CRC-BRAF where the error bar is\nwide, the effect is often driven by a single outlier, while the main cluster of FLEX results still shows superior\nperformance.\nMore importantly, a comprehensive assessment requires looking beyond a single metric. Our analysis of True\nPositive Rate (TPR) Disparity (Fig. R12c) offers a more direct measure of equitable performance. The violin\nplots for FLEX are visibly and consistently narrower and more tightly centered around the zero-disparity line\nacross nearly all tasks. This provides strong evidence that our method enables the model to perform more\nreliably across different demographic groups. This conclusion is further substantiated by the quantitative\nTPR Disparity RMSE in Fig. R12d, where FLEX achieves a significantly lower (i.e., better) disparity score.\nIn summary, while the confidence interval for one metric can be skewed, the totality of the evidence\ndemonstrates that FLEX consistently improves both fairness and model reliability. We have revised our\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nE P Y e c a R\noitar\npag C U A C U A C O R\nytirapsid\nRPT R R I I D a a LC c c C e e 0 0 . .\nO\n0 0 3 4 5 0 0 0 . . 0 0\nR\n3 39 3 0 0 . . 0 0\nF\n3 5 3 3 b\nreriaF\nc Race-wise AUROC\nT\nA C R B y rts e c n A oitar pag C U A C U A C O R ytirapsid RPT 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 A A A n n v I c c e I D L e e r C C a s s g t t r r e y y 0 0 0 . . . 1 2 1 5 0 0 0 7 3 0 0 0 . . . 1 2 1 3 0 03 9 2 0 0 0 . . . 0 0 1 8 3 7 7 6 2\nROC AUC IDC ILC\na b c d\nFairness gap Race-wise AUROC TPR disparity TPR disparity RMSE 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nBetter Separation\nreriaF\nC\nE P Y T-\ne\nc a R\noitar\npag C U A C U A\nytirapsid\nRPT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001\nL L R R U U a a A S c c C D e e 0 0 . .\nO\n0 0 9 8 1 1 0 0 . . 0 1\nR\n7 0 0 5 0 0 . . 0 0\nF\n7 6 5 2\nL C S Ny rts e\nc\nn\nA\noitar pag\nC\nU A ∗\nC U\nA\nytirapsid\nRPT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 A A\nA\nn n L L\nv\nc c U U\ne\ne e\nr\nA S\na\ns s C D\ng\nt t r r\ne\ny y 0 0\n0\n.\n.\n.\n0 0\n0\n9 7\n8\n7 4\n6\n0 0\n0\n.\n.\n.\n0 1\n0\n7 0\n8\n9 2\n9 0\n0 0\n.\n.\n.\n0\n0 0 7 5\n6\n9 7\n8\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nROC AUC Method LUAD LUSC\np < 0.001\nR P\ne\nc a R\noitar\npag C U\nA ∗\nC U A\nytirapsid\nRPT R\nR\nN\nP\na\na o\ne c\nc s\ng e\ne\n0\n0\n.\n.\nO\n0\n0\n9\n9\n0\n5\n0\n0\n.\n.\n1\n0\n8\n5\n2\n0\n0\n.\n.\n1\n0\n8\n5\nC R By\nrts e c n\nA\noitar\npag C\nU A\nC U A\nytirapsid\nRPT\nA\nA\nA\nn\nn\nv\nN\nP\nc\nc\ne\ne\ne o\nr\ne\na\ns\ns s\ng\ng\nt\nt\nr\nr\ne\ny\ny\n0\n0\n0\n.\n.\n.\n1\n1\n1\n1\n3\n0\n5\n1\n8\n0\n0\n0\n.\n.\n.\n1\n1\n1\n2\n0\n2\n7\n3\n0\n0\n0\n0\n.\n.\n.\n1\n1\n0\n2\n9\n0\n7\n7\nO\nR D\nr\nM\nig\nis\ni\nS\nn\np E\na\na\nl\nrity\n.\nI\n3\nc C\n5\ne\n3 0\nR\n.\nI\n0\na L\n9\ne\n0\nn\n.\nI c\n2\nt\n8\nry A\n0\nn\n.\nC I\n0\nt\n0\nry\nReinhard 0.0327 0.0390 0.1388 0.2017\nROC AUC Method Neg Pos FLEX 0.0331 0.0532 0.0873 0.1318\nF A R\ne\nc a R\noitar\npag C U\nA\nC U A\nytirapsid\nRPT R\nR M\nW a\na U\nc\nc\nT\nT\ne\ne\n0\n0.\n.\nO\n1\n28\n1\n7\n9 0\n0.\n.0\n0\n8\n0\n9 0\n0.\n.1\n2\n20\nC R C - y\nrts e c n A\noitar\npag C U A C U A\nytirapsid\nRPT\nA\nA A\nn\nn v M\nc\nc e\nW e\ne U ra\nT s\ns T g\nt\nt\nr\nr e\ny\ny\n0\n0 0\n.\n.\n.\n1\n2 2\n2\n7 0 7 2\n5 0\n0 0\n.\n.\n.\n0\n1 1\n9\n9 4\n8\n7 6\n0\n0 0\n.\n.\n.\n1\n2 1\n1\n4 7\n8\n6 7\nROC AUC Method WT MUT\nA\nE\nC\nP\nR\nY\nC B T - -e c a R\noitar\npag C U A ∗ C U A\nytirapsid\nRPT\np < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001 p < 0.001\nR R I I D a a LC c c C e e 0 0 . .\nO\n0 3 5 0 8 2 0 0. .0 3\nR\n6 0 2 2 0 0. . 2 0\nF\n5 9 7 7\nAverage 0.180 0.182 0.177\nROC AUC Method IDC ILC\nD\nR F\nA\nG\nU C E L - -e c a R\noitar\npag C U A ∗∗ C U A\nytirapsid\nRPT\np < 0.001 p < 0.001 p < 0.001 p < 0.001\nR R M W a a U c c T T e e 0 0 . .\nO\n1 1 5 2 0 9 0 0 . . 1 1\nR\n7 1 1 6 0 0 . . 0 1\nF\n6 0 8 1\nAverage 0.140 0.144 0.085\nROC AUC Method WT MUT\nO R F\nOriginal Reinhard FLEX White / EUR Black / AFR Asian / EAS White / EUR mean Black / AFR mean Asian / EAS mean Original Reinhard FLEX\nFigure R12: (Figure 3 in manuscript) Evaluation of FLEX’s effectiveness in improving\n𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001\nA C 3 K e c a R\noitar\npag C U A C U A C O R\nytirapsid\nRPT R R M W a a U c c T T e e 0 0 . .\nO\n1 1 0 35 4 0 0 . . 0 1\nR\n9 2 8 6 0 0 . . 1 1\nF\n1 3 4 5\nd\nc b r o\ne\no o\nm\ns t s s\no\n- t v\ng\nr a a\nr\nl p i\na\nd p\np\na i\nh\nt n i\ni\ng o\nc\nn (\nf\n5 f\na\n0 o\ni\n0 l\nr\nd\nn\nr s e .\ne\ns\ns\nF a\ns\nm o\n.\nr p\nF\ne l\na\ne a\ni\ns c\nr\n) h\nn\n.\ne\nf a\ns\ns\n, ld T\nis\n, h m\ne\ne\nv\ne\na\nf t a\nl\nr\nu\ni i r\na\nc n s\nt\ne\ne\nf s\nd\no s r\na\ng\nc\ne a\nr\na p\no\nc\ns\nh f\ns\no d r\nr\ne\na\nfi m\nc\nv\ne\no e\na\ng r\nn\nr e a\nd\np p r\na\nh e\nn\ni s c e\nc\n- n\ne\nla\ns\nt\nt\na b\nr\nt e\ny\ni l v\ng\ng e\nr\nr t\no\no a\nu\nu s\np\np k\ns\ns w .\nu\ne T\ns\nr\ni\ne\nn\nh\ng\ne es A\nr\nt\ne\ni U\ns\nm\nu\nC a\nlt\nt\ns\ng e a d\nfr\np\no\nv\nm\nr ia at\n1\nio\n5\nis the\nIP A C R B y rts e c n\nA\noitar pag C U\nA\nC U A C O R ytirapsid RPT 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 𝑃<0.001 A A\nA\nn n\nv\nM c c\ne\nW e e U\nra\nT s s T\ng\nt t r r\ne\ny y 0 0\n0\n.\n.\n.\n1 1\n1\n1 7\n3\n9\n4\n9 0 0\n0\n.\n.\n.\n1 1\n1\n2 8\n3\n6 1\n3 0\n0 0\n.\n.\n.\n1\n1 1 1 6\n3\n1 7\n2\np S i\nS\nn m\nt\ne d\na\nr a i f\nt\nc o\ni\nl\ns\na l r e\nt\nt m r\ni\ne\nc\ns a d\na\nn o\nl\nb c t\ns\ne e s\ni\nt\ng\nt r d\nn\ne e i r\ni\np ff\nfi\nr e f\nc\na e r\na\ns i e r\nn\ne n n n\nc\nc e\ne\nt e s i\n(\ns b n\n*\n. d e\np\nM t i w v\n<\na i e d r e\n0\nu g n\n.\ni a\n0\nn l\n5\nt a h\n,\nf l o e\n*\nl d\n*\nd b i\np\ns s e ; t s\n<\nr l t i a - b r\n0\nu a g\n.0\nn t e i\n1\nd o d\n)\nn o w\nf\ns t\nr\no s\no\ns r h\nm\nw s o t i - w t\na\np h e\np\nt r e h\na\nf r o e\ni\nr\nr\nr o\ne\nd m r\nd\nis i b n t\nW\na r g i r b\ni\ns\nl\ns u\nc\nu s\no\nt h b\nx\ni o o g\no\nw n r\nn\no o u t\ns\nh f p\nig\ne A s\nn\nm U r\ne\ne\nd\nC e l\n-\na a\nr\nt n g\na\niv a\nn\na e p\nk\nn t d r\nt\no a\ne\n9 t\ns\nt i\nt\n5 h o %\nd\ne s\ne\na o C\nn\nc v\no\nI r e .\nt\no r\ne\nA s a\ns\ns ll\nt\nl f o\nh\nA o w l\ne\nU d e s C r . . ratio\nROC AUC WT MUT improvement of FLEX over the Original method. b, The AUROC distribution across subgroups. Boxes\nindicate the interquartile range of the AUROC distribution (n=15 folds), with the center being the 50th\npercentile and the whiskers extending to data points within 1.5× the interquartile range. Tighter\ndistributions indicate more equitable performance. c, Violin plots of True Positive Rate (TPR) disparity,\ndefined as the difference between a subgroup’s TPR and the overall TPR. Distributions centered closer to\nthe zero-disparity line indicate higher fairness. P-values from the paired t-test are shown. The dashed line\nrepresents the overall TPR across three methods. d, Quantitative summary of TPR disparity, showing the\nRoot Mean Square Error (RMSE) for Original (O), Reinhard (R), and FLEX (F). Lower values indicate\nbetter fairness.\n28\nmanuscript to guide the reader through this multi-faceted interpretation to avoid any misimpression and to\nclarify the robust reliability of our method.\nComment 8\nWhiletheproposedmethodiscomparedagainstbaselineandthestandardReinhardstainnormalization\nmethod in Figure 3, no statistical tests (e.g., paired t-tests or Wilcoxon signed-rank tests) are reported\nto determine whether the observed performance differences are significant. The observed differences in\nresults between methods appear minimal at times.\nResponse:\nThank you for this crucial suggestion. In our revised manuscript, we have performed a comprehensive\nstatistical analysis, with results presented in the updated Figure 3 (Fig. R12). These tests underscore the\nsignificance of our method’s improvements.\nFor the AUC Gap Ratio, we applied the non-parametric paired Wilcoxon signed-rank test. As indicated\nby the newly added significance markers in Fig. R12a, FLEX achieves a statistically significant reduction\nin the fairness gap in multiple key tasks, including NSCLC-TYPE and BRCA-PR. This finding also holds\nfor the external CPTAC dataset (e.g., C-BRCA-TYPE and C-LUAD-EGFR), confirming that the fairness\nimprovements delivered by FLEX are robust and generalizable.\nFor the TPR Disparity, paired t-tests similarly confirm that FLEX reduces disparity in most tasks, with\neliminated significant differences between different groups on several individual tasks, such as NSCLC-TYPE\nand BRCA-PR.\nFurthermore, we have added comprehensive statistical tests to other analyses, such as the AUROC distribu-\ntions for each task (Fig. R11c, d, e) and performance gains across different MIL model variants (Fig. R14c).\nTheseanalysesprovideasolidfoundationfortheclaimsmadeinourmanuscript,demonstratingtherobustness\nof FLEX’s improvements.\nComment 9\nIn Figure 4b, UMAP plots for the features obtained from the proposed methods still show substantial\noverlap between class labels across sites. The visual evidence does not compellingly support the claim\nof improved separability of diagnostic categories.\nResponse:\nThank you for this valuable feedback. To provide a more objective evaluation, we have augmented Figure 4b\n(now Fig. R13) with quantitative metrics that directly measure FLEX’s two key objectives: harmonizing\nfeatures across sites and improving the separability of diagnostic classes.\nFirst, to quantify the removal of site-specific batch effects, we calculated the Local Inverse Simpson’s Index\n(LISI). The results demonstrate that FLEX increases the LISI score by 68% for PathGen-CLIP (from 1.75\nto 2.94) and 45% for CONCH (from 2.22 to 3.22), providing quantitative proof of a significantly more\nharmonized feature space.\nSecond, to objectively measure class separability, we used the Silhouette Score. The score for CONCH more\nthan doubles from 0.09 to 0.24, and other models also show consistent positive gains.\nWhile some visual overlap is expected in such challenging classification tasks, the LISI and Silhouette scores\ndemonstrate that FLEX systematically refines the underlying feature geometry to achieve both better site\nintegration and enhanced class separability. We have updated the manuscript and figure caption to include\nthis analysis, making our conclusions more objectively verifiable.\nComment 10\nIn Figure 5, performance improvements across different data scales and MIL model variants are\nrelatively small, sometimes between 0.000 and 0.005 in AUROC. The authors could discuss the\npractical benefits of integrating the proposed methods into real-world pathology diagnosis.\nLISI:3.22 (2.95-3.49) Silhouette Score :0.24 (0.19-0.29) LISI:2.94 (2.65-3.23) Silhouette Score :0.01 (-0.04-0.06) LISI:2.81 (2.56-3.04) Silhouette Score :0.02 (-0.02-0.05)\nlanigirO\nXELF\nUMAPvisualization\nCONCH PathGen-CLIP QuiltNet\nVQ HU FP R5 CG EQ D7 CD 3M IN MX F1 HF ZA KB ZQ B7 HJ BR RD SW EBVNegative EBV Positive\nFigure R13: (Figure 4b in manuscript) UMAP visualizations illustrating the effect of FLEX on the\npatch feature space for the STAD EBV task. For each VLM, parallel subplots are colored by site (left) to\nvisualize batch effect mitigation, and by EBV status (right) to visualize class separability. The Local Inverse\nSimpson’s Index (LISI) for site integration (higher is better) and the Silhouette Score for class separation\n(higher is better) provide quantitative evidence. The visualizations and scores demonstrate that FLEX\nreduces site-specific clustering while improving the discriminability of task-relevant classes.\nResponse:\nThank you for this important point. We have strengthened our manuscript to address the practical benefits\nof FLEX through two key lines of evidence: statistical robustness and superior generalization to unseen\nclinical data.\nFirst, we established the statistical significance of these gains First, we established the statistical significance\nof these gains within the TCGA dataset. Our learning curve analysis (Fig. R14b) shows that after correcting\nfor multiple comparisons, the improvements from FLEX are statistically significant across all tested data\nscales (q < 0.05), proving the gains are reliable. This benefit is also universal across architectures. As\nvisualized in Fig. R14c, FLEX consistently and significantly improves performance across five different MIL\nmodels, with a highly significant average improvement for every model (q <0.001). This confirms FLEX\nacts as a robust and broadly beneficial module.\nHowever,themostcrucialmeasureofpracticalbenefitliesinamodel’sabilitytogeneralizetonew,real-world\ndata. To this end, we conducted extensive zero-shot validation on two independent external cohorts: the\npublic CPTAC dataset (n=1,738 slides) and our in-house NFH dataset (n=5,318 slides). The results\n(Fig. R15) demonstrate that models trained with FLEX on TCGA data show a markedly superior ability to\ngeneralize. The average AUROC across all tasks significantly improved on both external cohorts (p<0.001,\nFig. R15a), and task-specific results show consistent outperformance (Fig. R15b,c, and d). This result is\ncritical, as it demonstrates that FLEX effectively mitigates dataset-specific biases, addressing a primary\nhurdle for deploying AI in real-world diagnostic settings.\nIn summary, the practical benefit of FLEX is twofold: it delivers a performance boost that is consistent and\nstatistically significant, and more importantly, it enhances generalization to unseen clinical data. We have\nincorporated this discussion into the manuscript to highlight the practical relevance of our method.\n30\nb\nNumber of IND Splits Number of IND Splits\nCORUA\nDifferentscalesoftrainingdata\nBRCA-TYPE BRCA-HER2 BRCA-PIK3CA\nq < 0.05\nq < 0.01\nq < 0.001\nFLEX\nOriginal\nNumber of IND Splits\nc\n1.0\nABMIL\nACMIL\nCLAM-SB\n0.9\nDTFD-MIL\nILRA-MIL\n0.8\n0.7\n0.6\nBRCA-TYP\nN\nE SCLC-TY\nS\nP\nT\nE AD-LAUREN BRCA-HER2 BRCA-ER BRCA-PR STAD-EBV STAD-M\nB\nS\nR\nI CA-PIK3CA BRCA-CDH1 LUAD-EGFR LUAD-STK11 STAD-TP5\nS\nCORUA\na\nOuterFold1 OuterFold2 OuterFold3\nSite 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 Site 8\nSite-preserved\nSplit IND IND\nSplits Splits …\n1IND Split 1OOD Split\n2IND Splits\nLabel A 3IND Splits OOD OOD\nSplit Split\nLabel B 4IND Splits\nDifferentMILmodels\nOriginal\nFLEX\nq<0.05\nq<0.01\nq<0.001\nFigure R14: (Figure 5 in manuscript) Performance of FLEX across training data scales and\nMIL models. a, Schematic of the experimental design for evaluating performance at different training data\nscales. The procedure follows our main Site-Preserved Monte Carlo Cross-Validation (SP-MCCV) strategy,\nwhich involves 5 outer folds and 3 inner Monte Carlo Cross-Validation (MCCV) folds, for a total of 15\nevaluation runs. Specifically for this experiment, the dataset is first partitioned into 5 site-preserved outer\nfolds. In each of the 5 outer loop iterations, one fold is held out as the OOD test set. Training sets of\nincreasing size (1, 2, 3, and 4 folds) are constructed from the remaining 4 folds to create a learning curve.\nFor each scale condition, 3 inner Monte Carlo Cross-Validation (MCCV) runs are performed to ensure stable\nresults, for a total of 15 evaluation runs per condition. b, OOD AUROC comparison between FLEX and\nbaseline (Original). Box plots show aggregated results from 15 runs for each training data scale.\nAsterisks indicate statistically significant improvement (Benjamini-Hochberg adjusted q-value: *< 0.05, **< 0.01,\n***< 0.001). Box plots show Q1–Q3 interval with the median; whiskers are 1.5× interquartile range. c,\nDumbbell plot showing OOD AUROC after integrating FLEX with five state-of-the-art MIL models across\n16 tasks. FLEX consistently improves performance across different MIL architectures.\nC-NSCLC-TYPE IDC vs ILC N-CRC-BRAF\nLUAD vs LUSC WT vs MUT\nTCGA Training FLEX E T x e t s e t r i n n a g l C N P F TA H C C-B W RC T v A s - P M I U K T 3CA N-L W U T A v D s M -E U G T FR\nC-BRCA-CDH1 N-BRCA-PR WT vs MUT Pos vs Neg\nC-LUAD-EGFR N-BRCA-ER\nWT vs MUT Pos vs Neg\nC-LUAD-STK11 N-BRCA-HER2\nWT vs MUT Pos vs Neg\nC-COAD-BRAF N-STAD-LAUREN\nWT vs MUT Intestinal vs Diffuse\nq < 0.05\nq q < < 0 0 . . 0 0 1 01 C- W CO T v A s D M -T U P T 53 N- I B D R C C v A s - I T L Y C PE N L - U N A S D C v L s C L - U T S Y C PE\nOriginal Reinhard FLEX CPTAC NFH\nc\nCPTAC\nOriginal\nReinhard\nFLEX\nq < 0.05\nq < 0.01\nq < 0.001\nC-BRCA-TYPE C-NSCLC-TYPE C-BRCA-PIK3CA C-BRCA-CDH1 C-LUAD-EGFR C-LUAD-STK11 C-COAD-BRAF C-COAD-TP53\nd\nNFH\nOriginal\nReinhard\nFLEX\nq < 0.05\nq < 0.01\nq < 0.001\nN-BRCA-TYPE N-NSCLC-TYPE N-STAD-LAUREN N-BRCA-HER2 N-BRCA-ER N-BRCA-PR N-LUAD-EGFR N-CRC-BRAF\n976.0 976.0 417.0\n0.72\n0.70\n0.68\n0.66\nO R F\n337.0 037.0 177.0\n0.78\n0.76\n0.74\n0.72\n0.70\nO R F\nCUA CUA\na b\nCPTAC Public\nCPTAC-NSCLC\nn=1,198\nCPTAC-BRCA\nn=323\nCPTAC-COAD CPTAC NFH\nn=217\np<0.001 p<0.001\nNFH\nIn-house\nNFH-NSCLC\nn=2,493\nNFH-BRCA\nn=882\nNFH-STAD\nn=395\nNFH-CRC\nn=1,548\nO Original R Reinhard F FLEX\nFigure R15: Zero-shot generalization of TCGA-trained models to external CPTAC and\nin-house NFH cohorts. Models were trained exclusively on TCGA data and evaluated directly on two\nindependent external datasets without fine-tuning. a, Overall experimental design and results. Bar plots\ndisplay the average AUROC across all tested tasks on the CPTAC (left) and in-house NFH (right) datasets.\nFLEX significantly improves the average performance of the TCGA-trained model on both external cohorts\n(p<0.001, paired t-test). “O”, “R”, and “F” denote Original, Reinhard, and FLEX, respectively. b, Radar\nplot comparing task-specific AUROC across models on CPTAC and NFH data. FLEX consistently expands\nthe performance radius across tasks. Asterisks indicate statistical significance of the improvement by FLEX\ncompared to the original model (*q <0.05, **q <0.01, ***q <0.001, Benjamini-Hochberg FDR correction).\nc, d, Detailed box plots showing task-by-task performance on the (c) CPTAC and (d) NFH cohorts. FLEX\nconsistently outperforms or matches the baseline and Reinhard-normalized models, demonstrating its robust\nability to enhance zero-shot transfer learning by mitigating dataset-specific biases. Boxes indicate quartile\nvalues of the AUROC distribution (n=15 folds), with the center line indicating the median and whiskers\nextending to 1.5× the interquartile range.\nKnowledge-Guided Adaptation of Pathology\nFoundation Models Effectively Improves Cross-domain\nGeneralization and Demographic Fairness\nAddressed Comments for Publication to\nNature Communications\nAuthors’ Response to Reviewer 2\nComment 1\nRemarks to the Author: The additional analysis of two large datasets and subsequent results and\nmanuscript updates largely address my concerns.\nResponse:\nThank you very much for your positive feedback and for acknowledging that our revisions have successfully\naddressed your concerns.\nComment 2\nRemarks on code availability: The code for the initial submission is included but code relating to\nresubmission updates is missing from the repository. The code is well organized and the repository\nhas a README with instructions to run the code.\nResponse:\nThank you for your careful examination of our code repository and for your positive comments on its\norganization. To make the codebase clearer for future users and to ensure full reproducibility, we have\nupdated the README.md file in our repository. This updated documentation now explicitly details the\nsteps and parameters required to replicate the results using existing code.\nAuthors’ Response to Reviewer 3\nComment 1\nRemarks to the Author: The authors have addressed the previously raised comments.\nResponse:\nWesincerelyappreciateyourconfirmationandarehappytoknowthatourrevisionshavemetyourexpectations.\nComment 2\nRemarks on code availability: The authors provided a README file with instructions for installing\nand running the application.\nResponse:\nWe appreciate your positive comment on the usability and documentation of our code.\n2"}, {"round": 2, "reviewer_comments": {"reviewer_2": "The additional analysis of two large datasets and subsequent results and manuscript updates largely address my concerns.", "reviewer_3": "The authors have addressed the previously raised comments. Thank you."}, "author_response": "Responses to Reviewers’ Comments for Manuscript NCOMMS-25-16314A\nKnowledge-Guided Adaptation of Pathology\nFoundation Models Effectively Improves Cross-domain\nGeneralization and Demographic Fairness\nAddressed Comments for Publication to\nNature Communications\nAuthors’ Response to Reviewer 2\nComment 1\nRemarks to the Author: The additional analysis of two large datasets and subsequent results and\nmanuscript updates largely address my concerns.\nResponse:\nThank you very much for your positive feedback and for acknowledging that our revisions have successfully\naddressed your concerns.\nComment 2\nRemarks on code availability: The code for the initial submission is included but code relating to\nresubmission updates is missing from the repository. The code is well organized and the repository\nhas a README with instructions to run the code.\nResponse:\nThank you for your careful examination of our code repository and for your positive comments on its\norganization. To make the codebase clearer for future users and to ensure full reproducibility, we have\nupdated the README.md file in our repository. This updated documentation now explicitly details the\nsteps and parameters required to replicate the results using existing code.\nAuthors’ Response to Reviewer 3\nComment 1\nRemarks to the Author: The authors have addressed the previously raised comments.\nResponse:\nWesincerelyappreciateyourconfirmationandarehappytoknowthatourrevisionshavemetyourexpectations.\nComment 2\nRemarks on code availability: The authors provided a README file with instructions for installing\nand running the application.\nResponse:\nWe appreciate your positive comment on the usability and documentation of our code.\n2"}]

Decision letter