Current Pathology Foundation Models are unrobust to Medical Center
  Differences
Authors: name: Edwin D. de Jong, name: Eric Marcus, name: Jonas Teuwen

Pathology Foundation Models (FMs) hold great promise for healthcare. Before
they can be used in clinical practice, it is essential to ensure they are
robust to variations between medical centers. We measure whether pathology FMs
focus on biological features like tissue and cancer type, or on the well known
confounding medical center signatures introduced by staining procedure and
other differences. We introduce the Robustness Index. This novel robustness
metric reflects to what degree biological features dominate confounding
features. Ten current publicly available pathology FMs are evaluated. We find
that all current pathology foundation models evaluated represent the medical
center to a strong degree. Significant differences in the robustness index are
observed. Only one model so far has a robustness index greater than one,
meaning biological features dominate confounding features, but only slightly. A
quantitative approach to measure the influence of medical center differences on
FM-based prediction performance is described. We analyze the impact of
unrobustness on classification performance of downstream models, and find that
cancer-type classification errors are not random, but specifically attributable
to same-center confounders: images of other classes from the same medical
center. We visualize FM embedding spaces, and find these are more strongly
organized by medical centers than by biological factors. As a consequence, the
medical center of origin is predicted more accurately than the tissue source
and cancer type. The robustness index introduced here is provided with the aim
of advancing progress towards clinical adoption of robust and reliable
pathology FMs.