Explainable AI-based analysis of human pancreas sections identifies traits of type 2 diabetes

Domain: neuroscience
Published: 2026-02-09
Editorial outcome: minor_revision

Abstract
Type2diabetes(T2D)isachronicdiseasecurrentlyaffectingaround500
millionpeopleworldwidewithoftenseverehealthconsequences.Yet,histo-
pathologicalanalysesarestillinadequatetoinfertheglycaemicstateofa
personbasedonmorphologicalalterationslinkedtoimpairedinsulinsecre-
tionandβ-cellfailureinT2D.Giga-pixelmicroscopycancapturesubtlemor-
phologicalchanges,butdatacomplexityexceedshumananalysiscapabilities.
Inresponse,wegenerateadatasetofpancreaswhole-slideimagesfromliving
donorswithmultiplechromogenicandmultipleximmunofluorescencestain-
ingsandtraindeeplearningmodelstopredicttheT2Dstatus.Usingexplain-
ableAI,wemakethelearnedrelationshipsinterpretable,quantifythemas
biomarkers,andassesstheirassociationwithT2D.Remarkably,thehighest
predictionperformanceisachievedbysimultaneouslyfocusingonisletα-and
δ-cellsandneuronalaxons,alongsidesubtlepancreaticalterationsinT2D
donorssuchaslargeradipocyteclusters,alteredislet-adipocyteproximityand
smallerislets.Thisdata-drivenapproachprovidesafoundationforfuture
researchintorelevantdiagnosticandtherapeutictargets,refiningseveral
hypothesesregardingtissuealterationsassociatedwithT2D.

Introduction
BasedonthecurrentWHOclassificationofdiabetes,over90%ofall
personswiththeconditionfallintothecategorydefinedastype2
diabetes(T2D).T2Disamajorglobalhealthissue,affectingmillions
andplacingasignificantburdenonhealthcaresystems1.Pathophy-
siologicdriversofT2Dareinsulinresistanceandimpairedinsulin
secretion,withdysfunctionofpancreaticisletβ-cellsbeingthekey
feature.Despiteextensiveresearch,theexactetiologyandnatureofβ-
cellfailureinthepathophysiologyofT2Dremainunclear.β-cellsreside
inspecializedmicro-organs,theisletsofLangerhans,withinthepan-
creas,butaccessinghumanpancreatictissueinlivingsubjectsis
challengingduetoanatomicalconstraintsandtherisksassociatedwith
pancreaticbiopsyortissueresection2.Consequently,theidentification
ofmorphologicalisletchangesassociatedwithT2Dhasreliedmainly
onthehistopathologicalanalysisofspecimensfromdeceaseddonors.
Suchstudiespointedtoseveralstructuralalterations,suchasamyloid
deposition3,4,reducedβ-cellmasswithdecreasedβ-cell/α-cellratio5,6,
increasednumberofisletresidentmacrophages7,8andfibrosis9,10,but
several of these findings remain controversial11–13. Given the high
degreeofinter-individualvariability,noneofthesetraitsissufficient
forpathologiststodiscriminatewithahighdegreeofcertaintywhe-
therapancreaticspecimenbelongstoasubjectwithorwithoutT2D.
Whilesuchdiscriminativepowerwouldnothavediagnosticrelevance,
itcouldpavethewaytouncoveryetunknownstructuralhallmarksof
thediseaseandtherebyprovidefurtherinsightintoitspathogenesis.
Thisknowledgecould,inturn,beleveragedtoimprovetheprevention
andtreatmentofdiabetes.
Recent advances in Machine Learning (ML) have garnered sig-
nificantattentioninthefieldofcomputationalpathologyduetotheir
abilitytoanalyzelargevolumesofhigh-dimensionalwhole-slideima-
ges (WSIs) with efficacy comparable to expert pathologists14. For
example,deeplearning(DL)technologiesareemployedforprostate
cancer grading14, cell segmentation15, gene expression prediction16,
andthediscoveryofnewmorphologicalbiomarkersinbreastcancer
histopathology17. However, most DL applications in histopathology
focusononcologicaldiseases,wheredistinctcancercellsareidenti-
fiable in tissue samples. In contrast, pancreatic tissue lacks clear
quantifiable features that unequivocally differ between individuals
withandwithoutT2D.
Hence,ourgoalwas the identificationofhistologic biomarkers
allowingustodistinguishapancreaticspecimenfromapatientwithor
without T2D and thus gain novel insights into disease mechanisms
linkedto,orarisingfrom,morphologicalalterationsofthepancreatic
tissue.Forthatpurpose,weintroducedadata-drivenapproachcom-
biningstate-of-the-artDLwithexplainableartificialintelligence(XAI).
XAIisoftenemployedtoenhancemodeltrustworthinesstoverifythat
amodelismakingdecisionsbasedonrelevantfeatures,however,this
workleveragesXAIforscientificdiscoverypurposes.WhileDLmodels
couldfindpatternsindatathatareinfeasibletodetectbytraditional
approaches, XAI methods subsequently rendered the learned rela-
tionshipshuman-understandable,revealingregionsofinterest(ROIs)
associatedwiththepresenceofT2D.Giventhelimitedfeasibilityof
manual analyses of large amounts of XAI results, we quantified the
attentiontotheROIsandsubsequentlycomputedspecifichistologic
biomarkersforthemostimportantones.Atlast,weanalyzedthese
biomarkersincombinationwithclinicalpatientdatausingstatistical
models.BesidestheintegrationofvastamountsofdiverseWSIs,our
approachofferedtheadvantagethatwedidnotbiasourselvestoprior
assumptionsregardingT2D,astheXAIapplicationcouldalsouncover
unanticipatedfindings,leadingtotheformulationofnewhypotheses
aboutthecondition.

Results
AuniquedatasetofIHCandmIFstainedpancreassections
DLmodelsthatcanreliablydistinguishpatientswithT2Dfromother
patientsareaprerequisiteforgainingnewinsightsintoT2Dthrough
XAImethods.ByuncoveringthepatternsusedbytheDLmodelsfor
prediction,weaimforabetterunderstandingofthedisease.
Tothisend,wefirstcollectedadatasetbyapplyingbothsingle
chromogenic immunohistochemistry (IHC) and multiplex immuno-
fluorescence(mIF)techniquestoimmunostainpancreassectionsfrom
100metabolicallyphenotypedlivingdonorswith(n=35)orwithout
(n=65)T2Dwhounderwentpancreatectomyforvariouspancreatic
disorders18–22attwoacademiccenters.Sectionswereimmunostained
forglucagon,insulin,andsomatostatinasmarkersofisletα-,β-andδ-
cells,respectively,aswellasPECAM1forendothelialcellstovisualize
islet vascularization, perilipin 1 for adipocytes to assess intra-
pancreatic steatosis and tubulin beta 3 for neuronal axons. In the
caseofIHCstainings,eachmarkerwasdetectedindividuallyinserial
sectionscounterstainedwithhematoxylin.InthecaseofmIFstainings,
serialsectionswereco-labeledforseveralmarkers,includingDAPIand
primaryantibodieseitheragainstglucagon,somatostatin,andtubulin
beta 3 (Staining set 1), or against insulin, PECAM1, and perilipin 1
(Stainingset2)(Fig.1Aand“Method”Section).Overall,eightimages
perpatientwereacquired.Thedatasetispubliclyavailable.

Classificationofdiabetesfromwhole-slideimagesofpancreatic
tissuesections
Subsequentbrightfieldandfluorescenceacquiredwholeslideimages
(WSIs) were used to train DL models distinguishing donors with or
withoutT2D(Fig.1b).Patientsweresplitintoatrainingset(n=75)and
a test set (n = 25), with the test set used only for the final model
evaluation.Duringmodeldevelopment,weemployedacrossvalida-
tionwhere15differenttraining(n=60each)andvalidation(n=15
each)splitswerecreatedtoavoidoverfittingonsinglevalidationsets.
Finaltestsetpredictionswereobtainedbyensemblingthe15indivi-
dualmodels,i.e.,averagingtheindividualpredictions.
WetrainedDLmodelsforeachofthesixIHCstainings,whileon
themIFdata,wetrainedDLmodelsforeachofthetwostainingsets
(Fig.2).EachofourDLmodelsconsistedoftwocomponents,thepre-
trainedfeatureextractor,andtheMultipleInstanceLearning(MIL)23
classifier.Generally,duetotheirsize,eachWSIwasfirstdividedinto
smaller-sizedsquares(patches)andeachpatchwasencodedbythe
featureextractor.Subsequently,allencodingsbelongingtooneWSI
werecombinedandfedtotheMILclassifier,whichlearnedtodistin-
guishbetweenT2Dandnon-diabeticpatientsbasedonthecombined
patch encodings. By doing so, not only small local features from
individualpatchesbutalsolargerstructuresspanningmultiplepatches
wereavailabletothemodel.
We compared the classification performance for different pre-
training approaches on the brightfield WSIs. The ImageNet21k pre-
trained24VisionTransformerasfeatureextractorcombinedwiththe
CLAM model architecture25 as classifier delivered the best average
predictionperformance(AUROC=0.833;Fig.2aandSupplementary
Table 1). Across all stainings except for PECAM1 and somatostatin,
therewasasignificantincreaseinAUROCwhenusingtheImageNet21k
pre-trained encoder. Of note, the IHC staining resulting in the best
predictionperformancewastubulinbeta3(AUROC=0.895),whilethe
three islet-typical labels, i.e., insulin, glucagon, and somatostatin
showed similar AUROC values (AUROC = 0.842;Fig. 2a and Supple-
mentaryTable1).
On the mIF WSIs, we compared three different color-channel
representations: “RGB”, “channel-wise”, and “channel-wise average”
(seesubsection4.3).Inshort,“RGB”treatedthethreestainingsina
staining set as rgb color channels processing all channels simulta-
neously,whilethetwo“channel-wise”representationsprocessedeach
channel individually and either appended the resulting encodings
(“channel-wise”)oraveragedthem(“channel-wiseaverage”).Thelatter
benefitedfromco-occurrenceinformationreflectingthespatialrela-
tionships between different stainings. When ImageNet21k/CLAM-
basedpredictionwasconductedusingmIFWSIs,Stainingset1,which
labeledα-cells,δ-cells,andneuronalaxons,yieldedsignificantlybetter
classificationresultsthanStainingset2.Thebestdiagnosticperfor-
mancewasreachedusingthe“channel-wiseaverage”representation
onStainingset1(EnsembleAUROC=0.956;Fig.2bandSupplementary
Table1).

AImodelsattendtospecificbiologicaltraits
Upon completion of model training, we aimed to understand the
biologicalfeaturesutilizedbythemodelsinpredictingT2D.Forthis
purpose,weemployedXAItechniquesspecificallywithinthedomains
of attention26 and attribution27–29 methods,to highlight regions and
featurescriticaltothemodelswhenpredictingtheT2Dstatus(Fig.1c).
Attributionmethodsingeneraldeterminehowmucheachinputfea-
tureattributestoamodel’soutput(i.e.,howimportanteachfeatureis
in a prediction) through e.g., gradients or perturbations. Attention
methods,asubsetofattributionmethods,leveragetheoutputofthe
attention module used e.g., in the transformer architecture30. Raw
attentionoutputsindicatefeaturesofgeneralimportancetothemodel
but lack a direct connection to specific outputs, requiring filtering
(e.g.,viagradientsorrelevancescoring)tohighlightfeaturesusedto
predictaspecificoutcome.SincetheregionshighlightedbytheXAI
methodsreflectthefeaturesdeemedimportantbythemodelforthe
predictiontask, we consistentlyapplied thesemethods to the best-
performing model for both mIF and IHC modalities. While these
methods show what regions in the WSI are important, they do not
showwhytheyareimportant.Tothisend,weconductedasystematic
biologicalinterpretationoftheXAIresultsthroughstructuredanalysis
offeatureattributions,biomarkerdeterminationandcross-validation
withestablishedbiologicalliterature.
Initially, we determined the significance of more global ROIs
within the tissues before focusing on local features within these
regions.Tothisend,weusethebuilt-inattentionmechanismofthe
MILclassifieronthespecificpatches,basedonwhichwecreateheat-
maps for the individual IHC and mIF WSIs of each patient (see
exemplaryFig.3).Westructuredthesubsequentresultsaccordingto
bothimmunostainingtechniques.

Quantifyingattentiontoregions-of-interest
Toquantitativelyassessthegeneralimportanceofspecificregions
acrossallheatmaps,weimplementedascalable,data-drivenmeth-
odology.Weexaminedhighlyattendedpatchestodeterminetheir
corresponding tissue structures. Those recurring were defined as
ROIsandsegmentedintheWSIusingaspecificallytrainedDLseg-
mentationmodeltoenablequantitativeanalysisoftheattentionto
different relevant tissue features.TheattentionscoreswithinROIs
were standardized for tissue size and total amount of attention.
However,resultsfromIHCandmIFstainingmodalitiesshouldnotbe
directly compared. The IHC models analyzed each staining inde-
pendently,whilemIFmodelsevaluatedallstainingswithinastaining
setsimultaneously.
IHCandmIFstainingsalsodifferintheirdynamicranges(1–2vs.
3–5 order of magnitude, respectively), and thus in regard to their
sensitivityandsaturationofthesignal.Moreover,theoptimizationof
themIFstainingsrequiredtheuseofantibodiesthatinseveralcases
differedfromthoseusedforIHCdetectionofsingleantigens.

IHC.TheanalysisofattentionscoresinFig.4isshownforeachspecific
stainingoftherespectiveROIsandsummarizedforthenon-specific
stainingsasamorerobustcomparisonbaseline.Itrevealedahetero-
geneousdistributionofattentiontowardstissuecomponentssuchas
islets, adipocytes, or connective tissue in the individual IHC WSIs.
Notably, islets exhibited the highest average attention scores for
insulin- and glucagon-stained WSIs, irrespective of diabetes status
(Fig.4a).Attentiontoadipocyteswassubstantiallyelevatedinperilipin
1-stainedWSIs,whichspecificallyhighlightadipocytes,comparedto
other non-specific stainings (Fig. 4b).Furthermore, the attention to
adipocyteswassignificantlyhigherinnon-diabeticpatientsforperili-
pin1-stainedWSIs.Incontrast,forconnectivetissue,whichwasnever
specifically stained, attention was significantly higher in T2D
patients(Fig.4c).

mIF.InthemIFWSIs,totalattentiontoselectedtissue,i.e.,allpatches
containingstainedtissue,differedbetweentheindividualstainingsof
thetwostainingsets,whilethemodelsattributedsignificantlyhigher
attentiontoglucagon,tubulinbeta3,perilipin1,andPECAM1inthe
WSIsofT2Ddonors(Fig.5a).Asmostofthehigherattendedsignals
alsolabelextra-isletstructures,wenextdeterminedtheratiobetween
attentiontoislets(Fig.5b)andtotalattentiontoselectedtissue,which
inthecaseofglucagon,insulin,andsomatostatinstainingscomprises
onlythepatchescontainingislets.Theratioshowsthat,comparedto
thesurroundingtissue,isletsareattendedaboveaveragefortubulin
beta3andPECAM1andbelowaverageforperilipin1staining(Fig.5c).
Interestingly,attention to perilipin1wassignificantlyhigher in T2D
patients,contrastingwiththefindingsfromIHCanalyses.Thisobser-
vationsuggestsapotentiallycriticalroleofadipocytesinT2Dpatho-
physiology,warrantingfurtherinvestigation.Strikingly,intheStaining
set2boththeattentiontoisletsandtheislet-to-tissueattentionratio
werehighestforDAPI,thenucleardye,suggestingthatT2Dpathology
isassociatedwithaglobalalterationofisletmorphology(Fig.5b,c).

Unbiasedidentificationandcomputationofhistologic
biomarkers
WhiletheglobalheatmapsfacilitatethevisualizationoflargerROIsin
the tissue, they do not provide detailed insights into the finer
biological features that the model considers significant. However,
muchmoredetailedvisualizationsarenecessarytoformulatespecific
hypotheses and subsequently biomarkers based on the features or
regionsthemodelattendsto.Theprocessofidentifyingbiomarkersis
largely unbiased, as the model, without prior knowledge, learns to
classifypatientssolelyfromtheimages,potentiallyuncoveringnovel
histologicalfeaturesbeyondpredefinedhypotheses.
Wefirstsampledthetopattendedregionsusingattentionscores
(see exemplary Supplementary Fig. 1), essentially zooming in from
image-leveltopixel-level.Tofurtherdeterminethespecificpixel-level
featuresutilizedbythemodelineachofthetop-attendedregions,we
employedmultipleattributionmethodstoquantifythecontributionof
individualpixelstothepredictedoutcomes,astheattentionmodules
onlyoperateonthepatchlevel.Theheatmapsconfirmthatthemodel
wasindeedabletorecognizebiologicaltraits,suchasnuclei,adipocyte
cytosolic compartments, inter-cellular structures, or islets (Supple-
mentaryFig.2).
Usingtheglobalandlocalheatmaps,weselectedhistologicbio-
markers potentially representing key characteristics of the features
attendedtobythemodelindistinguishingbetweenT2Dandnon-T2D
patients.Figure6providesselectedregionsfromIHCandmIFWSIs
withhistologicbiomarkersandthecorrespondingpixel-levelattribu-
tion heatmaps derived from these analyses. Specifically, Fig. 6a, d
reveal consistently high attention to the stained islets across both
immunostaining techniques, suggesting that the islets possess fea-
tures essential to T2D pathology, besides their number and their
averagesize.Ofnote,Fig.6dunderscorestherelevanceofneuronal-
axonalstructures,stainedbytubulinbeta3,oftenattendedespecially
within the islets.Furthermore, Fig. 6b highlights the significanceof
adipocytes,whereasFig.6cemphasizestheimportanceofconnective
orfibrotictissue.BasedontheseROIstotheDLmodel,wedefinedand
computedseveralhistologicbiomarkerstovalidatewhytheseregions
areimportanttothemodel.

Exploratoryanalysisofthehistologicbiomarkers.Thedefinedhis-
tologic biomarkers for the IHC and mIF modalities were computed
throughthequantitativeXAIresultsandthesegmentationmapsofthe
ROIs(Fig.1d).Notably,thesebiomarkersexhibitedvariationsbetween
bothmodalitiesascertainROIscouldnotbecomputedformIFWSIs,
e.g.,connectivetissueduetotheabsenceofadedicatedstaining.All
histologicbiomarkerswerestandardizedfortissuesize.
AfirstexplorativeanalysisofthehistologicbiomarkersacrossIHC
stainingsrevealedthatpersonswithoutdiabetesgenerallyexhibited
largerislets(Fig.7a).Inthecaseofperilipin1,thesegmentationofthe
islets was more challenging due to a lack of islet-specific staining,
resultinginconsiderablyfewerdetectedislets.Further,patientswith
T2Dtendedtohavelargeradipocyteclusters(Fig.7b)andconnective
tissue-enriched areas (Fig. 7c). An adipocyte cluster was defined as
eitheranaccumulationofadipocytesorasingleadipocyteinthetis-
sue.Whenassessingtheaverageminimumdistancebetweeneachislet
and the nearest adipocyte, computed only on the insulin staining
segmentations,weobservedthatinpatientswithT2D,theisletswere
significantlyclosertoadipocytesthaninpatientswithoutT2D(Fig.7d).

Statisticalanalysisrevealsasignificantassociationofhistologic
biomarkerswithdiabetesstatusandinsulinsecretion
Tostatisticallyassesstheimpactofthehistologicbiomarkersondia-
betes status and insulin secretion, measured through HOMA2B (c-
peptide),weconsideredseveralcontrolvariables,whichwereeither
directlypertainingtoindividualdonortraits,includingsex,age,body
massindex(BMI),underlyingpancreatictumor,chronicpancreatitis,
andinsulintherapy;orexperimentalconditions,includingcohort,and
tissuestaining(Fig.1d).Weanalyzedthedatausinggeneralizedmixed
linearmodels(GMLMs),accountingforrandomeffectslinkedtothe
cohortandstainingmethod.

IHC.FortheIHCstainings,regressionanalysisrevealedasignificant
positiveassociationofbothadipocyteclusterareas(0.266;p=0.035)
andconnectivetissue-richareas(0.524;p<0.001)withdiabetesstatus
(Table1).Weobservedaninverseassociationofdiabetesstatuswith
thenumberofadipocyteclusters(-0.401;p=0.011).Thedistance
betweenisletsandadipocytesalsoshowedaninverseassociation
(-0.633;p=0.007),suggestingthatasmallerdistancebetweenadi-
pocytesandisletsisassociatedwithT2D.Forthefixedeffectscontrol
variables, sex (males vs females) (0.610; p = 0.007), age (0.741;
p < 0.001),BMI(0.538;p < 0.001), and chronic pancreatitis (1.420;
p<0.001)hadasignificantassociationwithdiabetesstatus.
Whenregressingoninsulinsecretion,weobservedasignificant
negative association with the area of connective tissue (-0.132;
p=0.001)andthenumberofadipocyteclusters(-0.189;p<0.001)
(Table 2). Contrarily, there was a significant positive association of
insulinsecretionwiththenumberofislets(0.069;p=0.043)andthe
area of adipocyte clusters(0.216; p < 0.001).This associationturns
negativewheninteractingwithBMI,suggestinganattenuationofthis
associationinthecaseofhigherBMI(-0.071;p=0.020).Amongthe
controlvariables,sex(-0.179;p=0.008),age(-0.125;p=0.001),BMI
(0.218; p < 0.001), insulin therapy (-1.460; p < 0.001), and chronic
pancreatitis(0.254;p=0.041)hadasignificantassociationwithinsulin
secretion.Amongtherandomeffectsinbothmodels,therewerelarge
variancesbetweencohortinterceptsandvirtuallynovariancesacross
differentstainings.
Asonewouldexpectpotentialeffectsonhistologicalcharacter-
isticsofpancreastissueduetoincreaseddiseaseduration,wefurther
regressedthehistologicbiomarkersonthedurationofT2D.However,
we alsocontrolled for the diabetes status,asdiabetes duration isa
near-perfectpredictorofdiabetesstatus,exceptincaseswhereT2D
wasdiagnosedforthefirsttimefromthetissuesample.Weobserved
significantnegativecorrelationsfordiabetesdurationwiththeareaof
islets(-0.1741;p<0.001)andthenumberofislets(-0.1951;p=0.012),
aligningwiththegeneralprogressionofT2Dandfurthervalidatingthe
biomarkers.Notably, we found a significant positive correlation for
diabetesdurationwithareaofadipocyteclusters(0.2397;p=0.009;
Supplem.Fig12a).

mIF. Results from mIF WSIs are only partially comparable to those
fromIHCdata.Onereasonistheinabilitytodetectconnectivetissue-
richstructuresintheabsenceofspecificmIFstaining.Inthecaseofthe
IHC stained WSI, we compute each biomarker for every different
staining,resultinginsixsetsofimagingmarkersperpatient.However,
thetwomultiplexedWSIsresultedinonlyonesetofimagingmarkers
perpatient,asthebiomarkersofthetwostainingsetsweredistinct
exceptfortheareaorsizeoftheislets,whichwascomputedoverboth
sets, reducing the sensitivity of the mIF statistical models. When
regressing the mIF biomarkers on diabetes status, only the control
variablesage(0.832;p=0.012),BMI(0.593;p=0.024)andchronic
pancreatitis (1.915; p = 0.048) were significant (Supplementary
Table3a).However,whenanalyzinginsulinsecretion,weobserveda
significantassociationwiththeareaofislets(-0.207;p=0.030)and
thenumberofislets(0.280;p=0.008),aswellasthecontrolvariables
BMI(0.243;p=0.008)andinsulintherapy(-1.435;p<0.001)(Sup-
plementaryTable3b).WerefertoSupplementaryFigs.10and11for
theevaluationofallGMLMs.

Methods
Clinicalcohort
We analyzed clinical, laboratory, and histologic data from living
donors obtained within the “LIDOPACO” (LIving DOnor PAncreatic
COhort) programs at two academic sites of the German Center for
DiabetesResearchnetwork,theUniversityHospitalTübingenandthe
UniversityHospital Dresden (SupplementaryFigs.4and5).Patients
undergoing pancreatic surgery for different indications provided
writteninformedconsenttodonatebloodsamplesandpancreastis-
sue,andsharehealthrecordsandlaboratorydataforresearchpur-
poses at both study sites (Tübingen and Dresden). The study was
approved by the Ethical Committees of the Technische Universität
Dresden (Reference EK 151062008) and Eberhard Karls Universität
Tübingen (Reference 697/2011BO1). We obtained macroscopically
healthytissue resected duringsurgery,butnotrequired for further
pathologyworkup.AllpatientswereofEuropeanethnicity.Addition-
ally,fastingbloodwasdrawnpre-surgeryfordetailedmetabolicphe-
notyping. Fasting glucose and C-peptide levels were measured as
previouslydescribed63,andhomeostaticmodelassessment(HOMA)of
insulin secretion was calculated using the computer model-based
HOMA-2BusingglucoseandC-peptide64.Noneoftheparticipantshad
depletedendogenousinsulinproductionasmeasuredbyC-peptide-
based HOMA2B (lowest HOMA2B: 6 with a diabetes duration of 24
years),excludingtype1diabetesamongtheparticipants.Information
onmedicalhistorywascollectedbyaphysician.Documentedbytheir
healthrecords,T2DpatientswerediagnosedashavingT2Datleastone
yearbeforeadmissiontopancreaticsurgery.Thisexcludesdiabetesin
thecontextofexocrinepancreaticdisease.Incontrast,patientswith-
outdiabetesneitherhaddiabetesnordidtheyfulfilldiagnosticcriteria
ofT2Dbasedonglycatedhemoglobin(HbA1c)andfastingglucose,as
definedbytheAmericanDiabetesAssociation65.

Dataacquisition
Hospitalpatientdata.Thepatientcohort-relatedmetadataaresum-
marized in Supplementary Fig. 6 and Supplementary Table 2. The
quantitative analysis of these parameters in Supplementary Fig. 6
revealed an unbalanced distribution of sex, age, BMI, tumor type,
chronicpancreatitis,andcohortbetweenthepatientswithandwith-
outdiabetes.In9patients,fastingbloodsampleswerenotobtained
priortosurgery,andtherefore,HOMA2Bwasnotcalculated.Sexwas
indicatedasself-reportedsexandwasnotconsideredasafactorat
studydesign,andwedidnotperformsex-stratifiedanalysesduetothe
limitedsamplesize.

IHC immunostaining and brightfield microscopy. Formalin-fixed,
paraffin-embedded (FFPE) pancreatic sections (2–4μm thick) were
processedusinganautomaticslidestainerBenchmarkUltra(Ventana
Technology,RocheDiagnostics).Deparaffinizationwasperformedfor
four min at 72∘C using EZPrep (Roche Ventana, #5279771001), fol-
lowedbyantigen-retrieval(AR)for40minat100∘CwithTRIS-based
CC1-buffer (Roche Ventana, #5424569001). After peroxidase-
inhibitionwithI-ViewInhibitor(Roche,#06396500001),thesections
wereincubatedwithprimaryantibodiesagainstinsulin(1:1000;Dako,
#A0564), glucagon (1:600; Santa Cruz, #sc13091), somatostatin
(1:6000;Invitrogen,#14-9751-80),CD31(1:100;Dako,#M0823),peri-
lipin 1 (1:2000; Progen, #651156), and tubulin beta 3 (1:2500; R&D
Systems, #MAB1195). The secondary horseradish peroxidase-linked
antibodywasdetectedviaanOpti-ViewDABIHCdetectionkit(Roche
Ventana, #06396500001). The samples were counterstained with
hematoxylin. WSI acquisition was performed with a Hamamatsu
NanoZoomer2.0-HTusing20xmagnificationandNDP.scan2.5soft-
ware.Exemplaryzoom-insofWSIsareshowninSupplementaryFig.7.

mIF staining and fluorescence microscopy. Consecutive 2–4μm
FFPEsectionsderivedfromtheidenticalpancreaticspecimensused
fortheIHCstainingsunderwentfluorescentstainingviaanautomated
system DiscoveryUltra (Ventana Technology, Roche Diagnostics).
Deparaffinizationutilized EZPrep (Roche Ventana;#5279771001) for
32minat69∘C,andsubsequentARwasperformedwithTRIS-based
CC1-buffer(RocheVentana,#5424569001)at91∘Cfor48min.Primary
antibody cocktails were applied after incubation with a human FC-
receptor-blockingreagent(1:50;Miltenyi,#130-059-901).
The antibody cocktails for Staining set 1 included for the first
incubationantibodiesagainstglucagon(1:200;Abcamplc.#Ab10988)
and tubulin beta 3 (1:25; R&D Systems, MAB1195), for the second
incubation against mouse IgG1 con. AF555 (1:200; Invitrogen, #A-
21127)andmouseIgG2acon.AF647(1:50;Invitrogen,#A21241),andfor
thethirdincubationagainstsomatostatincon.AF750(1:200;Novus
Biologicals, #NBP2-99309 AF750) with DAPI (1ng/ml; Invitrogen,
#D1306).ThecocktailsforStainingset2includedforthefirstincu-
bationantibodiesagainstperilipin1/PLIN1(1:50;Progen,#690156)and
PECAM1/CD31 (1:33; Abcam, #ab134168), for the second incubation
againstmouseIgG1con.AF555(1:100;Invitrogen,#A21127)andrabbit
IgGcon.AF750(1:50;Invitrogen,#A21039),andforthethirdincuba-
tionagainstinsulincon.AF488(1:200;Invitrogen,#53-9769-82)with
DAPI (1ng/ml; Invitrogen, #D1306). WSI acquisition was performed
withaslidescanner(ZeissAxioScan.Z1equippedwithZEN3.10soft-
ware of the Light Microscopy Facility, a Core Facility of the CMCB
TechnologyPlatformatTUDresden.)at×20magnification.Exemplary
WSIsareshowninSupplementaryFig.8.

Deeplearningmodels
Model architectures. Training DL models on WSIs requires special
precautionsduetothelargefilesize.Sinceanentireimagedoesnotfit
intoGPU(graphicsprocessingunit)memory,apatch-basedapproach
is required,where patchesaresimilar-sized cropsofpixelvaluesof
tissueregionsintheWSI.Adifficultytothisapproachisthatonlya
globallabelperimageisavailable,butnotperpatch.Itisnotknownif
themanifestationoftherespectiveimage-levelclassisactuallygivenin
everypatchoriftherearealsopatchesthatarenotindicativeofthe
image-levelclassandwouldhenceconfusethemodelsduringtraining.
A Multiple Instance Learning Approach (MIL) is commonly used to
tacklesuchproblems23.InMILtheindividualpatchesarefirstencoded
byapre-trainedfeatureextractorandsubsequentlypooledandfed
togethertoaMILclassificationalgorithmthatistrainedtopredictthe
image-levelclasslabel.Ourpatchsizewas256×256pixelscovering
115.2μmoftheWSI.Wetestedtwodifferentfeatureextractorsonthe
IHC WSIs. The first is a Vision Transformer pre-trained on
ImageNet21k24,whichconsistsofnaturalimages,whilethesecondis
Phikon66,aVisionTransformerpre-trainedonoversix-thousandhis-
tologicWSIsandthereforespecificallytailoredtothedomain.
AstheMILalgorithm,weusedClustering-constrainedAttention
Multiple Instance Learning (CLAM)25 as well as Chowder67. Both
architecturesdealwiththevariabilityinpatchimportanceindifferent
ways.CLAMusesanattentionmechanismtofocusonthemostrele-
vant patcheswithin a bagwhilesimultaneously imposing clustering
constraintstoensurethatsimilarpatchesreceivecomparableatten-
tionweights,enhancinginterpretabilityandperformance.Specifically,
each patch receives a learned attention score indicating its impor-
tance,unlikeintransformers,whereattentioniscalculatedbetween
patches.Thefinalslide-levelrepresentationisobtainedbyweighting
thepatchfeaturesbytheirattentionscoresandaggregatingthem.We
adapted the CLAM algorithm to further improve generalizability by
onlyshowingavaryingrandomsubsetofpatchesofaWSI(usually5%)
tothemodelduringanepoch.Moreover,acosineannealinglearning
rate scheduler as well as gradient accumulation were added. The
Chowderarchitectureemploysamodularandhierarchicalapproach
toMIL,whereinstance-levelfeaturesareaggregatedthroughaseries
ofconvolutionalandpoolinglayerstocapturecomplexpatternsand
dependencies.It also incorporates importancesampling to dynami-
cally select the most informative instances from each bag, thereby
reducing computational load and enhancing the model’s ability to
focusoncriticaldatapoints.

Datarepresentations.PatchesofIHCsampleswereencodedasany
othernaturalimageasRGBimages(3colorchannels),similartonat-
uralimages.Incontrast,mIFimmunostainingsrequiredfurtherpre-
processing,since inthesecaseseachchannelcontainstheintensity
valuesofoneoftherespectivestainings,whichdonotcoincidewith
colorchannels.EachofthemIFWSIshas4channels,containing3of
thestainingset-specificstainingsaswellasDAPI,asanuclear(DNA)
label.Wetested3differentdatarepresentationsonthemIFWSIs:
(cid:129) RGBThisrepresentationtreatseachofthe3stainingsasRGBcolor
channelsandadditionallyoverlaystheDAPIchannelasagrayscale
imageoneachoftheother3channels,resultinginanRGBimage
similartowhatcanbeseenwithastandardvisualizationsoftware
when showing all stainings simultaneously. This representation
containstheco-occurrenceofthestainings,i.e.,themodelcan
learnwhichstainingsoccurtogetherintheWSI.
(cid:129) Channel-wiseThechannel-wiserepresentationtreatseachchan-
nel (staining) individually as a grayscale image. Consequently,
eachpatchgetsencoded4timesduringfeatureextraction.The
resulting features (1-dimensional vectors) are then appended,
leading to a representation where each channel is encoded in
detail, but no co-occurrence information is preserved. The
resultingfeaturevectorshavethesamefeatureembeddingsize
(1024) as the other representations but yield 4x more features
thantheothertworepresentations.
(cid:129) Channel-wiseaverageThechannel-wiseaveragerepresentationis
very similar to the channel-wise representation but aims to
include the co-occurrence of stainings by averaging the 4
different feature vectors respective to their occurrence in the
imagewhilelosingspecificdetailsofindividualstainings.

Training procedure. We split the data consisting of 100 unique
patientsintoatrain(75patients)andtest(25patients)setandper-
formedthemodeldevelopmentsolelyonthetraindata.Forthat,we
usedacross-validation,whereineachof15splitswerandomlyused60
patientsfortrainingand15patientsforvalidation.Toavoidoverfit-
tingonthesinglevalidationsets,weappliedbalancedsamplingduring
trainingonlyforTübingenpatients,whiletheDresdencohortwasalready
balanced.As the target metric, we chose the Area Under the Receiver
OperatingCharacteristicCurve(AUROC),sinceforthesubsequentXAI
stepsitismoreimportanttohaveamodelthathasahighseparability
andcangenerallydistinguishbetweenthetwoclasses(T2DornoT2D)
thanhavingamodelthatworkswellwithaspecificcutoffasmeasured
bytheF1-ScoreortheAccuracy.

ExplainableAI
Attention methods. Initially, we focused on determining the sig-
nificanceofspecificpatchesusingthebuilt-inattentionmechanismof
the classification head. Nevertheless, raw attention data pre-
dominantly indicate general importance rather than class-specific
relevance. Propagation (LRP)26, which filters the attention layer-wise
valuestohighlightpatchesdistinctlyassociatedwitheitherdiabeticor
non-diabeticstatuswithinthesameWSI.Theclass-specificrelevanceR
isiterativelycomputedforeachlayerl.Here,x(l)istheinputatlayerl
withjelementsandx(l−1)istheinputofthedownstreamlayerwithi
elements.W istherespectiveweightmatrixofthelayer.Non-linear
ji
functionslikeGELU68producebothpositiveandnegativeoutputs.To
accountforthis,relevancepropagationcanbeadjustedbyforminga
subsetofindicessresultinginthefollowingrelevancepropagation:

RðlÞ = xjWji Rðl(cid:2)1Þ, 
j fijði,jÞ2sg Σ fj0jðj0,iÞ2sgxj0Wj0i i ð1Þ
where s = fði,jÞjxW ≥0g:
j ji
For initialization r(0) is set to r(0) = 1, where k is the one-hot-
k
encodedoutcomeclass.Basedontherelevance,theaveragerelevance
filteredattentionmatrixA b iscomputedforeachtransformerblockb.
Tothisend,theexpectationacrossallattentionheadshoftheHada-
mardproductbetweentherelevanceandtheattentionhead’sgradient
∇A(b)istaken,consideringonlythepositiverelevance:

A bðbÞ =I+E
h
ð∇AðbÞ(cid:3)RðlbÞÞ +: ð2Þ
The final attentionattribution output is then computed by the
weightedattentionrelevanceacrossallBtransformerblocks.

Attribution methods. Compared to attention methods, attribution
methods are architecture agnostic and do not require attention
mechanismsinthemodel.Thesimplestformofattributionisthegra-
dientfromanoutcomeclassy
j
w.r.t.theinputx.Whenonlyconsidering
thepositivegradients,thismethodiscalledSaliency69.However,raw
gradientsarepronetogradientshattering70andvanishing71.Therefore,
weemployedIntegratedGradients(IG)29,amorerobusttechniquethat
offersgreatercontextualinformationforeachinputpixelbyincorpor-
atinggradientswithrespecttoabaselinevalue(wechoseablackpixelas
thebaseline).Tothisend,wecomputeapathintegralbetweenabaseline
valuex andthetruevaluex ofeachofthejinputpixels:
0 j
Z
IGðx,x ,fÞ=ðx (cid:2)x Þ 1 ∂fðx 0 +αðx(cid:2)x 0 ÞÞ dα: ð3Þ
j 0 j 0
α=0
∂x
j
However, the initial selection of a baseline value in IG can be
ambiguous, and computing multiple path integrals across various
baselinevaluesmaybeinefficient.Toaddressthis,ExpectedGradients
(EG)27avoidsthisselectionofabaselinevaluebyaveragingtheinte-
gratedgradientsoveradistributionofbaselineimagesx ~D(oftena
0
trainingdatadistributionornoise):

" #
∂fðx +αðx (cid:2)x ÞÞ
v
p
a
a
l
t
i
i
de
t
t
i
so
f
r
.o
D
m
u
t
he
to
tra
th
in
e
s
det
if
f
f
o
e
re
t
t
ra
t
in
di
i
n
s
g
tr
a
ib
n
u
t
ti
t
o
h
e
s
re
o
m
f
a
T
in
2D
ing
w
1
i
5
th
p
ati
t
e
h
ne
t
ts
tw
fo
r
EG j ðx,fÞ=E x0(cid:4)D,α(cid:4)Uð0,1Þ 0 ∂x
j
j 0 dα : ð4Þ
Inapplication,thisexpectationisapproximatedviaamini-batch
samplingapproachforx andα.
0
Morerelatedtorelevancecomputingmethods,DeepLIFT72works
bycomparingtheactivationofeachneurontoitsreferenceactivation
andassigningcontributionscoresbasedonthedifference.Defining
the difference between a target neuron t and the baseline t as
0
Δt=t−t (wechoset =0),thesummation-to-deltaofcontribution
0 0
scoresisestablished.Here,ηrepresentsthedownstreamhiddenlayer
neuron, K, the number of downstream neurons, and C, the single
contributionscore,whichquantifiestheimpactofdownstreamneuron
η onthedifferenceint.Basedonthecontributionscores,themulti-
i
plier mΔηΔt is constructed, inspired by the partial derivative in the
backpropagationalgorithm:

r R e e l l e e v v a a n n c c e e . P T r o op a a d g d a r t e io s n s ( t A he tte l n at t t io er n , L w R e P) a 26 p , p w li h e i d ch A fi tt l e te n r t s io t n he La a y tt e e r n -w ti i o s n e mΔη i Δt = CΔ Δ η t i Δt with XK CΔη i Δt =Δt ð5Þ
i=1
Basedonthismultiplier,wemeasurethecontributionscoresof
the input: mΔxΔt . To make results more robust, we advance all
attribution methods by SmoothGrad73. SmoothGrad computes the
heatmapsseveraltimesbasedonnoisyinputsampleswithGaussian
noise(withσ=1.0andn=10inourcase)andsubsequentlyaverages
the heatmaps. The heatmaps are smoothed with a Gaussian Kernel
withσ=1.0.Wefocusexclusivelyonpositiveattribution,emphasizing
factorsthatsupporta specific outcome. Exemplaryheatmaps ofall
methodsaredepictedinSupplementaryFig.9.

Histologicbiomarker
ROIsegmentation.Tocomputetheidentifiedbiomarkersasegmen-
tationmapoftherespectiveregionsisneeded.Duetotheamountand
sizeoftheWSIsweusedscribbleannotation74toefficientlylabel44
(outof600)IHCWSIswiththelabelsbackground,tissue,adipocytes,
fibroticpatterns,andislets.Scribbleannotationallowsforaveryfast
annotationbecauseregionsarenotannotateddensely(i.e.,eachpixel
getsannotated)butratheronlysmallerlines(scribbles)areusedto
approximatelylabeltherespectiveregions.Duringmodeltraining,we
usedtheignorelabelinnnU-Net75toonlyupdatethemodelweights
based on annotated scribbles while ignoring the remaining image.
Whileannotationisbasedonscribbles,themodelstillreturnsadense
prediction,whichmakesthisapproachwell-suitedforsegmentingthe
ROIs.Sincethenatureofscribblesonlyallowsaquantitativeevaluation
oftherespectiveannotatedareas,aqualitativeanalysisoftheresulting
segmentationmapswasconducted.Afteradjustingsomeannotations
and retraining the nnU-Net, a satisfactorysegmentation quality was
reached.

Quantifying attention. The attention to specific ROIs for the IHC
stainings was computed by summing up the raw attention within
specificsegmentationmasksandstandardizingitbythetotalattention
in the WSI and the tissue size. We summed up the attention per
stainingchannelforthemIFstainingsandstandardizeditbythetotal
attention overall stainings and the tissue size. For the exploratory
analysis, we z-standardized all continuous variables on non-
interpretable scales; however, we routinely z-standardized all con-
tinuouscovariatesforthestatisticalanalysis.

Biomarker quantification. For the IHC staining, we computed the
area-relatedhistologicbiomarkersbasedonthesegmentationmasks
andcountedthenumberofdistinctisletsandadipocyteclustersby
applying connected components to the respective segmentation
mask.Allbiomarkersarestandardizedbythetotalsizeofthetissue.
Fortheislet-relatedbiomarkers,weonlyincludedisletsegmentations
largerthan5000pixels,ignoringsegmentationartifactsorerrors.For
the mIF-specific intensity biomarkers, we computed the channel-
specificmeanintensityasanapproximationoftherespectiveareaof
the trait and standardized itagainby the tissue size. Inthe caseof
perilipin1,wefirstthresholdtheintensityvalues,includingonlyvalues
largerthan1000(intensitymeasuredas16-bitintegers),filteringout
non-specificstainedtraits.

Statisticalmethods
Hypothesis testing. To test differences between two groups, e.g.,
patientswithandwithoutdiabetes,weappliedatwo-sidedtwo-sample
t-test.Totestifatleastonegroupissignificantlydifferent,weapplied
theF-testwithinaone-wayANOVA.Totestifacoefficientinalinear
modelissignificantlydifferentfromzeroweusedthetwo-sidedz-test.
Foralltests,weconsideredp>0.05asnotsignificant.

Statisticalmodels.Forstatisticalanalysisofthebiomarkers,weuseda
generalizedmixedlinearmodel(GMLM),aswehaveseveraldepen-
dencestructureswithinthedata.Forthediabetesstatusastheout-
come, we selected the canonical binomial link function. For the
HOMA2Blevel,weselectedthecanonicalGaussianandGammalink
functions and reported results for the Gaussian link, as its Akaike
Information Criterion (AIC) was lower. We accounted for random
effectslinkedtothecohortandstainingmethod.Becausetherewasno
variationbetweentheoutcomes(i.e.,diabetesstatus)withinaperson,
wecouldnotmodelthepersonasarandomeffect.Inthiscase,the
interceptwouldbeenoughtofitonemodelperpatientperfectly.All
models were fitted via maximum likelihood and the L-BFGS-B
optimizer76. We tested for multicollinearity by computing variance
inflation factor (VIF) and removed “benign tumor” due to perfect
multicollinearity with “malignant tumor” combined with “chronic
pancreatitis”.

Statisticalmodelevaluation.Forallmodelswiththesamemodality
and response variable, we used the AIC for model selection. Addi-
tionally,weevaluatedthelogisticmodelsthroughaconfusionmatrix
and the linearmodels through parity plots, residualplots,and Q-Q
plotsoftheresiduals.Allevaluationresultsofthestatisticalmodelsare
showninSupplementaryFigs.10and11.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Overall, I believe the manuscript has improved significantly and has addressed many of my previous questions and\nconcerns. I now understand most of the rationales and logic of the paper and am generally satisfied with their arguments.\nHowever, I still have some confusion regarding certain figures and paragraphs in the paper that need to be explained more\nprecisely before its publication.\n1) In the legend of Figure 3, the authors stated they used * to indicate actual diabetes states. However, * appears only in front\nof \"No diabetes\" in both panels A and B, suggesting that these samples are non-diabetic. Paradoxically, the panel marked\nwith √, which indicates areas supporting a T2D diagnosis, seems to overlap with the regions labeled with x for \"No\ndiabetes.\" Therefore, I don’t understand what this figure means. Additionally, it is essential to include controls showing\nattention heatmaps from samples with diabetes to achieve a complete understanding.\n2) In the same sense, I don’t know what \"left side\" refers to in the sentence: “increased attention to areas abundant in\nconnective tissue (left side of the PECAM1-, glucagon-, insulin-, and somatostatin-stained WSIs) as well as a pronounced\nfocus on adipocytes (top left of the perilipin 1-stained WSI). For readers to better understand the differences, I also suggest\nthat the authors include enlarged ROIs to illustrate the detailed structures of the connective tissues and adipocytes\nmentioned.\n3) Compared to T2D samples, non-T2D samples showed higher attention scores for islets in insulin-stained WSIs, whereas\nthe opposite was observed for islets in somatostatin-stained WSIs (Fig. 4A). However, in Fig. 5B, the attention to islets\nlabeled with somatostatin or insulin did not differ between T2D and non-T2D samples. In contrast, attention to islets labeled\nwith glucagon showed significant differences. I wonder what causes these discrepancies.", "reviewer_2": "Explainable AI (XAI) is indeed important in biomedical research, but it may not be the most appropriate tool for this particular\nproblem. This work aims to identify histological features in pancreatic tissue that could predict type 2 diabetes (T2D),\nfocusing on both the pancreas's exocrine tissue and the pancreatic islets. However, T2D prediction is neither based on nor\ndiagnosed through histological sections, as the pancreas cannot be biopsied. What biological insights does the model\nprovide to deepen our understanding of new disease biomarkers for treating or reversing type 2 diabetes? In addition, the\nauthors made a substantial effort to revise the manuscript; however, several specific comments are below.\n1. It is concerning that the chromogen versus fluorescence-based method for islet detection yielded such different results\nregarding islet number and size on consecutive paraffin-embedded sections, particularly when considering diabetes status.\nThe data generated from fluorescence-based islet detection aligns with recent work demonstrating that the mass of beta,\nalpha, and delta cells, the three primary types of islet endocrine cells, is unchanged in early-stage type 2 diabetes.\n2. The most significant advantages of XAI are its ability to enhance model trustworthiness and reduce model complexity.\nSpecifically, attribution scores can help verify that a model is making decisions based on relevant features rather than being\nmisled by confounding factors. However, in this study, these advantages are not applicable since the predictive model used\nis unlikely to be widely adopted by other researchers - predicting disease states from images does not seem to be a\nstandard or practical approach.\n3. As the authors claim, XAI confirms hypotheses and lays the groundwork for future research. However, attribution and\nattention scores are complicated and need subjective human interpretation. The examined features in this manuscript still\nultimately relate to channel-level analyses and correlate with conventional features (e.g., islet area). There are no clear\nexamples of novel hypotheses generated by XAI in this work, particularly ones that would be difficult to analyze using\ntraditional methods.\n4. While the computation of attribution scores may be unbiased, the study’s novel findings and its demonstration of XAI’s\npower appear limited.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature\nCommunications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career\nResearchers who co-review manuscripts.", "reviewer_4": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature\nCommunications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career\nResearchers who co-review manuscripts."}, "author_response": "Rebuttal\nChanges within the manuscript are indicated by blue font color.\nReviewer #1\nQ1 What are the advantages of XAI over a large-scale analysis on pre-defined traits? For\nexample, if we pre-define a series of traits (e.g., intensity of marker X in islets, distance of marker\nX and marker Y), calculate all traits, and run statistical tests against diabetes states (also\nconsidering co-variants), can we also obtain the same key traits as Fig. 1D? It is unclear whether\nXAI offers higher efficiency or accuracy over traditional analytical methods.\nWe appreciate the reviewer’s insightful comments regarding the novelty of our approach. The\nkey advantage of an XAI-driven discovery process lies in its hypothesis-free nature, which\nreduces bias inherent in traditional biomarker identification methods. In the process described by\nthe reviewer, all extracted biomarkers must be predefined before the experiments, potentially\nconstraining the discovery to known features and limiting novel insights.\nIn contrast, our XAI-based approach allows for the unbiased identification of relevant regions of\ninterest (ROIs), enabling direct comparison between XAI-derived ROIs and those hypothesized a\npriori by human experts. Observing whether these regions overlap or diverge provides (crucial)\nvaluable information not only about type-2 diabetes but also about the limitations and biases of\nexisting experimental approaches.\nWe acknowledge that the novelty of our approach, particularly the distinct steps of human-XAI\ninteraction in biomarker identification and comparison, was not fully elaborated in the original\nmanuscript. To address this, we have now added a dedicated results subsection, \"Unbiased\nidentification and computation of histologic biomarkers\" (line 304), and revised the introduction to\nmore clearly highlight the innovative aspects of the XAI-driven discovery process.\nWe hope these clarifications and additions sufficiently address the reviewer’s concerns.\nQ2 The manuscript does not thoroughly explain the biological significance of the attention results\nto biologists. For example, in Fig 5a, DAPI receives high attention – does it suggest general cell\nmorphology also contributes to classification? In Fig 5b & 5c, what does high attention of tubulin\nbeta 3 imply? Should biologists focus on the spatial distribution of these markers or their\ninteractions with other markers/cell types?\nWe appreciate the reviewer’s feedback and acknowledge that the concept and implications of\nattention and attribution may not have been sufficiently explained for researchers without an AI\nbackground. Attribution methods in general determine how much each input feature attributes to\na model’s output (i.e. how important each feature is in a prediction) through e.g. gradients or\nperturbations. Attention methods, a subset of attribution methods, leverage the output of the\nattention module used e.g. in the transformer architecture. Raw attention outputs indicate\nfeatures of general importance to the model but lack a direct connection to specific outputs,\nrequiring filtering (e.g., via gradients or relevance scoring) to highlight features used to predict a\nspecific outcome.\nIn our approach, attention modules operate at the patch level rather than directly on individual\npixels, meaning attention-based methods can only highlight the importance of each patch. To\nachieve a finer-grained analysis at the pixel level, we must rely on alternative attribution methods\nthat can assess feature importance within each patch. As the reviewer correctly points out, high\nattribution to a specific feature—such as a particular cell type—indicates that the feature was\nutilized for prediction but does not inherently explain why it was important.\nTo address this, we went beyond classical attention analysis by applying novel strategies, such\nas summing attention within the DAPI channel in fluorescence microscopy to compare the\nrelative importance of different multiplex channels. Additionally, we filtered the attention using\nLayer-wise Relevance Propagation (LRP) to assess the significance of each channel in relation\nto specific predicted outcome classes, such as diabetes status.\nIn our framework, attention and attribution maps serve as an unbiased filter, guiding the\ndiscovery of spatial features that are most relevant in the vast dataset. From these identified\nregions of interest, we derive hypotheses about potential biomarkers. For instance, we observed\nthat the multiplex channel for tubulin beta 3 contributes significantly to the model’s classification\nof patients with T2 diabetes. Based on this finding, we hypothesized that alterations in tubulin\nbeta 3-stained structures, particularly within the islets, may be relevant. Consequently, we\ncomputed the biomarker “Tubulin beta 3 intensity within islets” to quantify this observation.\nTo better clarify the connection between attention results and specific biomarkers, and to provide\na more specific interpretation of the attention maps, we have expanded the manuscript with a\nnew results subsection, \"Unbiased identification and computation of histologic biomarkers\" (line\n304). Further, the definition and differences between attribution and attention is now indicated in\nline 213 and 315.\nWe believe these additions improve clarity and ensure that the AI-driven analytical process is\nmore accessible to a broader audience.\nQ3 The manuscript describes the technical necessity of training the model with subsets of the\nWSIs, named patches. What is the biological relevance of the selected patch size? How does it\ncompare to the size of cells in the images?\nSince each pixel in our WSIs covered ~0.45 µm and the patches covered 256x256 pixels, each\npatch had the biological size of 115.2 µm in height and width. Consequently, single patches\ncontained multiple exo-/endocrine cells, each typically measuring 10-15 µm in diameter. In\ncontrast, the diameter of the islets ranges from 50 µm for smaller islets to up to 200 µm for larger\nones, while adipocytes can reach diameters of up to 100 µm. Areas larger than the patch size\nare still seen by the model since in a later stage of the model (that is computationally less\nexpensive and therefore feasible) the patches are all pooled together again before the final\nclassification happens. Therefore, the model’s decision is based on all information from the\nrespective WSI and not only on a subset. We added this information to the Methods section (line\n679).\nQ4 How was the islet segmentation performed? Figure 6 shows the relative difference in\nquantified XAI-derived histologic biomarkers between the control and T2D groups. The methods\nsection describes quantifying these biomarkers (Lines 733 – 765). The ROI segmentation is\ndependent on the scribble annotation deep learning algorithm, which is a very recently described\napproach. Due to the novelty of this approach, particularly for application to pancreatic sections,\nthe segmentation method should be validated appropriately. The manuscript lacks an explicit\ndemonstration of successful islet segmentation. This is a concern, especially for sections with\nchromogenic stains that were not labeled for islet-specific markers.\nThe islet segmentation process was conducted using a standard dense labeling approach for\nsemantic segmentation. To ensure comprehensive coverage and accuracy, we randomly\nselected 226 regions (each approximately 2000×2000 pixels, i.e. 900 x 900 µm, or 0.81 mm2)\nand manually annotated them. For the chromogenic WSI dataset, we employed a 5-fold\ncross-validation strategy, and the final predictions were obtained using an ensemble of all five\nsegmentation models. The models achieved a mean Dice score of 0.8782 across all\ncross-validation folds, demonstrating the robustness of our approach.\nFor the fluorescent dataset, islet segmentation was performed completely manually. This was\nfeasible because matching islet locations across multiplex channels made manual annotation\nmore efficient compared to chromogenic slides, where each staining corresponds to a different\ntissue and WSI, increasing the annotation effort significantly.\nQ5 The description of donor demographics is unclear. Supplementary Table 2 describes the\ncohort donor metadata, but this table does not adequately describe important characteristics of\nthe cohort. It is unclear how the cohort is split by sex and how many (integer) Controls/T2D\npatients are in each sex category. The reporting of Hba1c and other relevant physiological\nparameters is not split by diabetes status.\nWe appreciate the reviewer’s question; however, it appears that Figures 4, 5, and 6 in the\nSupplementary Material directly address these concerns. Specifically, Supplementary Figure 4A\npresents the demographic breakdown of the cohort, showing that it consists of 45 female and 55\nmale donors. Furthermore, it illustrates that 15 out of 45 female patients and 20 out of 55 male\npatients had T2D, which directly answers the reviewer’s question.\nAdditionally, all relevant physiological parameters stratified by diabetes status are provided in\nSupplementary Figure 4, while demographic characteristics are detailed in Supplementary Figure\n6.\nQ6 How do the histological biomarkers identified by XAI relate to disease duration? One would\nexpect non-negligible effects on histological characteristics of pancreas tissue due to increased\ndisease duration. Are predictive histological biomarkers consistent between sub-cohorts of\ndiffering disease duration?\nInitial results addressing this question are summarized in Supplementary Figure 12A, B, where\ncorrelation matrices reveal several significant but modest associations between disease duration\nand biomarkers in both chromogenic and fluorescence WSIs.\nHowever, it is important to note that diabetes duration is a near-perfect predictor of diabetes\nstatus, except in cases where T2D was diagnosed for the first time from the biopsy. Specifically,\nall patients with a diabetes duration of zero are T2D-negative, while those with a nonzero\nduration are T2D-positive. Consequently, it is expected that biomarkers correlated with diabetes\nstatus also exhibit some correlation with diabetes duration. To assess whether these biomarkers\ncorrelate with disease duration independently of diabetes status, we performed a regression\nanalysis, incorporating both chromogenic and fluorescence biomarkers alongside diabetes\nstatus. The regression coefficients and p-values are summarized in the table below:\nBrightfield (duration_diabetes ~ diabetes_status + {variable}):\nVariable Coefficient p-value\nNumber of Islets -0.1951 0.0117\nArea of Islets -0.1741 0.0008\nArea of Adipocyte Clusters 0.2397 0.0098\nNumber of Adipocyte Clusters 0.0138 0.8589\nMin. Dist. Islets and Adipocytes -0.0913 0.3005\nArea of Connective Tissue -0.0876 0.3036\nFluorescence (duration_diabetes ~ diabetes_status + {variable}):\nVariable Coefficient p-value\nNumber of Islets -0.1079 0.2041\nArea of Islets -0.1599 0.0527\nTubulin beta 3 Intensity within Islets 0.0984 0.2359\nPerilipin 1 Intensity 0.1089 0.1893\nNotably, we observe significant negative correlations for Area of Islets (Chromogenic: -0.1741; p\n= 0.0008, Fluorescence: -0.1599; p = 0.0527) and Number of Islets (-0.1951; p = 0.0117), as well\nas a significant positive correlation for Area of Adipocyte Clusters (0.2397; p = 0.0098). These\nfindings align with the general progression of T2D, where pancreatic islet structures are reduced\nover time. However, the increase in adipocyte cluster area is also commonly observed with\naging, making it a potential confounding factor.\nOverall, given the variability in disease progression among individuals, structural changes in\npancreatic tissue may not always correlate with disease duration in a straightforward manner.\nThanks to the reviewers advice and the relevance of the resulting findings, we now addressed\nthis topic in the results (line 393) and discussion section (line 511). Further, we advanced the\ncorrelation matrices in Suppl. Figure 12A, B which is now colored based on the significance\nlevels.\nQ7 “(Line 153) We employed a 15-fold cross-validation, i.e., 15 different training (n=60 each) and\nvalidation (n=15 each) splits to avoid overfitting”. The authors used an incorrect definition of\nK-fold validation. Typically, 15-fold validation means the data is split into 15 groups (n=5 each in\nthis case) and one group is used as the test set in each run.\nWe acknowledge the reviewer’s observation that we did not strictly follow the definition of splitting\ninto 15 equally sized folds. As the reviewer correctly calculated, this approach would have\nresulted in validation folds containing only 5 cases, which we deemed too small for reliable\nvalidation. To address this, we instead created 15 different splits, ensuring that each validation\nset contained 15 cases, thereby improving the reliability of the validation process (see also reply\nto Q1 by Reviewer #3). To prevent any confusion, we have revised the wording in the\nmanuscript to explicitly clarify this approach (line 158).\nQ8 While patch-level attention and attribution reveal the differential patterns between control and\nT2D samples, pixel-level attribution needs to be more conclusive. Beyond Figure 4, is it possible\nto find differential patterns of marker distribution (e.g., colocalization of glucagon and\nsomatostatin) within the images when comparing T2D and control samples?\nWe appreciate the reviewer’s feedback and agree that the transition from slide-level heatmaps to\nspecific biomarkers was presented too abruptly, making it challenging to follow. To address this,\nwe have introduced a new subsection, \"Unbiased identification and computation of histologic\nbiomarkers\" (line 304), where we explicitly interpret the specific cellular and multicellular\nstructures that the model attends to and explain how we infer the respective biomarkers from\nthese interpretations.\nFor instance, in Figure 6(d), as the reviewer hypothesized, we observe that the model indeed\nattends to the colocalization of fluorescent glucagon and somatostatin markers within pancreatic\nislets. We further quantify these findings for subjects with T2D and in the control group in two\nsections: the subsection “Quantifying attention to regions-of-interest,” analyzes the differences in\nattention, and the subsection “Statistical analysis reveals a significant association of histologic\nbiomarkers with diabetes status and insulin secretion,” evaluates the differences between the\nresulting biomarkers.\nThese additions ensure a clearer, more structured transition from heatmaps to biomarker\nderivation, enhancing the interpretability of our approach.\nQ9 How well do attention/attribution models capture regions with lower marker intensities? And\nhow well do they disregard regions with high but less informative marker intensities?\nWe appreciate the reviewer’s insightful question, which intertwines two key aspects: (1) the\nmodel’s ability to detect lower marker intensities and (2) the role of attention and attribution\nmethods in visualizing the features the model is sensitive to.\nThe model training process inherently determines whether certain regions are captured based on\nwhether they provide a meaningful signal for solving the classification task. Once the model has\nlearned which image features are relevant for the distinction between T2D and no diabetes, the\nreasoning of the model is visualized by the attention and attribution methods. These methods are\nnot biased by the marker intensities but solely show how important the model deems certain\nfeatures for its task. For instance, Figure 6(d) illustrates that even weak tubulin beta 3 staining is\nhighlighted by the model, confirming that the model can attend to subtle but relevant features.\nGiven the importance of this discussion, we have incorporated these observations into the\nmanuscript to further clarify how the heatmaps relate to the underlying model (line 220).\nQ10 Could authors provide more details on Multiple Instance Learning (MIL) in the model,\nparticularly how attention is calculated? Is it the same as self-attention in Transformer? Is\nattention calculated between patches or from each patch to the final outputs?\nThe Multiple Instance Learning (MIL) approach in the model follows the widely used\nClustering-constrained Attention Multiple Instance Learning (CLAM) framework. Unlike naive MIL\napproaches, which often base its decision on a single patch, CLAM employs attention-based\npooling to aggregate all patch-level features to make a prediction based on the full representation\nof the slide.\nThus, the attention computation in CLAM is not the same as self-attention in Transformers.\nInstead of computing relationships between all patches as in a Transformer, CLAM assigns an\nattention score to each patch based on its relevance to the classification task. These scores\ndetermine the contribution of each patch to the final slide-level representation.\nAs attention is calculated per patch, each patch receives an attention score indicating its\nimportance. The final slide-level representation is obtained by weighting the patch features by\ntheir attention scores and aggregating them. Thus, attention is allocated to each patch, rather\nthan between patches, and the relationship between the patches is learned through the\ndownstream fully connected layers in the MIL classifier.\nBased on the reviewers feedback, we highlight this difference to the application of attention in\ntransformer architectures now in the method section (Line 689). For further information on CLAM,\nwe refer to the respective paper [1].\n[1] Lu, Ming Y., et al. \"Data-efficient and weakly supervised computational pathology on\nwhole-slide images.\" Nature biomedical engineering.\nQ11 The authors compared different methods of handling fluorescence WSI channels and\nclaimed “(Line 182-183) the latter benefited from co-occurrence information reflecting the spatial\nrelationships between different staining”. Does concatenating the encodings along another axis\n(resulting in Pn x 4096 encodings) preserve the co-occurrence relationship? In addition, the\nreason that the “channel-wise” model performs worse than the “channel-wise average” model\nmight also be overfitting due to a higher dimensionality.\nWe appreciate the reviewer’s feedback and apologize for not explaining the handling of different\nfluorescence channels better. In our approach, we did not concatenate the feature vectors of the\nsame patch; instead, we appended them as if they were independent patches. As a result, the\nmodel had no information about which feature vectors originated from the same patch. This\nensured that, while the amount of patches is four times larger for the channel-wise\nrepresentation, the feature length stays the same between all representations. This is important\nas the attention mechanism aggregates across the number of patches, and the final classification\nmodule only operates on one final vector that has the size of the feature length not influenced by\nthe number of patches. This feature length is the same across all representations.\nFurthermore, our results showed that the channel-wise average representation performed better\nfor one staining set but worse for the other. Together with our extensive cross validation and the\ntest set of 25% of the patients, these results clearly indicate that different levels of overfitting due\nto image representation were not present (see also reply to Q1 by Reviewer #3).\nTo clarify this in the manuscript, we have revised the methods section (Line 722) to ensure that\nour approach is more clearly described.\nReviewer #3\nQ1 I understand the challenges involved in obtaining pancreatic sections from living donors,\nwith 35 samples from T2D patients and 65 from non-T2D patients who underwent\npancreatectomy for various pancreatic disorders. However, I believe that simply labeling six\nwell-known proteins and applying a DL algorithm to such a small sample size is excessive.\nWith such a limited sample size, overfitting is almost inevitable. Additionally, the potential for\nindividual variability to dominate the sample reduces the likelihood of discovering significant\nnew findings.\nWe appreciate the reviewer’s concern regarding the sample size and the potential for\noverfitting. While the patient number in this study might appear small for a DL project, it is\none of the most extensive ones in the domain of diabetes microscopy. Further, we\nemphasize that the gigapixel-sized WSIs, including different specific markers and two\nmicroscopy modalities, provide a very diverse set of information even for one patient, as one\nWSI resulted on average in 4424 patches (i.e. around 3.5M patches in total).\nIn addition, this work represents, to our knowledge, the first DL-based approach for\nclassifying T2D using human pancreatic tissue WSIs from living donors. Given the absence\nof prior studies in this area, we carefully selected proteins and labels that are\nwell-established in T2D pathology to ensure that the AI model learns from clinically relevant\nhistological markers, thereby strengthening the robustness of our findings.\nWe recognize the challenges posed by a small sample size and took several measures to\nmitigate overfitting and account for inter-individual variability:\n1. Cross-validation strategy: Instead of conventional 15-fold cross-validation, which\nwould have resulted in validation sets that were too small, we created 15 individual\ntrain/validation splits, each with 15 patients in the validation set. This approach\nensures more reliable validation (see also reply to Q7 by Reviewer #1)\n2. Bias reduction in performance metrics: Rather than averaging the performance\nacross individual splits, we saved the predictions from all splits and computed the\nfinal metrics at the end. This prevents bias from outliers in any single split and\nprovides more reliable validation metrics for hyperparameter tuning.\n3. Independent test set evaluation: The final model was evaluated on a completely\nunseen test set of 25 patients. If significant overfitting had occurred, we would have\nobserved a drastic drop in performance on these cases, which was not the case (see\nalso reply to Q11 by Reviewer #1).\n4. Phikon foundation model: We also used the Phikon foundation model as a patch\nencoder, which was pretrained on 40M histologic patches. The idea behind\nfoundation models is that they are pre-trained on a large volume of data and then\nfine-tuned for specific downstream tasks, leveraging their broad knowledge and\nadaptability to achieve high performance with minimal task-specific training.\nAdditionally, while the study is based on a limited number of 100 unique patients, this is\ncounterbalanced by the rich dataset comprising multiple stainings and imaging modalities\n(chromogenic and fluorescence). Instead of pooling all patient data into a single model, we\nconducted independent experiments for each staining+modality combination, training\nseparate models for each. This diverse experimental setup significantly reduces the\nlikelihood that all models would overfit to the same features, as they were trained on different\ndata (even if derived from the same patient).\nGiven that our conclusions are based on consistent trends across multiple independently\ntrained models, rather than any single potentially overfitted model, we are confident that our\nfindings remain valid despite the small sample size. To further clarify these methodological\nconsiderations, we have revised the manuscript accordingly.\nQ2 Considering these points, I am perplexed about the rationale behind undertaking such a\ncomplex procedure to obtain human pancreatic slices with minimal labeling, only to then use\nDL to extract differences of limited confidence. I disagree with the authors' claim that the\ncurrent paper has identified \"reliable biomarkers with reliable associations with T2D.\"\nFurthermore, the study does not underpin \"key findings about pancreatic tissue alterations in\nT2D and provides novel targets for research \", as the authors have suggested.\nAs outlined in our response to Q1, we took careful steps to ensure the reliability of our DL\nmodels. Importantly, all biological conclusions were based exclusively on test set\ncases—meaning that all attention and attribution heatmaps originate from the test set, not the\ntraining set. This approach significantly reduces the risk of interpreting potentially overfitted or\nunreliable DL results.\nFurthermore, as detailed in Q1, all biological interpretations are supported by multiple\nindependent experiments, each involving separately trained DL models and statistical models on\ndifferent datasets and biomarkers. The statistical models underwent extensive evaluation (see\nTable 1, Supplementary Figures 10 & 11, and Supplementary Table 3), demonstrating a strong\nassociation between the computed biomarkers and diabetes status.\nWhile we acknowledge that overfitting can never be entirely ruled out in any deep learning study,\nwe have carefully designed our experiments to minimize this risk and limit our interpretations\nstrictly to the examined dataset. Given that our findings are consistently supported across\nmultiple independent analyses, we are confident in their reliability and robustness.\nWe believe that we have identified multiple important, even if subtle, morphological aspects\nrelated to the T2D pathology. Our analysis reveals that pancreatic adipocytes are located closer\nto islets in samples from patients with T2D. While this might initially seem like a minor finding, it\nholds significant implications due to the role of the pancreatic microenvironment in islet function.\nThis proximity likely influences the exocrine-endocrine crosstalk, which is critical for the islets.\nMost interesting, a previous study reported that \"fatty infiltration in the human pancreas creates a\nneurotrophic microenvironment and promotes remodeling of pancreatic innervation\" [1]. Given\nthat both adipocytes and beta cells, i.e. fatty acid release via lipolysis and insulin secretion, are\nstrongly regulated by neuronal signals, we believe that our observation opens a new avenue for\nexploring the pathogenesis of diabetes. By linking these observations, we offer a new\nperspective on the complex interaction between pancreatic fat, islet function and neuronal\nregulation. This leads us to another key finding: the best performance in diabetes classification\nwas achieved for the neuronal marker tubulin beta 3 in the chromogenic WSI, and for staining set\n1 (labeling neurons, alpha, and delta cells) in the fluorescence modality. This suggests that\nstructural changes in the neuronal network as well as in alpha and/or delta cells—changes that\nare not yet fully understood—may not only accompany but also actively contribute to the\nprogression of the disease. We address these findings in more detail in our discussion part (line\n441, 447, and 562).\n[1] Tang, Shiue-Cheng, et al. \"Human pancreatic neuro-insular network in health and fatty\ninfiltration.\" Diabetologia.\nQ3 In fact, quantifying attention to regions of interest for chromogenic WSIs (Figure 4) and\nfluorescence samples (Figure 5) from non-diabetic and T2D donors yielded inconsistent\nresults. For the chromogenic WSIs, only minimal differences were observed in PECAM1,\nPerilipin 1, and tubulin in some cases, while no differences were detected in insulin,\nglucagon, or somatostatin. I believe this indicates a failure or insensitivity of the method in\nconfirming well-known biomarkers. In contrast, the fluorescence samples showed some\ndifferences in glucagon as well as tubulin, PECAM1, and Perilipin 1. Therefore, I do not\nunderstand the authors' assertion that \"islets are attended above average for tubulin beta 3\nand PECAM1 and below average for Perilipin 1 staining compared to the rest of the tissue\n(Fig. 5C).\"\nWe appreciate the reviewer’s feedback but believe there is a fundamental misunderstanding\nregarding the presented results. The passage cited by the reviewer about chromogenic WSIs\npertains specifically to attention to connective tissue, not the general importance of stainings.\nThese results assess how relevant connective tissue is for each of the six models trained on\ndifferent stainings, given that none of these stainings are specific to connective tissue.\nOur standardized attention scores confirm that attention to connective tissue is consistent across\nall six stainings, which is expected. Furthermore, we observe that connective tissue becomes\nmore important when classifying a patient as T2D positive, with a significant difference for\nPECAM1, Perilipin 1, and tubulin beta 3. This is reasonable, as the non-significant stainings\nspecifically target islets, where attention is significantly higher compared to non-islet-specific\nstainings. The referred results in the fluorescence analyses then focus on attention to total tissue\nand specific marker channels, which are fundamentally different and not directly comparable to\nthe referred chromogenic WSI results.\nTo improve clarity and prevent misinterpretation, we have revised the manuscript by modifying\nFigure 4 for better readability and refining the corresponding text in the section.\nQ4 The identified differences in the DL network's attentions for PECAM1, Perilipin 1, and\ntubulin were later determined to be related to the sizes of the cells involved. I do not consider\nthis to be a novel finding, and I do not see the necessity of going through such extensive\nefforts to uncover such an apparent morphological feature. Technically, this may be due to\nthe fact that the authors divided the whole WSI into smaller patches to extract features. This\napproach tends to focus solely on obvious local features and may miss relatively large,\ndistributed features that DL algorithms could detect, but human observers might not.\nWe believe there is again a fundamental misunderstanding of our results. We are also\nunsure how the reviewer arrived at the conclusion that “the identified differences in the DL\nnetwork’s attentions for PECAM1, Perilipin 1, and tubulin were later determined to be related\nto the sizes of the cells involved,” as this is not stated in the manuscript.\nFor example, differences in attention to tubulin beta 3 in fluorescence-stained\nWSIs—particularly when comparing attention within islets versus the whole\ntissue—motivated the computation of the biomarker \"tubulin intensity within islets\" as a proxy\nfor the number of nerve cells in islets. Cell size is just one of many biomarkers computed\nacross different cell types in both chromogenic and fluorescent-stained WSIs, but none of\nthe cell size markers is specific for nerve cells (tubulin beta 3) or blood vessels (PECAM1).\nAdditionally, the assertion that our findings are due to patching of WSIs is incorrect. While\neach patch receives one attention score, the MIL classifier processes all patches from a WSI\nsimultaneously, enabling it to detect larger ROIs and critical spatial features—a\nwell-established approach in computational histopathology [1,2]. Furthermore, Figure 3\n(LRP-based attention heatmaps) clearly shows that the model identifies large ROIs, such as\nadipocyte clusters and connective tissue structures spanning across the tissue.\nOverall, we believe that reviewer #3 has misinterpreted both the methodology and the\npresented results. To improve clarity and to prevent this from happening again, we have\nadded an additional section explicitly explaining the connection between attention heatmaps\nand biomarker computation.\n[1] Lu, Ming Y., et al. \"Data-efficient and weakly supervised computational pathology on\nwhole-slide images.\" Nature biomedical engineering.\n[2] Ilse, Maximilian, Jakub Tomczak, and Max Welling. \"Attention-based deep multiple\ninstance learning.\" International conference on machine learning.\nQ5 Overall, I do not accept the assertion that “The AI-based T2D prediction models\ndemonstrated a highly reliable performance,” and I do not believe that the study represents a\ncarefully designed experiment or a significant technical advancement.\nWhile we appreciate the reviewer’s opinion, we respectfully disagree with her/his assessment, as\nher/his questions and statements indicate a fundamental misunderstanding of both the\nmethodology and results. Additionally, some referenced results were taken out of context, and\ncertain claims were made regarding findings that are not present in the manuscript.\nTo our knowledge, this study is the first to classify T2D directly from human pancreatic tissue\nwhile also introducing a novel approach for unbiased biomarker detection. The rigorous\nexperimental design, including extensive cross-validation and independent test set evaluation,\ndemonstrates the high predictive performance of our models across two microscopy modalities\nas described in our extensive answer to Q1.\nBeyond the results, the entire experimental pipeline was carefully designed—from patient\nselection and the choice of clinically relevant histological markers to DL model training, XAI\napplication, and statistical evaluation. Furthermore, this study goes beyond conventional\nexperiment pipelines, as each step involves interaction between human experts and AI models,\nmaking it a pioneering approach in scientific discovery.\nGiven these considerations, we strongly disagree with the reviewer’s conclusion. The novelty of\nour work is evident not only in the results but also in the innovative experimental process used to\nachieve them.\nAnswer to all Reviewers\nWe sincerely thank all reviewers for their valuable comments. We appreciate the general\nconsensus among the reviewers regarding the significance of our work for the community\nand the improvement since the last revision: “[...] I am generally satisfied with their\narguments” (“Reviewer 1”), “[...] the authors made a substantial effort to revise the\nmanuscript” (“Reviewer 2”).\nWe addressed all concerns in our point-by-point responses to each reviewer and revised the\nmain paper while retaining the overall structures. Based on the reviewers' feedback, we have\nmade the following improvements to the manuscript (location in manuscript in parentheses):\n1. Improving Figure 3 by highlighting WSI regions discussed in the text and showing an\nenlarged section below (Page 8).\n2. Adding another Figure similar to Figure 3 but with the WSIs of a T2D-positive patient\n(Suppl. Figure 13, Page 38).\n3. Adding a Figure showing the distribution of the number of islets and average islets\nsize computed based on the fluorescence segmentations only (Suppl. Figure 14,\nPage 39)\n4. Improved explanation of the limitations of attention heatmaps and how we include the\nscientist in the loop (Line 226).\n5. Specifying discussed ROIs attended by the model in more detail (Lines 240, 246, and\n348).\n6. Highlight more the discrepancy between modalities when interpreting results (Lines\n276 and 281)\n7. Strengthen the point that the novelty lies also in the methodology (Lines 118 and\n640)\n8. Highlighting that XAI is used for scientific discovery in this work and not solely for\nmodel trustworthiness (Line 104)\n9. Discussing the results with a more in-depth focus on new findings (Lines 560, 598,\n627, and 632)\nAnswers for Reviewer 1\nQ1: In the legend of Figure 3, the authors stated they used * to indicate actual diabetes\nstates. However, * appears only in front of \"No diabetes\" in both panels A and B, suggesting\nthat these samples are non-diabetic. Paradoxically, the panel marked with √, which indicates\nareas supporting a T2D diagnosis, seems to overlap with the regions labeled with x for \"No\ndiabetes.\" Therefore, I don’t understand what this figure means. Additionally, it is essential to\ninclude controls showing attention heatmaps from samples with diabetes to achieve a\ncomplete understanding.\nAnswer: We thank the reviewer for bringing up the seemingly paradoxical fact that heatmap\nareas indicating high attention of the model when predicting T2D can overlap with areas the\nmodel attends to when predicting no T2D. This is expected as the attention maps only show\nwhat spatial areas are important for each predicted outcome, not the specific traits within\nthese areas. In our setting, it is up to the biological interpretation, leveraging prior human\nknowledge, and further analysis of the heatmaps to determine these traits.\nFor example, the size of a cell can support one outcome but its shape another one. In both\ncases, the same area, i.e. the cell, is of importance but due to different traits. To further\nclarify this, we added an explanation in Line 226 to the manuscript.\nMoreover, we agree that it is essential to add respective heatmaps of a control case. We\nfocus on one case in the main manuscript to be able to show all different modalities (6\nindividual chromogenic stainings as well as both fluorescence staining sets). Consequently,\nwe added a new figure representing a control case to the supplementary material (page 40).\nQ2: In the same sense, I don’t know what \"left side\" refers to in the sentence: “increased\nattention to areas abundant in connective tissue (left side of the PECAM1-, glucagon-,\ninsulin-, and somatostatin-stained WSIs) as well as a pronounced focus on adipocytes (top\nleft of the perilipin 1-stained WSI). For readers to better understand the differences, I also\nsuggest that the authors include enlarged ROIs to illustrate the detailed structures of the\nconnective tissues and adipocytes mentioned.\nAnswer: We agree that Figure 3 needs further descriptions of the underlying WSI structure\nthat is shown as an example in this figure. We adjusted the figure so that the original WSIs\nwithout heatmaps are enlarged and specific areas such as fibrotic patterns or adipocytes are\nbetter visible.\nWe have clarified the mentioned sentence to specify that \"left side\" refers to the left side of\nthe original WSIs rather than the figure orientation in Lines 246-249. Beyond the enlarged\nROIs available in Figure 6, we also enhanced Figure 3 by adding bounding boxes to\nhighlight the specific areas referenced in the manuscript text. In addition, we also show the\nenlarged content of the bounding boxes now below the WSIs.\nQ3: Compared to T2D samples, non-T2D samples showed higher attention scores for islets\nin insulin-stained WSIs, whereas the opposite was observed for islets in\nsomatostatin-stained WSIs (Fig. 4A). However, in Fig. 5B, the attention to islets labeled with\nsomatostatin or insulin did not differ between T2D and non-T2D samples. In contrast,\nattention to islets labeled with glucagon showed significant differences. I wonder what\ncauses these discrepancies.\nAnswer: We appreciate the reviewer’s question and the chance to clarify these findings.\nFirst, it is important to mention that the differences in the attention to islets for insulin and\nsomatostatin-stained chromogenic WSIs (Fig. 4A) are not statistically significant, similar to\nthe fluorescence equivalents (Fig. 5B). For glucagon, we see a difference in the\nfluorescence staining.\nThis discrepancy may originate from the difference between the modalities. The\nfluorescence modality obtains information from multiple stainings in the model at the same\ntime (as it is a staining set). Therefore, one possible explanation could be that glucagon is\nonly relevant when interacting with the other two stainings from that staining set\n(somatostatin and tubulin) but not by itself. In addition, chromogenic stainings have a\ndynamic range of only 1-2 orders of magnitude compared to 3-5 orders of magnitude in the\ncase of immunofluorescence stainings. The optimization of the multiplex fluorescence\nprotocols required the use of different antibodies, for instance against glucagon (mouse\nmonoclonal, 1:200; Abcam plc. #Ab10988, mouse), that those used for chromogenic\nstainings (anti-glucagon rabbit polyclonal, 1:600; Santa Cruz, #sc13091). We highlight this\ndiscrepancy between the modalities when interpreting the results now more in Lines\n276-285.\nAnswers for Reviewer 2\nQ1: It is concerning that the chromogen versus fluorescence-based method for islet\ndetection yielded such different results regarding islet number and size on consecutive\nparaffin-embedded sections, particularly when considering diabetes status. The data\ngenerated from fluorescence-based islet detection aligns with recent work demonstrating\nthat the mass of beta, alpha, and delta cells, the three primary types of islet endocrine cells,\nis unchanged in early-stage type 2 diabetes.\nAnswer: We appreciate the reviewer’s concerns about the islet segmentation methods\napplied to the chromogenic and fluorescence WSIs. While we use a deep learning-based\nsemantic segmentation on the chromogenic WSIs and a manual annotation on the\nfluorescence WSIs, the results between both modalities are very similar (excluding Perilipin\nas the segmentation model failed here). When only visualizing the distribution of the\nstandardized islet size and number of islets for the fluorescence modality (Suppl. Figure 14),\nwe observe that their distribution is very similar to that of the other chromogenic stainings,\nagain excluding Perilipin. An additional t-test supports this observation. As we agree that this\nis an important question we added the distribution of the average islet number and size,\nincluding the statistical tests, to the Appendix page 40 as Supplementary Figure 14.\nThe average islet size detected in the fluorescence WSI is smaller in T2D patients compared\nto ND individuals, supporting the results of the chromogenic staining presented in Fig. 7A.\nWhile most published studies report no significant differences in islet size or number, i.e.\nbeta cell mass between early-stage T2D and ND patients, prolonged metabolic stress may\ncontribute to the reduction in beta cell mass in long-lasting diabetes patients [1]. As our\ncohort is heterogeneous in terms of diabetes duration including also individuals with\nlong-lasting diabetes (Supplem. Table 2), this might explain the observed differences.\nHowever, our statistical models (Table 1a for chromogenic and Suppl. Table 3a for\nfluorescence), as well as the correlation analysis (Suppl. Figure 12), demonstrate that islet\narea and number of islets exhibit comparable effects across both modalities, showing similar\ncoefficient signs and magnitudes. Notably, neither parameter reaches statistical significance\nin predicting T2D status.\n[1] Chen, C., Cohrs, C. M., Stertmann, J., Bozsak, R., & Speier, S. (2017). Human beta cell mass and function in diabetes:\nRecent advances in knowledge and technologies to understand disease pathogenesis. Molecular metabolism, 6(9), 943-957.\nQ2: The most significant advantages of XAI are its ability to enhance model trustworthiness\nand reduce model complexity. Specifically, attribution scores can help verify that a model is\nmaking decisions based on relevant features rather than being misled by confounding\nfactors. However, in this study, these advantages are not applicable since the predictive\nmodel used is unlikely to be widely adopted by other researchers - predicting disease states\nfrom images does not seem to be a standard or practical approach.\nAnswer: While one of the advantages of XAI is indeed increasing the trustworthiness of a\nmodel, as the reviewer correctly noted, the main goal is different in our study. We are not\ntraining models to be able to diagnose T2D from pancreas tissues since this is much easier\ndone using blood samples. Instead, we aim to use XAI to gain a better understanding of the\ndisease itself. Using XAI to understand data is the purpose of an emerging subfield of XAI\nfor knowledge or scientific discovery, where scientists use XAI to understand complex and\nhigh-dimensional data from domains such as semiconductor design [2], solar cell production\n[3], or antibiotics discovery [4].\nAs discovering spatial traits of pancreatic tissue related to T2D is a nontrivial objective, we\nhypothesized that this approach could also work with our data and problem. To this end, we\ntrain models that can predict the T2D status solely based on the pancreatic tissue, a task\nthat cannot be performed by a human, and then use XAI to see what features are important\nfor the model to distinguish between T2D and non-T2D. By interpreting these features, new\ninsights can be generated.\nWe have revised the manuscript to better clarify this important distinction in our XAI\nmethodology and now provide a more detailed statement in Line 104.\n[2] Choubisa, H., Todorović, P., Pina, J. M., Parmar, D. H., Li, Z., Voznyy, O., ... & Sargent, E. H. (2023). Interpretable discovery\nof semiconductors with machine learning. npj Computational Materials, 9(1), 117.\n[3] Klein, L., Ziegler, S., Laufer, F., Debus, C., Götz, M., Maier‐Hein, K., ... & Jäger, P. F. (2024). Discovering process dynamics\nfor scalable perovskite solar cell manufacturing with explainable AI. Advanced Materials, 36(7), 2307160.\n[4] Wong, F., Zheng, E. J., Valeri, J. A., Donghia, N. M., Anahtar, M. N., Omori, S., ... & Collins, J. J. (2024). Discovery of a\nstructural class of antibiotics with explainable deep learning. Nature, 626(7997), 177-185.\nQ3: As the authors claim, XAI confirms hypotheses and lays the groundwork for future\nresearch. However, attribution and attention scores are complicated and need subjective\nhuman interpretation. The examined features in this manuscript still ultimately relate to\nchannel-level analyses and correlate with conventional features (e.g., islet area). There are\nno clear examples of novel hypotheses generated by XAI in this work, particularly ones that\nwould be difficult to analyze using traditional methods.\nAnswer: We respectfully disagree with this assessment. Our approach is fundamentally\nnovel in reversing the traditional biomarker discovery paradigm: rather than starting with\nhuman assumptions, we systematically analyze which features the AI model prioritizes,\nfiltering potential biomarkers without initial bias before human interpretation.\nThe convergence of XAI-derived features with conventional markers represents validation,\nnot limitation. Two fundamentally different methodologies independently identifying similar\nbiomarkers provides mutual confirmation and demonstrates that our models learn\nbiologically meaningful patterns rather than artifacts.\nOur methodology strategically combines AI's strength in processing large-scale datasets and\ndetecting subtle patterns with human expertise in biological interpretation and abstraction.\nThis hybrid approach enables the discovery of features that purely traditional,\nhypothesis-driven methods might overlook, particularly complex spatial relationships across\nlarge tissue areas.\nThe correlation with known features establishes biological plausibility while our novel\nmethodology opens pathways for feature discovery that traditional approaches can often not\nachieve. To strengthen this point that the novelty lies also in the methodology we added a\nsentence in Lines 118 and 640.\nQ4: While the computation of attribution scores may be unbiased, the study’s novel findings\nand its demonstration of XAI’s power appear limited.\nAnswer: Our study addresses a fundamental methodological challenge in biomedical\nresearch: the analysis of large-scale and multi-modal histological datasets that have only a\nvery limited tractability using traditional approaches. The volume and complexity of\nmulti-modal pancreatic tissue data would be prohibitively expensive and time-consuming to\nanalyze comprehensively through conventional manual annotation methods.\nOur approach represents the first human-AI collaborative framework for pancreatic\nhistology-based biomarker discovery in diabetes research, where XAI and deep learning\nprovide a more unbiased computational filter to identify critical regions of interest from vast\nhistological datasets. This methodology reduces human bias in feature selection while\nmaintaining biological interpretability through XAI techniques.\nThis novel approach revealed several biological insights that either support or challenge\ncurrent assumptions in T2D pathophysiology. Most remarkably, insulin was not the most\nrelevant marker for T2D prediction, which is a counterintuitive finding suggesting that beta\ncell compartments beyond insulin-containing granules may be critically relevant for disease\npathogenesis. The model's attention to tubulin uncovered alterations in islet innervation, a\nnovel and poorly understood T2D aspect that provides a rationale for investigating\nparasympathetic synapses on beta cell primary cilia [5].\nAdditionally, our findings highlight fibrosis patterns and pancreatic fat infiltration as key\nalterations in T2D, which could have clinical implications in diabetes prevention and\ntreatment, by directing future research to address fibrotic processes and reduce pancreatic\nfat accumulation.\nThese discoveries exemplify the transformative potential of human-AI collaboration in\nuncovering previously unknown pathobiological mechanisms at scale, representing a\nsignificant methodological advancement with substantial biological and clinical implications.\n[5] Müller, A., Klena, N., Pang, S., Garcia, L. E. G., Topcheva, O., Aurrecoechea Duran, S., ... & Solimena, M. (2024). Structure,\ninteraction and nervous connectivity of beta cell primary cilia. Nature Communications, 15(1), 9168."}, {"round": 2, "reviewer_comments": {"reviewer_1": "Overall, I believe that the authors have adequately addressed my previous questions. I think the work is ready for the\npublication in Nature Communications.", "reviewer_5": "In the present study, the authors develop an artificial intelligence-based tool for the analysis of histological sections of the\npancreas from both healthy individuals and patients with T2D. The authors trained the model using a cohort of samples\nsubjected to both chromogenic and fluorescent stainings and validated the training on a separate cohort. This approach\nenables the prediction of the patient’s status (T2D or non-T2D), as well as the identification of other potentially disease-\ndetermining factors in an entirely unbiased manner, since artificial intelligence does not rely on the same visual criteria as\nthe human eye to detect regions of interest and can identify patterns that may initially appear irrelevant.\nMy role in this review is to evaluate the authors’ replies to the comments and concerns originally raised by referee 2, who\nwas unable to provide a subsequent response. This evaluation is conducted through a detailed analysis of each specific\nquestion and the corresponding answer provided by the authors, followed by my own assessment after each point, as well\nas a general evaluation of the revised manuscript.\nQ1. The referee expressed concern regarding the discrepancy between the number of islets and the size of consecutive\npancreatic sections that differed only in the detection technique used (chromogenic versus fluorescent)."}, "author_response": ""}]

Decision letter