Analysis of Healthcare Data Explainability using Language
Models and Rough-fuzzy
possibilities
Lipika Dey

IJCRS 2024

“AI in healthcare is said to be the hope, the hype, and the promise”- Nazar et al., 2021

Intrinsic XAI

Post hoc XAI
Accuracy
Reliability

Explainability

Interpretability

AI in healthcare
Personal Digital Healthcare

Prediction - Sepsis severity,
Medical ProcedureExplainable AI - XAI
Explainability of model - knowledge about the model-Important for a machine learning engineer to design optimalmodelsInterpretability - ability of the model to provide causal
association between input and output-

Meaningful insight about the logic behind the decision arrived
at using the data-

Important for Healthcare end-users

Healthcare Data
Patient Vitals-Heart rate
Respiratory rate
Temperature
Oxygen saturation
Intubated - Yes /No
Disease names
Drugs

Advantage of using text for prediction
1. Details about patient's health status

1. Interpretability is better

Nursing Notes-Detailed notes from - doctor, nurse, pathologist, medical technologist

Length of Stay at ICU / Hospital - a prediction problem
Can admission statistics
predict Length of Stay inICU / Medical procedurerequirement
Can these decisions beinterpreted by healthcare
professionals?
Nursing Notes

Deep Neural Networks for Predicting ICU LOSX-BERT

SOI Scores

Transformer +LSTM - handles
long notes

BioNER
TF-IDF weights of Top 2000 significant
Linguistic features fed additionally

Clinical BioBERT - developed
byAccuracy
further training theofBioBERT(base version; 110
millionparameters with 12Prediction
layers, 768 hidden units, and
12 attention heads) using
improves - butclinical text from the MIMIC-IIIcorpus.

can clinicians

BlueBERT - language modeltrainedon Biomedicalandinterpretwhy?
Clinical texts – PUBMED
abstracts and MIMIC III clinicalnotes

Can pre-trained LLMs be used to summarize the notes?
Summarize the health condition for the note within quotes and predict possible length ofstay in hospital "Recieved alert and oriented 46yr old male with acute onset sob andhypertension with bp 227/104 in ER that was refractory to hydralazine and metoprolol. Pt
arrived on nipride gtt set at 2mcg/kg/min. Currently he is in SR/no ectopy with bp currently
175/87. 02 has been titrated down to 2L nc with 02 sat 97%. Pt states his breathing "feelsbetter", lungs with exp wheezes anteriorly, decreased at bases. +multifocal pneumonia by
CT and vanco/cefampime initiated per ER report. Pt also states "cough since Wednesday"with scant production of white/yellow sputum. Temp on arrival 99, blood cultures pending.
GU; ESRD/HD with pt currently in transition to peritoneal dialysis. Last HD [**2109-3-25**]
with 2.7L off per patient. Left upper arm AV fistula with +bruit/neg thrill and "failed" per pt.
Right chest HD catheter in place with dressing clean and dry. Mature/healed peritoneal
dialysis catheter in place at left lower abdomen. Abd soft, bs present, last bm 4/17 per pt.
Wife is listed as hcp. PO antihypertensives initiated with plan to wean off nipride gtt andgoal sbp 140-160. Pt aware of and agreeable to plan of care. ?d/c out of ICU in am if
medically stable."Local Interpretable Model-Agnostic Explanations
Explains each individual
prediction
For a given black-box classifier
For an input - generates “similar”
samples through perturbation

minimizes the locality-aware loss L(f,g,Πx) measuring how unfaithful g approximates themodel to be explained f in its vicinity Πx with leastcomplexity

Understanding individual predictions

Patient 1

Patient 2

Contribution of each term in Clinical note towards the final prediction

Local interpretability not robust● LIME focuses on individual instances - overlooks global patterns● Clinicians require both pictures to make informed decision understand each patient with respect to a cohort● Good for structured - can’t handle text well
○ Discrete Words don’t provide the full picture■ “No fever” vs “fever” can change the sense■ Handling synonyms - “diabetes” vs “high blood sugar”

We want them all!

HighInterpretability
Local and Global
context - individual
patient along withcohort

03

Complex Data
Clinical notes plus
patient healthparameters

01

02

Powerful Model
Model accuracy
cannot becompromised

Unstructured → Structured Representation
Unified Medical Language

System (UMLS) - brings
together many health andbiomedical vocabularies andstandards to enableinteroperability

Generating Discrete
Representations fromtext
Number of features - very largeO(4000 - 5000) features for patients
admitted with one major disease

Autoencoders for representation of Nursing Notes
Binary vector of 4541 health conditions – Large and Sparse

HCHC2m-1

EntityStandardization
UMLS

Identifying
Negation
NegEx

BioNER
SciSpacy, MetaMap

ClinicalNotes

500 Dimensional vector

HC1

Clustering Patients –obtaining patient cohorts

Need Interpretation of clusters / cohorts

2000+ Pneumonia patientsSHAP explainers● Uses game theory concepts to compute feature contributions foreach prediction● Provides both local and global explanations○ Offers insights into individual predictions (local explanations)○ Quantifies feature importance across the entire dataset (globalexplanations)● Decomposes the model's prediction into the contributions ofindividual features based on their importanceSHAP value computation● Computes for members ofPower-set of S
● Finds features that maximize
intracluster and minimize
inter-cluster similarity

SHAP explanations for clusters
Each cluster could bedescribed uniquely using
top three disease /symptom as attributes(present / absent)
Cluster 1 – Influenza, LungConsolidation absent

Cluster 0 – Diabetic
Patients withEndometriosis

Cluster 2 – Acquired Abnormalityof Atrium, Left Atrial Abnormality

Clusters for Pneumonia Patients

Using Cohorts to understand new patients

New patients

Prediction for new patient - using cohort information

Cluster ID

F1

F2

Cohort closenessinformation

…

F500

Length of Stay

Accuracy Improves

Risk Prediction and Recovery Pathways
Day 1

Day k

Day n

Symptoms
Present on
Day 1

Symptoms
Present on
Day k

Symptoms
Present on DaynImprove /Deteriorate/
Persistent

Improve /Deteriorate/
PersistentTransition Probabilities can be computed for each cluster

Predicting Risk for Sepsis Patients
SEPSIS is a generic term - Severity score is determined based on multiple
conditions
Sepsis Severity score = f (Temp., Heart rate, Resp. rate, WBC count, Organfailures)●●●●

SIRS - Severity Score 1
Information gathered from clinical notes
Sepsis - Severity Score 2
Severe Sepsis - Severity Score 3
Septic Shock - Severity Score : 4

For a newly admitted patient compute severity score - Predict oneof the following outcomes after n days Improve (score-)/ Deteriorate (score +) / Persist (no change)/Discharge / Decease

Using Cohorts to predict outcome for newly admitted
patients after 3 days

New patients

MIMIC-III, v1.4, 1593 patients identified with SEPSIS on admission

Note - SEPSIS is used
in a generic sense - it
could be infection Initially grouped into 8 cohorts

It’s the severity scorethat matters

Distribution of features across clusters

Feature distribution across outcomes

Discharged
patients hadvery few
symptoms

Deceased andDeteriorate,
Persistent andImproved
have similar
distribution offeatures
No clearpictureemerging atindividual
symptom levelCluster descriptions obtained using SHAP explainersPredicting outcome after 3 days from admission
Model for predicting

Accuracy of prediction

BlueBERT representation ofadmission notes

43%

Autoencoded UML features(500 dim)70%

Autoencoded UML features(500 dim) + cohort
information75%

Transition probabilities to next stage for each cluster

Zoom in - symptoms responsible for state transitions
Interpretation
Patients whose hypotension wastreated effectively - - likely to
improve
Patients who has developed
Erythema and/or diabetic - likely toremain persistent or deteriorate

Green symptoms thathave effectively
cured
Red - Newsymptoms

Note - only two shown
here - there are othersIndividual PatientsPsis
No feverDyspnea
Hypotension
PainChills
AnemiaImproved

None

Atrial_premature_com
plexes
SepsisNo melanoma
Weakness
Muscle_weakness
Urinary_tract_infectio
nHematuria
Osteoporosis
No urosepsis
Benign_hematuria
Hypertensive_disease
No coughNo hypotension
Mental_depression
No_dehydration

Improved

AtrialArrhythmia

Thick_sputum pulmonary_edema#cough#nausea#diabetes#anasarca#agitation#yellow_sputum#hemodynamically_stable#anxiety#edema#able_to_communicate#dependent_for_dressing#fatigue#pressure_ulcer#premature_ventricular_contractions#body_substance_discharge#dyspne
a#pain#septic_shock#premature_ventricular_contraction#respiratory_distress#clear_sputum#tingling_in_fingers#intraventricul
ar_conduction_defect#feeling_relief#gastrointestinal_leak_nos#chest_pain

Thick_sputum
pulmonary_edema#cough#nausea#diabetes#anasarca#agitation#yellow_sputum#hemodynamically_stable#anxiety#edema#able_to_communicate#dependent_for_dressing#fatigue#pressure_ulcer#premature_ventricular_contractions#body_substance_discharge#dyspnea#pain#premature_ventricular_contraction#respiratory_distress#tingling_in_fingers#intraventricular_conduction_defect

Persistentcardiac problem
premature ventricular contractions#mediastinitis#thick sputum#empyema#ulceration#bacteremia#aortic coarctation#anxiety#premature ventricular contraction#infection#pleural effusion (disorder)#candidemia#deep vein thrombosis#edema#candidiasis#tachycardia#hypotension#yellow sputum#hernia#diabetes

Shapley Values for Building a Clinical Decision Support System(Bienefeld et al., 2023)
Predict the onset of Delayed
Cerebral Ischemia in patientswith aneurysmal subarachnoid
hemorrhage (aSAH)
Predicted score - 0.8
Based on static and dynamicfeatures
Pink - high risk
Blue - low risk

What next?

Can rough-fuzzy ideas be usedfor explanations?

T-norm and Implicator functions canhelp in quantifying the dependencies
between sets of attributes

To be explored ..
Contribution of combination of features to classes
For each member - predict outcome possibilities based onpresence of features

Track changes in symptoms and dynamically predict nextoutcome

Thank You!