Doctor XA v I er: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation (2022)

Abstract: We introduce Doctor XAvIer — a BERT-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. We present a novel performance plot and evaluation metric for feature attribution methods — Feature Attribution Dropping (FAD) curve and its Normalized Area Under the Curve (N-AUC). FAD curve analysis shows that integrated gradients outperforms Shapley values in explaining diagnosis classification. Doctor XAvIer outperforms the baseline with 0.97 F1-score in named entity recognition and symptom pertinence classification and 0.91 F1-score in diagnosis classification.

## introduction previous studies have shown that electronic medical record (emr) data are difficult to use in machine learning systems due to the lack of regulation in data quality -emr data are often incomplete and inconsistent @xcite . recently, there have been attempts to improve automated clinical note-taking by extracting relevant information directly from physicianpatient dialogues @xcite @xcite . this can alleviate physicians of tedious data entry and ensures more consistent data quality @xcite . due to the potential in reducing costs associated with collecting patient information and diagnostic errors, there is increasing interest in using information extraction techniques in automatic diagnostic systems @xcite . @xcite introduced a system that extracts pertinent medical information from clinical conversations for automatic note taking and diagnosis. however, their methodology did not explore state-of-the-art natural language processing (nlp) techniques -entity extraction was done by searching the transcript for entities from medical lexicons and tf-idf was used for text classification. although there is existing work that employs more sophisticated nlp techniques to patient-physician dialogues @xcite , there is a lack of end-to-end diagnostic systems that employ such techniques. furthermore, all of the previous works mentioned fail to address the black-box nature of deep learning in the medical industry. most physicians are reluctant to rely on opaque, ai-based medical technology -especially in high-risk decision-making involving patient wellbeing @xcite . in this work, we present doctor xavier -a bert-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. feature attribution methods are explainable ai (xai) methods that compute an attribution score for each input feature to represent its contribution to the model's prediction. we report feature attribution scores using integrated gradients (ig) @xcite and shapley values (lundberg and @xcite to provide insight as to which features are important in diagnosis classification. descriptions of integrated gradients and shapley values are provided in appendix a. feature attribution scores could potentially help physicians build confidence in the model's prediction or give additional insight about the relationships between different diseases and relevant patient information @xcite . finally, we present a novel performance plot and evaluation metric for feature attribution methods -the feature attribution dropping (fad) curve and its normalized area under the curve (n-auc). ## fad curve analysis we introduce feature attribution dropping (fad) curve analysis for evaluating feature attribution methods. fad curve analysis requires no modifications to the original machine learning model and is simple to implement. ## methods and experiments we introduce doctor xavier -a medical diagnostic system composed of joint named entity recognition (ner) and intent (i.e. symptom pertinence) classification, primary diagnosis classification, and fad curve analysis. in this section we discuss each component in detail and evaluate each component. ## conclusion doctor xavier yields significant improvements in ner, symptom pertinence classification, and diagnosis classification compared to previous work @xcite , while also explaining why the model made each diagnosis. we also present a novel performance plot and evaluation metric for feature attribution methods -fad curve analysis and its n-auc. fad curve analysis shows that integrated gradients outperforms shapley values in explaining diagnosis classification in the verilogue dataset. in our future work, we will calculate β in a data-driven manner to standardize fad curve analysis for a given dataset. we will also apply fad curve analysis to other feature attribution methods, ai domains, and datasets to evaluate its generalizability.