Leveraging Natural Language Processing and Clinical Notes for Dementia Detection (2023)

Abstract: Early detection and automated classification of dementia has recently gained considerable attention using neuroimaging data and spontaneous speech. In this paper, we validate the possibility of dementia detection with in-hospital clinical notes. We collected 954 patients’ clinical notes from a local hospital and assign dementia/non-dementia labels to those patients based on clinical assessment and telephone interview. Given the labeled dementia data sets, we fine tune a ClinicalBioBERT based on some filtered clinical notes and conducted experiments on both binary and three class dementia classification. Our experiment results show that the fine tuned ClinicalBioBERT achieved satisfied performance on binary classification but failed on three class dementia classification. Further analysis suggests that more human prior knowledge should be considered.

## introduction dementia describes a collection of symptoms that are caused by disorders affecting the brain. the global burden of dementia is large and expected to triple by 2050 in the absence of a treatment @xcite . the application of deep learning to early detection and automated classification of dementia has recently gained considerable attention @xcite , as rapid progress in neuroimaging techniques has generated large-scale multimodal neuroimaging data. the adress challenge @xcite released a benchmark dataset of spontaneous speech, which is acoustically pre-processed and balanced in terms of age and gender, defining a shared task through which different approaches to dementia recognition in spontaneous speech can be compared, several speech classification models were used for dementia detection, in which different types of linguistic features were extracted and fed into traditional statistical models. this study is an interesting proof of concept, with fewer than 100 patients. more recent studies @xcite measured the impact of linguistic features (e.g. verbal disfluency tags) on dementia detection. dementia can be an underlying cause of hospital admissions, for example due to increased rates of falls in dementia sufferers. however the diagnosis associated with the admission will be a fracture, rather than dementia. in this paper, we test the possibility of early detection for dementia patients based on the in-hospital clinical notes. specifically, we collected 954 patients' clinical notes from melbourne frankston hospital foot_0 and assign dementia/non-dementia/uncertainty labels. given the labeled dementia data sets, we develop a deep learning model based on clinicalbiobert @xcite . we experiment with both the binary (dementia/non-dementia) and the coarse (dementia/non-dementia/uncertainty) settings, and find that clinicalbiobert works well in the binary setting but performs poorly on the coarse setting, at the same time it still suffers from the low annotation problem, and the embedding representation is not effective as the structured representation (e.g. umls concept representation). "poor" in this context is in comparison to traditional statistical and machine learning classifiers (not discussed here). our main contributions are: ## dementia dataset construction in this section, we describe how we collected the medical notes and acquired the gold-standard labels, we also show some of the basic data statistics. ## methodology in this section, we describe the development of classification models using the data sets described above. there are two aspects to consider before the model development. first, what is the classifier's granularity? there could be three levels of input, i.e., sentence, document and patient (multidocument) level. typically, it is more challenging to achieve high performance when the input text is longer. however, developing a sentence or single document level classification model requires fine grained annotation, which is often time consuming and expensive in the medical setting. meanwhile, ## experiments experiment setup we experimented with two classification schemes: binary and three-class. for the binary case, only the medical notes for patients from cohorts 1a and 2a were selected, which is an exact binary classification (1a v.s. 2a) setting. in contrast, in the three-class setting, we regard the 1b and 2b cohorts as the uncertainty group, which returns the three class (1a v.s. 2a v.s. uncertain) setting. for the clinicalbiobert model, we keep the default settings and trained 20 epochs until convergence. like other biomedical settings, we use accuracy, precision, recall and micro f1 as the evaluation metrics. we also add a keyword method as the naive baseline, where we use a pre-recognized 245 umls concept names as a keyword list, these concepts are recognized by human experts to correlated with dementia. if any of those concept names appear in the document, we give a prediction of ## analysis even though bert based models show superior performance on most generic text classification tasks, the fine tuned clinicalbiobert does not exhibit satisfied results in the coarse setting in this study. we anticipate three reasons: first, the clinical notes are too long for clinicalbiobert to encode, since the standard bert models can only take an input length of 512 tokens. meanwhile, the dementia related text spans are quite sparse in the clinical notes, further text compression and selection heuristics are required. furthermore, the bert based modeling techniques cannot leverage expert prior knowledge, which in this study are the filtered umls concepts. to validate our hypothesis, we apply t-sne for the 954 patients' feature representation with either the 245 umls concepts or the [cls] embedding of the clinical note summary. as shown in figure 1 (a), the umls concept representation is more meaningful, as those positive dementia patients can be easily separated with those negative patients, while in figure 1 (b) there is no clear representation patterns for these three classes. ## conclusion in this work, we collected clinical text from a local hospital and leveraged deep neural models for dementia detection. we fine tuned a clinical-biobert and evaluated its their performance on dementia classification, experiment results showed that the fine tuned model works well on binary dementia classification but fails on three class dementia classification. as for the future work, we will leverage more human prior knowledge and experiment with both statistical and deep neural models. also, more structured patient representation using knowledge graphs will be considered.