Storyline-Centric Detection of Aphasia and Dysarthria in Stroke Patient Transcripts (2023)

Abstract: Aphasia and dysarthria are both common symptoms of stroke, affecting around 30% and 50% of acute ischemic stroke patients. In this paper, we propose a storyline-centric approach to detect aphasia and dysarthria in acute stroke patients using transcribed picture descriptions alone. Our pipeline enriches the training set with healthy data to address the lack of acute stroke patient data and utilizes knowledge distillation to significantly improve upon a document classification baseline, achieving an AUC of 0.814 (aphasia) and 0.764 (dysarthria) on a patient-only validation set.

## introduction published studies reported that about 30% of acute ischemic stroke patients are presented with aphasia as an initial symptom @xcite , while around half of these patients exhibit some form of dysarthria @xcite , with acute dysarthria specifically associated with small lacunar stroke primarily due to small vessel disease. the "cookie theft" picture description task shown in figure 1 is commonly used for language assessment in the nih stroke scale (nihss) to score the severity of aphasia and dysarthria among others. currently, the scoring is done by a certified healthcare worker. recent research has demonstrated the feasibility of deep learning-based stroke detection, using facial expression and voice data gathered from the "cookie theft" storytelling task (figure 1 ) that serves to differentiate mild/moderate stroke among stroke mimics in the emergency room @xcite . however, existing approaches to aienabled stroke prediction have only utilized the audio spectrum of patient recordings. the language content of these recordings is yet to be used for language assessment, even though the storytelling audio is often automatically transcribed. this motivates us to apply unstructured storytelling transcript and large-scale language model in order to predict the presence of aphasia or dysarthria in patients with stroke-like symptoms, using nihss subscores 9 and 10 as gold standard. in this paper, we present a new storyline-centric pipeline that uses transcribed patient descriptions alone to detect aphasia and dysarthria in patients with stroke-like symptoms. although no such study has been done in stroke to the best of our knowledge, these unlabeled patient transcripts are proven to be highly useful for language-related symptoms detection by a robust body of research in alzheimer's disease (ad) prediction and monitoring (de la fuente garcia et al., 2020). working with patient descriptions of the cookie theft picture amongst other transcripts, recent studies in ad discover that transformer-based language models that leverage a comprehensive language understanding @xcite @xcite tend to outperform models trained on syntactic, lexical, or pragmatic features alone @xcite . moreover, models that depend on syntax and pre-defined lexicons are more prone to racial and educational biases that 2 discriminate against patients who are non-native speakers or dialect users of english. the relative lack of nlp-enabled stroke detection could be due to the lack of patient textual data, while similar studies in alzheimer's disease could benefit from publicly available corpora of patient narratives, both on the cookie theft picture and otherwise (macwhinney 2019). to tackle this challenge, we are interested in exploring how data from cloud-sourced healthy volunteers, which is easier and more cost-effective to obtain, could be used to improve clinical nlp models. we experiment with two approaches to enrich our training set with healthy subject data, by including them first directly as the texts themselves and then indirectly as metadata representations in the form of knowledge graphs. by circumventing the data bottleneck, we believe that it is possible to improve nlp-enabled detection of language symptoms in stroke patients. ## knowledge distillation this section reports our experiment designs aimed at testing the hypothesis that knowledge distillation with storyline-encoded knowledge graphs, extracted from both healthy and patient transcripts, would transfer semantic knowledge to the enriched document classification model and improve the performance of detecting aphasia and dysarthria. as shown in figure 2 , the knowledge distillation pipeline has two components. first, we conduct unsupervised knowledge graph extraction with bert-based entity annotation and relation extraction models. second, we leverage these knowledge graphs to provide semantic reinforcement for the baseline electra document classification model defined in section 2.3. ## behavioral testing and discussion we conduct further qualitative testing to thoroughly evaluate the sensitivity of our aphasia and dysarthria models to different types of language errors and generalizability to external data. this is motivated by a recent surge in the behavioral testing of nlp models that challenge the effectiveness of common quantitative testing metrics @xcite . for nlp models in the clinical domain, @xcite highlight the need to simulate plausible real-life patient inputs to analyze model sensitivity directly. our main goals thus are to 1) verify that our models are in fact learning semantically, as expected by our methodology, 2) simulate "external" data to assess potential model overfitting to the specific clinical settings of the patient data, and 3) better understand the semantic limitations and boundary conditions of our models in order to make more accurate, informed, and measured claims about their clinical outcomes. for each patient transcript, we generate 9 versions of the original text that amplify types of language errors in both semantic and syntactic categories. ## conclusion and future work this work explores the under-researched area of applying nlp to unlabeled patient transcripts for supporting the triage and detection of stroke and stroke mimics. we introduce a storyline-centric approach that leverages data enrichment and knowledge distillation to overcome the lack of big clinical training datasets for automating aphasia and dysarthria detection. our experiments show that our approach to knowledge distillation has the potential to significantly improve the performance of patient document classification. nonetheless, we believe that it is possible to further enhance the results in figure 5 by designing more robust and effective knowledge distillation techniques to integrate transcripts, triples, and graph-theoretic aspects of kgs. our ongoing work include: 1) using sentence transformers @xcite to further denoise the output of kg extraction, 2) developing solutions to incorporate both semantic knowledge embeddings and graph embeddings in clinical document classification, and 3) recruiting spanish-speaking patients and healthy volunteers and expanding our storyline-centric pipeline to spanish language models @xcite , to better serve the clinical needs of the hispanic community in stroke triage and detection.