Who needs context? Classical techniques for Alzheimer’s disease detection
Year: 2023

Abstract:
1Institute of Biomedical Engineering, University of Toronto, Toronto, Canada 2KITEToronto Rehabilitation Institute, University Health Network, Toronto, Canada behrad.taghibeyglou@mail.utoronto.ca 3Faculty of Computer Science, Dalhousie University, Halifax, Canada 4Department of Computer Science, University of Toronto, Toronto, Canada 5Vector Institute for Artificial Intelligence, Toronto, Canada frank@dal.ca Abstract

heading: 1 Introduction
text: Alzheimer’s disease (AD) is the most prevalent form of dementia, a neurodegenerative disease that impairs cognitive functioning and is increasingly common in our aging society (Luz et al., 2021; Ilias and Askounis, 2022). According to the World Health Organization, approximately 55 million people currently suffer from dementia, with this number expected to surge to 78 million and 139 million by 2030 and 2050, respectively (Ilias and Askounis, 2022). Symptoms of AD include (but are not limited to) memory decline, disorientation, confusion, and behavioural changes. Importantly, AD progression can lead to loss of independence which significantly impacts patients, their families, and society as a whole (Pappagari et al., 2021). Given that late-stage AD progression is inevitable, early detection of AD through cost-effective and scal-
able technologies is critical. While most clinical diagnoses of AD rely on neuroimaging, there is a critical need for more accessible and efficient methods of diagnosis.
Accessible evaluation methods for AD include cognitive tests such as the Mini-Mental Status Examination (MMSE) (Kurlowicz and Wallace, 1999) and the Montréal Cognitive Assessment (MoCA) (Nasreddine et al., 2003). However, these methods still require active integration with an expert, and their specificity in early-stage diagnosis is questionable. During the course of AD, patients experience a gradual deterioration of cognitive function and accordingly may face a loss of lexical-semantic skills, including anomia, reduced word comprehension, object naming problems, semantic paraphasia, and a reduction in vocabulary and verbal fluency (Mirheidari et al., 2018; Pan et al., 2021; Chen et al., 2021). Speech processing and, consequently, natural language processing (NLP) can therefore provide new precision medicine tools for AD diagnosis that deliver objective quantitative analyses and reliable proof, analysis, comparison, and circulation for faster diagnosis.
The Alzheimer’s Dementia Recognition through Spontaneous Speech (ADReSS) challenge of INTERSPEECH 2020 is a shared database developed to advance research into automatic AD detection based on spontaneous speech and transcripts (Luz et al., 2020). Participants in the challenge were tasked with describing the Cookie Theft picture in English, which is part of the Boston Diagnostic Aphasia Exam (Guo et al., 2021). The first set of the ADReSS 2020 database comprises speech recordings and CLAN-annotated transcripts of 54 AD patients and 54 sex- and age-matched controls.
Various groups have worked with the ADReSS dataset, approaching the problem from different perspectives and leveraging available information. These studies typically combined speech processing and linguistic feature extraction or NLP-based
102
fine-tuning. The literature on speech processing mostly focused on zero-crossing rate, spectral bandwidth, roll-off, and centroids of audio recordings, as well as active data representation cluster-based feature extraction methods including the emobase (Eyben et al., 2010), ComParE (Eyben et al., 2013), and Multi-Resolution Cochleagram (MRCG) (Chen et al., 2014) feature sets. Meanwhile, linguistic features have extracted lexical richness, the proportion of various PoS tags, utterance duration, total utterances, type-token ratio, openclosed class word ratio, and similarity between consecutive utterances. NLP-based methods have comprised from-scratch training or fine-tuning contextbased models, such as bidirectional long shortterm memory (bi-LSTM) (Cummins et al., 2020), bi-directional Hierarchical Attention Network (biHANN) (Cummins et al., 2020), Convolutional Recurrent Neural Network (CRNN) (Koo et al., 2020), and Bidirectional Encoder Representations from Transformer (BERT) (Balagopalan et al., 2020). Despite excellent performance compared to baseline methods (Luz et al., 2020), the complexity of these methodologies and the need to implement them on high-memory GPUs highlights the need to explore simpler methodologies that can ensure ease and performance in AD detection.
In this paper, we present a novel approach for detecting AD in the first set of ADReSS dataset by integrating a new Word2Vec-based model and dimension reduction method. We not only implement and compare top-cited and recent state-of-the-art models on the same dataset, but also demonstrate that our approach outperforms these models. Our proposed approach is simple, easy to implement, and highly accurate.

heading: 2 Methodology

heading: 2.1 Other models
text: In order to evaluate the performance of our proposed language processing model, we have considered several publicly available models for comparison including:
• Linguistic-Based Features (LBF): In this study, we utilized the CLAN package to extract 34 linguistic-based features (LBFs) from transcripts, including duration, total utterances, mean length of utterance (MLU), typetoken ratio, open-closed class word ratio, and percentages of 9 parts of speech. We also incorporated demographic information such as
age and sex. To identify the most informative features for classification, we performed correlation and variance analyses on the extracted features using the FeatureWiz package (AutoViML, 2020). We set a correlation threshold of 0.6 and repeated the analyses 5 times with random seeds over all samples. We then selected the top 5 features that appeared in at least 3 iterations for further classification.
• BERT Models: Since BERT models have shown promising performance in different applications of NLP, in this study we leveraged some of BERT-based architectures with a maximum length of 512 tokens as a reference for our model. We tested three versions of uncased base BERT (Devlin et al., 2018): one with no extension in the last layers, called baseBERT1, another with two fully connected layers at the end (768 → 64 and 64 → 1), called baseBERT2, and the last one with three fully connected layers (768 → 128, 128 → 16, and 16 → 1), called baseBERT3. For baseBERT2, we varied the training epochs between 3 and 5. Additionally, we tested BioCLinical BERT (Alsentzer et al., 2019) with a batch size of 4 and 3 epochs, DistilBERT (Sanh et al., 2019) with a batch size of 4 and 3 epochs, and BioMed-RoBERTa-based (Gururangan et al., 2020) with a batch size of 4 and 3 epochs. We used binary cross-entropy as the loss function for all models and AdamW (Adam with weight decay) (Loshchilov and Hutter, 2017) as the optimizer with a learning rate of 2 × 10−5. To address potential issues with local optima, we applied a linear warm-up scheduler. Each transcript is classified as AD if the average of the probabilities (after the sigmoid layer) over all sentences in the transcript is greater than or equal to 0.5; otherwise, it is classified as control.

heading: 2.2 Pre-processing
text: To preprocess the data for our proposed model, we have neglected the first four sentences of each transcript, as the initial speaker is typically a member of the data collection team. Additionally, stop words were removed from each sentence using the Gensim library (Řehřek et al., 2011).

heading: 2.3 Proposed model
text: In this study, we used Wikipedia2Vec (Yamada et al., 2018), a tool that generates embeddings (or vector representations) of words and entities from Wikipedia, to convert tokens to vector embeddings. We used the skip-gram strategy for training, and the embedding dimension of the model was set to 500. We denote this model as W2V throughout this paper. Suppose that each participant’s transcript consists of Nk sentences, each comprising m words, where m varies from 1 to Mk (the maximum length among all sentences in the kth transcript). We input each word ⟨wi,k⟩ into the W2V model (W2V (⟨wi,k⟩)), which outputs the corresponding embedded vector xi,k ∈ R500. All embeddings of the kth transcript form the set Xk. We standardized each 500-dimensional vector across all embeddings of each subject using the following formula:
yk = med(Xk)
std(Xk) , (1)
where med is the median operator applied to each dimension independently, std is the standard deviation of embeddings, and yk denotes the standardized vector for the kth participant. So far, we developed the first framework and leveraged the previously introduced feature selection method by iteratively applying FeatureWiz five times. We then selected features that were chosen at least three times during the process to identify the most informative dimensions for AD detection. This feature selection procedure reduced the dimension from 500 to 64. We refer to this first framework as model 1, and Figure 1 illustrates the process. To further enhance our analysis, we concatenate linguisticsbased features from the previous section with W2Vbased feature vectors and apply feature selection in a similar manner to model 1. This second framework, called model 2, resulted in the selection of 86 features (out of 537 features). Prior to inputting the features into the classifiers of each model, the zero-mean-unit-variance standardization technique is applied to normalize the features.

heading: 2.4 Evaluation and Metrics
text: All results presented in this study were obtained using the leave-one-subject-out (LOSO) crossvalidation technique to evaluate the generalizability of the models. Thus, a total of 104 models were trained per architecture/classifier. For each model, accuracy, sensitivity, specificity, and F1 were re-
ported as performance metrics. For the featurebased models, such as linguist-based features and our proposed frameworks, we employed various classifiers including logistic regression (LR), decision tree (DT), linear and Nu-support vector classification (SVC), linear and quadratic discriminant analysis (LDA and QDA), Gaussian naive Bayes (GNB), extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), and extra trees classifier.

heading: 3 Results

heading: 3.1 Other models
text: We investigated different BERT models for AD classification, and the results are presented in Table 1. As expected, the performance of Bio-Clinical BERT and DistilBERT models were comparable; however, Bio-Clinical BERT showed superior sensitivity and was chosen as the best BERT model in this study. Additionally, as demonstrated in Table 2, integrating linguistic-based features with feature selection and a combination of classifiers achieved an accuracy of 0.81 in AD detection.

heading: 3.2 Proposed frameworks
text: The performance of our proposed frameworks is presented in Table 3. The best performance was achieved by model 2 with the help of the GNB classifier, which obtained an accuracy of 0.90. On the other hand, the best performance of model 1 was achieved by the ExtraTrees classifier.

heading: 3.3 Comparison with previous literature
text: Table 4 compares our proposed model with the existing models in the literature as well as the ones explored in this paper. Our model achieved significantly higher performance, including a 3% improvement in accuracy and an 8% improvement in sensitivity compared to one of the BERT-based
models on the same dataset (Balagopalan et al., 2020, 2021). It is worth noting that our proposed model also outperformed the baseline linguistic model introduced in the ADReSS challenge.

heading: 4 Discussion
text: By mapping each word into a 500-dimensional space where words with similar context are closer together, the proposed model can identify when all words in a transcript are focused on the same topic with minimal deviations. Coupled with the suggested standardization method, the results demonstrate a significant difference in performance between the proposed model and the only linguistbased model, which prioritizes utterances, pauses, and interactions between text and speech. The BERT models explored in this study are relatively massive and require significant computational resources, and training them requires delicate hyperparameter optimization. In this study, we followed the BERT authors’ recommendations to keep the model’s trainability on an Nvidia RTX 3080 GPU and to avoid changing the weights of the model by selecting smaller epoch numbers.

heading: 5 Conclusions
text: In this study, we introduced a word2vec-based model that combines pre-trained Wikipedia embeddings with linguistic features. We also employed correlation-based feature selection to reduce the dimensionality of the embeddings. The results demonstrated that our proposed model outperformed existing models on the same dataset. However, as BERT models offer diverse applicability, a potential future direction is to incorporate feature maps extracted from the hidden states of these networks to enhance the performance of our model.

heading: Acknowledgment
text: This research is supported by AI4PH, a Health Research Training Platform funded by the Canadian Institutes of Health Research (CIHR).

heading: Future Work
text: However, as BERT models offer diverse applicability, a potential future direction is to incorporate feature maps extracted from the hidden states of these networks to enhance the performance of our model