Venue: NAACL 2019

Multilingual Prediction of Alzheimer's Disease Through Domain Adaptation and Concept-Based Language Modelling

 Kathleen C. Fraser\({}^{1}\), Nicklas Linz\({}^{2}\), Bai Li\({}^{3}\), Kristina Lundholm Fors\({}^{4}\), Frank Rudzicz\({}^{3,5}\), Alexandra Konig\({}^{6}\), Jan Alexandersson\({}^{2}\), Philippe Robert\({}^{6}\), Dimitrios Kokkinakis\({}^{4}\)

###### Abstract

There is growing evidence that changes in speech and language may be early markers of dementia, but much of the previous NLP work in this area has been limited by the size of the available datasets. Here, we compare several methods of domain adaptation to augment a small French dataset of picture descriptions (_n_ = 57) with a much larger English dataset (_n_ = 550), for the task of automatically distinguishing participants with dementia from controls. The first challenge is to identify a set of features that transfer across languages; in addition to previously used features based on _information units_, we introduce a new set of features to model the order in which information units are produced by dementia patients and controls. These concept-based language model features improve classification performance in both English and French separately, and the best result (AUC = 0.89) is achieved using the multilingual training set with a combination of information and language model features.

National Research Council Canada, Ottawa, Canada

\({}^{2}\)German Research Center for Artificial Intelligence (DFKI), Saarbrucken, Germany

\({}^{3}\)University of Toronto and Vector Institute, Toronto, Canada

\({}^{4}\)University of Gothenburg, Gothenburg, Sweden

\({}^{5}\)St. Michael's Hospital, Toronto, Canada

\({}^{6}\)University of Cote d'Azur, Nice University Hospital, and INRIA, Nice, France

kathleen.fraser@nrc-cnrc.gc.ca, nicklas.linz@dfki.de, bai@cs.toronto.edu, kristina.lundholmfors@qu.se, frank@cs.toronto.edu, alexandra.konig@inria.fr, jan.alexandersson@dfki.de, probert@unice.fr, dimitrios.kokkinakis@gu.se

## 1 Introduction

According to the World Health Organisation, the largest global challenge facing the world today is the rapid increase of the population aged over 65 years. It is projected to increase from 524 million in 2010 to 1.5 billion in 2050, with the largest increase in the developing world [Suzman and Beard2011]. This demographic trend has profound societal implications; for example, the number of persons affected by dementia will increase worldwide from 46 million in 2015 to 131.5 million in 2050 [Prince et al.2015]. The most common underlying condition causing dementia is Alzheimer's disease (AD). Although no cure to this neurodegenerative disease has been found, experts agree that intervention in early stages is crucial to delay onset [Dubois et al.2016].

AD is characterised by a global impairment of cognitive functioning, with specific deficits in episodic memory, executive functioning, perceptual speed and language [Backman et al.2005, Weiner et al.2008].

Machine learning experiments using speech and language for the detection of dementia or related disorders have been conducted in many languages, including English [Roark et al.2011, Mirheidari et al.2016, Fraser et al.2016, Asgari et al.2017], French [Troger et al.2017, Konig et al.2018], German [Weiner et al.2016], Hungarian [Szatloczki et al.2015, Vincze et al.2016], Spanish [Meilan et al.2014], Greek [Satt et al.2013], Swedish [Lundholm Fors et al.2018, Fraser et al.2018a], Japanese [Shipata et al.2016], Portuguese [Alustio et al.2016], and Mandarin Chinese [Lai et al.2009]. Most studies acknowledge that small data sets are a limitation and describe the difficulties in gathering more data, including the challenges in patient recruitment, the expense of running clinically-based studies, and the manual effort required for transcription and annotation.

Here, we consider whether it could be possible to increase the amount of available data by augmenting a corpus in one language with data from another language, and thus improve predictive per 

formance without the need for new data collection. Specifically, we consider augmenting a relatively small French dataset with a much larger English one. The two aims of this study are: (1) to identify a set of features that are both useful for the detection of dementia and that we expect to transfer across different languages, and (2) to improve classification results on the French dataset by augmenting the training set with English data.

## 2 Background and Related Work

### Narrative Analysis in AD

One way to assess language is through narrative speech, such as that elicited by the _Cookie Theft Picture_ (CTP) task Goodglass et al. (2000). In this task, participants are asked to describe the content of a line drawing of a kitchen scene, where a boy can be seen standing on a stool, trying to reach a cookie jar, while a woman is preoccupied washing dishes. In this study, we analyse CTP narratives, due to the widespread use of the task in multiple languages.

Narrative speech can be analysed on a number of levels, including phonology, morphology, syntax, semantics, and pragmatics. Here, our goal is to extract features that both predict AD and are likely to transfer across different languages. Although other studies have used acoustic features for this task Meilan et al. (2014); Konig et al. (2018), there are well-documented differences in the phonology and prosody of French and English Bertran (1999); Vaissiere (2002). Syntax and morphology also differ across languages, and the degree to which they are impaired in mild to moderate AD is unclear Taler and Phillips (2008). Pragmatic ability in AD may be disrupted Chapman et al. (1998); Boschi et al. (2017); however, the CTP is not ideally suited for assessing pragmatics.

Instead, we focus on the semantic level, with the assumption that while the specific vocabulary will be different across languages, the underlying meanings or semantic concepts expressed should be the same. Features relating to semantic content are also motivated by the AD literature. Cuetos et al. (2007) reported a significant reduction in semantic units produced by pre-clinical AD participants, relative to controls, on the CTP task. Croisile et al. (1996) studied CTP descriptions from French participants, and found that the AD descriptions were shorter and less informative than the control descriptions. They measured information content by scoring the narratives against a gold standard list of 23 expected "information units", which have been widely used in subsequent research.

### NLP for AD Classification

Several recent studies have used NLP and machine learning to analyse speech samples from people with dementia and other cognitive disorders. Most relevant here, are those which focus on picture description tasks in English or French.

DementiaBank1 is a large database of CTP narratives from AD patients and controls, containing primarily English data. A number of recent papers report classification results on this corpus Prud'hommeaux and Roark (2015); Fraser et al. (2016); Al-Hameed et al. (2016); Yancheva and Rudzicz (2016); Sirts et al. (2017). Language analysis of English-language CTP data from other sources has also been used to differentiate between different underlying pathologies in AD Rentoumi et al. (2014), and variants of frontotemporal lobar degeneration Pakhomov et al. (2010).

Footnote 1: [https://dementia.talkbank.org/](https://dementia.talkbank.org/)

In French, picture description was one of multiple tasks used to elicit speech for the classification of participants with mild cognitive impairment and AD reported by Konig et al. (2015) and Konig et al. (2018), although only acoustic processing was used.

### Multi- and Cross-Lingual NLP

There has been very little prior work on multilingual or cross-lingual dementia classification. Renduomi et al. (2018) presented preliminary results suggesting that some language features from CTP samples could transfer across Greek and English, but did not report classification results. Fraser et al. (2018) studied a related task of detecting mild cognitive impairment (MCI), and found that classification results could be improved in both English and Swedish by incorporating multilingual topic modelling into the feature extraction pipeline; however, they did not consider multilingual classification directly.

More generally, multilingual NLP is an active and growing area of research. Some approaches to improving classifier performance on a resource-poor _target_ language by leveraging a resource-rich _source_ language include: translate the target language to the source language (or vice versa) and 

train a unilingual classifier (Wan, 2009); extract features from the two languages separately and then use domain adaptation techniques to train a classifier for the target language Blitzer et al. (2006); Prettenhofer and Stein (2010); or determine a common representation for both languages and then extract features from the combined corpus to train a multilingual classifier (Ammar et al., 2016). In the extreme case, one can also consider purely cross-lingual classification, in which the classifier is trained solely on the source language, but tested on the target language.

We use a supervised domain adaptation approach, similar to that of Daume III (2007), by considering each language to be a different domain. In related (though not multilingual) work, Masrani et al. (2017) also used this approach to adapt a dataset of AD narratives to their MCI classification task.

### Class-Based Language Modelling

In contrast to the previous work on AD classification, we measure not only which information units are mentioned, but also the order in which they are mentioned. Our approach has some similarity to class-based language models Brown et al. (1992), in which words are first grouped into classes (or clusters), and then the language model is trained on the classes rather than the individual words. One benefit to this approach is improved generalisability Hoidekr et al. (2006), and another is the ability of classes to span different languages Tackstrom et al. (2012).

## 3 Methodology

### Data

Data were taken from two corpora: a small French dataset (\(n=57\)), collected at the Memory Clinic and Research Centre of the University Hospital Nice, and the Pitt subcorpus of DementiaBank, containing 550 English samples2. Detailed information about the protocols for each study can be found in Troger et al. (2017) and Becker et al. (1994). In both cases, ethics approval for the data collection was obtained from the local governing bodies.

Footnote 2: In this analysis, we included all participants in the Dementia subfolder, regardless of specific diagnosis, to maximize the size of the source data.

The demographics for the participants in each language are shown in Table 1. In both studies, the participants were asked to perform the CTP task in their respective languages. In English, the image was shown on paper and speech was digitally recorded, while in the French study, the image was displayed on a tablet and speech was recorded via the tablet microphone.

### Features

The English and French audio samples were manually transcribed using the CHAT protocol MacWhinney (2014). A set of pre-defined information units found in the CTP was determined as an extension to Croisic et al. (1996), and is given in Table 2a. Mentions of information units were determined using keyword-spotting (based on manually-constructed word lists specific to each language), and used to translate the full narratives to sequences of information units. As an example, the English _A boy is standing on a stool_ and French _Le gargon est sur un tabouret_ would both be mapped to the sequence Boy Stool.

Features relating to the occurrence of each distinct information unit comprise the _info_ feature set, described in Table 2b. Additionally, new features are derived from language models build on the sequence of information units. To this end, concept-based language models are trained for English and French in a leave-one-out fashion, using the kenlm framework Heafield (2011). Models up to 5-grams were constructed. For each participant, two language models are constructed for each \(n\): one trained on the healthy control (HC) population and one trained on the AD population. The participant is left out of the model built on their associated diagnostic group. The trained language models are then applied to the held-out participant's sequence of information units and various language model (LM) features are extracted (Table 2c).

\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{English} & \multicolumn{2}{c}{French} \\  & HC & AD & HC & AD \\ \hline N & 241 & 309 & 25 & 33 \\ Gender & 154F/87M & 189F/12OM & 19F/6M & 22FF1M \\ Age & 64.8 (7.7) & 71.4 (8.4) & 75.4 (7.0) & 79.2 (6.6) \\ Education & 14.2 (2.6) & 12.8 (3.0) & 14.0 (2.6) & 11.3 (4.0) \\ MMSE (3/0) & 29.1 (1.1) & 19.8 (5.7) & 28.6 (1.4) & 18.9 (3.9) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Demographics of participants, where AD indicates Alzheimer’s disease, and HC indicates healthy control. The Mini Mental State Examination (MMSE) is global measure of cognitive status.

 



### Unilingual Classification

To evaluate the performance of the three proposed feature sets (_info_, \(LM\), and _info+LM_), we first train classifiers to distinguish between HC and AD participants within a given language. To examine the importance of certain features, we restrict ourselves to more explainable linear models, namely logistic regression (LR) and linear support vector machines (SVM) Pedregosa et al. (2011). In both cases, we use \(L_{1}\) regularisation to promote sparsity in the feature weights.

Area under the Receiver-Operator curve (AUC) is reported as the evaluation parameter. Due to the small size of the French dataset, we use leave-pair-out cross validation (LPO-CV), which has been shown to produce an unbiased estimate for AUC on small datasets Airola et al. (2009), and has also been used in related work Roark et al. (2011). However, since LPO-CV is computationally very costly, we instead use 10-fold cross-validation (10-CV) for English, making sure that any samples for a given participant occur in either the training set or the test set, but not both. For LPO-CV we compute AUC and its standard deviation as described by Roark et al. (2011); for 10-CV we compute the AUC in each test fold and then report the average and standard deviation over folds.

Feature scaling and hyper-parameter optimisation is done on the training set in each fold. Features are scaled using Maximum-Absolute Scaling to preserve the binary nature of the _info_ features. For both SVMs and LR, \(C\) was optimised between \(C\in[10^{-4},...,10^{4}]\) using a grid search.

### Multilingual Classification

Our goal is to improve classification in French, by incorporating training data from English. To this end, we examine multiple ways to combine data from both English and French in the training set.

We first consider _domain adaptation_, where we treat French as the target domain and English as the source domain. We implement the Augment method of Daume III (2007), which involves augmenting the feature space with source-specific, target-specific, and combined versions of all the original features, allowing the classifier to assign a higher weight to the combined version when that feature transfers well across domains, while also retaining source- and target-specific information where appropriate.

We consider as well as the baseline methods described in Daume III (2007): Weight, in which the samples from the source domain are assigned reduced weights in the model; PRED, in which the prediction made by the source classifier is used as an additional feature in the target model; LinInt, in which the predictions from the source and target models are linearly interpolated; and All, in which target and source data are simply combined in a single training set. Due to the limited size of our data, we do not optimise the weighting factors in Weight and LinInt, but rather assume the two languages should be given equal importance, and use a weighting factor of 0.1 in Weight (since the English data is 10 times the size of the French data), and 0.5 in LinInt.

Another option is to combine the French

\begin{table}
\begin{tabular}{l l} \hline
**Actions** steal, fall, wash, overflow, girl’s action, woman’s indifference & \\
**Actors** boy, girl, child(bren), woman & \\
**Places** kitchen, exterior & \\  & \\
**Objects** cookie, jar, stool, sink, dishcloth, water, window, cupboard, dish, curtain, counter & \\ \hline \end{tabular}
\end{table}
Table 2: Top, the information units extracted from CTP narratives. Bottom, the _info_ and _LM_ features that are computed from the resulting sequence of information units.



and English datasets before extracting features. Specifically, we first replace the word-level transcripts with the sequence of information units, and then combine the two datasets and train the language models over the multilingual corpus, thus generating _multilingual language models_.

### Cross-Lingual Classification

To understand how well a trained classification model in one language could be applied to another, we also perform cross-lingual experiments. For this, we train language and classification models in one language and test it on the other.

## 4 Results

The results of the classification experiments are presented in Figure 1.

### Unilingual Classification

In French, for both LR and SVM, using _LM_ features leads to higher AUC than the _info_ features, and the combination of features is more effective than either feature set alone. In the English case, the _LM_ and _info_ features lead to equivalent performance individually, but the AUC is again marginally improved when the feature sets are combined, suggesting that they are capturing at least somewhat complementary information.

### Domain Adaptation Results

For French, the _LM_ features generally do not benefit from domain adaptation, with equivalent or poorer AUC relative to the unilingual case. The best result with the _LM_ features is achieved in the Augment scenario, where the classifier can select the French _LM_ features only (although this result holds only for the SVM classifier). In contrast, the _info_ features do benefit from the additional data available through domain adaptation, and lead to better results than the unilingual baseline. The best overall result of AUC = 0.89 is achieved by combining the feature types in the All configuration.

For English, we do not expect to see much benefit from including the (much smaller) French dataset. The Weight adaptation technique is not feasible when the source data is smaller than the target data, and the LinInt technique performs poorly, as it assigns too much importance the smaller and out-of-domain dataset. However, we do see marginal improvements using All and Augment, reflecting the value of increasing the training set size by roughly 10%. The best result of AUC = 0.84 is achieved in the All condition, using the combined feature set.

### Multilingual LM Results

Using the multilingual LM does not affect the _info_ features, and therefore Figure 1 shows only the _LM_ and _info+LM_ results. Clearly, the multilingual LM approach does not work well here. Unlike in domain adaptation, combining the datasets using this method assumes that information units will be produced in the same order in the two languages. While French and English are similar in this respect, there are many possible counter-examples, such as _cookie jar_ (Cookie Jar) versus _boite a biscuits_ (Jar Cookie).

### Cross-Lingual Classification

When training entirely on English data and testing on French, the results using _info_ and _info+LM_ features are significantly improved over the unilingual baseline, while the _LM_ results are reduced, once again indicating that the _info_ features transfer better across languages. The results are very similar to those using the All technique for domain adaptation, suggesting that in that case, model training is dominated by the English data.

To further explore the similarity in performance in the All and cross-lingual cases, we examine the effect of incrementally increasing the amount of English data in the training set, when testing on French data. Figure 2 displays the classification performance of SVM and LR classifiers trained either using the All method of domain adaptation or cross-lingually with increasing amounts (10% at a time) of the English data. Considering first the All method (red and blue), at \(x=0\) there is no English data, and so we recover the French unilingual baseline. As we increase the amount of English data in the training set, performance slowly increases, eventually reaching the values reported in Figure 1. Considering next the cross-lingual case (yellow and green), we see that training on only 10% of the English data (55 samples) results in much poorer AUC values. However, each further 10% increases the classification performance. At 80% of English data (440 samples) the multi- and cross-lingual cases converge in performance. Thus, it would appear that domain adaptation is more data-efficient, as we achieve close to optimal results with a smaller proportion of English data, 

but that the cross-lingual approach can be equally effective, given a large enough corpus.

### Feature Analysis

Finally, we examine the features to determine which features are most useful to the task of dementia detection, and to compare the selected features in the unilingual and multilingual cases. Figure 3 shows the median absolute value of the weights assigned to each feature, for English and French, in the unilingual and multilingual All condition. The \(L_{1}\) regularisation serves to set many feature weights to zero.

As a high-level observation, in both the uni- and multilingual cases, relatively more _info_ features are selected, and relatively fewer _LM_ features. Of the _LM_ features that are selected, those which relate to the maximum perplexity or minimum probability appear to be more useful. These features

Figure 1: Results of uni-, multi- and cross-lingual classification experiments. Left panel displays results for English, right panel for French. Labels in the middle indicate the classification scenario and method of domain adaptation. Colours indicate the feature set and classifier. Bars indicate the AUC; error bars represent standard deviation.

 

capture locally anomalous speech patterns, relative to either the AD or control language models.

In the unilingual case, the French models show a preference for the binary "has" features (indicating whether or not an information unit has been mentioned). Only 4 of the "ratio" features and none of the density or efficiency features have a median value greater than zero. However, these features _are_ relevant to the task, and potentially more generalisable (e.g., total concept efficiency differs between the French AD and HC groups with \(p<0.001\) on a \(t\)-test, and represents an aggregate score rather than depending on the presence or absence of a single information unit). Such features are selected more often in the multilingual case, and lead to improved performance. One explanation for this could be that in the small French training set, spurious correlations due to noise can overpower the real signal, and lead to less relevant features being assigned high weights, while correlated (but perhaps actually more relevant) features are suppressed. By increasing the size of the training set with English data, the signal-to-noise ratio is improved, and a better set of features is selected.

Generally, the feature values (not shown) support the intuition that controls mention more of the information units in the image (higher "has" feature values), convey information more efficiently, with fewer off-topic words (higher density and efficiency scores), and organize the narrative in a more predictable way (narratives have lower perplexity and higher probability) than the AD participants. Again, these trends are more apparent in the English data than the French data, likely due to the relatively larger number of samples.

## 5 Discussion

One perhaps surprising result of this study was that naively combining features in the All condition led to better results than the Augment algorithm. However, this is in line with the original findings of Daume III (2007), where he identified a set of tasks where Augment performed sub-optimally: specifically, those cases where training on source-only data was better than training on target-only data. This is precisely the case we have here, as training cross-lingually (on English source data) leads to better results than training unilingually (on French target data). The explanation offered by Daume III is, "If the domains are so similar that a large amount of source data outperforms a small amount of target data, then it is unlikely that blowing up the feature space will help." In some sense, then, these results are confirmation that we have indeed identified a set of features over which the two languages (i.e. domains) are very similar.

The fact that the All configuration is optimal in both French and English has an added practical benefit: since there is no distinction between source and target features, the resulting classifier is language-agnostic. This means that test data could come from either language, in a hypothesized future screening application.

While our goal in this paper was not to push the state-of-the-art on the DementiaBank dataset, we do find that our best English result (AUC=0.84, which corresponds to an accuracy of 75% and \(F_{1}\) score of 0.77) is comparable to the other published results on this dataset Prud'hommeaux and Roark (2015); Yancheva and Rudzicz (2016); Sirts et al. (2017); Fraser et al. (2016); Hernandez-Dominguez et al. (2018). There are no previously published results on the French dataset.

## 6 Conclusion and Future Work

In this work, we have shown that there are features which can both distinguish AD patients from healthy controls with a high degree of accuracy,

Figure 2: _AUC_ as a function of the amount of English data used in the training set, for both multi- and cross-lingual cases. Error bars indicate 95% confidence intervals.

 

Figure 3: Visualisation of feature weights for uni- and multilingual experiments. Median feature importances over LPO- and 10-CV are displayed. The left panel displays the English and the right panel the French data sets. Unilingual experiments are given in blue and multilingual in yellow.

 

and also generalize across languages. By incorporating a large English dataset, we were able to improve the AUC on the French dataset from 0.85 to 0.89. We also developed a new set of features for this task, using concept-based language modelling, which improved AUC from 0.80 to 0.85 in the unilingual case, and 0.88 to 0.89 in the multilingual case.

Future work will involve extending the set of features involved, incorporating data from other languages, and testing whether similar techniques can be effective for detecting earlier stages of cognitive decline, such as MCI. Other work from our group has also begun to explore the use of unsupervised methods and out-of-domain data sources Li et al. (2019).

Technical challenges aside, collaborations of this nature can be difficult due to the sensitive nature of the data, and the need to respect ethical guidelines and participant consent when sharing and storing data. With this in mind, we recommend to other researchers working in similar domains to consider from the outset whether their data could eventually be shared, and to make suitable provisions in their ethics protocols and participant consent forms. We look to DementiaBank as a model for this kind of data-sharing and openness, and hope that researchers can continue to find ways to share resources of this nature.

## Acknowledgements

This research was partially funded by the Riksbankens Jubileumsfond - The Swedish Foundation for Humanities and Social Sciences, grant no: NHS 14-1761:1, and the EIT Digital Wellbeing Activity 17074, ELEMENT. The French data was collected during the ELEMENT project and the FP7 Dem@Care project (grant number 288199). The original acquisition of the DementiaBank data was supported by NIH grants AG005133 and AG003705 to the University of Pittsburgh, and the data archive is supported by NIHNIDCD grant R01-DC008524 to Carnegie Mellon University.

## References

* Airola et al. (2009) Antti Airola, Tapio Pahikkala, Willem Waegeman, Bernard De Baets, and Tapio Salakoski. 2009. A comparison of AUC estimators in small-sample studies. In _Machine Learning in Systems Biology_, pages 3-13.
* Al-Hameed et al. (2016) Sabah Al-Hameed, Mohammed Benaissa, and Heidi Christensen. 2016. Simple and robust audio-based detection of biomarkers for Alzheimer's disease. In _7th Workshop on Speech and Language Processing for Assistive Technologies (SLPAT)_, pages 32-36.
* Aluisio et al. (2016) Sandra Aluisio, Andre Cunha, and Carolina Scarton. 2016. Evaluating progression of Alzheimer's disease by regression and classification methods in a narrative language test in Portuguese. In _International Conference on Computational Processing of the Portuguese Language_, pages 109-114. Springer.
* Ammar et al. (2016) Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, and Noah A Smith. 2016. Massively multilingual word embeddings. _arXiv preprint arXiv:1602.01925_.
* Asgari et al. (2017) Meysam Asgari, Jeffrey Kaye, and Hiroko Dodge. 2017. Predicting mild cognitive impairment from spontaneous spoken utterances. _Alzheimer's & Dementia: Translational Research & Clinical Interventions_, 3(2):219-228.
* Backman et al. (2005) Lars Backman, Sari Jones, Anna-Karin Berger, Erika Jonsson Laukka, and Brent J Small. 2005. Cognitive impairment in preclinical Alzheimer's disease: A meta-analysis. _Neuropsychology_, 19(4):520-531.
* Becker et al. (1994) James T. Becker, Francois Boiler, Oscar L. Lopez, Judith Saxton, and Karen L. McGonigle. 1994. The natural history of Alzheimer's disease: Description of study cohort and accuracy of diagnosis. _Archives of Neurology_, 51(6):585-594.
* Bertran (1999) Antonio Pamie Bertran. 1999. Prosodic typology: On the dichotomy between stress-timed and syllable-timed languages. _Language Design: Journal of Theoretical and Experimental Linguistics_, 2:103-130.
* Blitzer et al. (2006) John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In _Proceedings of the 2006 conference on empirical methods in natural language processing_, pages 120-128. Association for Computational Linguistics.
* Boschi et al. (2017) Veronica Boschi, Eleonora Catricala, Monica Consonni, Cristiano Chesi, Andrea Moro, and Stefano F Cappa. 2017. Connected speech in neurodegenerative language disorders: A review. _Frontiers in psychology_, 8:269.
* Brown et al. (1992) Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. _Computational Linguistics_, 18(4):467-479.
* Chapman et al. (1998) Sandra Bond Chapman, Amy Peterson Highley, and Jennifer L Thompson. 1998. Discourse in fluent aphasia and Alzheimer's disease: Linguistic and pragmatic considerations. _Journal of Neurolinguistics_, 11(1-2):55-78 

Bernard Croisile, Bernadette Ska, Marie-Josee Brabant, Annick Duchene, Yves Lepage, Gilbert Aimard, and Marc Trillet. 1996. Comparative study of oral and written picture description in patients with Alzheimer's disease. _Brain and Language_, 53(1):1-19.
* Cuetos et al. [2007] Fernando Cuetos, Juan Carlos Arango-Lasprilla, Claramonika Uribe, Claudia Valencia, and Francisco Lopera. 2007. Linguistic changes in verbal expression: A preclinical marker of Alzheimer's disease. _Journal of the International Neuropsychological Society_, 13(3):433-439.
* Daume III [2007] Hal Daume III. 2007. Frustratingly easy domain adaptation. _Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 256-263.
* Dubois et al. [2016] Bruno Dubois, Harald Hampel, Howard H Feldman, Philip Scheltens, Paul Aisen, Sandrine Andrieu, Horvagim Bakardjian, Habib Benali, Lars Bertram, Kaj Blennow, et al. 2016. Preclinical Alzheimer's disease: Definition, natural history, and diagnostic criteria. _Alzheimer's & Dementia_, 12(3):292-323.
* Fraser et al. [2018a] Kathleen C. Fraser, Kristina Lundholm Fors, Marie Eckstrom, Charalampos Themistokleous, and Dimitrios Kokkinakis. 2018a. Improving the sensitivity and specificity of MCI screening with linguistic information. In _Proceedings of the 2nd Workshop on Resources and processing of linguistic, paralinguistic and extra-linguistic data from people with various forms of cognitive/psychiatric impairments (RaPID)_, pages 19-26.
* Fraser et al. [2018b] Kathleen C Fraser, Kristina Lundholm Fors, and Dimitrios Kokkinakis. 2018b. Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment. _Computer Speech & Language_, 53:121-139.
* Fraser et al. [2016] Kathleen C. Fraser, Jed A. Meltzer, and Frank Rudzicz. 2016. Linguistic features identify Alzheimer's disease in narrative speech. _Journal of Alzheimer's Disease_, 49(2):407-422.
* Goodglass et al. [2000] Harold Goodglass, Edith Kaplan, and Barbara Barresi. 2000. _Boston Diagnostic Aphasia Examination_. Lippincott Williams & Wilkins.
* Heafield [2011] Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In _Proceedings of the Sixth Workshop on Statistical Machine Translation (WMT)_, pages 187-197.
* Hernandez-Dominguez et al. [2018] Laura Hernandez-Dominguez, Sylvie Ratte, Gerardo Sierra-Martinez, and Andres Roche-Bergua. 2018. Computer-based evaluation of Alzheimer's disease and mild cognitive impairment patients during a picture description task. _Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring_, 10:260-268.
* Hoidekr et al. [2006] Jan Hoidekr, Josef V Poutka, Ales Prazak, and Josef Poutka. 2006. Benefit of a class-based language model for real-time closed-captioning of TV ice-hockey commentaries. In _Proceedings of the Language Resources and Evaluation Conference (LREC)_, pages 2064-2067.
* Konig et al. [2015] Alexandra Konig, Aharon Satt, Alex Sorin, Ran Hoory, Alexandre Derreumaux, Renaud David, and Philippe H Robert. 2018. Use of speech analyses within a mobile application for the assessment of cognitive impairment in elderly people. _Current Alzheimer Research_, 15(2):120-129.
* Konig et al. [2015] Alexandra Konig, Aharon Satt, Alexander Sorin, Ron Hoory, Orith Toledo-Ronen, Alexandre Derreumaux, Valeria Manera, Frans Verhey, Pauline Aalten, Phillipe H Robert, and Renaud David. 2015. Automatic speech analysis for the assessment of patients with predementia and Alzheimer's disease. _Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring_, 1(1):112-124.
* Lai et al. [2009] Yi-hsiu Lai, Hsiu-hua Pai, et al. 2009. To be semantically-impaired or to be syntactically-impaired: Linguistic patterns in Chinese-speaking persons with or without dementia. _Journal of Neurolinguistics_, 22(5):465-475.
* Li et al. [2019] Bai Li, Yi-Te Hsu, and Frank Rudzicz. 2019. Detecting dementia in Mandarin Chinese using transfer learning from a parallel corpus. In _The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)_.
* Fors et al. [2018] Kristina Lundholm Fors, Kathleen C. Fraser, and Dimitrios Kokkinakis. 2018. Automated syntactic analysis of language abilities in persons with mild and subjective cognitive impairment. In _Proceedings of the Medical Informatics Europe (MIE) Conference_, pages 705-709.
* MacWhinney [2014] Brian MacWhinney. 2014. _The CHILDES project: Tools for analyzing talk, Volume I: Transcription format and programs_. Psychology Press.
* Masrani et al. [2017] Vaden Masrani, Gabriel Murray, Thalia Shoshana Field, and Giuseppe Carenini. 2017. Domain adaptation for detecting mild cognitive impairment. In _Proceedings of the Canadian Conference on Artificial Intelligence_, pages 248-259.
* Meilan et al. [2014] Juan Jose G Meilan, Francisco Martinez-Sanchez, Juan Carro, Dolores E Lopez, Lymanie Millian-Morell, and Jose M Arana. 2014. Speech in Alzheimer's disease: Can temporal and acoustic parameters discriminate dementia? _Dementia and Geriatric Cognitive Disorders_, 37(5-6):327-334.
* Mirheidari et al. [2016] Bahman Mirheidari, Daniel Blackburn, Markus Reuber, Traci Walker, and Heidi Christensen. 2016. Diagnosing people with dementia using automatic conversation analysis. In _Proceedings of Interspeech_, pages 1220-1224. ISCA.

 

Serguei VS Pakhomov, Glenn E Smith, Dustin Chacon, Yara Feliciano, Neill Graff-Radford, Richard Caselli, and David S Knopman. 2010. Computerized analysis of speech and language to identify psycholinguistic correlates of frontotemporal lobar degeneration. _Cognitive and Behavioral Neurology_, 23(3):165.
* Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. _Journal of Machine Learning Research_, 12:2825-2830.
* Prettenhofer and Stein (2010) Peter Prettenhofer and Benno Stein. 2010. Cross-language text classification using structural correspondence learning. In _Proceedings of the 48th annual meeting of the Association for Computational Linguistics (ACL)_, pages 1118-1127.
* Prince et al. (2015) Martin Prince, Anders Wimo, Maelenen Guerchet, Gemma-Claire Ali, Yu-Tzu Wu, and Matthew Prina. 2015. World Alzheimer report 2015: The global impact of dementia. _Alzheimer's Disease International (ADI)_.
* Proud'hommeaux and Roark (2015) Emily Proud'hommeaux and Brian Roark. 2015. Graph-based word alignment for clinical language evaluation. _Computational Linguistics_, 41(4):549-578.
* Rentoumi et al. (2018) Vassiliki Rentoumi, George Paliouras, Dimitra Arfani, Katerina Fragkopoulou, Spyridoula Varlokosta, and Peter Garrard. 2018. Automatic detection of linguistic indicators as a means of early prediction of Alzheimer's and of related dementias: A cross-linguistics analysis. _Alzheimer's & Dementia: The Journal of the Alzheimer's Association_, 14:1302.
* Rentoumi et al. (2014) Vassiliki Rentoumi, Ladan Raoufan, Samrah Ahmed, Celeste A. de Jager, and Peter Garrard. 2014. Features and machine learning classification of connected speech samples from patients with autopsy proven Alzheimer's disease with and without additional vascular pathology. _Journal of Alzheimer's Disease_, 42(S3):S3-S17.
* Roark et al. (2011) Brian Roark, Margaret Mitchell, John-Paul Hosom, Kristy Hollingshead, and Jeffery Kaye. 2011. Spooken language derived measures for detecting mild cognitive impairment. _IEEE Transactions on Audio, Speech, and Language Processing_, 19(7):2081-2090.
* Satt et al. (2013) Aharon Satt, Alexander Sorin, Orith Toledo-Ronen, Oren Barkan, Ioannis Kompatsiaris, Athina Kokonozi, and Magda Tsolaki. 2013. Evaluation of speech-based protocol for detection of early-stage dementia. In _Proceedings of Interspeech_, pages 1692-1696.
* Shibata et al. (2016) Daiskaku Shibata, Shoko Wakamiya, Ayae Kinoshita, and Eiji Aramaki. 2016. Detecting Japanese patients with Alzheimer's disease based on word category frequencies. In _Proceedings of the Clinical Natural Language Processing Workshop (ClinicalNLP)_, pages 78-85.
* Sirts et al. (2017) Kairit Sirts, Olivier Piguet, and Mark Johnson. 2017. Idea density for predicting Alzheimer's disease from transcribed speech. In _Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)_, pages 322-332.
* Suzznan and Beard (2011) Richard Suzznan and John Beard. 2011. Global health and aging. _National Institute on Aging and National Institutes of Health, WHO_, 11-7737.
* Szatloczki et al. (2015) Greta Szatloczki, Ildiko Hoffmann, Veronika Vincze, Janos Kalman, and Magdolna Pakaski. 2015. Speaking in Alzheimer's disease, is that an early sign? Importance of changes in language abilities in Alzheimer's disease. _Frontiers in Aging Neuroscience_, 7:1-7.
* Tackstrom et al. (2012) Oscar Tackstrom, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In _Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT)_, pages 477-487.
* Taler and Phillips (2008) Vanessa Taler and Natalie A Phillips. 2008. Language performance in Alzheimer's disease and mild cognitive impairment: A comparative review. _Journal of Clinical and Experimental Neuropsychology_, 30(5):501-556.
* Troger et al. (2017) Johannes Troger, Nicklas Linz, Jan Alexandersson, Alexandra Konig, and Philippe Robert. 2017. Automated speech-based screening for Alzheimer's disease in a care service scenario. In _Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare_, pages 292-297.
* Vaissiere (2002) Jacqueline Vaissiere. 2002. Cross-linguistic prosodic transcription: French versus English. In N.B. Volskaya, N.D. Svetozarous, and P.A. Skrelin, editors, _Problems and Methods of Experimental Phonetics. In honour of the 70th anniversary of Pr. L.V. Bondavko_, pages 147-164. St Petersburg State University Press.
* Vincze et al. (2016) Veronika Vincze, Gabor Gosztolya, Laszlo Toth, Ildiko Hoffmann, Greta Szatloczki, Zoltan Banretti, Magdolna Pakaski, and Janos Kalman. 2016. Detecting mild cognitive impairment by exploiting linguistic information from transcripts. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, volume 2, pages 181-187.
* Wan (2009) Xiaojun Wan. 2009. Co-training for cross-lingual sentiment classification. In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1_, pages 235-243 

Jochen Weiner, Christian Herff, and Tanja Schultz. 2016. Speech-based detection of Alzheimer's disease in conversational German. In _Proceedings of Interspeech_, pages 1938-1942.
* Weiner et al. (2008) Myron F Weiner, Katherine E Neubecker, Mary E Bret, and Linda S Hynan. 2008. Language in Alzheimer's disease. _The Journal of Clinical Psychiatry_, 69(8):1223.
* Yancheva and Rudzicz (2016) Maria Yancheva and Frank Rudzicz. 2016. Vector-space topic models for detecting Alzheimer's disease. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 2337-2346.