Identifying robust markers of P arkinson’s disease in typing behaviour using a CNN - LSTM network (2020)

Abstract: There is urgent need for non-intrusive tests that can detect early signs of Parkinson’s disease (PD), a debilitating neurodegenerative disorder that affects motor control. Recent promising research has focused on disease markers evident in the fine-motor behaviour of typing. Most work to date has focused solely on the timing of keypresses without reference to the linguistic content. In this paper we argue that the identity of the key combinations being produced should impact how they are handled by people with PD, and provide evidence that natural language processing methods can thus be of help in identifying signs of disease. We test the performance of a bi-directional LSTM with convolutional features in distinguishing people with PD from age-matched controls typing in English and Spanish, both in clinics and online.

## introduction parkinson's disease is a neurodegenerative disease that affects approximately 1% of people over the age of 60 @xcite . its cardinal manifestations include bradykinesia (slowness of movement), tremor and rigidity. these result from the degeneration of dopaminergic neurons in the basal ganglia (an area of the brain responsible for action selection). a particular challenge in the treatment of pd is that by the time such motor signs are present, over 50% of neurons in the affected area of the basal ganglia (the substantia nigra) have been lost @xcite . while neuroimaging can pick up on these changes @xcite , such procedures are prohibitively expensive and cannot be performed on whole populations. there is thus an urgent need for cheap and easy-to-administer measures that can be used for the identification of at-risk individuals. a long-used simple motor test for pd is the alternating finger tapping test @xcite . this test involves asking a person to alternately tap an index finger in two locations a set distance apart on a surface or on a keyboard @xcite . people with pd are typically able to perform fewer taps over a 30 second period than people with no diagnosis. while such measures have proved useful, they suffer from a clear lack of specificity -slowing of movement is also a strong predictor of other neurodegenerative disorders, such as alzeimer's disease @xcite . furthermore, neural degeneration is unlikely to be detected by as coarse-grained a measure as tapping rate until the disease is relatively advanced. if specificity and earlier detection is to be achieved more targeted tests will be required. there is good theoretical reason to think that more pd-specific markers will be present in recordings of learned serial order behaviours, such as making a cup of tea, driving or typing. analysis of the production of such frequently-performed behaviours, and their underlying neurobiology, often distinguishes between habit (the automatic production of routinised movements) and goal-directed responses (behaviours that involve top-down planning; dolan and dayan 2013). there is substantial evidence that the degeneration of the basal ganglia in pd primarily affects areas responsible for automatic behaviours @xcite , and results in a shift in the balance of habitual and goal-directed control @xcite . @xcite predict that people with early-stage or prodromal pd will have a problem initiating their automatic behaviours. this paper focuses on the detection of markers of pd in one such behaviour -that of typing. it is motivated by the prediction that people with pd will, from very early on and potentially prodromally (before the emergence of the acute symptoms that allow conventional diagnosis), change the way that they type, as they lose capacity for automatic control. natural language processing provides us with techniques that we can use to pick up on those changes. as a motor behaviour, typing has been the focus of previous work on detecting or monitoring pd (see related work section). some such work has focused on coarse-grained measures such as typing speed. these suffer from the lack of specificity associated with tapping measures. other promising work has looked at more detailed timing measures. however this has continued to ignore the identity of the sequences being typed. different sequences of keys present different motor challenges due to the position of the keys on the keyboard and the hand used. the extent to which typing these will be facilitated by prior automatisation depends on the relative frequency with which they have been typed @xcite . we therefore expect consideration of the content of typing to be critical in picking up on pd-related changes. we describe a method for using a convolutional neural network (cnn) long short term memory (lstm) network to distinguish people with pd from age-matched people with no diagnosis. one motivation for this choice is that we want to pick up on the fine temporal details of the sequential data, in order to provide a measure that will be specific to pd. however the difficulty in picking up on such subtle information is that any typing dataset will also contain cruder information, such as the average differences in overall timing across participants. in order to tackle this, we normalise all temporal variables using robust scaling -subtracting each participant's median value from all their datapoints and dividing by their interquartile range. we thereby require our network to pick up on more subtle and potentially disease-specific information. the contributions of this article are as follows: ## related work a simple motor test that we might consider a precursor to the use of typing is the alternating finger tapping test @xcite . this test involves asking a person to alternately tap an index finger in two locations a set distance apart on a surface or on a keyboard @xcite . @xcite report that the number of key taps in 30 seconds (averaged across hands) can be used to distinguish patients from controls, identifying 50% of true positives with only 15% false positives. using the same measure (selecting the worst performing limb in patients and comparing it with the best performing limb in controls), @xcite report an auc of 0.87. while tapping tests are widely used they suffer from a lack of specificity to pd. while they distinguish people with pd from control participants with considerable success, there is good reason to think that they will struggle to distinguish pd from other neurodegenerative disorders. @xcite report an auc of 0.68 in distinguishing people with pd from people with alzheimer's disease using a single tapping test. in pursuit of an easier-to-gather alternative to finger-tapping tests, @xcite examine the interkey intervals (ikis) of people typing usernames while logging-in to a website. they found a moderate-to-strong correlation between the participants' median ikis during typing and the mean time between taps finger taps in a 10 second period. building on this, @xcite used key-hold times during transcription typing in order to distinguish 42 people with recently-diagnosed pd (off medication) from 43 controls. properties of the distribution of hold times for each patient was used in an -support regression to generate a unique score. in a two-fold cross-validation this achieved a combined auc of 0.81, comparable to an auc of 0.75 achieved with an alternating finger tapping test on the same sample. adams (2017) logged key events during regular computer use over an extended period by 20 patients and 33 controls. information about hold times and ikis, including measures of variance and of asymmetry between hands was used in a classification ensemble of eight different classification methods. this ensemble, trained on the new data, achieved an auc of 0.97 on the 85 participants from @xcite . all of the work described above has represented typing behaviour with summary statistics rather than as sequences. furthermore they have analysed the timing of keystrokes without considering what is being typed. the one exception to this latter point is the work of @xcite who look at the accuracy of typing while copying text using engineered features. they predict that people with pd, while making more errors in general, should make fewer 'habit slips'. this is when a well-learned sequence of key presses is produced in an inappropriate context, such as typing t-h-i-n-g when the intended word is t-h-i-n because -i-n-g is a frequent sequence. they find that is the case, and that adding this information to a generalised additive regression model predicting disease progression gives an improvement in fit relative to a model just including timing information. ## datasets we perform analyses of the following three datasets, representing two different usage contexts (recruited and tested in a clinic, and recruited and tested remotely online) and two different languages: english and spanish. all participants were tested via a browser-based app which presents a series of sentences to be copy-typed, and collects the identity and timing of each key-press. all datasets contain information about key down timing (when the typist pressed each key), and the online-recruited dataset additionally contains information about about key up timing (when they released each key). summary statistics are found in ## method we implement a neural language model (nlm) which receives two different types of information in variety of combinations: (1) character identity information: one-hot encoded character sequences; continuous bag-of-words model @xcite encoded character sequences; ## experimental setup the purpose of our experiments is to understand the effect of including character information in the classification of pd patients, when employing copy-typing as a diagnosis medium. using the model discussed in §4 we conduct multiple binaryclassification experiments to distinguish sentences written by people with pd (pwpd), from those written by age-matched controls. the same exercise is undertaken to classify participants themselves. we evaluate performance by measuring the area under the receiver operating characteristic curve (auc). this is a common approach when dealing with a two-class prediction problem (binary classification), in which the outcomes are labelled either as positive (pwpd) or negative (control). the auc scores reported in §6 are calculated on the test sets. for sentence classification we use participant level five-fold cross-validation ensuring that sentences from any one participant do not exist in both the train and test set. we report the mean and standard deviation over folds. for participant classification we aggregate the sentence classification probabilities using logistic regression with leave-one-out cross validation and employ bootstrapping to report mean and standard deviation. the model is applied to the datasets described in detail in §3 with summary statistics given in we conduct hyperparameter search, model introspection and ablation studies. each dataset is preprocessed according to the procedures outlined in §4.1 and §4.2, and split into train, test and validation sets. this partitioning reduces the number of samples which can be used for learning the model. our datasets are small compared to those typically used for deep learning. we deal with this in multiple ways, as detailed in appendix c. ## results our main experiments, as outlined above, involve the use of timing information that has been robustly scaled at the participant level in order to remove coarse-grained differences between groups. to aid understanding of the data, however, we will first report the performance of a classifier that uses the information that we have removed -the median and interquartile range of keypresses -as the sole features. the aucs for logistic regressions using these features as predictors can be seen in ## conclusion in this paper we have provided evidence that natural language processing techniques and in particular cnn-lstm networks can identify markers of parkinson's disease in logged typing behaviour. critically there is good reason to think that the markers identified will have high specificity with regards to parkinson's disease. while simple motor tests like the finger tapping test, and summary timing statistics from typing data, are widely used to distinguish pwpd from people without the disease, they rely on disease signs that pd has in common with other disorders -namely general slowing. in this work we first remove this disease sign from the data and then use a cnn-lstm to pick up on more subtle changes in performance. we report very promising performance using this approach. we further report on an analysis of the gradients in our model which suggests that it is picking up on plausible effects of pd seen in the data. previous work has sought to distinguish pwpd from controls by observing how rapidly and consistently they press keys when typing. however, such work begins by discarding potentially valuable information -the identity of the keys pressed. we found that including key identify in our data/model provided a performance improvement relative to timing-only models. we found an improvement in performance (increased auc) in identifying patients among participants tested in clinics in both english and spanish. furthermore we found a substantial leap in performance on the more difficult task of discriminating pd patients from controls in a new large dataset recruited and tested online. these results suggest that nlp techniques allows us to identify theoretically-motivated markers of pd @xcite in typing data. these incorporate both speed and character information, and so may be more robust than currently-used markers. future work will of course require that we test this directly, by collecting typing data from people with other neurological disorders and using these markers for multi-class classification. this work is only the tip of the iceberg in terms of the contribution that nlp can make to the task of detecting signs of parkinson's disease, and potentially other movement disorders, in typing data.