# AI-Based Clinical Decision Support Tool on Mobile Devices for Neurodegenerative Diseases

Annamaria Demarinis Loiotile, Vincenzo Dentamaro, Paolo Giglio, Donato Impedovo

► To cite this version:

Annamaria Demarinis Loiotile, Vincenzo Dentamaro, Paolo Giglio, Donato Impedovo. AI-Based Clinical Decision Support Tool on Mobile Devices for Neurodegenerative Diseases. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.139-148, 10.1007/978-3-030-85623-6_10. hal-04330973

HAL Id: hal-04330973

https://inria.hal.science/hal-04330973v1

Submitted on 8 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
AI-based Clinical decision support tool on Mobile Devices
for Neurodegenerative Diseases

Annamaria Demarinis Loiotile¹ [0000-0003-3503-9952], Vincenzo Dentamaro² [0000-0003-1148-332X], Paolo Giglio² [0000-0002-0426-7349], Donato Impedovo³ [0000-0002-9285-2555]

¹ Dipartimento di ingegneria elettrica e dell'informazione, Politecnico di Bari, Via Orabona 4,
70126 Bari, Italy

² Department of Computer Science, Università degli studi di Bari “Aldo Moro”, via Orabona 4 70126 Bari, Italy.

³ Digital Innovation S.R.L., via Orabona 4 70126 Bari, Italy.

**Abstract.** Recently, the role of AI in the development of eHealth is becoming increasingly ambitious since AI is allowing the development of whole new healthcare areas. In many cases, AI offers the possibility to support patient screening and monitoring through low-cost, non-invasive tests. One of the most relevant sectors in which a great contribution from AI is expected is that of neurodegenerative diseases, which represent one of the most important pathologies in Western countries with very serious follow up not only clinical, but also social and economic.

In this context, AI certainly represents an indispensable tool for effectively addressing aspects related to early diagnosis but also to monitoring patients suffering from various neurodegenerative diseases. To achieve these results, AI tools must be made available in test applications on mobile devices that are also easy to use by a large part of the population. In this sense, the aspects related to human-machine interaction are of paramount relevance for the diffusion of these solutions.

This article presents a mobile device application based on artificial intelligence tools for the early diagnosis and monitoring of patients suffering from neurodegenerative diseases and illustrates the results of specific usability tests that highlight the strengths but also the limitations in the iteration with application users. Some concluding remarks are highlighted to face the actual limitations of the proposed solution.

**Keywords:** Human-Computer Interaction, Artificial Intelligence, Neurodegenerative Disease, Mobile Applications.
# 1 Introduction

AI is assuming an even more relevant role for the Digital Transitions for a wide range of application area, such as cybersecurity [1], urban safety [2], sustainable mobility [3] and higher education [4].

In Healthcare 4.0 domain, the increasing frequency with which both patients and health professionals can access and use connected devices for clinical care and for monitoring patients has created abundant opportunities to deliver health and wellness services [5]. Recently, the role of AI in the development of eHealth is becoming more and more ambitious since AI is allowing the development of whole new healthcare areas. In many cases, AI offers the possibility to support patient screening and monitoring through low-cost, non-invasive tests [6]. One of the most relevant sectors in which a great contribution from AI is expected is that of neurodegenerative diseases, which represent one of the most important pathologies in Western countries with very serious follow up not only clinical, but also social and economic.

In this context, AI certainly represents an indispensable tool for effectively addressing aspects related to early diagnosis but also to monitoring patients suffering from various neurodegenerative diseases. To achieve these results, AI tools must be made available in test applications on mobile devices that are also easy to use by a large part of the population. In this sense, the aspects related to human-machine interaction are of paramount relevance for the diffusion of these solutions.

Preemptively diagnosing a form of neurodegeneration is critical to improving the patient's quality of life [7, 8].

This article presents a mobile device application based on artificial intelligence tools for the early diagnosis and monitoring of patients suffering from neurodegenerative diseases. In particular, the article illustrates the smartphone version of some of the pen and paper tasks used by neurologists, psychologists and so on, for monitoring and diagnosis of neuro-muscular degenerative diseases.

The tests considered refer to different cognitive areas and are based on the patient's abilities in constructive praxis, concentration and language. They aim to facilitate and reduce the time of diagnosis, for patients, their relatives and caregivers.

# 2 Related Work

## 2.1 Dementia

Dementia is defined as a syndrome of acquired and persistent impairment in cognition and intellectual functioning. The most common cause of dementia is Alzheimer's disease (AD) and the most common dementia syndrome is Alzheimer dementia (AD) [9], which is characterized by a major impairment of episodic memory, while in other types of dementia, other domains are more primarily affected, such as executive functioning in vascular dementia (VaD). Dementia can also develop in people with Parkinson's disease, differing in the relationship between motor and cognitive impairment [10].
Dementia affects memory, thinking, orientation, comprehension, calculation, learning ability, language, and judgment. Consciousness is not impaired. Disorders of cognitive function are commonly accompanied, sometimes preceded, by impairment of emotional control, social behavior, or motivation, which interfere with a person's normal relational and work life.

Doctors diagnose dementia when one or more cognitive functions have been impaired at a significant level. The specific symptoms experienced by a person with dementia depend on the area of the brain that is damaged.

It is increasingly recognized that mixed cases of dementia are commonly encountered, especially in elderly. It is less common for people under age 65 to develop dementia, and in this case, it is referred to as "early-onset dementia." More challenging, however, is distinguishing dementia from more subtle patterns of cognitive impairment that do not fit into standard definitions of dementia but may represent a "pre-clinical" state of dementia, called "mild cognitive impairment" syndrome [11].

Early detection of symptoms is important, as some causes can be treated. In many cases, a person's specific type of dementia may not be confirmed until the person is dead, and the brain is examined [12].

A medical evaluation for dementia generally includes:

- **Medical history:** typical questions generally include taking medications that could cause or worsen a symptom, how and when symptoms occurred, and whether or not there is dementia in the family.

- **Physical exam:** measuring blood pressure and other vital signs can help detect conditions that could cause a dementia state.

- **Neurological testing:** assessing reflexes, cognitive function, balance and sensory response helps identify symptoms early for a more accurate diagnosis.

## 2.2 Assessment of cognitive functioning

Within the neurological testing, there are cognitive tests which are used to assess memory, problem solving, language skills, math skills, and other abilities related to mental functioning. The Mini-Mental State Examination (MMSE) is one of the most frequently used and widely studied with regard to precision and accuracy. A score below 23 has generally been associated with the presence of cognitive impairment [13]. The best screening exam for memory is a short-term memory test: patients with dementia are unable to perform this exam. Neuropsychological testing should be performed when the history and mental status tests performed by the patient are inconclusive. In addition, recent studies point out that the sensitivity of the MMSE is lower when a patient in the early stages of dementia is tested [13].

Another pen and pencil set of tests used by neurologists is the Milan Overall Dementia Assessment (MODA). It is composed by three sections: behavioral and two section of pen and paper testing [14].

This work is organized as follows: section 2 shows the protocol developed, section 3 describes the tasks selected to be implemented as mobile version of the pen and paper.
Section 4 shows the SUS usability test results while section 5 sketches conclusions and future remarks.

## 3 Acquisition protocol

This protocol can be divided mainly into three phases: an initial screening, a battery of standard cognitive and functional tests and some more relevant graphic and vocal tasks extracted from the medical literature [15].

Initial screening allows the examiner to collect all personal information about the patient, including patient data, caregiver interview, and preliminary testing.

The details of the patient whose cognitive impairment is to be assessed are collected, such as: first name, last name, place of birth, date of birth, social security number, sex, schooling, medications taken and type of dementia. These are sensitive personal data that must be kept confidential; moreover, their treatment, according to the law in force, must be authorized by the user of the test. For this reason, this new mobile version saves the multimedia files (voice and screenshots of tasks) of the patient's exam in a folder with a unique name, to distinguish it from other exams and different from the patient's generalities, to guarantee confidentiality.

Health care providers (caregivers) are involved in gathering information about the patient. Their role is critical to the accuracy of the test outcome. The questions they are asked are regarding the most common symptoms of dementia such as: difficulty in remembering recent conversations; inability to respond to trivial problems; lack of respect for social behavioral rules; tendency to get lost in familiar places, etc.

If the patient, according to the caregiver, would result negative to all the questions indicated, the test would have no reason to continue, because it results a healthy person. If there would be a positive response to at least one question, the test should continue.

## 4 Implemented tasks

In this section the tasks that are possible to implement as a smartphone app are discussed. The selection of these tasks is performed, on one hand, keeping in mind the constraints of the device, a smartphone with Android operating system and a touch screen support, and the difficulty of elderly people in approaching the mobile technology. On the other hand, the fact that an app is easily accessible by the majority of people and no special device (Wacom tablet with pen and software) is necessary, would increase the number of people using it and decrease the entry barriers.

Particular attention is dedicated on the design of positive experiences when products and services with AI are developed, using a human-centered design approach [5].

An EC Expert Group defined the potential as well as the need for designing AI systems human-centered, including the ethically and trustworthy development of AI. The Group stated that “AI is not an end in itself, but rather a promising means of enhancing individual and societal well-being and bringing progress and innovation” [16]. This commitment indicates that human-centered approaches have a pivotal role in the development and use of AI technology for the well-being of people [17].
From a Human Computer Interaction perspective, it has been decided to analyze people (mostly elderly) using a smartphone. It has been noticed that elderly tend to bring the device closer to their eyes, they tend to touch the screen several times waiting for feedback, too complex interfaces increase the abandonment of using the app so also interfaces that are too colorful or with low contraction contribute to the abandonment.
Thus, it has been decided to focus on tasks that are feasible given the small screen space of a smartphone.

## 4.1 Preliminary tests

Prior to testing, the patient undergoes a series of preliminary tests to ensure that the examinee becomes familiar with the device and application.
They are asked to perform some preliminary tests such as:

- repeating the word "HELLO" aloud;

- connecting two horizontal dots with a straight line;

- connecting two vertical dots with a straight line.

## 4.2 Attention matrix

Attention matrix test is a very important test based on an attention task. The patient is shown two matrices and asked to cross out, as quickly as possible, the target numbers.
The first two lines, performed by the operator who is reviewing the test, are rehearsal and are used to show the patient how to perform the task.

In Figure 1, the user is asked to cross all the “6” numbers. A second attention matrix task is to cross 2 numbers at same time: number “2” and “8” at same time.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Attention Matrix where user is asked to cross one single number, in this case “6”</figcaption></figure>

## 4.3 Dots connect

There are several dot linking tasks in the test. They allow the clinician to assess the trait and strategy adopted by the patient. The tasks are listed below:

- Connect 4 points;

- Connect points in numerical order;

- Connect points in alphabetical order.
## 4.4 Geometrical figure copying task.

The patient is asked to copy the figure shown. It is usually asked to copy a square or a rectangular figure given the constraints of the device.

## 4.5 Copy and trace of Archimedes' spiral.

The Archimedes spiral is one of the most popular handwriting exercises. In fact, the spiral drawing has frequently been used for motor performance assessment in various movement disorders, particularly Parkinsonians. In these tests, patients are asked to copy an Archimedes spiral and in a second moment to trace it. The chosen Archimede' spiral is the simpler one with only 3 convolving curves, as shown in fig. 2.

<figure><img src="image_4.png" /><figcaption>**Fig.2.** Archimede' spiral copying (left) and tracing (right)</figcaption></figure>

## 4.6 “I” and “e” sequence

The patterns of the cursive letters "I" and "e" are another common exercise in handwriting assessment, this is because such letters are easy to write recurrently and continuously; moreover, their use minimizes the processes of linguistic comprehension. User is asked to draw with one finger three times “le”, as shown in Fig.3:

<figure><img src="image_5.png" /><figcaption>**Fig.3.** “le le” drawing task</figcaption></figure>

## 4.7 Repeat the indicated word aloud

Speech impairment is a disorder that prevents patients from expressing and/or understanding words. It can result from neuro-muscular alterations. Thus, gradual loss of language is a symptom indicative of Alzheimer's and Parkinson's disease.

Nuances in tone of voice, as well as hand gestures or facial expressions, can be critical in helping to determine the meaning behind the words spoken. In Parkinson's disease, the voice is monotone, raspy, and characterized by low volume; in addition, reduced facial expressions also contribute to greater difficulty in managing the vocal tract. In specific terms, there is a 40% reduction in loudness (sound perception).

For this reason, it has been decided to allow the patient to repeat some very simple words and record the voice, to allow more experienced people to analyze it. The words are administered by visual feedback or by listening to audio.
## 4.8 Description of the content of an image

This task is very similar to the previous one, but the patient is asked to look carefully at a picture and repeat the content orally aloud. The user interface (UI) developed is presented in Fig.4.

<figure><img src="image_6.png" /><figcaption>Fig.4 The UI developed for capturing the aloud description of the image.</figcaption></figure>

# 5 Usability tests

Prior to beginning actual data acquisition, a usability assessment was performed, adopting two methodologies:

- SUS (System usability Scale) questionnaire conducted on five young adults;

- Thinking aloud protocol conducted on five healthy older people.

More experiments are of course in progress and data acquisition is still ongoing. In addition, it is important to mention that for both evaluation criteria, users were required to perform the tasks (speech and drawing) without any prior knowledge of the application, the only way to become familiar with the system being the preliminary trials.

The SUS questionnaire is structured by the ten following items:

1. I think I would like to use this [tool] frequently.

2. I find the [tool] unnecessarily complex.

3. The [tool] is easy to use.

4. I think I would need support from a technical person to use this [tool].

5. I find that the various functions of this [tool] were well integrated.

6. I think there is too much inconsistency in this [tool].

7. I imagine that most people would learn to use this [tool] very quickly.

8. I find the [tool] very cumbersome to use.

9. I feel very confident using the [tool].

10. I need to learn a lot of things before I could start this [tool].

The SUS results are presented in Table 1. The overall score is 97 out of 100, which is very high but, because of limited number of people and reflecting only a young population, it is not representative.

<table><thead><tr><td>Subject</td><td>Age</td><td>Job</td><td>Data test</td><td>Informatic skill</td><td>Score</td></tr></thead><tbody><tr><td>1</td><td>24</td><td>Unemployed</td><td>30/03/2021</td><td>very Good</td><td>97,5</td></tr></tbody></table>
<table><caption>**Table 1. SUS Test results**</caption><tr><td>2</td><td>23</td><td>Policeman</td><td>02/04/2021</td><td>Good</td><td>95</td></tr><tr><td>3</td><td>21</td><td>Student</td><td>01/01/2021</td><td>Good</td><td>95</td></tr><tr><td>4</td><td>25</td><td>Student</td><td>27/03/2021</td><td>very Good</td><td>100</td></tr><tr><td>5</td><td>24</td><td>Healthcare professional</td><td>26/03/2021</td><td>very Good</td><td>100</td></tr></table>

The SUS test is less suitable for elderly people, as they are less familiar with technological equipment. For this reason, an assessment based on thinking aloud was adopted only for the sample of elderly people.

The user is encouraged to act and verbalize any thoughts that come to mind while performing the assigned tasks.

The results in Table 2 showed that the users have a good familiarity with the system. The main problem encountered relates to the small screen size. Again, given the low amount of people, reasoning about usability is limited.

<table><caption>**Table 2. Think Aloud tests results**</caption><thead><tr><td>Subject</td><td>Age</td><td>Job</td><td>Data test</td><td>Informatic skill</td></tr></thead><tbody><tr><td>1</td><td>78</td><td>Retiree (former public employee)</td><td>30/03/2021</td><td>Low</td></tr><tr><td>2</td><td>69</td><td>Housewife</td><td>02/04/2021</td><td>Low</td></tr><tr><td>3</td><td>73</td><td>Retiree (former engineer)</td><td>01/04/2021</td><td>Sufficient</td></tr><tr><td>4</td><td>68</td><td>Retiree (former teacher)</td><td>28/03/2021</td><td>Good</td></tr><tr><td>5</td><td>72</td><td>Retiree (former teacher)</td><td>01/04/2021</td><td>Very low</td></tr></tbody></table>

The aforementioned application is the ground basis for data acquisition, which in turn is represented by the results of the tests previously described. The tests outcomes represent the input to AI algorithms, specifically Convolutional Neural Networks, in order to provide an additional score in terms of probability and confidence of the plausible presence of neurodegenerative pathology [7]. This outcome shall not substitute the physician diagnosis, instead it shall be an additional and objective outcome that the physician could consider during a patient examination.
# 6 Conclusion and future remarks

In this work a mobile app has been developed with the aim to implement some of the most famous pen and pencil tests present in MMSE, IADL and other clinically vali-dated battery tests for assessment of neurodegenerative disease. Taking into account the human centered design approach, it has been decided which tasks to integrate with respect to the constraints of the device and the user. The choice allowed to develop a protocol that, given the widespread availability of smartphones, will be easier to access, at low cost and able to store a great amount of data to be clinically vali-dated in a second stage by merging all information collected. Extended data acquisition and preprocessing session are in progress.

In future, pattern recognition techniques could be applied to the patient data in order to verify that, even though the use of mobile devices, it will be possible to perform early diagnosis of neurodegenerative diseases.

# References

[1] Dentamaro V., Convertini N., Galantucci S., Giglio P., Impedovo D., Pirlo G., (2021). Ensemble Consensus: an unsupervised algorithm for anomaly detection in network security data. Itasec 2021.

[2] Convertini, N., Dentamaro, V., Impedovo, D., Pirlo, G., & Sarcinella, L. (2020). A controlled benchmark of video violence detection techniques. Information, 11(6), 321.

[3] Nikitas, A.; Michalakopoulou, K.; Njoya, E.T.; Karampatzakis, D. Artificial Intelligence, Transport and the Smart City: Definitions and Dimensions of a New Mobility Era. Sustainability 2020, 12, 2789. https://doi.org/10.3390/su12072789

[4] Popenici, S.A.D., Kerr, S. Exploring the impact of artificial intelligence on teaching and learning in higher education. RPTEL 12, 22 (2017). https://doi.org/10.1186/s41039-017-0062-8

[5] Adarsha A.S.; Reader K., Erban S. AIS Transactions on Human-Computer Interaction, Volume 11, Issue 4 Special Issue on User Experience-driven Innovation (2019). User Experience, IoMT, and Healthcare, Article 6
[6] Dentamaro, V., Impedovo, D., & Pirlo, G. (2020, October). Sit-to-Stand Test for Neurodegenerative Diseases Video Classification. In International Conference on Pattern Recognition and Artificial Intelligence (pp. 596-609). Springer, Cham.

[7] Impedovo, D., & Pirlo, G. (2018). Dynamic handwriting analysis for the assessment of neurodegenerative diseases: a pattern recognition perspective. IEEE reviews in biomedical engineering, 12, 209-220.

[8] De Stefano, C., Fontanella, F., Impedovo, D., Pirlo, G., & di Freca, A. S. (2019). Handwriting analysis to support neurodegenerative diseases diagnosis: A review. Pattern Recognition Letters, 121, 37-45.

[9] Olson E. Dementia and neurodegenerative diseases - Geriatric palliative care, 2003

[10] Rosenblum, S., Samuel, M., Zlotnik, S., Erikh, I., & Schlesinger, I. (2013). Handwriting as an objective tool for Parkinson's disease diagnosis. Journal of neurology, 260(9), 2357-2361.

[11] Cummings, J. L., & Benson, D. F. (1992). Dementia: A clinical approach. Butterworth-Heinemann Medical.

[12] «What Is Dementia? Symptoms, Types, and Diagnosis». https://www.nia.nih.gov/health/what-dementia-symptoms-types-and-diagnosis.

[13] Cockrell, J. R., & Folstein, M. F. (2002). Mini-mental state examination. Principles and practice of geriatric psychiatry, 140-141.

[14] Brazzelli, M., et al. "A neuropsychological instrument adding to the description of patients with suspected cortical dementia: the Milan overall dementia assessment." Journal of Neurology, Neurosurgery & Psychiatry 57.12 (1994): 1510-1517.

[15] Impedovo, D., Pirlo, G., Vessio, G., & Angelillo, M. T. (2019). A handwriting-based protocol for assessing neurodegenerative dementia. Cognitive Computation, 11(4), 576-586.

[16] EC. (2019). ETHICS GUIDELINES FOR TRUSTWORTHY AI. Retrieved from https://ec.europa.eu/digitalsingle-market/en/news/ethics-guidelines-trustworthy-ai

[17] Auernhammer, J. (2020) Human-centered AI: The role of Human-centered Design Research in the development of AI, in Boess, S., Cheung, M. and Cain, R. (eds.), *Synergy - DRS International Conference 2020*, 11-14 August, Held online. https://doi.org/10.21606/drs.2020.282