# Scoping review of clinical decision support systems for multiple sclerosis management: Leveraging information technology and massive health data

Stanislas Demuth, Chadia Ed-Driouch, Cédric Dumas, David Laplaud, Gilles

Edan, Nicolas Vince, Jérôme de Sèze, Pierre-antoine Gourraud

► To cite this version:

Stanislas Demuth, Chadia Ed-Driouch, Cédric Dumas, David Laplaud, Gilles Edan, et al.. Scoping review of clinical decision support systems for multiple sclerosis management: Leveraging information technology and massive health data. European Journal of Neurology, 2024, pp.e16363. 10.1111/ene.16363 . inserm-04676399

HAL Id: inserm-04676399

https://inserm.hal.science/inserm-04676399v1

Submitted on 23 Aug 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
REVIEW ARTICLE

european journal
of neurology

The official journal of the european academy of neurology

# Scoping review of clinical decision support systems for multiple sclerosis management: Leveraging information technology and massive health data

Stanislas Demuth<fnref n="1" />,<fnref n="2" /> | Chadia Ed-Driouch<fnref n="2" />,<fnref n="3" /> | Cédric Dumas<fnref n="3" /> | David Laplaud<fnref n="2" />,<fnref n="4" /> |
Gilles Edan<fnref n="5" /> | Nicolas Vince<fnref n="2" /> | Jérôme De Sèze<fnref n="1" />,<fnref n="6" /> | Pierre-Antoine Gourraud<fnref n="2" />,<fnref n="7" />

**Correspondence**
Pierre-Antoine Gourraud, INSERM, CR2TI-Center for Research in Transplantation and Translational Immunology, Nantes Université, Nantes F-44000, France.
Email: pierre-antoine.gourraud@universantes.fr

**Funding information**
European Union's Horizon 2020 Research and Innovation Programme, Grant/Award Number: 754995; Agence Nationale de la Recherche, Grant/Award Number: ANR-17-RHUS-0010 and ANR-21-RHUS-0014; Conseil Régional des Pays de la Loire, Grant/Award Number: 2019_11235; European Commission, Grant/Award Number: 754995

**Abstract**

**Background and purpose:** Multiple sclerosis (MS) is a complex autoimmune disease of the central nervous system, with numerous therapeutic options, but a lack of biomarkers to support a mechanistic approach to precision medicine. A computational approach to precision medicine could proceed from clinical decision support systems (CDSSs). They are digital tools aiming to empower physicians through the clinical applications of information technology and massive data. However, the process of their clinical development is still maturing; we aimed to review it in the field of MS.

**Methods:** For this scoping review, we screened systematically the PubMed database. We identified 24 articles reporting 14 CDSS projects and compared their technical and software development aspects.

**Results:** The projects position themselves in various contexts of usage with various algorithmic approaches: expert systems, CDSSs based on similar patients' data visualization, and model-based CDSSs implementing mathematical predictive models. So far, no project has completed its clinical development up to certification for clinical use with global release. Some CDSSs have been replaced at subsequent project iterations. The most advanced projects did not necessarily report every step of clinical development in a dedicated article (proof of concept, offline validation, refined prototype, live clinical evaluation, comparative prospective evaluation). They seek different software distribution options to integrate into health care: internal usage, "peer-to-peer," and marketing distribution.

**Conclusions:** This review illustrates the potential of clinical applications of information technology and massive data to support MS management and helps clarify the roadmap for future projects as a multidisciplinary and multistep process.

**KEYWORDS**

artificial intelligence, big data, clinical decision support system, multiple sclerosis, precision medicine

This is an open access article under the terms of the [Creative Commons Attribution-NonCommercial License](https://creativecommons.org/licenses/by-nc), which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

© 2024 The Author(s). *European Journal of Neurology* published by John Wiley & Sons Ltd on behalf of European Academy of Neurology.

<footnote n="1">¹INSERM CIC 1434, Clinical Investigation Center, University Hospital of Strasbourg, Strasbourg, France</footnote>

<footnote n="2">²INSERM, CR2TI-Center for Research in Transplantation and Translational Immunology, Nantes Université, Nantes, France</footnote>

<footnote n="3">³Département Automatique, Productique et Informatique, IMT Atlantique, CNRS, LS2N, UMR CNRS 6004, Nantes, France</footnote>

<footnote n="4">⁴Department of Neurology, University Hospital of Nantes, Nantes, France</footnote>

<footnote n="5">⁵Department of Neurology, University Hospital of Rennes, Rennes, France</footnote>

<footnote n="6">⁶Department of Neurology, University Hospital of Strasbourg, Strasbourg, France</footnote>

<footnote n="7">⁷Data Clinic, Department of Public Health, University Hospital of Nantes, Nantes, France</footnote>
# INTRODUCTION

The use of information technology (IT) to support clinical decision-making through human-computer interactions has been envisioned with the concept of augmented intelligence. The technologies required for that only reached sufficient maturity recently with massive datasets and the breakthroughs of artificial intelligence (AI). Clinical decision support systems (CDSSs) are the archetypical translation of these technologies into digital tools to empower physicians [1]. They integrate (i) individual data of a given patient with (ii) computerized knowledge to provide (iii) a personalized assessment or recommendation to the physician [2]. It may be symptom rating or an individual prognosis to support clinical decisions such as diagnosis, treatment selection, or monitoring. This value proposition must be differentiated from perception aids like dashboards (i.e., actionable visualization of a given patient's data with no integration with computerized knowledge), automatic measurement tools (i.e., assessment of naturally quantitative measures), and discussion support for shared decision-making (i.e., nonindividualized recommendation) [3]. General-purpose CDSSs have been widely implemented as embedded modules in electronic health records (EHRs), such as contraindication warnings or computerized physician ordering entry [4]. However, more specialized CDSS projects, whose developments require significant scientific expertise and dedicated web platforms, are less disseminated [5]. These projects embody the development of a computational approach to precision medicine. There is a vast amount of preclinical publications on predictive models using various AI techniques [6, 7]. The clinical development consists of implementing these proofs of concepts in CDSSs to be integrated into routine health care. Here, we aim to review the clinical development of CDSSs in the field of multiple sclerosis (MS).

MS is the most frequent autoimmune disease of the central nervous system (CNS), affecting approximately 2.8 million people worldwide with a rising prevalence [8]. Its typical natural history starts during early adulthood and evolves until advanced ages. Its precise pathophysiology is unsolved, associated to various degrees with relapsing–remitting inflammatory and progressive neurodegenerative processes. There are no well-defined disease subtypes, apart from the traditional clinical distinction of the primary progressive course from the relapsing–remitting course (RR-MS) sometimes transitioning to a secondary progressive course (SP-MS). In other diseases, molecular biomarkers such as autoantibodies or somatic mutations have enabled a mechanistic approach to precision medicine. In MS, the paucity of molecular biomarkers available in routine examination and the heterogeneity of individual natural histories and therapeutic responses make MS challenging for precision medicine [9]. Nonetheless, numerous immunoactive disease-modifying treatments are now approved with various efficacies [10]. Yet, none is fully curative. As the brain tissue has low repair capacities, MS management consists of preventing brain tissue injury and personalizing the therapeutic intervention by choosing the right treatment at the right time, making predictions ethical and desirable.

There are three main algorithmic approaches to CDSSs. (i) Expert systems (a.k.a., knowledge-based CDSSs) implement human

knowledge into knowledge bases to support deductive reasoning [11]. (ii) CDSSs based on data visualization query and visualize the data of similar patients recorded in reference databases to support analogy reasoning. Such on-demand accessible reference databases are called “data marts” [12]. (iii) Model-based CDSSs implement mathematical predictive models fitted with training datasets. They are commonly termed “AI-powered” or “AI-driven” and aim to support inductive reasoning [13]. The development of a CDSS is a multidisciplinary effort, and its evaluation is multidimensional. Ideally, the CDSS would provide the physician with an individual-level prediction that satisfies its needs in clinical practice [14]. Beyond having high and externally validated predictive performances, the context of usage of the CDSS should be along an already existing clinical pathway with relevant input data to be clinically meaningful [15]. The quality of the reference data used as a data mart or used to develop and validate models contributes to its credibility [16]. The algorithm should be understandable by the end-user (either transparent by design or explainable), and the interface should be usable (i.e., ergonomic, intelligible). The result of the CDSS should be safe concerning the emotional impact and have a decisional utility instead of only concurring with the user's prior beliefs. All these aspects would contribute to acceptance and certification for trustworthy clinical usage and accessibility through market distribution. There have been recent efforts to clarify a clinical development roadmap for CDSSs [13]. In this scoping review, we identify the historical and current CDSS projects in MS, compare their characteristics in the light of the aforementioned evaluation dimensions and clinical development roadmap, and discuss the current state of the art.

# METHODS

## Literature search and CDSS project selection

We searched PubMed up to 15 April 2024, with a query comprising the keywords identified by a first screening of the topic (Appendix S1-S2.). We also extracted authors of the field from the oral communications at ECTRIMS (2018-2023; Appendix S2) and searched their articles indexed in PubMed. We postulated a working definition of a CDSS project based on a wide-sense definition [2] (Box 1) and screened the articles for eligibility accordingly (Figure 1). Borderline cases were adjudicated by S.D., J.D.S., and P.-A.G. CDSS projects were identified from the articles, and we searched the web for their subsequent advances.

## Data charting process and synthesis

The report follows the PRISMA guideline for scoping reviews [17]. We describe and compare the technical and software development aspects of the CDSS projects in Tables 1 and 2, respectively. The projects are also represented graphically according to their chronology (Figure 2), their contexts of usage in MS management (Figure 3), and the stage of clinical development inspired by the
roadmap conveyed by the DECIDE-AI expert group [13] (Figure 4). We also propose a classification of their software integration approach (Figure 5) and distribution (Figure 6). Finally, the findings are synthesized narratively.

## BOX 1 The working definition of a CDSS used as a selection criteria during the literature search.

Working definition of a CDSS:

* A software/digital tool whose calculations are too complex to be implemented as a paper tool

* Input: characteristics of an individual patient

* Matched to computerized clinical knowledge

* Patient-specific assessments or recommendations are then presented to the clinician for a decision

* Not part of the basic features of a general-purpose EHR commercial platform

Additional selection criteria:

* The end-user is the neurologist

* Decision support for multiple sclerosis management

* Reported or disclosed efforts of clinical development: reported as an original article referenced in PubMed

Note: The items of the wide-sense definition postulated by Sim et al. 2001 are in bold. Abbreviations: CDSS, clinical decision support system; EHR, electronic health record.

# RESULTS

We screened 17,447 articles based on information in the title and 360 based on information in the abstract (Figure 1). The remaining 36 articles were subjected to full-text review. The final review included 24 articles reporting 14 CDSS projects, which satisfied our working definition (Box 1). Of note, among the excluded articles, 152 reported preclinical predictive models with various modeling approaches, but without reported efforts to implement them in a digital tool for neurologists. The bulk of AI-driven support reported in MS was about magnetic resonance imaging (MRI) postprocessing (automatic volu-metry or lesion segmentation; 238 articles). CDSS projects appeared as a small area in comparison to the entirety of digital health in MS. Hereafter, we synthesize narratively the 14 CDSS projects. We grouped them into sections according to their algorithmic approach: expert systems, data visualization, or model-based.

## Expert systems (knowledge-based CDSSs)

Expert systems are the archetypical software to navigate knowledge bases, which may be elicited from decision tables, decision trees, or ontologies. They may support physicians at diagnosis (i.e., computer-assisted diagnosis). The Clinical Decision Support System

<figure><img src="image_2.png" /><figcaption>**FIGURE 1** PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart. Articles were primarily identified with a search based on keywords and key authors (the last search was performed on 15 April 2024). CDSS, clinical decision support system; ECTRIMS, European Committee for Treatment and Research in Multiple Sclerosis.</figcaption></figure>
<table><caption>TABLE 1 Technical aspects of the CDSS projects.</caption><thead><tr><td>CDSS name</td><td>Context of usage</td><td>Input data</td><td>Algorithm technology</td></tr></thead><tbody><tr><td>AEDSS</td><td>Assessment during monitoring</td><td>Functional system scores</td><td>Expert system: production rules, extended Backus-Naur form syntax<br>More recently: ontology (Operational Conceptual Modeling Language)</td></tr><tr><td>EBDiMS</td><td>Prognostic counseling Initial prognosis</td><td>Short-term version: MS course, age at onset, EDSS, disease duration, number of relapses in the past 12 months<br>Long-term version: MS course, gender, age at onset, number of attacks in the first 2 years</td><td>Data visualization: weighted distance-based patient matching; statistical analysis in R</td></tr><tr><td>MS Prediction Score</td><td>Transition from RR-MS to SP-MS</td><td>Age, time since the last relapse, type of the last relapse, remission from the latest relapse</td><td>Model-based: continuous hazard functions estimated by Poisson regression models</td></tr><tr><td>MS BioScreen</td><td>Personalized monitoring</td><td>Any biomarker collected in the MS EPIC cohort</td><td>Data visualization: patients matching algorithm with customizable variables</td></tr><tr><td>MS Prognosis Simulation</td><td>First relapse, first year of RR-MS, monitoring the consecutive years</td><td>Age at onset of disease, gender, sphincter onset, pure motor onset, motor-sensory onset, sequel after onset, number of involved functional systems at onset, number of sphincter plus motor relapses, EDSS ≥ 4</td><td>Expert system: agent-based modeling, NetLogo 5.0.4</td></tr><tr><td>Function Watch (SMSreg)</td><td>Assessment during monitoring</td><td>Gender, age, disease duration, treatment; maximum 2-year-old reference data</td><td>Data visualization: patients matching algorithm with customizable variables</td></tr><tr><td>Bloodwatch, RiskMx</td><td>Alemtuzumab treatment monitoring</td><td>Laboratory results (in HL7 format)</td><td>Expert system: RiskMxTM system, matching against monitoring schedule and reference ranges</td></tr><tr><td>Clinical Decision Support System for Multiple Sclerosis Diagnosis</td><td>Diagnosis of RR-MS</td><td>45 demographic, clinical, and paraclinical items</td><td>Expert system: production rules</td></tr><tr><td>MSProDiscuss</td><td>Transition from RR-MS to SP-MS</td><td>Multiple demographic, clinical, and paraclinical items</td><td>Model-based: scoring algorithm, multiple logistic regression</td></tr><tr><td>MS TreatSim, UISS-MS</td><td>Initial prognosis</td><td>Presence of oligoclonal bands, age, lesion load, treatment</td><td>Model-based: agent-based modeling, Protégé OWL, UML modeling</td></tr><tr><td>PHREND (DESTINY)</td><td>Treatment selection</td><td>Age, gender, EDSS, index treatment, past treatment, disease duration, time since last relapse, relapse count, DMT count, efficacy class of the past treatment, duration of the past treatment, duration of the index treatment, and clinical site</td><td>Model-based: hierarchical Bayesian generalized linear model</td></tr><tr><td>Prognosis for patients with RR-MS<sup>a</sup></td><td>Prognosis counseling</td><td>Age, gender, disease duration, EDSS, number of GdE lesions, number of previous relapses during the previous 2 years, months since the last relapse, whether it is on treatment</td><td>Hierarchical Bayesian generalized linear model</td></tr><tr><td>MS Vista (PRIMUS)</td><td>Treatment selection</td><td>Age, sex, age at onset, disease duration, MS type, EDSS at the last visit, number of relapses within the past 12 months, number of T2 lesions on current MRI, GdE lesions on current MRI, number of new T2 lesions within the past 12 months</td><td>Data visualization: filter-based patient matching algorithm</td></tr><tr><td>sNfL reference app<sup>a</sup></td><td>Assessment during monitoring<br>Assessment of treatment response</td><td>Age, body mass index, sNfL level</td><td>Model-based: GAMLSS</td></tr></tbody></table>

Note: Platform names are mentioned in parentheses.

Abbreviations: AEDSS, Automatic Expanded Disability Status Scale; AUC, area under the curve; CDMS, clinically definite MS; CDSS, clinical decision support system; DMT, disease-modifying treatment; EBDiMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; EDSS, Expanded Disability Status Scale; EPIC, Epigenetics, Proteomics, Imaging, Clinical; EQ5D, EuroQoL group health questionnaire; FSMC, Fatigue Scale for Motor and Cognitive Functions; FSS, Fatigue Severity Scale; GAMLSS, generalized additive model for location, scale, and shape; GdE, gadolinium-enhancing; MRI, magnetic resonance imaging; MS, multiple sclerosis; MSFC, Multiple Sclerosis Functional Composite; MSIS-29, Multiple Sclerosis Impact Scale-29; MSSS, Multiple Sclerosis Severity Scale; OS, observational study; OWL, Web Ontology Language; PHREND, Predictive Healthcare with Real-World Evidence for Neurological Disorders; PRIMUS, Projections in Multiple Sclerosis; RCT, randomized clinical trial; RR-MS, relapsing-remitting MS; SDMT, Symbol Digit Modalities Test; SF36-1, the first question from the SF-36 Health Survey; SLCMSR, Sylvia Lawry Centre for MS Research; SMSreg, Swedish MS Registry; sNfL, serum neurofilament light chain; SP-MS, secondary progressive MS; UISS-MS, Universal Immune System Simulator (MS extension); UML, unified modeling language.

<footnote><sup>a</sup>The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. Inferential models are modeling approaches that do not assess predictive performance metrics, like for a classification or regression task.</footnote>
<table><thead><tr><td>Reference data</td><td>Individual assessment: prediction outcome</td><td>Predictive performance metric</td><td>Year of first report</td></tr></thead><tbody><tr><td>Expert knowledge (Kurtze rules)</td><td>EDSS</td><td>Kappa coefficient</td><td>2002</td></tr><tr><td>SLCMSR database reusing RCT and OS data; 45 sources</td><td>Individual risk profile: EDSS course, time to sustained progression, time to EDSS = 6</td><td>Brier Score</td><td>2007</td></tr><tr><td>Gothenberg incidental cohort Uppsala cohort from the SMSreg</td><td>Yearly probability of transition to SP-MS</td><td>Inferential model</td><td>2014</td></tr><tr><td>MS EPIC cohort at the University of California, San Francisco</td><td>Overlay of the patient's trajectory with the distribution of similar patients for any biomarker in the reference database</td><td>None</td><td>2014</td></tr><tr><td>Aggregated results in literature and 50 patients from the Hospital Egas Moniz, Lisboa, Portugal</td><td>Conversion to CDMS at 10 and 20 years, conversion to SP-MS at 10 years, risk of reaching EDSS = 6 at 10 and 20 years</td><td>Pearson correlation coefficient</td><td>2014</td></tr><tr><td>Sweden MS Registry</td><td>Function Watch diagram: overlay of the patient's metrics with the distribution of similar patients for EDSS, MSSS, MSFC, MSIS-29-Physical, MSIS-29-Psychological, SDMT, FSMC, FSS, EQ5D, activity/work capacity, MS-checklist, SF36-1</td><td>None</td><td>2015</td></tr><tr><td>Laboratory reference ranges</td><td>Alert the patient and neurologist in case of abnormal value or missed blood draw</td><td>None</td><td>2019</td></tr><tr><td>Decision tables, decision trees, and semantic networks according to the 2004 MS Diagnosis Guideline and McDonald's 2017 diagnostic criteria 130 medical records from the Shahid Beheshti Hospital of Kashan, Iran</td><td>Diagnosis of RR-MS</td><td>AUC and kappa coefficient</td><td>2020</td></tr><tr><td>Observational study of 3294 MS patients in the USA</td><td>Likelihood of progression</td><td>Inferential model with subsequent ranking and weighting of the predictors by physicians</td><td>2020</td></tr><tr><td>Aggregated data from MS literature, AFFIRM trial dataset</td><td>Relapses (as oligodendrocyte loss), cytokines, and immune cell population dynamics</td><td>Statistical comparison of the in silico results versus the real results</td><td>2020</td></tr><tr><td>NeuroTransData MS registry, CONFIRM, DEFINE, REGARD, TRANSFORMS, AFFIRM, CLARITY, OPERA I/II, and TEMSO trials.</td><td>Number of relapses, progression-free MRI, and confirmed disability progression up to 4.5 years</td><td>Mean squared error, negative log-likelihood, and Harrell's concordance statistic (C-index)</td><td>2020</td></tr><tr><td>Swiss Multiple Sclerosis Cohort</td><td>Relapses at 2 years</td><td>C-statistic</td><td>2021</td></tr><tr><td>ADVANCE trial dataset</td><td>Relapses, new MRI lesions, confirmed disability progression at 1 year</td><td>None</td><td>2022</td></tr><tr><td>Swiss Multiple Sclerosis Cohort, SMSreg, normative dataset of 4532 persons in the USA</td><td>Age- and body mass index-adjusted sNfL percentile values and z-scores</td><td>Odds ratio</td><td>2022</td></tr></tbody></table>
<table><caption>TABLE 2 Software development aspects of the CDSS projects.</caption><thead><tr><th>CDSS name</th><th>Institutional academic/ health care stakeholder</th><th>IT stakeholder</th><th>Industrial pharmaceutical stakeholder</th><th>Stage of development</th><th>Regulatory framework</th><th>Accessibility</th><th>Access/URL</th></tr></thead><tbody><tr><td>AEDSS</td><td>University of Bologna, Italy</td><td>None</td><td>None</td><td>Offline validation</td><td>Not reported</td><td>Online demonstration version for experimentation</td><td>http://cs.unibo.it/~gaspari/www/aedss/</td></tr><tr><td>EBDMS</td><td>Sylvia Lawry Center for MS Research, Munich, Germany</td><td>None</td><td>None</td><td>Offline validation against expert prognosis, user tests</td><td>Free, anyone may use the service for noncommercial purposes</td><td>Online, publicly available demo with a fraction of the reference database, restricted access to the whole reference database</td><td>http://slcmsr.net/ebdms-lo/</td></tr><tr><td>MS prediction score</td><td>University of Gothenburg, Sweden</td><td>None</td><td>None</td><td>POC</td><td>Not reported</td><td>Online</td><td>https://msprediction.com/</td></tr><tr><td>MS BioScreen</td><td>University of California, San Francisco, USA</td><td>None</td><td>None</td><td>Standalone beta version project of institution-wide platform interacting with the EHR: Bridge</td><td>Not reported</td><td>Institutional deployment</td><td>https://bioscreen.ucsf.edu/</td></tr><tr><td>MS Prognosis Simulation</td><td>Department of Anesthesiology, Lisbon, Portugal</td><td>None</td><td>None</td><td>POC validated against literature</td><td>Not reported</td><td>Online</td><td>http://www.arn.pt/Multiple_Sclerosis/Prognostic_Models_files/MSprognosisSimulation.html</td></tr><tr><td>Function Watch (SMSreg)</td><td>Karolinska Institute, Sweden</td><td>Carmona</td><td>None</td><td>Final release</td><td>GDPR</td><td>National deployment, Sweden</td><td>https://www.neuroreg.se/en/multi-pel-skleros/</td></tr><tr><td>Bloodwatch, RiskMx</td><td>University of Sydney, Australia</td><td>Medical Safety Systems (RMX)</td><td>Sanofi-Genzyme</td><td>Commercial release</td><td>Australian RMP, industrial service, not an MD</td><td>National deployment, Australia</td><td>https://bloodwatch.com.au/csp/riskmx/bloodwatch.csp</td></tr><tr><td>MSProDiscuss</td><td>Center of Clinical Neuroscience, University of Dresden, Germany</td><td>None</td><td>Novartis</td><td>Commercial release</td><td>GDPR, currently being assessed as software as an MD for CE mark eligibility</td><td>Online</td><td>https://www.msprodiscuss.com</td></tr><tr><td>TreatSim, UISS-MS</td><td>Combine Group, University of Catania, Italy</td><td>InSilicoTrials Technologies, Mimesis</td><td>None</td><td>Commercial release, but for RCT simulation</td><td>Not reported</td><td>Online for experimentation, SaaS for RCT simulation, with pay-per-use pricing</td><td>https://combine.dmi.uni.it/, https://mstreat.insiliconeuro.com/</td></tr><tr><td>PHREND (DESTINY)</td><td>NeuroTransData health care network, Germany</td><td>PricewaterhouseCoopers Digital Services</td><td>None</td><td>Certification as MD underway</td><td>CE mark as a class 1 MD; class 2a certification is currently underway</td><td>Institutional deployment Online demonstration version</td><td>https://www.neurotransdata.com/en/destiny</td></tr><tr><td>Prognosis for patients with RR-MS<sup>a</sup></td><td>University of Bern, Switzerland</td><td>None</td><td>None</td><td>POC, not externally validated</td><td></td><td>Online calculator, for academic research and educational purposes only</td><td>https://cinema.ispm.unibe.ch/shines/rrms/</td></tr><tr><td>MS Vista (PRIMUS)</td><td>University of Nantes, France</td><td>Pixyl, B-com</td><td>Biogen, Merck</td><td>POC of concept, user tests</td><td>GDPR, currently being assessed as software as an MD for CE mark eligibility</td><td>Not deployed</td><td>https://www.chu-rennes.fr/rechercherche-innovation/rhu-primus.html</td></tr><tr><td>sNfL reference app<sup>a</sup></td><td>University Hospital Basel, Switzerland</td><td>None</td><td>None</td><td>Offline validation</td><td>Not reported</td><td>Online, for academic research and educational purposes only</td><td>https://shiny.dkfb Basel.ch/basel/nfireference/</td></tr></tbody></table>

Note: Platform names are mentioned in parentheses.

Abbreviations: AEDSS, Automatic Expanded Disability Status Scale; CDSS, clinical decision support system; CE, European Conformity; EBDMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; EHR, electronic health record; GDPR, General Data Protection Regulation; IT, information technology; MD, medical device; MS, multiple sclerosis; PHREND, PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; POC, proof of concept; PRIMUS, Projections in Multiple Sclerosis; RCT, randomized clinical trial; RMP, risk management plan; RR-MS, relapsing–remitting MS; SaaS, software as a service; SMSreg, Swedish MS Registry; sNfL, serum neurofilament light chain; UISS-MS, Universal Immune System Simulator (MS extension).

<footnote>aThe "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review.</footnote>
<figure><img src="image_3.png" /><figcaption>**FIGURE 2** Chronology of the clinical decision support system (CDSS) projects and their technological basis. The years indicate the first publication or the first release (if specified in the article). Expert systems (also known as knowledge-based CDSSs) implement human knowledge into knowledge bases. CDSSs based on data visualization query and visualize the data of similar patients recorded in reference databases. Model-based CDSSs implement mathematical predictive models. They are commonly termed "AI-powered" or "AI-driven." *The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. AEDSS, Automatic Expanded Disability Status Scale; EBDIMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; GAMLSS, generalized additive model for location, scale and shape; GLM, generalized linear model; MS, multiple sclerosis; PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; RR-MS, relapsing–remitting MS; sNfL, serum neurofilament light chain; UISS-MS, Universal Immune System Simulator (MS extension).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**FIGURE 3** Contexts of usage of the clinical decision support system projects along the management of multiple sclerosis (MS). Peaks represent clinical relapses. Platform names are mentioned in parentheses. *The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. AEDSS, Automatic Expanded Disability Status Scale; EBDIMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; PRIMUS, Projections in Multiple Sclerosis; RR-MS, relapsing–remitting MS; SMSreg, Swedish MS Registry; sNfL, serum neurofilament light chain; SP-MS, secondary progressive multiple sclerosis; UISS-MS: Universal Immune System Simulator (MS extension).</figcaption></figure>

for Multiple Sclerosis Diagnosis is a proof of concept of a CDSS implementing the logic of the 2004 MS Diagnosis Guideline and McDonald's 2017 diagnostic criteria [18–20]. Decision tables, decision trees, and a semantic network have been elicited from these

sources to represent the knowledge and implemented as production rules in a knowledge base. The user may select the clinical and paraclinical findings from a list of 45 items. The CDSS then gives the degree of certainty of MS diagnosis or suggests the diagnosis of
<figure><img src="image_5.png" /><figcaption>**FIGURE 4** Advancement of the clinical development of the clinical decision support system (CDSS) projects. The roadmap has been inspired by the one conveyed by the DECIDE-AI working group. Circles represent the reported development steps and are colored according to the accessibility of the CDSS. Multiple steps could be reported by a single article. Platform names are mentioned in parentheses.
*The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. AEDSS, Automatic Expanded Disability Status Scale; EBDiMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; MD, medical device; MS, multiple sclerosis; MVP, minimum viable product; PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; POC, proof of concept; PRIMUS, Projections in Multiple Sclerosis; RR-MS, relapsing–remitting MS; SMSreg, Swedish MS Registry; sNfL, serum neurofilament light chain; UISS-MS, Universal Immune System Simulator (MS extension).</figcaption></figure>

another disease with clues for the examination. The diagnosis performance for RR-MS has been assessed over 130 medical records from the Shahid Beheshti Hospital of Kashan, Iran. The patients of this validation set had RR-MS (n=91), stroke, neuromyelitis optica, CNS tumors, or acute encephalomyelitis. User tests with 10 neurologists confirmed the ease of learning, memorability, and satisfaction. The CDSS has been deployed at the Neurology Department of the Shahid Beheshti Hospital of Kashan without a later report of external distribution.

Expert systems may serve as assessment tools for the patient's health status. Quantifying health status with metrics helps follow slow evolutions and detect subtle worsening. In MS, the level of disability is globally assessed with the Expanded Disability Status Scale (EDSS) [21]. The logic of EDSS scoring is complex, leading to a risk of interrater variability and intrarater inconsistencies. The Automatic EDSS project provides decision support to rate the EDSS [22]. This expert system implements the logic of the original Kurtzke rules [21]. The proof of concept used production rules [22], and a further version used an ontology of the EDSS domain [23]. The tool is thus able to detect inconsistencies in the input data and to suggest corrections to the user interactively. The latest version reduced interrater variability by 3.75% and corrected errors in 14.5% of cases, showing its

utility. A demonstration version for "research use only" is available online (Table 2). International validation experiments were planned to validate its routine clinical use but never reported. Instead, the Neurostatus EDSS has been globally accepted as an electronic data capture system for randomized clinical trials (RCTs) in MS [24]. The structured input is the Neurostatus per se, a standardized neurological examination. The Neurostatus items are then mapped to the Kurtzke functional system scores and the ambulatory score to compute the EDSS. However, in our sense, the logic of the algorithm does not go beyond the traditional electronic implementation of a paper tool with automatic data quality controls. Therefore, we did not consider Neurostatus EDSS as a CDSS.

Other use cases focus on automating the monitoring. "Bloodwatch" is a mobile application to monitor remotely patients who underwent an immune reconstitution therapy by alemtuzumab [25]. The application automates the interpretation of laboratory test results by sending them electronically to the RiskMx backend. RiskMx compares the values to reference ranges and to the monitoring agenda. It sends alerts to the neurologist if it identifies autoimmune adverse events during the immune reconstitution or if the patient misses a blood draw. The system has been evaluated by the Alemtuzumab in MS Safety System study [25].
<figure><img src="image_6.png" /><figcaption>**FIGURE 5** Software integration approaches with a proposed classification of the covered clinical decision support system (CDSS) projects. Local applications are installed and contained in a single computation unit (desktop computer, smartphone, etc.). Applications may also be installed over a whole institution's network, typically with a client-server architecture. Web-based applications do not require any installation by the customer. They are provided as "software as a service." Standalone applications require manual data input, whereas interoperable applications have automatic data exchanges typically according to standard data formats. *The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. AEDSS, Automatic Expanded Disability Status Scale; EBDiMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; EHR, electronic health record; MS, multiple sclerosis; PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; RR-MS, relapsing-remitting MS; sNfL, serum neurofilament light chain; UISS-MS, Universal Immune System Simulator (MS extension).</figcaption></figure>

For the 10 patients enrolled in the study, the system was significantly faster at identifying and prompting reactions by the medical team than standard paper-mail-based laboratory test monitoring. The software has been developed in collaboration between the University of Sydney, an IT corporation (Medical Safety Systems), and Sanofi-Genzyme. It is released commercially at the national scale in Australia. It has been certified as an industrial service, not a medical device (MD).

## CDSSs based on data visualization

Besides expert-driven knowledge, there has been a paradigm shift toward data-driven knowledge. It conveys a disruption in the intentionality of data, shifting from a primary use as health records to

secondary uses as sources of empirical knowledge. This led to the constitution of institutional or national observational disease registries collecting clinical data, often through dedicated observational studies (OSs), apart from the real-world data collection in EHRs. Several CDSS projects are based on the transformation of these databases into data marts, from which data of similar patients may be queried and visualized on demand.

At the institutional scale, the MS BioScreen is an iPad-based application providing a multimodal visualization of clinical, MRI, biological, and genetic data for an MS patient on a single user interface [26]. The patient data are recorded in the Epigenetics, Proteomics, Imaging, Clinical (EPIC) cohort, which is the institutional MS cohort at the University of California, San Francisco. The system fetches them to visualize the trajectory of any marker. Moreover, the system features the "contextualization" of the
<figure><img src="image_7.png" /><figcaption>**FIGURE 6** Software distribution options with a proposed classification of the covered clinical decision support system (CDSS) projects. MS Prognosis Simulation and MS Vista have no reported distribution. "Internal usage" is the case when a health care institution implements an algorithm as a CDSS with no external distribution. It relies on internal or outsourced software development. "Peer-to-peer" is an external distribution in a noncertified framework. For instance, software may be shared through URL links to access a web application or a download but not be certified for medical usage. "Marketing" distribution requires the CDSS to be certified for medical usage as a medical device or another qualification depending on the local regulation and the intended use. Although MSProDiscuss was intended for marketing distribution, it is currently unavailable through this channel. *The "Prognosis for patients with RR-MS" and "sNfL reference app" names were given by the authors of this review. AEDSS, Automatic Expanded Disability Status Scale; EBDiMS, Evidence-Based Decision Support Tool in Multiple Sclerosis; MS, multiple sclerosis; PHREND, Predictive Healthcare With Real-World Evidence for Neurological Disorders; RR-MS, relapsing–remitting MS; SaMD: Software as a Medical Device; sNfL, serum neurofilament light chain; UISS-MS, Universal Immune System Simulator (MS extension).</figcaption></figure>

patient data. Concretely, patient metrics may be expressed as quantiles and visualized overlaid with the summary of a reference population. In this case, the reference population is restricted to a subgroup of patients from the EPIC cohort sharing a set of similar characteristics as chosen by the neurologist. This enables personalized monitoring by personalizing the reference ranges in meaningful pathological contexts. The BioScreen was first reported as a standalone physician-facing tool. Since then, it has been replaced by the Open MS BioScreen web portal, which made the concept accessible to patients of the institution to support self-management [27]. The portal also refers to several web-based calculators such as a progressive multifocal leukoencephalopathy (PML) risk calculator for patients treated with natalizumab. The Bridge project currently extends the concept to a cross-specialty institution-wide EHR-agnostic platform [5]. It features modular data integration of the different modalities of records in the institution as well as modular clinic-specific interoperable dashboards and CDSS modules that may be embedded in the EHR (e.g., for fall

telemonitoring in MS [28] or perception aid for the evaluation of Parkinson disease [29]).

At the national scale, the Function Watch is a tool embedded in the data visualization application of the Swedish MS Registry [30]. Several national or international MS registries have developed data visualization applications that summarize graphically the history of a patient as an individual timeline. The Function Watch is a decision support module that contextualizes patient data among a subgroup of similar patients recorded in the registry. Twelve markers are represented in a radar plot with an overlay summarizing the subgroup data (Table 1). Such insight is cross-sectional, without individual prognosis. It has been developed in collaboration between the Karolinska Institute and an IT corporation (Carmona) and is deployed at the national scale in Sweden.

Other projects use RCT data as reference databases. The Evidence-Based Decision Support in Multiple Sclerosis (EBDiMS) project is a CDSS describing the natural evolution of MS. Two versions have been developed by the Sylvia Lawry Centre for
Multiple Sclerosis Research, Munich, Germany: a short-term and a long-term tool. The first describes the near-term natural evolution, using 45 separate datasets of academic and industrial RCTs [31]. The reported data mart is a population of 1059 placebo patients pooled from 30 RCT placebo arms. The full data mart including 20,000 patients is accessible only to authorized staff. The CDSS queries similar patients with a patient-matching algorithm and describes their natural history over 24-36 months. The second version describes the long-term evolution using OS data [32]. The data mart is a population of 717 patients from the London/Ontario natural history cohort [33]. The CDSS estimates an individual long-term prognosis of up to 30 years. The estimated individual prognoses of the long-term version have been validated against experts' predictions using the integrated Brier Score [32]. Experts and EBDiMS had similar Brier Scores, indicating similar predictive performances. The usability and safety (here: the emotional impact for patients) have been assessed for both versions [14, 34]. They appeared understandable and acceptable, without causing relevant anxiety. However, they appeared of modest utility, as they did not change patients' prognostic estimates.

MS Vista introduces the comparison of two therapeutic scenarios [35]. The data mart is a population of approximately 800 patients from the ADVANCE trial (peginterferon beta-1a vs. placebo) [36]. Similar patients may be queried by filtering the data mart. Continuous variables were discretized according to the decision thresholds derived from epidemiological studies and validated by expert neurologists. These thresholds reflect the mental decision trees used in practice and are meant to favor adoption by neurologists. MS Vista describes the evolution of the selected subgroup at a 1-year horizon. The software is a proof of concept for the Projections in Multiple Sclerosis (PRIMUS) project. There is ongoing work by the authors with subsequent prototypes named "PRIMUS" to integrate several RCTs and observational studies to develop a CDSS able to inform all therapeutic scenarios [37].

## Model-based CDSSs

Model-based CDSSs implement mathematical predictive models into physician-facing software. The mathematical model may be formulated by humans (e.g., mechanistic differential equations) or learned automatically by computers (e.g., deep learning). Although less interpretable than data visualization, their inferential nature enables model-based CDSSs to inform yet unseen situations. The decision options may be explicit or not. Prediction scores that do not integrate precise treatment options in the predictors and yield absolute statistics (e.g., risk of disease activity or progression) give prospective insights and have implicit decisional consequences such as stratification of treatment, frequency of monitoring, or scale rating.

The MS Prognosis Simulation aims at simulating long-term prognosis using agent-based modeling [38]. It is a system science approach where the "agents" model entities in a complex system.

The simulation aims at predicting emergence phenomena at the system level. In this work, the system was the MS population and the agents were individuals. NetLogo software was used to program the simulation model, and the aggregated data from the scientific literature were used to parametrize the population behavior [39]. From the characteristics of MS onset and of the current visit (Table 1), the software predicts the conversion from a first demyelinating event to clinically definite MS, the conversion to SP-MS, and the risk of reaching EDSS=6 at 10 and 20 years. The CDSS has been validated with a cohort of 50 patients from the Hospital Egas Moniz, Lisboa, Portugal by assessing the correlation of the predicted and real EDSS, the accuracy of the predicted MS severity score decile, and the accuracy of the disability classification (MS Severity Rank calculator).

Likewise, the MS extension of the Universal Immune System Simulator relies on agent-based modeling [40]. In this CDSS project, the simulation is mechanistic and aims at predicting the main features and dynamics of the immune system activities at the cellular and molecular level. The system level is the individual patient, and the agents are nervous cells, immune cells, and cytokines. Based on characteristics at onset, the simulation predicts cytokines and immune cell population dynamics, as well as relapses (as oligodendrocytes loss). It may simulate the therapeutic effects of teriflunomide, fingolimod, beta interferon, ocrelizumab, and natalizumab. The interactions in the computational model were parametrized with immunology literature results. A preliminary validation reproduced the relapse occurrence of six patients [40]. The CDSS project evolved toward an in silico clinical trials simulator named "MS TreatSim," involving a collaboration between the Combine Group at the University of Catania, Italy, and an IT corporation (InSilicoTrials Technologies, Mimesis) [41]. The simulator has been validated by reproducing in silico the AFFIRM trial (natalizumab vs. placebo) [42]. TreatSim is currently intended to support clinical trial design, but the authors foresee its use as clinical decision support for treatment selection.

The DESTINY project proposes an integrative platform for MS management at the scale of the health care network NeuroTransData in Germany [43]. Its predictive module PHREND (Predictive Healthcare With Real-World Evidence for Neurological Disorders) implements a Bayesian statistical model [44]. It has been fitted using data from the NeuroTransData MS registry (approximately 25,000 patients). It is refitted quarterly and has been validated externally with nine RCT datasets [45]. The model's inputs are the therapeutic history and the clinical assessment. It predicts the number of relapses and the occurrence of confirmed disability progression (and more recently, progression-free MRI) at a horizon that may be up to 4.5 years. The model has low performance in predicting the exact number of relapses and confirmed disability progression, but makes consistent relative predictions, that is, personalized ranking of the different treatments' benefits. This personalized ranking refines the relative benefits of the treatments to individual-level predictions using interaction terms compared to the population-level assessment in RCTs or network meta-analyses. Clinical utility has been assessed by comparing the evolutions of patients taking the
highest ranked treatment with others. The software is certified as a class I MD by the European Medicines Agency and is deployed in the NeuroTransData network (Table 2). Further work is ongoing to certify the software as a class IIa MD.

MSProDiscuss is a CDSS supporting the discussion during the visit as a physician-completed digital checklist to detect early signs of transition to SP-MS [46]. The development of the tool has been driven by physician-reported and patient-reported data to assess the feasibility and elicit the relevant variables [47]. The importance of the variables has been stratified by a quantitative analysis over a reference dataset of 3294 patients in the USA using a LASSO (least absolute shrinkage and selection operator) penalized logistic regression. This drove the design of the questionnaire provided by the web application. A scoring algorithm weighing the questionnaire items was derived from this multiple logistic regression. Beyond the classically prominent impairment of lower limb motricity, subtle signs have been identified as suggestive of the transition to the progressive phase. Once the form is completed, the CDSS computes a score estimating the likelihood of progression. The CDSS has been developed in collaboration between the Center of Clinical Neuroscience (University of Dresden, Germany) and Novartis. It has been released globally to evaluate its usability and usefulness, which were good [48]. However, Novartis, which owns the CDSS, has stopped its industrialization to certify it as a "software as a medical device" and stopped online access.

Likewise, the MS Prediction Score calculator aims at predicting the transition from RR-MS to SP-MS [49]. It implements a continuous hazard function estimated by a Poisson regression model fitted on the Gothenburg incidence cohort from Sweden (n=157) and was later validated with the Uppsala cohort from the Swedish MS Registry. The CDSS computes the yearly probability of transition to SP-MS. It is released globally as a free-access online calculator but without certification for medical usage.

Another statistical model has been implemented as an online calculator to predict the risk of relapse at 2 years (here referred to as "prognosis for patients with RR-MS") [50]. It is based on a mixed-effect Bayesian generalized linear model fitted on the Swiss MS cohort. It proposes to assess the clinical usefulness of the CDSS through decision curve analysis, which calculates the net benefit of the CDSS compared to default therapeutic strategies. The authors foresee external validation as the next step of development. Yet, they do not define a precise context of usage of the CDSS along the clinical pathways of MS management.

Model-based CDSSs may also help integrate quantitative molecular biomarkers in clinical practice. Serum neurofilament light chain (sNfL) is a molecular biomarker gaining popularity in detecting subclinical neuroinflammatory activity. However, numerous confounding factors are hindering the definition of a universal decisional threshold. To interpret sNfL levels, an online calculator (here referred to as "sNfL reference app") has been proposed to compute an age- and body mass index-adjusted z-score according to a reference population of US and European control persons [51]. This reference database served to fit a generalized additive model for location,

scale, and shape to capture the distribution of sNfL levels in healthy subjects. Using longitudinally collected samples obtained from 1313 MS patients participating in the Swiss MS registry, z-scores > 1.5 were associated with a 3.15-fold increase in future MS activity. The findings were validated externally using the Swedish MS Registry. The generalizability of the tool requires that the sNfL level is measured with the standard kit and on the same hardware platform as the one of the reference database. The authors envision the use of this calculator in RCTs where sNfL is an endpoint measure rather than in routine practice.

## DISCUSSION

CDSSs make knowledge accessible to physicians "on the fly," whether it is derived from experts, data from similar patients, or predictive models. We reviewed the CDSS projects with reported clinical developments in the field of MS. Our literature screening confirms the bottleneck between preclinical predictive model development and the clinical development of CDSSs (Figure 1). The projects we identified have various algorithmic approaches (Figure 2), position themselves for various contexts of usage (Figure 3), have various stages of advancement (Figure 4), and have various strategies to reach integration in health care (Figures 5 and 6).

There are some limits to this study. The first is the necessity to postulate a more stringent working definition of a CDSS than the wide-sense definition postulated in the early 2000s [2] (Box 1). The wide-sense definition could apply to any online calculator implementing a paper-based prognosis score or symptoms scale. In our sense, such cases do not reflect the translational efforts behind CDSS projects. We restricted our screening to the PubMed database, as we were only interested in reports of clinical development disclosing technical details about the algorithmic approach. This is seldom the case of CDSSs reported on software distribution channels only. There were borderline digital tools, possibly identifying themselves as decision support, requiring adjudication. Likewise, we included some projects in the early phase that did not necessarily identify themselves as CDSS projects. The most advanced projects did not necessarily report every step of clinical development in a dedicated article (proof of concept, offline validation, refined prototype, live clinical evaluation, comparative prospective evaluation). Some projects are only reported with a summary review article [30]. Overall, it shows yet maturing report practices. The development of a CDSS is iterative by nature, which means that some prototypes we reviewed have been replaced at subsequent project iterations. The updates on project activity, industrialization, and distribution may not be publicly available.

CDSS development is a multidisciplinary translational effort up to integration in health care and could use different distribution options (Figure 6). The most advanced projects involved collaborations between an institutional stakeholder providing the scientific input, an IT corporation to implement the analytics in an industrialized fashion up to certification, and/or a pharmaceutical company
to deploy and maintain the CDSS in the long run. So far, no project has completed its clinical development up to certification for medical use with market distribution (Figure 4). MS TreatSim evolved toward RCT design support and MS BioScreen toward self-management support instead of clinical decision support [27, 41]. None is currently recommended in international practice guidelines, probably due to a lack of externally validated models [52]. The ones that are distributed did not necessarily seek certification as an MD to be marketed, which we propose to call “peer-to-peer” distribution (Figure 6). Bloodwatch and MS TreatSim are marketed, but not as MDs. In the sense of the European Union Medical Device Regulation, certification as an MD is necessary to distribute software for specific medical usage, like CDSSs [53, 54]. Even if the CDSS algorithm is validated scientifically, its implementation as a digital tool is at risk of technical flaws. The certification as MD acts as a guarantee for physicians to trust the implementation. However, certifying a CDSS involves an important financial investment and thus a technology transfer to the private sector, such that Software as medical devices (SaMDs) are usually proprietary instead of freeware. The transfer to the private sector is usually thought to bring sustainability to the CDSS, but the case of MSProDiscuss showed that the distribution may be stopped unilaterally by the CDSS owner for economic reasons. “Peer-to-peer” distribution appears as a way to bypass the market and the private sector (Figure 6). For instance, the CDSS may be shared as freeware through URL links if web-based (e.g., the PML risk score calculator referred by the Open MS Bioscreen platform) or sent through peer-to-peer correspondence if desktop. In an academic setting, it is a means to share proof of concepts, but it is officially restricted to nonmedical usage. Still, certifying relatively simple calculators as MDs seems overkill and likely increases the implementation gap of digital tools compared to paper tools. A third distribution option could be that an institution implements an algorithm itself for internal usage only (Figure 6), which is the case of CDSS projects that do not seek external distribution.

Besides accessible implementation, the adoption of CDSSs in clinical practice involves human factors including but not limited to usability and utility [1]. Most of the CDSS projects report user tests to assess the system's usability. Small-scale user tests are reported at the proof of concept step, whereas a minority of projects report large-scale user tests in dedicated reports [48]. The assessment of the utility of the CDSS appears more critical. Although EBDiMS is the most accessible CDSS with a freely accessible demonstration version, its assessment showed a low utility because its individual prognoses did not change the physician's or patient's prior beliefs. This highlights the importance of CDSSs not only to be confirmatory of the user's prior beliefs. At most, confirmatory CDSSs may be used as discussion support with the patient. As such, the EBDiMS positioned its context of usage as “prognosis counseling” (Figure 3) [34].

Likely, the context of usage of a CDSS has already been explored by epidemiological studies with population-level models. Therefore, a CDSS should seek its utility from the individual-level granularity of its insight. Utility could be measured by prospective

comparative efficacy studies, such as RCTs comparing CDSS-aided care and conventional care [55]. Another perspective for CDSS utility is to facilitate the integration of quantitative biomarkers in clinical practice. It may be molecular biomarkers (e.g., sNfL) [51], radiomic biomarkers (e.g., the rate of whole brain atrophy), or digital biomarkers (e.g., patient-reported outcome measures or signals from wearable sensors) [56]. Traditionally, the integration of quantitative biomarkers requires the definition of a decisional threshold by population-level studies [57]. Defining a one-size-fits-all threshold may be challenged by confounding factors. The ability of CDSS algorithms to integrate many variables to compute individual-level predictions is likely to overcome this limitation. Finally, the assessment of safety also includes the assessment of the emotional impact the prediction may have on the patient, thus calling for patient-centric designs.

## CONCLUSIONS

The clinical development of CDSSs is an emerging multidisciplinary and translational field whose processes are still maturing. The complexity of MS and the high amount of therapeutic options have aroused several CDSS projects, with various algorithmic approaches. They illustrate the potential of clinical applications of IT and massive data to support MS management. Their review helps clarify the roadmap for future projects, as a multidisciplinary and multistep process. One can expect large-scale RCTs testing the utility and safety of CDSSs in the coming years, similar to the assessment of regular MDs. Although the development of IT brought various algorithmic techniques, there is always a critical need for reference datasets informing the context of usage of the CDSS. As a result, data collection is increasingly integrated into care [58], often through the platform hosting the CDSS [43]. Data become more collected from patients themselves, either actively or passively. The progress of data architecture, shifting practices toward interoperability, will ease the integration of modular CDSSs into platforms, as well as the integration of multiple modalities of data.

## AUTHOR CONTRIBUTIONS

**Stanislas Demuth:** Conceptualization; investigation; writing – original draft; writing – review and editing. **Chadia Ed-Driouch:** Conceptualization; investigation; writing – review and editing. **Cédric Dumas:** Supervision; writing – review and editing. **David Laplaud:** Writing – review and editing; resources; funding acquisition. **Gilles Edan:** Resources; writing – review and editing; supervision; funding acquisition. **Nicolas Vince:** Writing – review and editing; resources. **Jérôme De Sèze:** Supervision; resources; writing – review and editing. **Pierre-Antoine Gourraud:** Conceptualization; supervision; resources; writing – review and editing; funding acquisition.

## FUNDING INFORMATION

This work has been supported by a government grant managed by the National Research Agency under the program “Investissements d'avenir” with the reference KTD-Innov (ANR-17-RHUS-0010). This
project has received funding from the European Union's Horizon 2020 Research and Innovation Program under grant agreement No. 754995, and the scholarship "Bourse Région Pays de la Loire" number 2019_11235. This review is part of the PRIMUS project, which was supported in part by the French National Research Agency (Agence Nationale de la Recherche [ANR]) as its third PIA, integrated into the France 2030 plan under reference ANR-21-RHUS-0014.

## CONFLICT OF INTEREST STATEMENT

P.-A.G. is the founder of Methodomics (2008) and the cofounder of Big Data Santé (2018). He consults for major pharmaceutical companies, all of which are handled through academic pipelines (AstraZeneca, Biogen, Boston Scientific, Cook, Edimark, Ellipses, Elsevier, Methodomics, Merck, Mérieux, Sanofi-Genzyme, Octopize). P.-A.G. is a volunteer board member at AXA not-for-profit mutual insurance company (2021). He has no prescription activity with either drugs or devices. None of the other authors has any conflict of interest to disclose.

## DATA AVAILABILITY STATEMENT

Data-sharing is not applicable to this article, as no new data were created or analyzed in this study.

## REFERENCES

1. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. *Npj Digit Med*. 2020;3(1):1-10. doi:10.1038/s41746-020-0221-y

2. Sim I, Gorman P, Greenes RA, et al. Clinical decision support systems for the practice of evidence-based medicine. *J Am Med Inform Assoc*. 2001;8(6):527-534. doi:10.1136/jamia.2001.0080527

3. Col NF, Solomon AJ, Alvarez E, et al. Implementing shared decision-making for multiple sclerosis: the MS-SUPPORT tool. *Mult Scler Relat Disord*. 2023;80:105092. doi:10.1016/j.msard.2023.105092

4. Peikari HR, Zakaria MS, Yasin NM, Shah MH, Elhissi A. Role of computerized physician order entry usability in the reduction of prescribing errors. *Healthc Inform Res*. 2013;19(2):93-101. doi:10.4258/hir.2013.19.2.93

5. Bove R, Schleimer E, Sukhanov P, et al. Building a precision medicine delivery platform for clinics: the University of California, San Francisco, BRIDGE experience. *J Med Internet Res*. 2022;24(2):e34560. doi:10.2196/34560

6. Arani LA, Hosseini A, Asadi F, Masoud SA, Nazemi E. Intelligent computer systems for multiple sclerosis diagnosis: a systematic review of reasoning techniques and methods. *Acta Inform Med*. 2018;26(4):258-264. doi:10.5455/aim.2018.26.258-264

7. Alshamrani R, Althbiti A, Alshamrani Y, Alkomah F, Ma X. Model-driven decision making in multiple sclerosis research: existing works and latest trends. *Patterns*. 2020;1(8):100121. doi:10.1016/j.patter.2020.100121

8. Walton C, King R, Rechtman L, et al. Rising prevalence of multiple sclerosis worldwide: insights from the atlas of MS, third edition. *Mult Scler*. 2020;26(14):1816-1821. doi:10.1177/1352458520970841

9. Roxburgh RHSR, Seaman SR, Masterman T, et al. Multiple sclerosis severity score: using disability and disease duration to rate disease severity. *Neurology*. 2005;64(7):1144-1151. doi:10.1212/01.WNL.0000156155.19270.F8

10. De Angelis F, John NA, Brownlee WJ. Disease-modifying therapies for multiple sclerosis. *BMJ*. 2018;363:k4674. doi:10.1136/bmj.k4674

11. Shortliffe EH, Buchanan BG, Feigenbaum EA. Knowledge engineering for medical decision making: a review of computer-based clinical decision aids. *Proc IEEE*. 1979;67(9):1207-1224. doi:10.1109/PROC.1979.11436

12. Inmon W. Data Mart does not equal data warehouse. 2000. DMReview Com; 7.

13. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. *Nat Med*. 2022;28(5):924-933. doi:10.1038/s41591-022-01772-9

14. Heesen C, Gaissmaier W, Nguyen F, et al. Prognostic risk estimates of patients with multiple sclerosis and their physicians: comparison to an online analytical risk counseling tool. *PLoS One*. 2013;8(5):e59042. doi:10.1371/journal.pone.0059042

15. Voigt I, Inojosa H, Dillenseger A, Haase R, Akgün K, Ziemssen T. Digital twins for multiple sclerosis. *Front Immunol*. 2021;12:669811. doi:10.3389/fimmu.2021.669811

16. FDA. Assessing the credibility of computational modeling and simulation in Medical Device Submissions. U.S. Food and Drug Administration. Published January 25, 2022. Accessed August 3, 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/assessing-credibility-computational-modeling-and-simulation-medical-device-submissions

17. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. *Ann Intern Med*. 2018;169(7):467-473. doi:10.7326/M18-0850

18. Hosseini A, Asadi F, Arani LA. Development of a knowledge-based clinical decision support system for multiple sclerosis diagnosis. *J Med Life*. 2020;13(4):612-623. doi:10.25122/jml-2020-0182

19. Thompson AJ, Banwell BL, Barkhof F, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. *Lancet Neurol*. 2018;17(2):162-173. doi:10.1016/S1474-4422(17)30470-2

20. National Collaborating Centre for Chronic Conditions (UK). Multiple sclerosis: National Clinical Guideline for Diagnosis and Management in Primary and Secondary Care. Royal College of Physicians (UK); 2004. Accessed October 9, 2023. http://www.ncbi.nlm.nih.gov/books/NBK48919/

21. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an Expanded Disability Status Scale (EDSS). *Neurology*. 1983;33(11):1444-1452. doi:10.1212/wnl.33.11.1444

22. Gaspari M, Roveda G, Scandellari C, Stecchi S. An expert system for the evaluation of EDSS in multiple sclerosis. *Artif Intell Med*. 2002;25(2):187-210. doi:10.1016/S0933-3657(02)00015-5

23. Gaspari M, Saletti D, Scandellari C, Stecchi S. Refining an automatic EDSS scoring expert system for routine clinical use in multiple sclerosis. *IEEE Trans Inf Technol Biomed*. 2009;13(4):501-511. doi:10.1109/TITB.2008.926498

24. D'Souza M, Yaldizli Ö, John R, et al. Neurostatus e-scoring improves consistency of expanded disability status scale assessments: a proof of concept study. *Mult Scler*. 2017;23(4):597-603. doi:10.1177/1352458516657439

25. Reddel SW, Barnett MH, Riminton S, et al. Successful implementation of an automated electronic support system for patient safety monitoring: the alemtuzumab in multiple sclerosis safety systems (AMS3) study. *Mult Scler*. 2019;25(8):1124-1131. doi:10.1177/1352458518783673

26. Gourraud PA, Henry RG, Cree BAC, et al. Precision medicine in chronic disease management: the multiple sclerosis BioScreen. *Ann Neurol*. 2014;76(5):633-642. doi:10.1002/ana.24282

27. Schleimer E, Pearce J, Barnecut A, et al. A precision medicine tool for patients with multiple sclerosis (the open MS BioScreen): human-centered design and development. *J Med Internet Res*. 2020;22(7):e15605. doi:10.2196/15605

28. Block VJ, Koshal K, Wijangco J, et al. A closed-loop falls monitoring and prevention app for multiple sclerosis clinical practice:
human-centered design of the multiple sclerosis falls InsightTrack. *JMIR Hum Factors*. 2024;11(1):e49331. doi:10.2196/49331

29. Brown EG, Schleimer E, Bledsoe IO, et al. Enhancing clinical information display to improve patient encounters: human-centered design and evaluation of the Parkinson disease-BRIDGE platform. *JMIR Hum Factors*. 2022;9(2):e33967. doi:10.2196/33967

30. Hillert J, Stawiarz L. The Swedish MS registry - clinical support tool and scientific resource. *Acta Neurol Scand*. 2015;132(199):11-19. doi:10.1111/ane.12425

31. Daumer M, Neuhaus A, Lederer C, et al. Prognosis of the individual course of disease-steps in developing a decision support tool for multiple sclerosis. *BMC Med Inform Decis Mak*. 2007;7:11. doi:10.1186/1472-6947-7-11

32. Galea I, Lederer C, Neuhaus A, et al. A web-based tool for personalized prediction of long-term disease course in patients with multiple sclerosis. *Eur J Neurol*. 2013;20(7):1107-1109. doi:10.1111/ene.12016

33. Scalfari A, Neuhaus A, Degenhardt A, et al. The natural history of multiple sclerosis: a geographically based study 10: relapses and long-term disability. *Brain*. 2010;133(Pt 7):1914-1929. doi:10.1093/brain/awq118

34. Kosch R, Schiffmann I, Daumer M, et al. Long-term prognostic counselling in people with multiple sclerosis using an online analytical processing tool. *Mult Scler*. 2021;27(9):1442-1450. doi:10.1177/1352458520964774

35. Ed-Driouch C, Chéneau F, Simon F, et al. Multiple sclerosis clinical decision support system based on projection to reference datasets. *Ann Clin Transl Neurol*. 2022;9(12):1863-1873. doi:10.1002/acn3.51649

36. Calabresi PA, Kieseier BC, Arnold DL, et al. Pegylated interferon beta-1a for relapsing-remitting multiple sclerosis (ADVANCE): a randomised, phase 3, double-blind study. *Lancet Neurol*. 2014;13(7):657-665. doi:10.1016/S1474-4422(14)70068-7

37. Demuth S, Ed-Driouch C, Rousseau O, et al. PRIMUS-Alpha: prototype de médecine de précision dans la sclérose en plaques contextualisant l'évolution des patients dans des données de référence multi-sources. *Rev Neurol*. 2023;179:S152. doi:10.1016/j.neurol.2023.01.670

38. Veloso M. A web-based decision support tool for prognosis simulation in multiple sclerosis. *Mult Scler Relat Disord*. 2014;3(5):575-583. doi:10.1016/j.msard.2014.04.005

39. Veloso M. An agent-based simulation model for informed shared decision making in multiple sclerosis. *Mult Scler Relat Disord*. 2013;2(4):377-384. doi:10.1016/j.msard.2013.04.001

40. Pappalardo F, Russo G, Pennisi M, et al. The potential of computational modeling to predict disease course and treatment response in patients with relapsing multiple sclerosis. *Cells*. 2020;9(3):586. doi:10.3390/cells9030586

41. Sips FLP, Pappalardo F, Russo G, Bursi R. In silico clinical trials for relapsing-remitting multiple sclerosis with MS TreatSim. *BMC Med Inform Decis Mak*. 2022;22(Suppl 6):294. doi:10.1186/s12911-022-02034-x

42. Polman CH, O'Connor PW, Havrdova E, et al. A randomized, placebo-controlled trial of Natalizumab for relapsing multiple sclerosis. *N Engl J Med*. 2006;354(9):899-910. doi:10.1056/NEJMoa044397

43. Bergmann A, Stangel M, Weih M, et al. Development of registry data to create interactive doctor-patient platforms for personalized patient care, taking the example of the DESTINY system. *Front Digit Health*. 2021;3:633427. doi:10.3389/fdghs.2021.633427

44. Stühler E, Braune S, Lionetto F, et al. Framework for personalized prediction of treatment response in relapsing remitting multiple sclerosis. *BMC Med Res Methodol*. 2020;20(1):24. doi:10.1186/s12874-020-0906-6

45. Braune S, Stuehler E, Heer Y, van Hoevell P, Bergmann A, NeuroTransData Study Group. PHREND®-a real-world data-driven tool supporting clinical decisions to optimize treatment in relapsing-remitting multiple sclerosis. *Front Digit Health*. 2022;4:856829. doi:10.3389/fdghs.2022.856829

46. Ziemssen T, Vandercappellen J, Jordan Mondragon V, Giovannoni G. MSProDiscuss™ clinical decision support tool for identifying multiple sclerosis progression. *J Clin Med*. 2022;11(15):4401. doi:10.3390/jcm11154401

47. Tolley C, Piani-Meier D, Bentley S, et al. A novel, integrative approach for evaluating progression in multiple sclerosis: development of a scoring algorithm. *JMIR Med Inform*. 2020;8(4):e17592. doi:10.2196/17592

48. Ziemssen T, Giovannoni G, Alvarez E, et al. Multiple sclerosis progression discussion tool usability and usefulness in clinical practice: cross-sectional, web-based survey. *J Med Internet Res*. 2021;23(10):e29558. doi:10.2196/29558

49. Skoog B, Tedeholm H, Runmarker B, Odén A, Andersen O. Continuous prediction of secondary progression in the individual course of multiple sclerosis. *Mult Scler Relat Disord*. 2014;3(5):584-592. doi:10.1016/j.msard.2014.04.004

50. Chalkou K, Steyerberg E, Bossuyt P, et al. Development, validation and clinical usefulness of a prognostic model for relapse in relapsing-remitting multiple sclerosis. *Diagn Progn Res*. 2021;5(1):17. doi:10.1186/s41512-021-00106-6

51. Benkert P, Meier S, Schaedelin S, et al. Serum neurofilament light chain for individual prognostication of disease activity in people with multiple sclerosis: a retrospective modelling and validation study. *Lancet Neurol*. 2022;21(3):246-257. doi:10.1016/S1474-4422(22)00009-6

52. Reeve K, On BI, Havla J, et al. Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis. *Cochrane Database Syst Rev*. 2023;9(9):CD013606. doi:10.1002/14651858.CD013606.pub2

53. Ludvigsen K, Nagaraja S, Daly A. When is software a medical device? Understanding and determining the "intention" and requirements for software as a medical device in European Union Law. *Eur. J. Risk Regul*. 2022;13(1):78-93. doi:10.1017/err.2021.45

54. European Commission. Guidelines on the qualification and classification of stand alone software used in healthcare within the regulatory framework of medical devices. Published July 1, 2016. https://health.ec.europa.eu/system/files/2020-09/md_meddev-guidance-216_en_0.pdf

55. Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. *Nat Med*. 2020;26(9):1351-1363. doi:10.1038/s41591-020-1037-7

56. Dillenseger A, Weidemann ML, Trentzsch K, et al. Digital biomarkers in multiple sclerosis. *Brain Sci*. 2021;11(11):1519. doi:10.3390/brainsci11111519

57. Sormani MP, Gasperini C, Romeo M, et al. Assessing response to interferon-β in a multicenter dataset of patients with MS. *Neurology*. 2016;87(2):134-140. doi:10.1212/WNL.00000000000002830

58. Ziemssen T, Kern R, Voigt I, Haase R. Data collection in multiple sclerosis: the MSDS approach. *Front Neurol*. 2020;11:445. doi:10.3389/fneur.2020.00445

## SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.

**How to cite this article:** Demuth S, Ed-Driouch C, Dumas C, et al. Scoping review of clinical decision support systems for multiple sclerosis management: Leveraging information technology and massive health data. *Eur J Neurol*. 2024;00:e16363. doi:10.1111/ene.16363