between 70 and 130. Scores between 70 and 85 and those between 115 and 130 indicate borderline intelligence and medium-to-higher intelligence, respectively. Finally, only a little more than 2% of people are located in the two tails, respectively. An IQ below 70 is therefore considered pathological and indicative of intellectual disability. An IQ above 130 is indicative of superior intelligence.

Another reason a new test is administered to a normative sample is to evaluate its psychometric properties to understand whether it is suitable for clinical or research use [23]. The two main properties worth mentioning are reliability and validity [24].

Reliability indicates the consistency of a measure or in other words the proportion of variance in the observed scores attributable to the actual variance of the measured function, and not to measurement errors [25]. Reliability may be assessed in various ways. Internal consistency, for example, indicates whether the items of a test all measure the same cognitive function. A common procedure to evaluate it is to randomly divide the test into two halves and calculate the correlation between them. Test-retest reliability indicates the ability of a test to provide the same score consistently over time. No undesirable event, such as a pathological event, should have occurred between the two assessments and cause the patient to score worse (or better) on the second one. Another bias that could undermine test-retest reliability is practice effect, which refers to a gain in scores that occurs when the respondent is retested with the same cognitive test. This gain does not reflect a real improvement in the function assessed [26]. Parallel forms of the same test are often used to avoid these problems. Another measure of reliability is the consistency between different examiners (inter-rater reliability). In fact, despite the standardization described above, some degree of variance may remain between examiners [27].

Validity is the capacity of a test to measure what it actually proposes to measure and not similar constructs [28]. The validity of a test can be assessed by calculating the correlation between the score of interest with another measure that is theoretically supposed to be correlated. The following are some types of validity commonly assessed when developing or validating a new neuropsychological test: content validity (i.e., the test only measures what it is supposed to measure), substantive validity (i.e., the test is developed on the basis of theoretical knowledge and empirical evidence), convergent validity (i.e., individuals belonging to a certain homogeneous group have a similar score on the same test), and divergent validity (i.e., individuals belonging to two different groups have different scores on the same test, e.g., patients versus controls).
## 3.3 Realization of a Neuropsychological Assessment and Interpretation of Its Results

During an assessment, the neuropsychologist chooses the most appropriate tests for the patient, ensures that they are performed correctly, and interprets their results. Indeed, each neuropsychological assessment is tailored to the patient's needs. To assess a certain cognitive function, the clinician can choose a specific test depending on the patient's level of education, the presence of any sensory deficits (e.g., tests involving verbal material will be proposed to a visually impaired patient), as well as the diagnostic hypothesis.

Once anamnestic data has been collected and the cognitive scores have been obtained, the goal is to interpret these results and define the patient's cognitive profile. Defining a cognitive profile means identifying which cognitive functions are preserved and which are impaired. In the event that one or more impaired cognitive deficits are detected, it is necessary to specify at what level the deficit is located and its severity. For example, a patient may have a memory disorder whose severity can be identified by comparing their score to normative data as described above. Depending on the test used, the neuropsychologist will be able to define whether this memory disorder is due to difficulties in creating new memory traces (linked to the medial temporal lobe [14]), or to difficulties in retrieving existing traces (linked to the prefrontal lobe [16]), and so on. By describing the impaired and preserved cognitive mechanisms and by referring to what we know about brain correlates of cognitive function, the neuropsychologist will be able to detect a pattern. This may be a cortical syndrome, such as in the event of alteration of language or visuospatial functions [29]; a subcortico-frontal profile, involving, for example, impaired executive functions [30]; a subcortical profile, often involving slow information processing [31]; etc.

It is important to clarify that the aim of the neuropsychological assessment is not to diagnose a disease, but to describe a cognitive profile. This is only one of the elements taken into account by a physician, often a neurologist, to make the diagnosis. The physician will determine which disease or pathological condition underlies the cognitive impairment, by combining the evidence from other tests, such as laboratory tests, imaging, and neurological examination, as described above.

## 3.4 The Example of a Cognitive Test: The Mini-Mental State Examination (MMSE)

The Mini-Mental State Examination, also known as MMSE, is one of the most widely used tools in both clinical practice and research, validated in many languages and adapted to administration in many countries. It is a screening tool for adults, which allows assessing global cognition quickly and easily through a paper–pencil test lasting 5–10 min.
Box 4 MMSE Questions and Scoring System

**Temporal orientation** [5 points, 1 per item]

The respondent is asked to say the day of the week, the day of the month, the month, the year, and the season

**Spatial orientation** [5 points, 1 per item]

The respondent is asked to say the floor and the name of the hospital or practice, district, town, and country.

**Short-term memory** [3 points, 1 per word]

The examiner names three objects (apple, table, and penny in the English version), and the respondent repeats them immediately

**Attention** [5 points, 1 per subtraction]

The respondent subtracts 7 from 100 five times

**Verbal learning** [3 points, 1 per word]

The respondent recalls the three previously learned words

**Denomination** [2 points, 1 per object]

The respondent names two objects indicated by the examiner, often a pen and a watch

**Repetition** [1 point]

The respondent repeats the sentence "No ifs, ands, or buts"

**Listening comprehension** [3 points, 1 per task]

The respondent is asked to take a sheet with their right hand, fold it in half, and throw it on the ground

**Written comprehension** [1 point]

The respondent executes a written command, often "Close your eyes"

**Writing** [1 point]

The respondent writes a sentence that contains a verb and a subject

**Praxico-constructive and visuospatial skills** [1 point]

Copy of two intersecting pentagons showed by the examiner

The MMSE includes 30 questions, each with a binary score (0 for wrong answer and 1 for correct answer). More details are presented in Box 4. The total score ranges from 0 to 30. An MMSE score of 18 or less indicates severe impairment of cognitive functions. A score between 18 and 24 indicates moderate to mild impairment. A score of 25 is considered borderline. And a score of 26–30 indicates cognitive normality. Different diagnostic thresholds have been proposed as they depend—mainly—on age, education, and setting [32]. In clinical settings, a score below 24 is commonly considered pathological [33]. In research contexts, it is more common to use a cut-off of 26 (pathological if <26) [34]. The MMSE is therefore very useful for getting an idea of the patient's cognitive functioning, also facilitating effective communication between professionals.
Concerning psychometric properties, internal consistency is reported to vary significantly according to the setting. Alpha coefficient was around 0.30 in the general population [35] and 0.96 in a clinical setting [36]. Lower coefficients may be related to lower variability in community-based samples where the majority of participants are healthy and often highly educated. Regarding test-retest reliability, healthy individuals scored better at retest (about one point higher) when they repeated the MMSE about 3 months after the first assessment. Patients with cognitive impairment, on the contrary, did not show such learning. In [10], the MMSE also had good validity in discriminating patients with Alzheimer's dementia, depression, and schizophrenia.

# 4 Clinical Examination by Pathology

Neurology is a broad branch of medicine that deals with all pathologies affecting the central and peripheral nervous system, also including blood vessels and muscles, such as neurodegenerative diseases, epilepsy, sleep disorders, vascular diseases, headaches, movement disorders, neuro-oncology, etc. Clinical evaluation is therefore tailored to the complaint and symptoms. The purpose is to propose a treatment or follow the evolution of the disease. There is therefore a need for sensitive clinical tests that allow for early detection of abnormalities, so that treatment can be administered more promptly.

## 4.1 Diversity of Brain Disorders and Clinical Evaluation

As science advances, medicine is getting increasingly specialized. Although “general neurologists” are the majority in the domain, the field is segmented in different subspecialties in university hospitals, each with their topic and diseases of interest, and dedicated tools for innovative studies. We briefly describe these subspecialties below (see Box 5).

### Box 5 Non-exhaustive List of the Main Neurological Diseases

<table><tr><td>Neurodegenerative disorders affecting mostly cognition or behavior</td><td>Alzheimer's disease<br>Frontotemporal dementia<br>Lewy body dementia<br>Primary progressive aphasia</td></tr><tr><td>Movement disorders</td><td>Parkinson's disease<br>Essential tremor<br>Dystonia</td></tr></table>

(continued)
<table><tr><td>Epilepsy</td><td>Generalized idiopathic epilepsy<br>Absence<br>Partial idiopathic epilepsy<br>Secondary epilepsy (post-traumatic, post-stroke, etc.)</td></tr><tr><td>Stroke or neurovascular diseases</td><td>Ischemic stroke<br>Brain hemorrhage<br>Cerebral venous thrombosis</td></tr><tr><td>Neuro-oncology</td><td>Meningioma<br>Oligodendroglioma<br>Astrocytoma<br>Glioblastoma<br>Brain metastasis</td></tr><tr><td>Peripheral nerve diseases</td><td>Mononeuropathy<br>Polyneuropathy<br>Radiculopathy<br>Plexopathy</td></tr><tr><td>Headaches</td><td>Migraine<br>Tension-type headache</td></tr><tr><td>Sleep disorders</td><td>Sleep apnea<br>Narcolepsy</td></tr><tr><td>Inflammatory and demyelinating brain diseases</td><td>Multiple sclerosis<br>Sarcoidosis</td></tr><tr><td>Neurogenetic diseases</td><td>Huntington's chorea<br>Spinocerebellar ataxia</td></tr><tr><td>Neuromuscular disorders</td><td>Amyotrophic lateral sclerosis<br>Myasthenia<br>Myopathies</td></tr></table>

### 4.1.1 Neurodegenerative Disorders Affecting Mostly Cognition or Behavior

They include Alzheimer's disease, Lewy body and frontotemporal dementias, as well rarer conditions such as primary progressive aphasias. This field relies heavily on neuropsychological evaluation. Although progress has been achieved in diagnosis of these conditions (especially Alzheimer's disease) these last decades, therapeutic unmet needs remain high.

### 4.1.2 Movement Disorders

These include Parkinson's disease but also dystonia, myoclonus, tics, and tremors. Different treatment options have emerged for this group of diseases in the last years. These include drugs based on the dopamine levels in the brain (one of the main neurotransmitters for movement) and deep brain stimulation which requires the implantation of electrodes to stimulate or inhibit specific regions of the basal ganglia.
4.1.3 Epilepsy
This broad term refers to the abnormal electric activity of neurons in brain regions or in the whole brain inducing seizures. They are defined by the co-occurrence of symptoms or signs, and these electric abnormalities are detected by electroencephalography (EEG). Many anti-epileptic drugs exist to decrease the seizure frequency in these patients. Some patients present with pharma-coresistant epilepsy. For such patients, surgery, which aims at resecting part of the brain in order to suppress seizures, can be a treatment option.

4.1.4 Stroke or Neurovascular Diseases
Acute stroke is managed in stroke emergency units. A stroke can be either a brain infarction or a hemorrhage. They are not primary diseases of the brain tissue but of the arteries, capillaries, and veins that irrigate it. Treatment options range from rapid clot removal in ischemia (whether by thrombolysis or neuroradiological intervention), anti-aggregating or anticoagulation therapy, and physical or speech rehabilitation.

4.1.5 Neuro-oncology
This specialty deals with brain tumors, which may be malignant or benign. There are close connections with neurosurgery units and neuropathology which play a valuable role in analyzing the micro-structure of the tumor in order to achieve a precise diagnosis. Treatments typically rely on a combination of surgery, radiotherapy, and chemotherapy.

4.1.6 Peripheral Nerve Diseases
They include all the diseases of the nerves outside of the brain, brainstem, or spine. These diseases induce motor, sensory, and autonomous impairments and are diagnosed through a combination of medical examination and electromyographic (EMG) recordings. Treatment options are very dependent on the cause of the disease which can range from simple mechanic compression of a nerve requiring mild surgery (carpal syndrome) to hepatic graft in some rare conditions (TTR mutation causing familial transthyretin amyloidosis).

4.1.7 Headaches
Although headaches are highly prevalent, specialists are rare in university hospital as these conditions (including migraine) are often cared for in private practice offices, except for the most urgent causes which are managed by emergency units. Treatments aim to decrease the frequency of the crisis (preventative treatments) for the most severe cases or the pain during a given crisis.

4.1.8 Sleep Disorders
Sleep disorders are sometimes managed by neurologists for some diseases (like narcolepsy) or pneumologists (since sleep apneas are among the most frequent cause of sleep impairment) or psychiatrists (tackling insomnia, often associated with psychiatric comorbidities). A sleep recording called polysomnography is sometimes
required to assess the most complex problems. Physicians can prescribe continuous positive airway pressure devices which keep the airways opened during sleep.

### 4.1.9 Inflammatory and Demyelinating Brain Diseases

The most emblematic of this group is multiple sclerosis in which the autoimmune system turns against the individual, penetrates the blood-brain barrier, and attacks the myelin which allows the rapid diffusion of the neuronal electric signal along the axons. This is one of the most advanced fields of neurology regarding treatment. Since the start of the twenty-first century, specific therapies preventing the crossing of the blood-brain barrier of lymphocytes revolutionized the management of multiple sclerosis [37].

### 4.1.10 Neurogenetic Diseases

Neurogenetic diseases are a group of rare diseases (like Huntington's chorea) due to a genetic mutation. These diseases usually follow a Mendelian mode of inheritance. They have the particularity to be detectable (through genetic testing after a specific counseling) which gives the opportunity to study them in their premorbid phase (i.e., before the onset of typical symptoms in a group of mutation carriers). Innovative gene therapies are actually being developed in some of these neurogenetic conditions [38]. Note that there also exist genetic forms of diseases which are in majority sporadic (e.g., familial forms of Alzheimer's disease).

### 4.1.11 Neuromuscular Disorders

These are diseases affecting the motor neurons such as amyotrophic lateral sclerosis, the neuromuscular synapse like myasthenia, or specifically the muscles in myopathies. To the exception of myasthenia, few treatment options exist in this particular field of neurology.

## 4.2 Importance of a Correct and Timely Diagnostic Classification

Neurologists have a saying: "time is brain." The correct and timely identification of a neurological disease is indeed crucial to be able to mitigate and sometimes reverse the signs and symptoms. As such, machine learning techniques may be very useful tools both in the context of slow-paced diseases such as Alzheimer's which are often diagnosed quite late or not at all [39] and to optimize the patient flow in emergency care, in case of stroke, for instance. This framework is theoretical as in practice some diseases can interact to induce symptoms. For instance, dementia is often of mixed origin, due to the association of degenerative (Alzheimer's disease) and vascular alterations. A walking deficit can be due to Parkinson's disease but also in part to arthrosis, etc. The correct identification of a disease is in part probabilistic, and this can lead to heterogeneity in the collected data from the clinical assessment.
# 5 Conclusion

Clinical assessment is central in neurology for the assessment of the patient because it is the direct reflection of what he/she feels and experiences. Indeed, according to regulatory agencies, a treatment is deemed effective if it has an effect on the clinical expression of the disease (e.g., on cognition, motor skills, sensitivity, autonomy, and survival) and not on intermediate markers such as imaging, biology, or others.

Machine learning is bringing clinical evaluation into a new era because it allows to go beyond the intuitions of the individual physician and could associate signs that were previously not seen as part of a disease type or subtype. However, the researcher should always remember that the best algorithm is only as good as the data it runs on, which depends on the clinician's understanding of how and why these particular data are collected and will be used for. So, for discovery, validation, and clinical implementation of new machine learning techniques, basic knowledge of the possible discrepancies and biases one may experience going from research setting to clinical practice is paramount.

# Acknowledgments

The research leading to these results has received funding from the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-10-IAIHU-06 (Agence Nationale de la Recherche-10-IA Institut Hospitalo-Universitaire-6).

# References

1. Lage JMM (2006) 100 years of Alzheimer's disease (1906–2006). J Alzheimers Dis 9(s3): 15–26

2. McKhann G, Drachman D, Folstein M et al (1984) Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease. Neurology 34(7): 939–944. https://doi.org/10.1212/wnl.34.7.939. [doi]

3. Öhman F, Hassenstab J, Berron D et al (2021) Current advances in digital cognitive assessment for preclinical Alzheimer's disease. Alzheimers Dement 13(1):e12217

4. Dillenseger A, Weidemann ML, Trentzsch K et al (2021) Digital biomarkers in multiple sclerosis. Brain Sci 11(11):1519

5. Rodríguez-Gómez O, Rodrigo A, Iradier F et al (2019) The MOPEAD project: advancing patient engagement for the detection of "hidden" undiagnosed cases of Alzheimer's disease in the community. Alzheimers Dement 15(6):828–839

6. Daly T, Mastroleo I, Gorski D et al (2020) The ethics of innovation for Alzheimer's disease: the risk of overstating evidence for metabolic enhancement protocols. Theor Med Bioeth 41(5–6):223–237

7. Penfield W, Rasmussen T (1950) The cerebral cortex of man; a clinical study of localization of function. Macmillan

8. Tsai AC, Hong SY, Yao LH et al (2021) An efficient context-aware screening system for Alzheimer's disease based on neuropsychology test. Sci Rep 11(1):18570
9. Cacciamani F, Houot M, Gagliardi G et al (2021) Awareness of cognitive decline in patients with Alzheimer's disease: a systematic review and meta-analysis. Front Aging Neurosci 13

10. Folstein MF, Folstein SE, McHugh PR (1975) "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12(3):189-198. https://doi.org/10.1016/0022-3956(75)90026-6. [pii]

11. Ramaker C, Marinus J, Stiggelbout AM et al (2022) Systematic evaluation of rating scales for impairment and disability in Parkinson's disease. Mov Disord 17(5):867-876

12. Goetz CG, Tilley BC, Shaftman SR et al (2008) Movement Disorder Society-sponsored review of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov Disord 23(15):2129-2170

13. Vallar G (2000) The methodological foundations of human neuropsychology: studies in brain-damaged patients. In: Boller F, Grafman J (eds) Handbook of neuropsychology, 2nd edn. Elsevier, Amsterdam, pp 459-502

14. Preston AR, Eichenbaum H (2013) Interplay of hippocampus and prefrontal cortex in memory. Curr Biol 23(17):R764-R773

15. Indefrey P, Levelt WJM (2000) The neural correlates of language production. In: Gazza-niga MS (ed) Anonymous the new cognitive neurosciences, 2nd edn. MIT Press, Cam-bridge, MA, pp 845-865

16. Robinson H, Calamia M, Gläscher J et al (2014) Neuroanatomical correlates of executive functions: a neuropsychological approach using the EXAMINER battery. J Int Neurop-schol Soc 20(1):52-63

17. Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15(1):20-25

18. Benton AL (1988) Neuropsychology: past, present and future. In: Boller F, Grafman J (eds) Anonymous handbook of neuropsychol-ogy. Amsterdam, pp 3-27

19. Schmidt S, Pardo Y (2014) Normative data. In: Michalos AC (ed) Encyclopedia of quality of life and well-being research. Springer, Dordrecht

20. Guilmette TJ, Sweet JJ, Hebben N et al (2020) American Academy of Clinical Neuropsychol-ogy consensus conference statement on uniform labeling of performance test scores. Clin Neuropsychol 34(3):437-453

21. Wechsler D (2008) Wechsler adult intelligence scale—fourth edition administration and scor-ing manual. Pearson, San Antonio, TX

22. Wechsler D (2003) The Wechsler intelligence scale for children—fourth edition. Pearson, London

23. Portney LG, Watkins MP (2009) Foundations of clinical research: applications to practice. Pearson Education, New Jersey

24. Anastasi A, Urbina S (1997) Psychological test-ing. Prentice Hall, Pearson Education

25. Nunnally JC, Bernstein IH (1994) Psychomet-ric theory. McGraw-Hill, New York

26. Cacciamani F, Salvadori N, Eusebi P et al (2017) Evidence of practice effect in CANTAB spatial working memory test in a cohort of patients with mild cognitive impairment. Appl Neuropsychol Adult 22:1-12. https://doi.org/10.1080/23279095.2017.1286346

27. Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quantit Methods Psychol 8(1):23-34

28. Messick S (1989) Validity. In: Linn RL (ed) Educational measurement American Council on education and Macmillan, New York, NY, pp 13-104

29. Huber SJ, Shuttleworth EC, Paulson GW et al (1986) Cortical vs subcortical dementia: neu-ropsychological differences. Arch Neurol 43(4):392-394

30. Bonelli RM, Cummings JL (2007) Frontal-subcortical circuitry and behavior. Dialogues Clin Neurosci 9(2):141-151

31. Mayeux R, Stern Y (1987) Subcortical demen-tia. Arch Neurol 44(2):129-131

32. Arevalo-Rodriguez I, Smailagic N, Roqué Fig-uls M et al (2015) Mini-Mental State Examina-tion (MMSE) for the detection of Alzheimer's disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev 3

33. Crum RM, Anthony JC, Bassett SS et al (1993) Population-based norms for the Mini-Mental State Examination by age and educational level. JAMA 269(18):2386-2391

34. Meiran N, Stuss DT, Guzman DA et al (1996) Diagnosis of dementia. Methods for interpre-tation of scores of 5 neuropsychological tests. Arch Neurol 53(10):1043-1054

35. Hopp GA, Dixon RA, Grut M et al (1997) Longitudinal and psychometric profiles of two cognitive status tests in very old adults. J Clin Psychol 53(7):673-686
36. Foreman MD (1987) Reliability and validity of mental status questionnaires in elderly hospitalized patients. Nurs Res 36(4):216-220

37. Polman CH, Uitdehaag BM (2003) New and emerging treatment options for multiple sclerosis. Lancet Neurol 2(9):563-566

38. Hinderer C, Miller R, Dyer C et al (2020) Adeno-associated virus serotype 1-based gene

therapy for FTD caused by GRN mutations.
Ann Clin Transl Neurol 7(10):1843-1853

39. Epelbaum S, Paquet C, Hugon J et al (2019) How many patients are eligible for disease-modifying treatment in Alzheimer's disease? A French national observational study over 5 years. BMJ Open 9(6)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
**Neuroimaging in Machine Learning for Brain Disorders**

Ninon Burgos

**Abstract**

Medical imaging plays an important role in the detection, diagnosis, and treatment monitoring of brain disorders. Neuroimaging includes different modalities such as magnetic resonance imaging (MRI), X-ray computed tomography (CT), positron emission tomography (PET), or single-photon emission computed tomography (SPECT).

For each of these modalities, we will explain the basic principles of the technology, describe the type of information the images can provide, list the key processing steps necessary to extract features, and provide examples of their use in machine learning studies for brain disorders.

**Key words** Magnetic resonance imaging, Computed tomography, Positron emission tomography, Single-photon emission computed tomography, Neuroimaging, Medical imaging, Machine learning, Deep learning, Feature extraction, Preprocessing

# 1 Introduction

Medical imaging plays a key role in brain disorders. In clinical care, it is vital for detection, diagnosis, and treatment monitoring. It is also an essential tool for research to characterize the anatomical, functional, and molecular alterations in brain disorders, to better understand the pathophysiology, or to evaluate the effects of new treatments in clinical trials, for instance. Medical imaging of the brain is referred to as neuroimaging and involves different modalities such as X-ray computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), or single-photon emission computed tomography (SPECT).

Most neuroimaging modalities have been developed in the 1970s (Fig. 1). The first CT image of a brain was acquired in 1971 [1, 2]. This technology results from the discovery of X-rays by Wilhelm Röntgen in 1895 [3]. A few years later, PET [4] and then SPECT [5, 6] cameras were developed. Both modalities result from the discovery of natural radioactivity in 1896 by Henri Becquerel [7]. The first MR image of a brain goes back to 1978 [8]
<figure><img src="image_123.png" /><figcaption>**Fig. 1 Timeline of the main developments in neuroimaging**</figcaption></figure>

following the discovery of nuclear magnetic resonance in 1946 by Felix Bloch [9]. Some of these imaging modalities were later combined into hybrid scanners. The first prototype combining PET and CT was introduced into the clinical arena in 1998 [10], while the first PET and MR images of a brain simultaneously acquired were reported in 2007 [11, 12]. The first commercial SPECT/CT system dates back to 1999 [13], while SPECT/MR systems are still under development [14].

CT and MRI are the modalities of choice when studying brain anatomy, while SPECT and PET are used to image particular biological processes. Note that MRI is a versatile modality that allows studying both the structure and function of the brain, through the acquisition of different sequences. The use of these imaging modalities differs between clinical practice and research contexts. For example, CT is the main modality used in hospitals on adults [15], while MRI is by far the modality the most used for the study of brain disorders with machine learning (Fig. 2, top). The two most studied disorders with machine learning are brain tumors and dementia, mainly Alzheimer's disease (Fig. 2, bottom).

This chapter will start by shortly describing the nature of neuroimages, detailing the type of features that can be extracted from them, and listing software tools that can be used to do so. We will then briefly describe the principles of the imaging modalities the most used in machine learning studies: anatomical, diffusion, and functional MRI, CT, PET, and SPECT. For each modality, we will report the processing steps often performed to extract features, explain the type of information provided, and give examples of their use in machine learning studies.
<figure><img src="image_124.png" /><figcaption>**Fig. 2** Distribution by imaging modality (top) and brain disorder (bottom) of 1327 articles presenting a study using machine learning. Note that these numbers should only be taken as rough indicators as they result from a non-exhaustive literature search. The Scopus query and the resulting articles (after some manual filtering) are available as a public Zotero library (https://www.zotero.org/groups/4623150/neuroimaging_with_ml_for Brain_disorders/library)</figcaption></figure>

## 2 Manipulating Neuroimages

In clinical routine, neuroimages are primarily exploited through visual inspection by a radiologist (or a neuroradiologist, who is a radiologist with an additional specialization in brain imaging, in expert hospitals) or a nuclear medicine physician. This results in a radiological report that is a written text describing the characteristics of the brain of the patient, its alterations, and possibly the most likely diagnosis. Note that neuroimaging exploration is usually requested by a neurologist or a psychiatrist and is associated with an indication that may correspond to the exploration of a set of symptoms (for instance, the exploration of a dementia syndrome) or to the confirmation of a potential diagnosis. Neuroimaging
alone will thus usually not provide a diagnosis. It will rather bring arguments in favor, or against, a potential diagnosis (for instance, in the exploration of a dementia syndrome, MRI can bring positive arguments for a diagnosis of Alzheimer's disease due to the observed atrophy in specific areas or on the contrary exclude this diagnosis by showing that the syndrome is due to a different cause such as a brain tumor). Overall, the diagnosis will generally be made by the neurologist or the psychiatrist based on a combination of clinical examination and a set of multimodal data (clinical and cognitive tests, radiological report, biomarkers, etc.).

However, the use of neuroimages goes way beyond visual inspection and is subject to quantification using image processing procedures. This is particularly true in research even though image processing tools are also increasingly used in clinical routine. A characteristic of these tools that differentiates them from general purpose image processing tools is their ability to handle three-dimensional (3D) images.

## 2.1 The Nature of 3D Medical Images

Most medical imaging devices acquire 3D images. This is the case of all the ones presented in this chapter (MRI, CT, PET, and SPECT). If 2D images are essentially 2D arrays of elements called pixels (for picture elements), 3D images are 3D arrays of elements called voxels (for volume elements). Depending on the imaging modality, voxel values will represent different properties of the underlying tissues. For example, in a CT image, they will be proportional to linear attenuation coefficients. The shape and size of a voxel will also depend on the imaging modality (or the type of sequence in MRI). When its three dimensions are of equal lengths, the voxel is isotropic; otherwise, it is anisotropic (see Fig. 3). For instance, a typical voxel size for a T1-weighted MR image is about 1 × 1 × 1 mm<fnref n="3" />, while it is about 3 × 3 × 3 mm<fnref n="3" /> for a functional MR image. Most neuroimaging modalities will have a voxel dimension between 0.5 mm and 5 mm.

Even though most neuroimages are 3D, they are visualized as 2D slices along different planes: axial, coronal, or sagittal (see Fig. 4). Multiple tools exist to visualize neuroimages. Several are available within suites such as FSLeyes,<fnref n="1" /> Freeview,<fnref n="2" /> or medInria,<fnref n="3" /> while others are independent such as Vinci,<fnref n="4" /> Mango,<fnref n="5" /> or Horos.<fnref n="6" /> Note that viewers may interpolate the images they display, which may be misleading (see Fig. 5 for an illustration).

<footnote n="1">¹ FSLeyes: https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FSLeyes.</footnote>

<footnote n="2">² Freeview: https://surfer.nmr.mgh.harvard.edu/fswiki/FreeviewGuide.</footnote>

<footnote n="3">³ medInria: https://med.inria.fr.</footnote>

<footnote n="4">⁴ Vinci: https://vinci.sf.mpg.de.</footnote>

<footnote n="5">⁵ Mango: http://ric.uthscsa.edu/mango.</footnote>

<footnote n="6">⁶ Horos: https://horosproject.org.</footnote>
<figure><img src="image_125.png" /><figcaption>**Fig. 3** Most neuroimaging modalities are three-dimensional. Left: volume rendering of an excavated T1-weighted MR image. Middle: voxel grid with isotropic, i.e., cubic, voxels overlaid on the MRI. Right: voxel grid with anisotropic, i.e., rectangular, voxels overlaid on the MRI</figcaption></figure>

<figure><img src="image_126.png" /><figcaption>**Fig. 4** Axial, coronal, and sagittal slices extracted from a T1-weighted MR image</figcaption></figure>

## 2.2 Extracting Features from Neuroimages

When using machine learning to analyze images, one will often extract features. These features can be grouped into four categories that we will now describe and are illustrated in Fig. 6. Note that these features are conceptually the same for the different modalities but their actual content will differ (e.g., volume of a region for anatomical MRI vs average uptake in this region for PET). Modality-specific preprocessing and corrections often need to be applied before neuroimages can be analyzed; these will be described in Subheadings 3, 4, and 5.
<figure><img src="image_127.png" /><figcaption>**Fig. 5** Axial slice of a T1-weighted MRI with an isotropic voxel size originally of 1 × 1 × 1 mm³ (left) and downsampled to 2 × 2 × 2 mm³ (right) displayed without interpolation or with linear interpolation. If the difference with or without interpolation is subtle at 1 × 1 × 1 mm³, it is well visible at 2 × 2 × 2 mm³</figcaption></figure>

**Voxel-Based Features** As mentioned previously, all the imaging modalities described in this chapter produce volumetric images. The whole 3D image can be used as input of a machine learning algorithm. In that case, each subject is seen as a collection of values at each voxel of the image. These values can simply be the intensity of the image at each voxel after some minimal preprocessing (which is very often what is used in deep learning) or some more complex value extracted from the image (for instance, gray-level density from anatomical MRI; see Subheading 3.1). A prerequisite is often to align the images studied in a common space, by registering each image to a template and/or by performing a group-wise registration, thus guaranteeing a voxel-wise correspondence across subjects [16]. Note that this correspondence becomes particularly important when using a machine learning algorithm that takes as input a vector in which each element implicitly represents the same information for each subject (e.g., logistic regression or support vector machine).

**Vertex-Based Features** Studying the surface of the cortex is natural given its shape: it is a convoluted ribbon delimited by inner and outer surfaces. Moreover, surface-based characteristics can provide useful information such as for developmental or neurodegenerative diseases. Surfaces can be represented as meshes consisting of vertices, edges, and faces. The vertices encode position and properties such as cortical thickness. In the vertex-based feature scenario, each subject is seen as a collection of values at each vertex of the surface. Classical values computed at each vertex include cortical thickness and local surface area (see Subheading 3.1). As for voxel-based features, images studied are usually aligned in a common space to ensure a vertex-wise correspondence across subjects [17, 18].
### Voxel-based features

<figure><img src="image_128.png" /></figure>

### Vertex-based features

<figure><img src="image_129.png" /></figure>

### Regional features

<figure><img src="image_130.png" /></figure>

### Graph-based features

<figure><img src="image_131.png" /><figcaption>**Fig. 6** Examples of voxel, vertex, regional, and graph features that can be extracted from neuroimages. It is, for instance, possible to extract voxel-based features from CT and SPECT images, vertex-based features from anatomical T1-weighted (T1w) MRI or PET images, regional features from diffusion MRI, and graph-based features from functional MRI. Note that the modalities are just examples. For instance, voxel-based features can be extracted for any modality. See Subheadings 3, 4, and 5 for a description of the imaging modalities</figcaption></figure>
**Regional Features** The brain can be divided into subregions according to different criteria that can be anatomical or functional [16]. When considering regional features, each subject is seen as a collection of values for each region of the brain defined by an atlas. Many atlases exist, either anatomical or functional, with different degrees of granularity. A list can be found online.<fnref n="7" /> Classical values include the volume of a given region or the average image signal within a region.

**Graph-Based Features** A last way to represent an image is through a graph where nodes will correspond to brain regions and edges will encode a particular property (for instance, anatomical or functional connections, possibly together with their strength). Graphs can directly be used as features, but network indices characterizing global and local graph topology, e.g., efficiency or degree, can also be computed [19].

## 2.3 Neuroimaging Software Tools

The features described above can be obtained using neuroimaging software tools. However, an important step before any preprocessing or analysis is to properly organize data. The neuroimaging community proposed the Brain Imaging Data Structure [20], which specifies how to organize data in folders and sub-folders on disk and how to name the files. It also details the metadata necessary to describe neuroimaging experiments.

Many tools exist to process neuroimages.<fnref n="8" /> The historical generic frameworks include SPM<fnref n="9" /> [21], FSL<fnref n="10" /> [22], FreeSurfer<fnref n="11" /> [23], or ANTs<fnref n="12" /> [24]. Some tools are modality-specific such as MRtrix<fnref n="13" /> [25], dedicated to diffusion MRI, or AFNI<fnref n="14" /> [26], dedicated to functional MRI. Recent initiatives aim to make the use of neuroimaging tools easier by distributing them in containers (e.g., BIDSApps<fnref n="15" /> [27]), by providing in a single environment tools from preprocessing to machine learning (e.g., Nilearn<fnref n="16" /> [28]), or by providing automatic pipelines that do not require a particular

<footnote n="7">⁷ List of atlases: https://www.lead-dbs.org/helpsupport/knowledge-base/atlasesresources.</footnote>

<footnote n="8">⁸ List of open source medical imaging software tools: https://idoimaging.com.</footnote>

<footnote n="9">⁹ SPM: https://www.fil.ion.ucl.ac.uk/spm.</footnote>

<footnote n="10">¹⁰ FSL: https://fsl.fmrib.ox.ac.uk.</footnote>

<footnote n="11">¹¹ FreeSurfer: https://surfer.nmr.mgh.harvard.edu.</footnote>

<footnote n="12">¹² ANTs: http://stnava.github.io/ANTS.</footnote>

<footnote n="13">¹³ MRtrix: https://www.mrtrix.org.</footnote>

<footnote n="14">¹⁴ AFNI: https://afni.nimh.nih.gov.</footnote>

<footnote n="15">¹⁵ BIDSApps: https://bids-apps.neuroimaging.io/apps.</footnote>

<footnote n="16">¹⁶ Nilearn: https://nilearn.github.io.</footnote>
expertise in image processing (e.g., Clinica<fnref n="17" /> [29]). Other tools facilitate the application of deep learning approaches to neuroimages or medical images in general: for instance, MONAI,<fnref n="18" /> TorchIO<fnref n="19" /> [30], or ClinicaDL<fnref n="20" /> [31].

# 3 Magnetic Resonance Imaging

Magnetic resonance imaging is the modality of choice to study brain anatomy, thanks to its high-resolution and excellent soft-tissue contrast, but the applications of MRI go well beyond studying anatomy. This technique can be used to examine tissue micro-architecture (diffusion MRI, covered in Subheading 3.2) or neuronal activity (functional MRI, covered in Subheading 3.3) but also to visualize the brain vasculature (MR angiography), study tissue perfusion and permeability (perfusion MRI), assess iron deposits and calcifications (susceptibility-based imaging), or measure the levels of different metabolites (MR spectroscopy). Note that MRI is an extremely versatile modality and that new sequences are constantly developed to study other brain characteristics.

## 3.1 Anatomical MRI

### 3.1.1 Basic Principles

In MRI, most images are obtained by exploiting a magnetic property, called spin, of the hydrogen atomic nuclei found in the water molecules present in the human body. In the absence of a strong external magnetic field, the directions of the proton's spins are random, thus cancelling each other out (Fig. 7a). When the spins enter a strong external magnetic field (B0), they align either parallel or antiparallel, and they all precess around the B0 axis, referred to as the z axis (Fig. 7b). As a result, they cancel each other out in the transverse (x, y) plane, but they add up along the z axis. The result of this vector addition, called net magnetization M0, is proportional to the proton density (Fig. 7c). With the application of a radio frequency pulse denoted as B1, the system of spins and the net magnetization are tipped by an angle determined by the strength and duration of the radio frequency pulse. For a 90° radio frequency pulse, the magnetization along the z axis (Mz) becomes zero and the magnetization in the transverse plane (Mxy) becomes equal to M0 (Fig. 7d). As this radio frequency pulse provides energy, or excites, the spins, we also talk of radio frequency excitation.

<footnote n="17">¹⁷ Clinica: https://www.clinica.run.</footnote>

<footnote n="18">¹⁸ MONAI: https://monai.io.</footnote>

<footnote n="19">¹⁹ TorchIO: https://torchio.readthedocs.io.</footnote>

<footnote n="20">²⁰ ClinicaDL: https://clinicadl.readthedocs.io.</footnote>
<figure><img src="image_132.png" /><figcaption>**Fig. 7** MRI physics in a nutshell. (a) In the absence of a magnetic field, the directions of the proton's spins are random. (b) When the spins enter a strong external magnetic field (B0), they align either parallel or antiparallel, and they all precess around the B0 axis. (c) The net magnetization M0 is proportional to the proton density. (d) With the application of a radio frequency pulse, the system of spins is tipped</figcaption></figure>

When the radio frequency pulse is then turned off, two phenomena occur. First, the system of spins relaxes back to its preferred energy state of being parallel with B0 in a time T1, called longitudinal or spin-lattice relaxation time, and the longitudinal magnetization Mz slowly recovers to its original magnitude M0. Second, each spin starts precessing at a frequency that is slightly different from the one of its neighboring spins because the field of the MRI scanner is not uniform and because each spin is influenced by the small magnetic fields of the neighboring spins. When the spins are completely dephased, they are evenly spread in the transverse plane, and Mxy becomes zero. Mxy decreases at a much faster rate than that at which Mz recovers to M0. The transverse relaxation time T2, also called spin-spin relaxation time, describes the Mxy decrease because of interference from neighboring spins, while T2* describes the decrease because of both spin-spin interactions and nonuniformities of B0. Finally, the MRI signal is obtained by measuring the transverse magnetization as an electrical current by induction.

The contrast in MR images depends on three main parameters: the proton density, the longitudinal relaxation time T1, and the transverse relaxation time T2. These parameters can be adjusted by changing the time at which the signal is recorded, called echo time, and the interval between successive excitation pulses, called repetition time. A T1-weighted image is created by choosing a short repetition time, a T2-weighted image by choosing a long echo time, and a proton density (PD)-weighted image by minimizing both T1 and T2 weighting of the image (long repetition time and short echo time). The corresponding images are referred to as T1-weighted MRI, T2-weighted MRI, and PD-weighted MRI. Note that many variations of these sequences exist (for instance, gradient-echo vs spin-echo) and the corresponding
<figure><img src="image_133.png" /><figcaption>**Fig. 8** Example of anatomical MR images. T1-weighted, T2-weighted, and T2-FLAIR images of a patient with multiple sclerosis from the MSSEG MICCAI 2016 challenge data set [32, 33]</figcaption></figure>

implementation by different manufacturers usually comes with a
specific commercial name (e.g., MPRAGE is a T1-weighted
sequence available on Siemens scanners). Furthermore, many
more anatomical sequences exist including T2*-weighted,
T2-FLAIR (fluid-attenuated inversion recovery), or DIR (double
inversion recovery). Examples are displayed in Fig. 8. The set of
sequences chosen by the radiologist will depend on the potential
disease that is being investigated. Some examples in the context of
machine learning are given in Subheading 3.1.3.

3.1.2 Extracting Features
from Anatomical MRI

Several preprocessing steps are often necessary before analyzing
anatomical MR images to correct imperfections and ease their
comparison.

**Bias Field Correction** MR images can be corrupted by a low-frequency and smooth signal caused by magnetic field inhomogeneity. This bias field induces variations in the intensity of the same tissue in different locations of the image, which deteriorates the performance of image analysis algorithms such as registration or segmentation. Several methods exist to correct these intensity inhomogeneities, the most popular being the N4 algorithm [34] available in ANTs [24].

**Intensity Rescaling and Standardization** As MRI is usually not a quantitative imaging modality, MR images can have different intensity ranges, and the intensity distribution of the same tissue type may be different between two images, which might affect the subsequent image preprocessing steps. The first point can be dealt with by globally rescaling the image, for example, between 0 and
1, using the minimum and maximum intensity values. More robust choices exist such as the z-score normalization (at each voxel, one subtracts the mean intensity of the image, and the result is divided by the standard deviation across the image), which can be made even more robust by only considering a percentile of the intensities for computing the mean and standard deviation. Intensity standardization, to solve the second point, can be achieved using techniques such as histogram matching [35].

**Skull Stripping** Extracranial tissues can be an obstacle for image analysis algorithms [36]. A large number of methods have been developed for brain extraction, also called skull stripping. Some are available in neuroimaging software platforms, such as FSL [22] or BrainSuite [37], and others as independent tools<sup>21,22</sup> [38, 39]. Note that these methods can be sensitive to the presence of noise and artefacts, which can result in over or under segmentation of the brain.

**Image Registration** Medical image registration consists in spatially aligning two or more images, either globally (rigid and affine registration) or locally (nonrigid registration), so that voxels in corresponding positions contain comparable information. A large number of software tools have been developed for MRI-based registration [40]. They are available in all the major platforms (e.g., SPM [21], FSL [22], FreeSurfer [23], or ANTs [24]).

**Image Segmentation** Medical image segmentation consists in partitioning an image into a set of nonoverlapping regions. When processing brain images, these regions can correspond to tissue types, e.g., gray matter, white matter, and cerebrospinal fluid [41], but also to anatomical (e.g., hippocampus, pons) or functional (e.g., language network, sensorimotor network) regions defined by an atlas [42]. As for registration, many tools have been developed for MRI-based segmentation and are available, among others, in SPM [21], FSL [22], FreeSurfer [23], or ANTs [24].

**Resulting Features** Based on the combination of one, several, or all, of the previously mentioned preprocessing steps, various types of features can be extracted that correspond to those described in Subheading 2.2. For deep learning algorithms, which usually exploit voxel-based features, it is quite common to perform only very basic preprocessing. At the simplest, it can be intensity normalization (this step is mandatory for deep learning methods to work correctly). It is often combined with a bias field correction

<footnote n="21"><sup>21</sup> HD-BET: https://github.com/MIC-DKFZ/HD-BET.</footnote>

<footnote n="22"><sup>22</sup> SynthStrip: https://surfer.nmr.mgh.harvard.edu/docs/synthstrip.</footnote>
and a linear registration to a common space. Another common type of voxel-based features is that of tissue density maps (e.g., gray matter or white matter density) [43]. Their extraction involves bias field correction, registration to a common space, and tissue segmentation. Common vertex-based features are the local thickness and the local surface area [44]. Regional features are usually the volume of different regions of the brain, but they can also be the average intensity within the region or the average of another image-derived value. They can as well be related to lesions (for instance, the volume of multiple sclerosis lesions or of different compartments of a brain tumor) rather than anatomical regions. Finally, graph-based features can also be computed from anatomical MRI [45] even though this representation is more common for diffusion MRI and functional MRI.

### 3.1.3 Examples of Applications in Machine Learning Studies

T1-weighted MRI is the sequence the most commonly found in machine learning studies applied to brain disorders. Several features can be extracted from T1-weighted MRI such as the volume of the whole brain or of regions of interest; the density of a particular tissue, e.g., gray matter; or the local cortical thickness and surface area. All these features, as well as the raw T1-weighted MR images, have, for example, largely been used for the computer-aided diagnosis of dementia, in particular Alzheimer's disease, as they highlight atrophy, i.e., the neuronal loss that is a marker of neurodegenerative diseases [46–49].

T1-weighted MR images acquired with and without the injection of a contrast agent are often used in the context of brain tumor detection and segmentation, progression assessment, and survival prediction as they allow distinguishing active tumor structures [50]. Such tasks also typically rely on another sequence called T2-weighted fluid-attenuated inversion recovery (T2-FLAIR) that allows visualizing a wide range of lesions on top of tumors [51], such as those appearing with multiple sclerosis [52, 53] or age-related white matter hyperintensities (also called leukoaraiosis, which is linked to small vessel disease).

## 3.2 Diffusion-Weighted MRI

### 3.2.1 Basic Principles

Diffusion MRI [54, 55] allows visualizing tissue microarchitecture, thanks to the diffusion of water molecules. Depending on their surroundings, water molecules are able to either move freely, e.g., in the extracellular space, or move following surrounding constraints, e.g., within a neuron. In the former situation, the diffusion is isotropic, while in the later it is anisotropic. Contrast in a diffusion MR image originates from the fact that following the application of an excitation pulse, water molecules that move in a particular direction, and so the protons they contain, do not have the same magnetic properties as the ones that move randomly but not far from their origin point. The excitation pulse is parametrized by a weighting coefficient *b*: the higher the *b*-value, the more
<figure><img src="image_134.png" /><figcaption>**Fig. 9** Example of diffusion-weighted MR images. Top: diffusion volumes acquired using different *b*-values (0 and 1000 s/mm²) and gradient directions. Bottom: parametric maps resulting from diffusion tensor modeling (fractional anisotropy, FA; axial diffusivity, AD; radial diffusivity, RD; and mean diffusivity, MD)</figcaption></figure>

sensitive the acquisition is to water diffusion, but the lower the signal-to-noise ratio. Several diffusion MRI volumes, each volume corresponding to a particular *b*-value and gradient direction, are usually acquired. See examples in Fig. 9 (top row).

### 3.2.2 Extracting Features from Diffusion MRI

Diffusion MR images are typically acquired with echo-planar imaging, a technique that spatially encodes the MRI signal in a way that enables fast acquisitions with a relatively high signal-to-noise ratio. However, echo-planar imaging induces geometric distortions and signal losses known as magnetic susceptibility artifacts. Other artifacts include eddy currents (due to the rapid switching of diffusion gradients), intensity inhomogeneities (as for anatomical MRI), and potential movements of the subject during the acquisition. These artifacts need to be corrected before further analyzing the images. Various methods exist to do so; they are reviewed in [56]. Two widely used tools enabling the preprocessing of diffusion MR images are FSL [22] and MRtrix [25], but others exist<fnref n="23" /> [56].

<footnote n="23">²³ List of tools and software packages to process diffusion MRI: https://github.com/dmripreprocessing/neuroimage-review-2022.</footnote>
Once artifacts have been corrected, diffusion MR images can be analyzed in different ways. One of the earliest strategy for modeling water diffusion is the diffusion tensor imaging (DTI) model [57]. Such model can output parametric maps describing several diffusion properties: fractional anisotropy (FA, directional preference of diffusion), mean diffusivity (MD, overall diffusion rate, also called apparent diffusion coefficient), axial diffusivity (AD, diffusion rate along the main axis of diffusion), and radial diffusivity (RD, diffusion rate in the transverse direction). Examples of parametric maps are displayed in Fig. 9 (bottom row). DTI tractography [58] goes one step further by reconstructing white matter tracts. Other diffusion models have been developed to better characterize tissue micro-architecture. This is, for example, the case of neurite orientation dispersion and density imaging (NODDI) [59], which enables the study of neurite morphology by disentangling neurite density and orientation dispersion that both independently influence fractional anisotropy.

One can then again compute most of the different types of features covered in Subheading 2.2. Voxel-based features will represent the value of a given parametric map (e.g., FA, MD). Surface-based features are seldom used because diffusion MRI often focuses on the white matter even though it is in principle possible to project maps that are of interest in the gray matter onto the cortical surface. Regional features represent the average of a given map (e.g., FA, MD) in a set of anatomical regions. Graph-based features can be computed as follows, vertices are often regions of the cortex, and edges correspond to the connection strength, which can be derived, for instance, from the number of tracts connecting two regions or the average FA within those tracts.

### 3.2.3 Examples of Applications in Machine Learning Studies

Machine learning studies have mainly used diffusion MRI to assess white matter integrity. This has been done in a very wide variety of disorders. For example, fractional anisotropy and mean diffusivity have been used to differentiate cognitively normal subjects from patients with mild cognitive impairment or Alzheimer's disease [60, 61]. Diffusion MRI has also been exploited to perform tumor grading or subtyping [62] following the assumption that the cellular structure may differ between cancerous and healthy tissues.

## 3.3 Functional MRI

### 3.3.1 Basic Principles

When a region of the brain gets activated by a cognitive task, two phenomena occur: a local increase in cerebral blood flow and changes in oxygenation concentration [63]. Functional MRI (fMRI) is used to measure the latter phenomenon. The blood-oxygen-level-dependent (BOLD) contrast originates from the fact that hemoglobin molecules that carry oxygen have different magnetic properties than hemoglobin molecules that do not carry oxygen.
Task fMRI consists in inducing particular neural states, for example, by performing tasks involving the visual or auditory systems and then comparing the signals recorded during the different states. As the differences observed are small, it is important to preserve at best the signal-to-noise ratio that could be degraded because of head motion or polluted by fluctuations of the cardiac and respiratory cycles. This is done by quickly acquiring multiple image volumes with echo-planar imaging. The BOLD signal also varies when the brain is not performing any particular task [64]. These spontaneous fluctuations are studied with resting-state fMRI.

### 3.3.2 Extracting Features from Functional MRI

The preprocessing of functional MR images has two main objectives: limit the effect of nonneural sources of variability and correct acquisition-related artifacts [65]. Preprocessing steps can, for example, include susceptibility distortion correction (as for diffusion MRI); head motion correction, by registering each volume in the time series to a reference volume (e.g., the first volume); slicing correction, to eliminate differences between the time of acquisition of each slice in the volume; or physiologic noise correction, by temporal filtering [63, 65]. These preprocessing steps can be performed using tools such as SPM [21], FSL [22], or AFNI [26], but also using the dedicated fMRIPrep workflow [65].

The majority of machine learning studies in brain disorders focuses on resting-state rather than task fMRI [66]. This can be explained by the fact that the resting-state protocol is simpler and allows multi-site studies (as it is less sensitive to changes in local experimental settings) [66], which should result in larger samples. Depending on the application, preprocessed resting-state fMRI data may be further processed to extract features. One can directly use voxel-based features (or vertex-based features by projecting the functional MRI signal onto the cortical surface) [67]. Nevertheless, to the best of our knowledge, the most common features are graph-based. Indeed, most supervised algorithms for classification or regression use brain networks extracted from resting-state time series. In these networks, also called connectomes, the vertices correspond to brain regions, which size may vary, and the edges encode the functional connectivity strength, which corresponds to the correlation between time series.

### 3.3.3 Examples of Applications in Machine Learning Studies

Machine learning methods exploiting resting-state fMRI data have been used to investigate brain development and aging, but also neurodegenerative and psychiatric disorders [66]. Functional connectivity patterns have, for instance, been used to distinguish patients with schizophrenia from healthy controls [68] or discriminate schizophrenia and bipolar disorder from healthy controls [69].
# 4 X-Ray Imaging

X-ray imaging is built on the work of Röntgen who observed that if a “hand be held before the fluorescent screen, the shadow shows the bones darkly, with only faint outlines of the surrounding tissues” [3].

## 4.1 X-Ray and Angiography

When an X-ray beam passes through the body, part of its energy is absorbed or scattered: the number of X-ray photons is reduced by attenuation (Fig. 10, left). On the opposite side of the body, detectors capture the remaining X-ray photons, and an image is generated. In an X-ray image, the contrast, defined as the relative intensity change produced by an object, originates from the variations in linear attenuation coefficient with tissue type and density.

X-ray imaging provides excellent contrast between bone, air, and soft tissue but very little contrast between the different types of soft tissue, hence its limited use when studying brain disorders. However, coupled with the injection of an iodine-based contrast agent, X-ray imaging enables visualizing cerebral blood vessels and detecting potential abnormalities such as aneurysm. This technique is called X-ray angiography.

## 4.2 Computed Tomography

### 4.2.1 Basic Principles

Although the X-ray images produced were originally in 2D, X-ray computed tomography enables the reconstruction of 3D images by rotating the X-ray source and detectors around the body (Fig. 10, right). Rather than using the absolute values of the linear attenuation coefficients, CT image intensities are expressed in a standard

<figure><img src="image_135.png" /><figcaption>**Fig. 10** Left: attenuation of X-rays by matter. As it passes through a material of thickness $\Delta x$ and linear attenuation coefficient $\mu$, the X-ray beam is attenuated. Its intensity decreases exponentially with the distance travelled: $I_o = I_i e^{-\mu\Delta x}$, where $I_i$ and $I_o$ are the input and output X-ray intensities. Right: third-generation CT. A 3D image is created by rotating the X-ray source and detectors around the body</figcaption></figure>
unit, the Hounsfield unit (HU). The tissue attenuation coefficient is compared to the attenuation value of water and displayed on the Hounsfield scale:

$$x_{HU} = 1000 \times \frac{x_{\mu} - \mu_{water}}{\mu_{water} - \mu_{air}}$$

where $\mu_{water}$ and $\mu_{air}$ are the linear attenuation coefficients of water and air, respectively. For example, air has an attenuation of – 1000 HU, water of 0 HU, and cortical bone between 500 and 1900 HU.

As for 2D X-ray imaging, the injection of an iodine-based contrast agent improves the visualization of cerebral blood vessels. This technique, called CT angiography, is not the only one relying on a contrast agent. CT perfusion tracks the bolus of contrast agent over time and measures the resulting change in signal intensity. Perfusion parameters such as the cerebral blood flow or volume can then be derived [70].

Contrary to MRI, CT images usually do not require extensive preprocessing steps [71]. It can however be useful to extract the head from the hardware elements visible on the image (e.g., the bed or pillow) or extract the brain. This can be done using thresholding and morphological operators. Another common step is spatial normalization.

In the context of stroke, non-contrast CT is useful to detect an intracranial hemorrhage, which appears brighter than the surrounding tissues, or to estimate the extent of early ischemic injury, which results in a loss of gray-white matter differentiation. CT angiography can help identify a potential intracranial arterial occlusion, and CT perfusion allows differentiating the regions with nonviable/non-salvageable tissue, which have very low cerebral blood flow and volume, from the viable and potentially salvageable regions [70]. These techniques may also be employed in the context of brain tumors. In particular, contrast-enhanced CT can detect areas presenting a blood-brain barrier breakdown [72]. An example of CT acquired before and after contrast injection is displayed in Fig. 11.

To the best of our knowledge, CT is most often used in machine learning in the form of voxel-based features (the image intensities after some minimal preprocessing steps).

The vast majority of machine learning studies relying on CT images, particularly non-contrast CT, focus on cerebrovascular disorders [73, 74]. Non-contrast CT images were, for example, used for the detection of intracranial hemorrhage and its five subtypes [75]. A first neural network was in charge of identifying the presence or absence of intracranial hemorrhage and a second of determining the intracranial hemorrhage subtype, which depends
<figure><img src="image_136.png" /><figcaption>**Fig. 11** Example of CT images. Non-contrast CT images, whose window levels were adjusted to better visualize bone or brain tissues and contrast-enhanced CT image of a patient with lymphoma. Case courtesy of Dr Yair Glick, Radiopaedia.org, rID: 94844</figcaption></figure>

on the bleeding location [75]. In [76], non-contrast CT and CT perfusion images were used to segment the core of stroke lesions, as the lesion volume is a key measurement to assess the prognosis of acute ischemic stroke patients.

# 5 Nuclear Imaging

In X-ray CT imaging, the photons that are detected originate from an X-ray source. In nuclear imaging, and more precisely emission computed tomography, the photons detected are emitted from a radiopharmaceutical that has been intravenously injected to the patient.

## 5.1 Positron Emission Tomography

### 5.1.1 Basic Principles

Positron emission tomography is an imaging technique that requires the injection of a substance labeled with a positron-emitting radioactive isotope [77]. The labeled substance is distributed throughout the patient's body by the blood circulation and accumulates in target regions. The positrons emitted by the radioactive isotope combine with the electrons present in the tissues and annihilate. Each annihilation produces two nearly collinear photons (Fig. 12). The two photons are simultaneously detected by two opposing detectors, and a coincidence event is assigned to a line of response connecting the two detectors.

Note that the most common isotope in clinical routine is fluorine-18 ($^{18}$F), which has the advantage of a relatively long half-life (110 min) and thus does not require the presence of a cyclotron at the scanning site. Nevertheless, other isotopes are
<figure><img src="image_137.png" /><figcaption>**Fig. 12** PET annihilation. When a positron ($e^+$) and an electron ($e^-$) collide, they annihilate and create a pair of collinear gamma rays ($\gamma$)</figcaption></figure>

<figure><img src="image_138.png" /><figcaption>**Fig. 13** Illustration of PET data detection. Without time-of-flight, the annihilation is located with equal probability along the line of response, while with time-of-flight it is located in a limited portion of the line of response</figcaption></figure>

used. In particular, carbon-11 ($^{11}$C), which has a shorter half-life (20 min), is often used in research facilities equipped with a cyclotron.

In a time-of-flight PET system, the difference in arrival times between the two coincident photons is measured. Without time-of-flight information, the annihilation is located with equal probability along the line of response, while with time-of-flight information, the annihilation site can be reduced to a limited range (Fig. 13), thus decreasing the spatial uncertainty and increasing the signal-to-noise ratio. Once reconstructed, the PET image is a map of the radioactivity distribution throughout the body.

Two main protocols exist when acquiring PET data. Most acquisitions are static: the radiotracer is injected several minutes before the acquisition (e.g., between 30 and 60 min), which gives
the tracer time to diffuse in the body and accumulate in the target regions. The subject is then placed in the scanner and the acquisition lasts typically around 15 min. In the dynamic protocol, the subject is first installed in the scanner, and the acquisition starts at the same time the tracer is being injected. This allows recording how the tracer diffuses in the body. Dynamic acquisitions are less common than static ones because of their duration of 60–90 min, which reduces patient throughput. In both static and dynamic protocols, the acquisition is often split in frames of fix (in the static case) or increasing (in the dynamic case) duration. A static acquisition of 15 min can typically be split into three frames of 5 min, resulting in three PET volumes, each corresponding to the average amount of radioactivity detected at each voxel during the time frame.

$^{18}$F-fluorodeoxyglucose (FDG) is the most widely used PET radiopharmaceutical [77, 78]. As an analogue of glucose, FDG is transported to a cell, but, unlike glucose, it remains trapped in the cell. This radiopharmaceutical is an excellent marker of changes in glucose metabolism. In the brain, FDG acts as an indirect marker of synaptic dysfunction and is part of the diagnosis of epilepsy and neurodegenerative diseases, such as Alzheimer's disease [79].

If $^{18}$F-FDG is a nonspecific tracer, other radiopharmaceuticals target specific molecular or biological processes and are thus preferentially used for studying specific diseases. Amyloid tracers, such as the $^{11}$C Pittsburgh compound B, $^{18}$F-florbetapir, $^{18}$F-florbetaben and $^{18}$F-flutemetamol, which bind to fibrillar Aβ plaques, or tau tracers, such as $^{18}$F-flortaucipir, and $^{18}$F-MK-6240, which bind to neurofibrillary tangles, are, for example, used in the diagnosis of dementia syndromes [80]. Examples are displayed in Fig. 14. Of note, the so-called amyloid tracers are in fact not specific of amyloid and also bind to myelin in the white matter, making them of

<figure><img src="image_139.png" /><figcaption>**Fig. 14** Example of PET images. Left: $^{18}$F-FDG PET displaying brain glucose metabolism. Middle: $^{18}$F-florbetapir PET displaying the presence of tau neurofibrillary tangles. Right: $^{18}$F-florbetapir PET displaying the presence of amyloid plaques. All the images correspond to the same Alzheimer's disease patient from the ADNI study [83]</figcaption></figure>
interest for demyelinating disorders such as multiple sclerosis [81]. ¹¹C-methionine and ¹⁸F-fluoroethyltyrosine are both used in neuro-oncology [82]. Note that these are just examples of tracers and dozens of tracers exist for imaging specific molecular or biological processes.

### 5.1.2 Extracting Features from PET Images

The reconstruction procedure of the PET signal already includes several corrections (e.g., attenuation and scatter corrections), but several processing steps can be performed before further analyzing PET images. The first one is often motion correction. This is typically done by rigidly registering each frame to a reference frame. The registered frames are then averaged to form a single volume. To allow for intersubject comparison, brain PET images need to be intensity normalized, for example, to compensate for variations in the patients' weight or dose injected. Standardized uptake value ratios (SUVRs) are generated by dividing a PET image by the mean uptake in a reference region. This region can be obtained from an atlas, and in this case chosen depending on the tracer and disorder suspected, or in a data-driven manner [84]. Partial volume correction can be performed to limit the spill out of activity outside of the region where the tracer is meant to accumulate [85] using tools such as PETPVC [86]. Finally, PET images can also be spatially normalized. If an anatomical image (preferably MRI but also CT) of the subject is available, the PET image is rigidly registered to the anatomical image, and the anatomical image is registered to a template, often in standard space. By composing the two transformations, the PET image is spatially normalized. Alternatively, if no anatomical image is available, the PET image can directly be registered to a PET template, for example, as implemented in SPM [87]. Dynamic PET images are further processed to extract quantitative physiological data using kinetic modeling, which is introduced in [77, 78].

One can then obtain different types of features, as described in Subheading 2.2. Voxel-based features will very often be the SUVR at each voxel, usually after spatial normalization. Vertex-based features will generally be the SUVR projected onto the cortical surface [88]. Regional features will usually correspond to the average SUVR in each region of a parcellation. Graph-based features are less used than for diffusion or functional MRI but are still employed to study the so-called metabolic connectivity [89].

### 5.1.3 Examples of Applications in Machine Learning Studies

Machine learning studies have mainly exploited brain PET images in the context of dementia [90]. For example, the usefulness of ¹⁸F-FDG PET to differentiate patients with Alzheimer's disease from healthy controls and patients with stable mild cognitive impairment from those who subsequently progressed to Alzheimer's disease has been shown in [48, 91, 92]. ¹⁸F-FDG PET has also been used to differentiate frontotemporal dementia from
<figure><img src="image_140.png" /><figcaption>**Fig. 15** Illustration of a two-head SPECT system with a parallel hole collimator. The photons whose emission direction is perpendicular to the detector heads have a higher probability of being detected (solid lines)</figcaption></figure>

Alzheimer's disease [93]. In neuro-oncology, ¹¹C-methionine has been used to predict glioma survival [94] or to differentiate recurrent brain tumor from radiation necrosis [95].

**5.2 Single-Photon Emission Computed Tomography**

**5.2.1 Basic Principles**

Single-photon emission computed tomography is an imaging technique that requires the injection of a substance labeled with an isotope that directly emits gamma radiation. Typical isotopes employed in neurology are technetium-99m (⁹⁹mTc) and iodine-123 (¹²³I). As for PET, the labeled substance is distributed throughout the patient's body by the blood circulation and accumulates in target regions. The photons emitted are detected by one to three detector heads, called gamma cameras, that rotate around the patient. Having multiple heads allows reducing image acquisition time and improving sensitivity as more photons can be detected. Collimators are placed in front of the detector heads to localize the origin of the gamma rays: a gamma ray moving from the patient toward the camera has a higher probability of being detected if its direction aligns with the collimator (Fig. 15) [96]. Once reconstructed, the SPECT image is a map of the radioactivity distribution throughout the body. Both dynamic and static protocols exist when acquiring SPECT data.

SPECT is able to visualize and quantify changes in cerebral blood flow and neurotransmitter systems, such as the dopamine system [97, 98]. To image cerebral blood flow, the two most widely used tracers are ⁹⁹mTc-HMPAO and ⁹⁹mTc-ECD [97, 99]. These tracers can, for example, be employed in the context of dementia as a decrease in neural function will result in a decrease in cerebral blood flow in different regions. SPECT plays a key role when studying Parkinsonian syndromes, which are characterized by a loss of dopaminergic neurons. In this context, tracers targeting the dopaminergic system, such as ¹²³I-β-CIT and ¹²³I-FP-CIT
<figure><img src="image_141.png" /><figcaption>**Fig. 16** Examples of SPECT images. Left: <sup>99m</sup>Tc-HMPAO SPECT images of a normal control and an epileptic patient (http://spect.yale.edu) [100]. Right: <sup>123</sup>I-FP-CIT SPECT images of a normal control and a patient with Parkinson's disease from the PPMI study [101]</figcaption></figure>

(also called DaTscan), are employed to differentiate essential tremor from neurodegenerative Parkinsonian syndromes or distinguish dementia with Lewy bodies from other dementias [98]. Examples of SPECT images are displayed in Fig. 16.

**5.2.2 Extracting Features from SPECT Images**

After the reconstruction of a SPECT image, which includes several corrections, two processing steps are typically performed: intensity normalization and spatial normalization [97, 98]. As for PET, the intensity of a SPECT image can be normalized using a reference region, and the image can be spatially normalized by directly registering it to a SPECT template or by registering it first to an anatomical image.

As for PET, the most common feature types are voxel-based (the normalized signal at each voxel) and regional features (often the average normalized signal within a region). To the best of our knowledge, vertex-based and graph-based features are rarely used although they could in principle be computed.

**5.2.3 Examples of Applications in Machine Learning Studies**

Machine learning studies have mainly exploited brain SPECT images for the computer-aided diagnosis of Parkinsonian syndromes [102]. <sup>123</sup>I-FP-CIT SPECT was, for instance, used to distinguish Parkinson's disease from healthy controls [103, 104], predict future motor severity [105], discriminate Parkinson's disease from non-Parkinsonian tremor [104], or identify patients clinically diagnosed with Parkinson's disease but who have scans without evidence of dopaminergic deficit [104].

In studies targeting dementia, both <sup>99m</sup>Tc-HMPAO [106] and <sup>99m</sup>Tc-ECD [107] tracers were used to differentiate between images from healthy subjects and images from Alzheimer's disease patients.
<figure><img src="image_142.png" /><figcaption>Fig. 17 Example of ¹⁸F-FDG PET, CT, T1-weighted MRI, and fused images</figcaption></figure>

# 6 Conclusion

Neuroimaging plays a key role for the study of brain disorders. If some modalities provide information regarding the anatomy of the brain (CT and MRI), others provide functional or molecular information (MRI, PET, and SPECT). To provide a complete picture of biological processes and their alterations, it is often necessary to combine multiple brain imaging modalities (Fig. 17). This can be done by acquiring images with multiple standalone systems or with hybrid systems such as SPECT/CT, PET/CT, or PET/MRI scanners [108].

When analyzing neuroimages, both modality-specific and modality-agnostic processing steps must often be performed. These should be performed with care to obtain reliable features. Machine learning and deep learning are widely used to analyze neuroimaging data. The most common tasks are classification for computer-aided diagnosis, prognosis and disease subtyping, and segmentation to characterize anatomical structures and lesions.

# Acknowledgements

The author would like to thank the editor for useful suggestions. The research leading to these results has received funding from the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute) and reference ANR-10-IAHU-0006 (Agence Nationale de la Recherche-10-IA Institut Hospitalo-Universitaire-6).

# References

1. Ambrose J (1973) Computerized transverse axial scanning (tomography): Part 2. Clinical application. Br J Radiol 46(552):1023-1047. https://doi.org/10.1259/0007-1285-46-552-1023

2. Hounsfield GN (1973) Computerized transverse axial scanning (tomography): Part 1. Description of system. Br J Radiol 46(552):1016-1022. https://doi.org/10.1259/0007-1285-46-552-1016
3. Röntgen WC (1896) On a New Kind of Rays. Science 3(59):227-231

4. Ter-Pogossian MM, Phelps ME, Hoffman EJ, Mullani NA (1975) A Positron-Emission Transaxial Tomograph for Nuclear Imaging (PETT). Radiology 114(1):89-98. https://doi.org/10.1148/114.1.89

5. Jaszczak RJ, Murphy PH, Huard D, Burdine JA (1977) Radionuclide emission computed tomography of the head with 99mTc and a Scintillation Camera. J Nucl Med 18(4): 373-380

6. Keyes JW, Orlandea N, Heetderks WJ, Leonard PF, Rogers WL (1977) The Humongotron—A Scintillation-Camera Transaxial Tomograph. J Nucl Med 18(4): 381-387

7. Becquerel H (1903) Recherches Sur Une Propriété Nouvelle de La Matière: Activité Radiante Spontanée Ou Radioactivité de La Matière. Mémoires de l'Académie Des Sciences de l'Institut de France, L'Institut de France

8. Young IR, Bailes DR, Burl M, Collins AG, Smith DT, McDonnell MJ, Orr JS, Banks LM, Bydder GM, Greenspan RH, Steiner RE (1982) Initial clinical evaluation of a whole body nuclear magnetic resonance (NMR) Tomograph. J Comput Assist Tomogr 6(1):1-18. https://doi.org/10.1001/97/00004728-198202000-00001

9. Bloch F (1946) Nuclear induction. Phys Rev 70(7-8):460-474. https://doi.org/10.1103/PhysRev.70.460

10. Townsend DW, Beyer T, Blodgett TM (2003) PET/CT scanners: A hardware approach to image fusion. Semin Nucl Med 33(3): 193-204. https://doi.org/10.1053/snuc.2003.127314

11. Schlemmer HP, Pichler B, Wienhard K, Schmand M, Nahmias C, Townsend D, Heiss WD, Claussen C (2007) Simultaneous MR/PET for brain imaging: First patient scans. J Nucl Med 48(supplement 2):45P-45P

12. Schmand M, Burbar Z, Corbeil J, Zhang N, Michael C, Byars L, Eriksson L, Grazioso R, Martin M, Moor A, Camp J, Matschl V, Ladebeck R, Renz W, Fischer H, Jattke K, Schnur G, Rietsch N, Bendriem B, Heiss WD (2007) BrainPET: First human tomograph for simultaneous (functional) PET and MR imaging. J Nucl Med 48(supplement 2): 45P-45P

13. Patton JA, Delbeke D, Sandler MP (2000) Image fusion using an integrated, dual-head coincidence camera with X-ray tube-based

attenuation maps. J Nucl Med 41(8): 1364-1368

14. Hutton BF, Occhipinti M, Kuehne A, Máthé D, Kovács N, Waiczies H, Erlandsson K, Salvado D, Carminati M, Montagnani GL et al (2018) Development of clinical simultaneous SPECT/MRI. Br J Radiol 91(1081):20160690. https://doi.org/10.1259/bjr.20160690

15. Smith-Bindman R, Kwan ML, Marlow EC, Theis MK, Bolch W, Cheng SY, Bowles EJA, Duncan JR, Greenlee RT, Kushi LH, Pole JD, Rahm AK, Stout NK, Weinmann S, Miglioretti DL (2019) Trends in use of medical imaging in US Health Care Systems and in Ontario, Canada, 2000-2016. JAMA 322(9):843-856. https://doi.org/10.1001/jama.2019.11456

16. Evans AC, Janke AL, Collins DL, Baillet S (2012) Brain templates and atlases. Neuroimage 62(2):911-922. https://doi.org/10.1016/j.neuroimage.2012.01.024

17. Dale AM, Fischl B, Sereno MI (1999) Cortical surface-based analysis: I. Segmentation and surface reconstruction. Neuroimage 9(2):179-194. https://doi.org/10.1006/nimg.1998.0395

18. Fischl B, Sereno MI, Dale AM (1999) Cortical surface-based analysis: II: inflation, flattening, and a surface-based coordinate system. Neuroimage 9(2):195-207. https://doi.org/10.1006/nimg.1998.0396

19. de Vico Fallani F, Richiardi J, Chavez M, Achard S (2014) Graph analysis of functional brain networks: practical issues in translational neuroscience. Philos Trans R Soc B Biol Sci 369(1653):20130521. https://doi.org/10.1098/rstb.2013.0521

20. Gorgolewski KJ, Auer T, Calhoun VD, Craddock RC, Das S, Duff EP, Flandin G, Ghosh SS, Glatard T, Halchenko YO, Handwerker DA, Hanke M, Keator D, Li X, Michael Z, Maumet C, Nichols BN, Nichols TE, Pellman J, Poline JB, Rokem A, Schaefer G, Sochat V, Triplett W, Turner JA, Varoquaux G, Poldrack RA (2016) The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Sci Data 3: sdata201644. https://doi.org/10.1038/sdata.2016.44

21. Penny WD, Friston KJ, Ashburner JT, Kiebel SJ, Nichols TE (2011) Statistical parametric mapping: the analysis of functional brain images. Elsevier, Amsterdam

22. Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM (2012) FSL.
NeuroImage 62(2):782-790. https://doi.org/10.1016/j.neuroimage.2011.09.015

23. Fischl B (2012) Freesurfer. NeuroImage 62(2):774-781. https://doi.org/10.1016/j.neuroimage.2012.01.021

24. Avants BB, Tustison N, Song G (2009) Advanced normalization tools (ANTS). The Insight Journal 2(365):1-35. https://doi.org/10.54294/uvnhin

25. Tournier JD, Calamante F, Connelly A (2012) MRtrix: diffusion tractography in crossing fiber regions. Int J Imaging Syst Technol 22(1):53-66. https://doi.org/10.1002/ima.22005

26. Cox RW (1996) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29(3):162-173. https://doi.org/10.1006/cbmr.1996.0014

27. Gorgolewski KJ, Alfaro-Almagro F, Auer T, Bellec P, Capotà M, Chakravarty MM, Churchill NW, Cohen AL, Craddock RC, Devenyi GA, Eklund A, Esteban O, Flandin G, Ghosh SS, Guntupalli JS, Jenkinson M, Keshavan A, Kiar G, Liem F, Raamana PR, Raffelt D, Steele CJ, Quirion PO, Smith RE, Strother SC, Varoquaux G, Wang Y, Yarkoni T, Poldrack RA (2017) BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods. PLoS Comput Biol 13(3): e005209. https://doi.org/10.1371/journal.pcbi.1005209

28. Abraham A, Pedregosa F, Eickenberg M, Gervais P, Mueller A, Kossaifi J, Gramfort A, Thirion B, Varoquaux G (2014) Machine learning for neuroimaging with scikit-learn. Front Neuroinform 8:14. https://doi.org/10.3389/fninf.2014.00014

29. Routier A, Burgos N, Díaz M, Bacci M, Bottani S, El-Rifai O, Fontanella S, Gori P, Guillon J, Guyot A, Hassanaly R, Jacquemont T, Lu P, Marcoux A, Moreau T, Samper-González J, Teichmann M, Thibeau-Sutre E, Vaillant G, Wen J, Wild A, Habert MO, Durrleman S, Colliot O (2021) Clinica: an open-source software platform for reproducible clinical neuroscience studies. Front Neuroinform 15:39. https://doi.org/10.3389/fninf.2021.689675

30. Pérez-García F, Sparks R, Ourselin S (2021) TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Comput Methods Prog Biomed 208:106236. https://doi.org/10.1016/j.cmpb.2021.106236

31. Thibeau-Sutre E, Díaz M, Hassanaly R, Routier A, Dormont D, Colliot O, Burgos N (2022) ClinicaDL: An open-source deep learning software for reproducible neuroimaging processing. Comput Methods Prog Biomed 220:106818. https://doi.org/10.1016/j.cmpb.2022.106818

32. Commowick O, Istace A, Kain M, Laurent B, Leray F, Simon M, Pop SC, Girard P, Améli R, Ferré JC, Kerbrat A, Tourdias T, Cervenansky F, Glatard T, Beaumont J, Doyle S, Forbes F, Knight J, Khademi A, Mahbod A, Wang C, McKinley R, Wagner F, Muschelli J, Sweeney E, Roura E, Lladó X, Santos MM, Santos WP, Silva-Filho AG, Tomas-Fernandez X, Urien H, Bloch I, Valverde S, Cabezas M, Vera-Olmos FJ, Malpica N, Guttmann C, Vukusic S, Edan G, Dojat M, Styner M, Warfield SK, Cotton F, Barillot C (2018) Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure. Sci Rep 8(1):13650. https://doi.org/10.1038/s41598-018-31911-7

33. Commowick O, Kain M, Casey R, Ameli R, Ferré JC, Kerbrat A, Tourdias T, Cervenansky F, Camarasu-Pop S, Glatard T, Vukusic S, Edan G, Barillot C, Dojat M, Cotton F (2021) Multiple sclerosis lesions segmentation from multiple experts: The MICCAI 2016 challenge dataset. NeuroImage 244:118589. https://doi.org/10.1016/j.neuroimage.2021.118589

34. Tustison NJ, Avants BB, Cook PA, Zheng Y, Egan A, Yushkevich PA, Gee JC (2010) N4ITK: Improved N3 Bias Correction. IEEE Trans Med Imaging 29(6): 1310-1320. https://doi.org/10.1109/TMI.2010.2046908

35. Nyúl LG, Udupa JK, Zhang X (2000) New variants of a method of MRI scale standardization. IEEE Trans Med Imaging 19(2): 143-150. https://doi.org/10.1109/42.836373

36. Kalavathi P, Prasath V (2016) Methods on skull stripping of MRI head scan images—a review. J Digit Imaging 29(3):365-379. https://doi.org/10.1007/s10278-015-9847-8

37. Shattuck DW, Leahy RM (2002) BrainSuite: an automated cortical surface identification tool. Med Image Anal 6(2):129-142. https://doi.org/10.1016/S1361-8415(02)00054-3

38. Isensee F, Schell M, Pflueger I, Brugnara G, Bonekamp D, Neuberger U, Wick A, Schlemmer HP, Heiland S, Wick W et al (2019)
Automated brain extraction of multisequence MRI using artificial neural networks. Hum Brain Mapp 40(17):4952–4964. https://doi.org/10.1002/hbm.24750

39. Hoopes A, Mora JS, Dalca AV, Fischl B, Hoffmann M (2022) SynthStrip: skull-stripping for any brain image. NeuroImage 260: 119474. https://doi.org/10.1016/j.neuroimage.2022.119474

40. Oliveira FP, Tavares JMR (2014) Medical image registration: a review. Comput Meth Biomech Biomed Eng 17(2):73–93. https://doi.org/10.1080/10255842.2012.670855

41. Dora L, Agrawal S, Panda R, Abraham A (2017) State-of-the-art methods for brain tissue segmentation: A review. IEEE Rev Biomed Eng 10:235–249. https://doi.org/10.1109/RBME.2017.2715350

42. González-Villà S, Oliver A, Valverde S, Wang L, Zwiggelaar R, Lladó X (2016) A review on brain structures segmentation in magnetic resonance imaging. Artif Intell Med 73:45–69. https://doi.org/10.1016/j.artmed.2016.09.001

43. Ashburner J, Friston KJ (2000) Voxel-Based Morphometry—The Methods. NeuroImage 11(6):805–821. https://doi.org/10.1006/nimg.2000.0582

44. Fischl B, Dale AM (2000) Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc Natl Acad Sci 97(20):11050–11055. https://doi.org/10.1073/pnas.200033797

45. He Y, Chen ZJ, Evans AC (2007) Small-world anatomical networks in the human brain revealed by cortical thickness from MRI. Cereb Cortex 17(10):2407–2419. https://doi.org/10.1093/cercor/bhl149

46. Klöppel S, Stonnington CM, Chu C, Draganski B, Scahill RI, Rohrer JD, Fox NC, Jack Jr CR, Ashburner J, Frackowiak RS (2008) Automatic classification of MR scans in Alzheimer's disease. Brain 131(3): 681–689. https://doi.org/10.1093/brain/awm319

47. Rathore S, Habes M, Iftikhar MA, Shacklett A, Davatzikos C (2017) A review on neuroimaging-based classification studies and associated feature extraction methods for alzheimer's disease and its prodromal stages. NeuroImage 155:530–548. https://doi.org/10.1016/j.neuroimage.2017.03.057

48. Samper-González J, Burgos N, Bottani S, Fontanella S, Lu P, Marcoux A, Routier A, Guillon J, Bacci M, Wen J, Bertrand A, Bertin H, Habert MO, Durrleman S, Evgeniou T, Colliot O (2018) Reproducible evaluation of classification methods in Alzheimer's disease: Framework and application to MRI and PET data. NeuroImage 183:504–521. https://doi.org/10.1016/j.neuroimage.2018.08.042

49. Wen J, Thibeau-Sutre E, Diaz-Melo M, Samper-González J, Routier A, Bottani S, Dormont D, Durrleman S, Burgos N, Colliot O (2020) Convolutional neural networks for classification of Alzheimer's disease: Overview and reproducible evaluation. Med Image Anal 63:101694. https://doi.org/10.1016/j.media.2020.101694

50. Bakas S, Reyes M, Jakab A, . . ., Davatzikos C, van Leemput K, Menze B (2018) Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:181102629

51. Zeineldin RA, Karar ME, Coburger J, Wirtz CR, Burgert O (2020) DeepSeg: Deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images. Int J Comput Assist Radiol Surg 15(6):909–920. https://doi.org/10.1007/s11548-020-02186-z

52. Sweeney EM, Vogelstein JT, Cuzzocreo JL, Calabresi PA, Reich DS, Crainiceanu CM, Shinohara RT (2014) A comparison of supervised machine learning algorithms and feature vectors for MS lesion segmentation using multimodal structural MRI. PLoS One 9(4): e95753. https://doi.org/10.1371/journal.pone.0095753

53. La Rosa F, Abdulkadir A, Fartaria MJ, Rahmanzadeh R, Lu P, Galbusera R, Barakovic M, Thiran J, Granziera C, Cuadra MB (2020) Multiple sclerosis cortical and WM lesion segmentation at 3T MRI: A deep learning method based on FLAIR and MP2RAGE. NeuroImage: Clinical 27: 102335. https://doi.org/10.1016/j.nicl.2020.102335

54. Le Bihan D, Breton E, Lallemand D, Grenier P, Cabanis E, Laval-Jeanet M (1986) MR imaging of intravoxel incoherent motions: application to diffusion and perfusion in neurologic disorders. Radiology 161(2):401–407

55. Tournier JD (2019) Diffusion MRI in the brain–Theory and concepts. Prog Nucl Magn Reson Spectrosc 112:1–16. https://doi.org/10.1016/j.pnmrs.2019.03.001

56. Tax CM, Bastiani M, Veraart J, Garyfallidis E, Irfanoglu MO (2022) What's new and what's
next in diffusion MRI preprocessing. Neuro-
Image 249:118830. https://doi.org/10.101
6/j.neuroimage.2021.118830

57. Soares JM, Marques P, Alves V, Sousa N (2013) A hitchhiker's guide to diffusion tensor imaging. Front. Neurosci. 7:31. https://doi.org/10.3389/fnins.2013.00031

58. Jeurissen B, Descoteaux M, Mori S, Leemans A (2019) Diffusion MRI fiber tractography of the brain. NMR Biomed 32(4):e3785. https://doi.org/10.1002/nbm.3785

59. Zhang H, Schneider T, Wheeler-Kingshott CA, Alexander DC (2012) NODDI: practical in vivo neurite orientation dispersion and density imaging of the human brain. NeuroImage 61(4):1000-1016. https://doi.org/10.1016/j.neuroimage.2012.03.072

60. Maggipinto T, Bellotti R, Amoroso N, Diacono D, Donvito G, Lella E, Monaco A, Scelsi MA, Tangaro S (2017) DTI measurements for Alzheimer's classification. Phys Med Biol 62(6):2361. https://doi.org/10.1088/1361-6560/aa5dbe

61. Wen J, Samper-González J, Bottani S, Routier A, Burgos N, Jacquemont T, Fontanella S, Durrleman S, Epelbaum S, Bertrand A, Colliot O (2021) Reproducible evaluation of diffusion MRI features for automatic classification of patients with Alzheimer's disease. Neuroinformatics 19(1): 57-78. https://doi.org/10.1007/s12021-020-09469-5

62. Park YW, Oh J, You SC, Han K, Ahn SS, Choi YS, Chang JH, Kim SH, Lee SK (2019) Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging. Eur Radiol 29(8): 4068-4076. https://doi.org/10.1007/s00330-018-5830-3

63. Glover GH (2011) Overview of functional magnetic resonance imaging. Neurosurg Clin 22(2):133-139. https://doi.org/10.1016/j.nec.2010.11.001

64. Fox MD, Raichle ME (2007) Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci 8(9):700-711. https://doi.org/10.1038/nrn2201

65. Esteban O, Markiewicz CJ, Blair RW, Moodie CA, Isik AI, Erramuzpe A, Kent JD, Goncalves M, DuPre E, Snyder M, Oya H, Ghosh SS, Wright J, Durnez J, Poldrack RA, Gorgolewski KJ (2019) fMRIPrep: A robust preprocessing pipeline for functional MRI. Nat Methods 16(1):111-116. https://doi.org/10.1038/s41592-018-0235-4

66. Khosla M, Jamison K, Ngo GH, Kuceyeski A, Sabuncu MR (2019) Machine learning in resting-state fMRI analysis. Magn Reson Imaging 64:101-121. https://doi.org/10.1016/j.mri.2019.05.031

67. Fischl B, Sereno MI, Tootell RB, Dale AM (1999) High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum Brain Mapp 8(4):272-284. https://doi.org/10.1002/(SICI)1097-0193 (1999)8:4%3C272::AID-HBM10%3E3.0.CO;2-4

68. Kim J, Calhoun VD, Shim E, Lee JH (2016) Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: evidence from whole-brain resting-state functional connectivity patterns of schizophrenia. NeuroImage 124:127-146. https://doi.org/10.1016/j.neuroimage.2015.05.018

69. Rashid B, Arbabshirani MR, Damaraju E, Cetin MS, Miller R, Pearlson GD, Calhoun VD (2016) Classification of schizophrenia and bipolar patients using static and dynamic resting-state fMRI brain connectivity. NeuroImage 134:645-657. https://doi.org/10.1016/j.neuroimage.2016.04.051

70. Wannamaker R, Buck B, Butcher K (2019) Multimodal CT in Acute Stroke. Curr Neurol Neurosci Rep 19(9):63. https://doi.org/10.1007/s11910-019-0978-z

71. Muschelli J (2019) Recommendations for processing head CT data. Front Neuroinform 13:61. https://doi.org/10.3389/fninf.2019.00061

72. Chourmouzi D, Papadopoulou E, Marias K, Drevelegas A (2014) Imaging of Brain Tumors. Surg Oncol Clin 23(4):629-684. https://doi.org/10.1016/j.soc.2014.07.004

73. Yeo M, Tahayori B, Kok HK, Maingard J, Kutaiba N, Russell J, Thijs V, Jhamb A, Chandra RV, Brooks M, Barras CD, Asadi H (2021) Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. Journal of NeuroInterventional Surgery 13(4):369-378. https://doi.org/10.1136/neurintsurg-2020-017099

74. Buchlak QD, Milne MR, Seah J, Johnson A, Samarasinghe G, Hachey B, Esmaili N, Tran A, Leveque JC, Farrokhi F, Goldschlager T, Edelstein S, Brotchie P (2022) Charting the potential of brain computed tomography deep learning systems. J
Clin Neurosci 99:217-223. https://doi.org/10.1016/j.jocn.2022.03.014

75. Ye H, Gao F, Yin Y, Guo D, Zhao P, Lu Y, Wang X, Bai J, Cao K, Song Q, Zhang H, Chen W, Guo X, Xia J (2019) Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur Radiol 29(11):6191-6201. https://doi.org/10.1007/s00330-019-06163-2

76. Clerigues A, Valverde S, Bernal J, Freixenet J, Oliver A, Lladó X (2019) Acute ischemic stroke lesion core segmentation in CT perfusion images using fully convolutional neural networks. Comput Biol Med 115:103487. https://doi.org/10.1016/j.biomed.2019.103487

77. Hooker JM, Carson RE (2019) Human positron emission tomography neuroimaging. Annu Rev Biomed Eng 21:551-581. https://doi.org/10.1146/annurev-bioeng-062117-121056

78. Heurling K, Leuzy A, Jonasson M, Frick A, Zimmer ER, Nordberg A, Lubberink M (2017) Quantitative positron emission tomography in brain research. Brain Res 1670:220-234. https://doi.org/10.1016/j.brainres.2017.06.022

79. Guedj E, Varrone A, Boellaard R, Albert NL, Barthel H, van Berckel B, Brendel M, Cecchin D, Ekmekcioglu O, Garibotto V, Lammertsma AA, Law I, Peñuelas I, Semah F, Traub-Weidinger T, van de Giessen E, Van Weehaeghe D, Morbelli S (2022) EANM procedure guidelines for brain PET imaging using [18F]FDG, version 3. Eur J Nucl Med Mol Imaging 49(2): 632-651. https://doi.org/10.1007/s00259-021-05603-w

80. Cho H, Choi JY, Hwang MS, Kim YJ, Lee HM, Lee HS, Lee JH, Ryu YH, Lee MS, Lyoo CH (2016) In vivo cortical spreading pattern of tau and amyloid in the Alzheimer disease spectrum. Ann Neurol 80(2): 247-258. https://doi.org/10.1002/ana.24711

81. Stankoff B, Freeman L, Aigrot MS, Chardain A, Dollé F, Williams A, Galanaud D, Armand L, Lehericy S, Lubetzki C et al (2011) Imaging central nervous system myelin by positron emission tomography in multiple sclerosis using [methyl-11C]-2-(4'-methylaminophenyl)-6-hydroxybenzothiazole. Ann Neurol 69(4):673-680. https://doi.org/10.1002/ana.22320

82. Galldiks N, Lohmann P, Albert NL, Tonn JC, Langen KJ (2019) Current status of pet imaging in neuro-oncology. Neuro-Oncology

Advances 1(1):vdz010. https://doi.org/10.1093/noajnl/vdz010

83. Weiner MW, Veitch DP, Aisen PS, Beckett LA, Cairns NJ, Green RC, Harvey D, Jack CR, Jagust W, Morris JC, Petersen RC, Salazar J, Saykin AJ, Shaw LM, Toga AW, Trojanowski JQ (2017) The Alzheimer's Disease Neuroimaging Initiative 3: Continued innovation for clinical trial improvement. Alzheimers Dement. 13(5):561-571. https://doi.org/10.1016/j.jalz.2016.10.006

84. López-González FJ, Silva-Rodríguez J, Paredes-Pacheco J, Niñerola-Baizán A, Efthimiou N, Martín-Martín C, Moscoso A, Ruibal A, Roé-Vellvé N, Aguiar P (2020) Intensity normalization methods in brain fdg-pet quantification. Neuroimage 222: 117229. https://doi.org/10.1016/j.neuroimage.2020.117229

85. Erlandsson K, Buvat I, Pretorius PH, Thomas BA, Hutton BF (2012) A review of partial volume correction techniques for emission tomography and their applications in neurology, cardiology and oncology. Phys Med Biol 57(21):R119. https://doi.org/10.1088/0031-9155/57/21/R119

86. Thomas BA, Cuplov V, Bousse A, Mendes A, Thielemans K, Hutton BF, Erlandsson K (2016) PETPVC: A toolbox for performing partial volume correction techniques in positron emission tomography. Phys Med Biol 61(22):7975-7993. https://doi.org/10.1088/0031-9155/61/22/7975

87. Della Rosa PA, Cerami C, Gallivanone F, Prestia A, Caroli A, Castiglioni I, Gilardi MC, Frisoni G, Friston K, Ashburner J, Perani D (2014) A Standardized 18F-FDG-PET Template for spatial normalization in statistical parametric mapping of Dementia. Neuroinformatics 12(4):575-593. https://doi.org/10.1007/s12021-014-9235-4

88. Marcoux A, Burgos N, Bertrand A, Teichmann M, Routier A, Wen J, Samper-González J, Bottani S, Durrleman S, Habert MO, Colliot O (2018) An automated pipeline for the analysis of PET Data on the cortical surface. Front Neuroinform 12:94. https://doi.org/10.3389/fninf.2018.00094

89. Yakushev I, Drzezga A, Habeck C (2017) Metabolic connectivity: methods and applications. Curr Opin Neurol 30(6):677-685. https://doi.org/10.1097/WCO.0000000000000494

90. Duffy IR, Boyle AJ, Vasdev N (2019) Improving PET imaging acquisition and analysis with machine learning: a narrative review with focus on Alzheimer's disease and oncology. Mol Imaging 18:1536012119869070.
https://doi.org/10.1177/1536012119869070

91. Gray KR, Wolz R, Heckemann RA, Aljabar P, Hammers A, Rueckert D, Initiative ADN et al (2012) Multi-region analysis of longitudinal FDG-PET for the classification of Alzheimer's disease. *NeuroImage* 60(1):221-229.
https://doi.org/10.1016/j.neuroimage.2011.12.071

92. Lu D, Popuri K, Ding GW, Balachandar R, Beg MF (2018) Multiscale deep neural network based analysis of FDG-PET images for the early diagnosis of Alzheimer's disease. *Med Image Anal* 46:26-34. https://doi.org/10.1016/j.media.2018.02.002

93. Higdon R, Foster NL, Koepp RA, DeCarli CS, Jagust WJ, Clark CM, Barbas NR, Arnold SE, Turner RS, Heidebrink JL, Minoshima S (2004) A comparison of classification methods for differentiating fronto-temporal dementia from Alzheimer's disease using FDG-PET imaging. *Stat Med* 23(2): 315-326. https://doi.org/10.1002/sim.1719

94. Papp L, Pötsch N, Grahovac M, Schmidbauer V, Woehrer A, Preusser M, Mitterhauser M, Kiesel B, Wadsak W, Beyer T et al (2018) Glioma survival prediction with combined analysis of in vivo 11C-MET PET features, ex vivo features, and patient features by supervised machine learning. *J Nucl Med* 59(6):892-899. https://doi.org/10.2967/jnumed.117.202267

95. Hotta M, Minamimoto R, Miwa K (2019) 11C-methionine-PET for differentiating recurrent brain tumor from radiation necrosis: radiomics approach with random forest classifier. *Sci Rep* 9(1):1-7. https://doi.org/10.1038/s41598-019-52279-2

96. Accorsi R (2008) Brain Single-photon emission CT physics principles. *Am J Neuroradiol* 29(7):1247-1256. https://doi.org/10.3174/ajnr.A1175

97. Kapucu ÖL, Nobili F, Varrone A, Booi J, Vander Borght T, Nägren K, Darcourt J, Tatsch K, Van Laere KJ (2009) EANM procedure guideline for brain perfusion SPECT using 99mTc-labelled radiopharmaceuticals, version 2. Eur J Nucl Med Mol Imaging 36(12):2093. https://doi.org/10.1007/s00259-009-1266-y

98. Morbelli S, Esposito G, Arbizu J, Barthel H, Boellaard R, Bohnen NI, Brooks DJ, Darcourt J, Dickson JC, Douglas D, Drzezga A, Dubroff J, Ekmekcioglu O, Garibotto V, Herscovitch P, Kuo P, Lammertsma A, Pappata S, Peñuelas I, Seibyl J, Semah F, Tossici-Bolt L, Van de

Giessen E, Van Laere K, Varrone A, Wanner M, Zubal G, Law I (2020) EANM practice guideline/SNMMI procedure standard for dopaminergic imaging in Parkinsonian syndromes 1.0. Eur J Nucl Med Mol Imaging 47(8):1885-1912. https://doi.org/10.1007/s00259-020-04817-8

99. Yeo JM, Lim X, Khan Z, Pal S (2013) Systematic review of the diagnostic utility of SPECT imaging in dementia. *Eur Arch Psychiatry Clin Neurosci* 263(7):539-552. https://doi.org/10.1007/s00406-013-0426-z

100. McNally KA, Paige AL, Varghese G, Zhang H, Novotny Jr EJ, Spencer SS, Zubal IG, Blumenfeld H (2005) Localizing value of ictal-interictal SPECT analyzed by SPM (ISAS). *Epilepsia* 46(9):1450-1464. https://doi.org/10.1111/j.1528-1167.2005.06705.x

101. Marek K, Chowdhury S, Siderowf A, Lasch S, Coffey CS, Caspell-Garcia C, Simuni T, Jennings D, Tanner CM, Trojanowski JQ, Shaw LM, Seibyl J, Schuff N, Singleton A, Kieburtz K, Toga AW, Mollenhauer B, Galasko D, Chahine LM, Weintraub D, Foroud T, Tosun-Turgut D, Poston K, Arnedo V, Frasier M, Sherer T (2018) The Parkinson's progression markers initiative (PPMI)—establishing a PD biomarker cohort. *Ann Clin Transl Neurol* 5(12): 1460-1477. https://doi.org/10.1002/acn3.644

102. Khachnaoui H, Mabrouk R, Khlifa N (2020) Machine learning and deep learning for clinical data and PET/SPECT imaging in Parkinson's disease: A review. *IET Image Process* 14(16):4013-4026. https://doi.org/10.1049/iet-ipr.2020.1048

103. Prashanth R, Roy SD, Mandal PK, Ghosh S (2014) Automatic classification and prediction models for early Parkinson's disease diagnosis from SPECT imaging. *Expert Systems with Applications* 41(7):3333-3342. https://doi.org/10.1016/j.eswa.2013.11.031

104. Choi H, Ha S, Im HJ, Paek SH, Lee DS (2017) Refining diagnosis of parkinson's disease with deep learning-based interpretation of dopamine transporter imaging. *NeuroImage: Clinical* 16:586-594. https://doi.org/10.1016/j.nicl.2017.09.010

105. Rahmim A, Huang P, Shenkov N, Fotouhi S, Davoodi-Bojd E, Lu L, Mari Z, Soltanian-Zadeh H, Sossi V (2017) Improved prediction of outcome in parkinson's disease using radiomics analysis of longitudinal DAT SPECT images. *NeuroImage: Clinical* 16: 539-544. https://doi.org/10.1016/j.nicl.2017.08.021
106. Fung G, Stoeckel J (2007) SVM feature selection for classification of spect images of alzheimer's disease using spatial information. Knowl Inf Syst 11(2):243-258. https://doi.org/10.1007/s10115-006-0043-5

107. Górriz J, Segovia F, Ramírez J, Lassl A, Salas-Gonzalez D (2011) GMM based SPECT image classification for the diagnosis of Alzheimer's disease. Appl Soft Comput 11(2):

2313-2325. https://doi.org/10.1016/j.asoc.2010.08.012

108. de Galiza Barbosa F, Delso G, Ter Voert E, Huellner M, Herrmann K, Veit-Haibach P (2016) Multi-technique hybrid imaging in PET/CT and PET/MR: what does the future hold? Clin Radiol 71(7):660-672. https://doi.org/10.1016/j.crad.2016.03.013

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
# Electroencephalography and Magnetoencephalography

Marie-Constance Corsi

## Abstract

In this chapter, we present the main characteristics of electroencephalography (EEG) and magnetoencephalography (MEG). More specifically, this chapter is dedicated to the presentation of the data, the way they can be acquired and analyzed. Then, we present the main features that can be extracted and their applications for brain disorders with concrete examples to illustrate them. Additional materials associated with this chapter are available in the dedicated Github repository.

**Key words** Electroencephalography, Magnetoencephalography, Evoked activity, Oscillatory activity, Brain-computer interfaces

# 1 Introduction

This chapter aims at providing an overview of electroencephalography (EEG) and magnetoencephalography (MEG) to help the reader with no previous experience with these modalities to understand the information that can be extracted and their neurophysiological meaning in the perspective to be used for brain disorders. These two modalities, which share common characteristics, are often designated together with the acronym M/EEG.

To this end, instead of providing an exhaustive presentation of the M/EEG clinical applications, we focused on the main aspects related to these modalities. As a result, this chapter is organized as follows: We first describe the basic principles in terms of origins of the signals and electrophysiological activity exploited in M/EEG (Subheading 2). We then present the principles of M/EEG experiments (Subheading 3), the data analysis techniques (Subheading 4), and in particular features that can be extracted from the data (Subheading 5). The last part of this chapter presents illustrations of M/EEG applications to brain disorders (Subheading 6). To go further, additional resources are provided to the reader in Boxes 1 and 2 and in a dedicated Github repository.
**2 Basic Principles**

Being able to extract the information of interest to perform a classification from M/EEG data requires to have some neurophysiological background knowledge to assess the relevance of the selected features. This paragraph aims at providing some general elements regarding the origin of the signals and the recorded activity.

**2.1 Origin of the Signals**

Neurons create electrical signals, transmitted to other cells via synapses. First, an action potential (AP) arrives at a synaptic cleft (step 1 in Fig. 1) where it will transmit chemical information via neurotransmitters (step 2 in Fig. 1) that generate postsynaptic potentials (PSPs) and local currents (step 3 in Fig. 1). A PSP will create a current sink and will propagate until the cell body to generate a current source (step 4 in Fig. 1). As a result, the PSP creates an electrical dipole consisting in a negative pole (i.e., the sink) and a positive pole (i.e., the source). This dipole will generate primary (intracellular) currents and secondary (extracellular) currents. M/EEG signals result from postsynaptic potentials. More specifically, M/EEG signals result from the spatial and temporal summation of the activity of a large population of synchronous neurons. But notable differences exist between MEG and EEG.

<figure><img src="image_143.png" /><figcaption>**Fig. 1** Origin of M/EEG signals</figcaption></figure>
<table><caption>**Table 1**
Main features to compare MEG and EEG</caption><thead><tr><th>Items</th><th>MEG</th><th>EEG</th></tr></thead><tbody><tr><td>Measurement</td><td>Magnetic field, + intracellular currents</td><td>Difference of potentials, + extracellular currents</td></tr><tr><td>Spatial resolution</td><td>1 cm</td><td>2–3 cm</td></tr><tr><td>Temporal resolution</td><td>1 ms or less</td><td></td></tr><tr><td>Amplitudes</td><td>≈ 100 fT</td><td>≈ 100 μVolts</td></tr><tr><td>Advantages</td><td>– Absolute values<br>– Less affected by bone – Focal<br>– Focal</td><td>– Portable<br>– Cost</td></tr><tr><td>Drawbacks</td><td>– Financial constraints<br>– Mechanical constraints</td><td>– Need of a reference<br>– Affected by bone<br>– Diffuse</td></tr></tbody></table>

Firstly, regarding the signals themselves, MEG signals are mainly caused by intracellular currents generated by the PSP at the dendrite level and less by the extracellular currents; EEG signals correspond to a difference between electrical potentials, mainly due to extracellular currents. Secondly, regarding the sensitivity toward the dipole orientation, EEG is sensitive both to radial currents (activity located at the gyrus level) and to tangential currents (generated within sulci) even though it has stronger sensitivity to radial currents, whereas MEG is more sensitive to tangential currents. Finally, regarding the sensitivity toward the conductivity, EEG is strongly attenuated and deformed by crossing through the skull, whereas MEG is less sensitive to the different layers crossed (i.e., skull, brain, etc.). Such differences between MEG and EEG have an impact on the way data are preprocessed, analyzed, and, therefore, interpreted. The differences between MEG and EEG are summarized in Table 1.

**2.2 Evoked and Oscillatory Activity**

There are two main types of electrophysiological activity of interest that are exploited in the M/EEG domain: the evoked and the oscillatory activity. Evoked responses are weak variations of electromagnetic activity resulting from a stimulation (for instance, in response to a task performance by the participant). Given their amplitude, it is often necessary to average signals over chunks of signals, referred as epochs, to reduce noise. To identify and describe these evoked responses, there is a specific way to name them according to their latency, their amplitude, their shape, and the
<figure><img src="image_144.png" /><figcaption>**Fig. 2** Evoked activity. Results from a simulation where two sources, located in the visual area, generated an activity after a stimulus. On the right, we plotted the associated time course over the scalp (synthetic signals), resulting from the averaging of 1000 repetitions. One can observe notably a positive wave around *t* = 300 ms. The code to generate this figure is accessible via the dedicated Github repository</figcaption></figure>

polarity. Let's take an example (see Fig. 2), which represent evoked responses from a study where we simulated a visual stimulation. We first see a positive deflection occurring 300 ms after the presentation of the stimulation, which is referred to as P300. These waves can reflect different mechanisms: the early components are mostly exogenous and are related to the stimulus characteristics; the late components are endogenous and are related to the performed task and to the subject's state.

The oscillatory activity, or induced activity, results from the summation of the activity in a given brain region. These rhythms are mainly defined by their frequency, their amplitude, their shape, their location, and their duration. In Fig. 3, we provided examples of the main rhythms found in the literature. Each frequency band is referred to by a Greek letter. Delta (0.5-3 Hz] and Theta (3-7 Hz]) rhythms are, respectively, detected in deep and slight sleeps. Alpha (8-12 Hz] in posterior areas) and Mu (7-13 Hz] in central areas) rhythms are both observed in quiet watch and resting state (with the eyes closed for Alpha). Beta (13-30 Hz]) rhythm is detected during the active watch and during cognitive tasks such as motor imagery, for instance. Gamma rhythm (divided into two sub-rhythms: slow in 30-70 Hz and fast beyond 70 Hz) is observed during specific cognitive processing.

## 3 M/EEG Experiments

This section provides an overview of the devices currently used and the main steps that constitute an M/EEG experiment. As a take-home message, in Table 1, we propose a comparison of the main features of MEG and EEG.
<figure><img src="image_145.png" /><figcaption>**Fig. 3** We plotted the time course associated with the main rhythms that one can observe from M/EEG recordings. These plots were obtained from synthetic signals. The code to generate this figure is accessible via the dedicated Github repository</figcaption></figure>

**3.1 Instrumentation**

3.1.1 EEG

EEG signals are recorded through the use of electrodes placed over the scalp. The EEG relies on the difference of potentials. The first EEG recordings have been performed by Hans Berger in 1924. He described the oscillatory activity at 8 Hz occurring in the posterior area of the scalp when the subject is awake with his eyes closed. There are different types of electrodes: wet/dry electrodes and active/passive electrodes. Wet electrodes are generally made of tin, silver, or silver chloride material (Ag/AgCl). They need an electrolytic gel to enable the conduction between the skin and the electrode. Dry electrodes are made of stainless steel that behaves as a conductor between the skin and the electrode. The active electrodes contain an electronic module that performs a pre-amplification of the signal to ensure the stability of the system toward changes in impedance and noise. The passive electrodes do not use a pre-amplification module.

*Naming* Even though some differences may be found from one EEG device to another, there are some standardized ways to name and localize EEG sensors (also called channels). Each channel is often referred to by a letter and a number. Most of the time, odd channels are located on the left hemisphere and the even ones on the right hemisphere. The letters correspond to the area: frontal, temporal, parietal, central, and occipital. In addition to the sensors themselves, one can also find landmarks: nasion, inion, and pre-auricular points. An example of such naming is shown in Fig. 4b.
A

<figure><img src="image_146.png" /></figure>

B

<figure><img src="image_147.png" /></figure>

C

<figure><img src="image_148.png" /></figure>

<figure><img src="image_149.png" /><figcaption>**Fig. 4** M/EEG instrumentation. (a) EEG experimental setup. (b) Example of EEG montage. For an illustrative purpose, each color corresponds to a brain area. Each circle represents either a sensor or a landmark. Sensors appear in color while landmarks appear in gray. Sensors are designated with a letter and a number. The letter is indicative of the brain region. Odd numbers correspond to the left hemisphere and even ones to the right hemisphere. (c) MEG experimental setup</figcaption></figure>
**List of Montages** Depending on the scientific question to be addressed and, therefore, the brain areas of interest, different montages can be found. One can build an EEG montage from less than 5 electrodes to up to 256 channels. EEG measurements rely on a difference of electrical potentials. For this purpose, two montages can be considered: the referential montage and the bipolar montage. In the referential montage, each difference of electrical potentials considers an electrode placed over the scalp and a reference. As a result, each electrode placed over the scalp is compared to the reference electrode. The choice of the reference is crucial. The most commonly chosen locations for the reference electrodes are the mastoids (i.e., temporal bone behind the ears), even though several studies prefer placing the reference at the vertex (Cz, i.e., midline central of the scalp). Again, the location depends on the scientific question to be addressed. The bipolar montage consists of performing the difference between two electrodes placed over the scalp, after the experiment. Another electrode, referred to as ground electrode, is used. Among the privileged locations is the scapula (i.e., shoulder blade). An example of an EEG setup and a standard montage are proposed in Fig. 4a, b. For a complete description of the standardized EEG electrode arrays, the reader can refer to [1].

**Future of EEG Hardware** In the past years, there has been an increased interest in developing wearable EEG, to remove wires and to reduce its dimension but also to enable long-lasting recordings in a less constrained environment. Three bottlenecks need to be overcome: the EEG electrodes, hard to put on and to keep in place on the head; the EEG hardware, to make it less power-consuming and miniaturized; and the EEG software, to propose the most intelligible and reliable information regarding the captured brain activity [2]. In particular, EEG systems that rely on dry EEG electrodes get more and more attention. By not requiring conductive gel, it reduces the preparation time. Recent studies relying on commercialized dry electrodes systems show performances close to those obtained with wet electrodes [2].

### 3.1.2 MEG

**Sensors and Main Devices** The difficulty here is to detect signals that are $10^9$ weaker than the Earth magnetic field. The current devices rely on superconducting quantum interference devices (SQUIDs) that can detect small MEG signals [3]. One of the first proof of concept was made by D. Cohen in the 1970s [4]. The SQUIDs present a sensitivity, defined here as the smallest variation of magnetic field that can be detected by the sensor, of $1 \text{ fT}/\sqrt{\text{Hz}}$. To obtain such performance, a magnetic shielding room is required to remove the environmental noise, and a part of the device needs to be cooled via a cryogenic system (see Fig. 4c). Two types of
sensors are used to record MEG signals: magnetometers and gradiometers. Magnetometers measure the magnetic field, whereas the gradiometers measure the gradient of the magnetic field. They are used for noise elimination and consist in a combination of magnetometers. The main difference from one manufacturer to another lies in the type of gradiometers used:

* CTF manufacturer: radial gradiometers consisting of two magnetometers placed one above the other

* MEGIN manufacturer: planar gradiometers consisting of two magnetometers placed side by side

The type of gradiometer has an influence on the way brain activity is recorded and, therefore, on how to interpret the recorded signal [5]. Magnetometers and radial gradiometers are more sensitive to sources around the sensor, whereas planar gradiometers are more sensitive to sources located right below the sensor (Fig. 5).

*New Generation of Sensors* The current devices rely on a cryogenic cooling system that engenders technical and financial constraints. New cryogenic-free sensors have recently emerged: the optically pumped magnetometers (OPMs) [6, 7]. Developing cryogenic-free sensors presents two main advantages: an increase in the amplitude of the signal recorded by the sensor and a reduction of the dimension of the magnetic shielding room. Recent studies proved that OPMs present a better signal-to-noise ratio than EEG [8], can detect deep sources [9], and can be suited for pediatric or movement disorder studies [10]. Promising results could be obtained with triaxial measurements obtained from OPMs [11, 12].

## 3.2 Data Acquisition
Depending on the tasks and on the hardware used, the duration of an M/EEG experiment may vary. This section aims to present the main steps that constitute the data acquisition.

The first step consists in preparing all the materials to perform the experiment. For EEG, it will consist in cleaning the locations where electrodes will be in contact with the skin (e.g., forehead and mastoids). The electrodes and the EEG cap are then placed. Several key distances can be measured to verify that the cap is well-placed or to record fiducial points to be matched to other modalities afterward (e.g., MRI). Then, the experimenter needs to ensure that the communication between the electrodes and the scalp is established. For that purpose, an assessment of the impedance is made for each electrode. The lower it is, the better it is. In the case of wet electrodes, the experimenter has to inject gel at each sensor location. Once the impedances are lower than a certain threshold, typically a few kOhms, then the experiment can start. Regarding MEG, the experimenter places head-tracking coils to measure the head position before each recording. It helps preventing from large
A

<figure><img src="image_150.png" /></figure>

B

<figure><img src="image_151.png" /></figure>

C

<figure><img src="image_152.png" /></figure>

**Fig. 5** Examples of artifacts in M/EEG. (a) Cardiac artifacts recorded with magnetometers. (b) Ocular artifacts recorded with EEG. (c) Power line noise recorded with gradiometers. Given its characteristics, plotting the power spectra enables to elicit it easily. The code to generate this figure is accessible via the dedicated **Github** repository
head movements that could lead to motion artefacts and error in the localization of source activity. The locations of fiducial points (nasion, left and right preauricular points) are registered. The information is stored in each data file. The subject is then placed in the magnetic shielding room after taking off all the elements that could generate magnetic interference with the device (e.g., jewels, belt). The experimenter helps the subject to place his/her head in the MEG helmet. Once the subject is in a comfortable position, the experimenter will save the head position that will be used as reference during the whole session.

Once the subject is correctly installed, the experimenter can start some pre-recordings to check the quality of the signal and give specific instructions to the subjects accordingly (e.g., loosening the jaw to avoid muscular artifacts). Finally, the experimenter can give further instructions regarding the task to perform before starting the recordings. After the end of session, the data are stored in specific servers to be processed.

# 4 Data Analysis

This section aims at providing recommendations for analyzing M/EEG data. An overview of the main steps of the M/EEG data analysis is provided in Fig. 6.

**4.1 Types of Artifacts/Noise**
The notion of artifacts depends strongly on the signal of interest. Here, we consider as artifacts the signals that make the recording more difficult and may hamper the analysis of the brain activity recorded with EEG and/or MEG. Such artifacts can be divided into two categories: the neurophysiological artifacts and the environmental noise. This section aims at presenting their main features.

**4.1.1 Neurophysiological Artifacts**
This category of artifacts corresponds to noise generated by the subjects themselves, whether it is voluntarily or not. In a nutshell, it is important to bear in mind that the brain is far from being the only organ that generates electromagnetic activity. In particular, the eyes and the heart produce electromagnetic activity, which shows an amplitude higher than that of the brain. As a result, the main neurophysiological artifacts are related to cardiac activity and ocular activity (via blinks and saccades) and can be visually spotted out during an M/EEG recording (see Fig. 5a, b). A possible way to reduce the ocular artifacts is to instruct the subject to avoid moving their eyes and, for short recordings only, to avoid eyes blinking. Another neurophysiological artifact may be induced by the subjects' voluntarily motion. Indeed, motion engenders muscular activity that can distort the recorded brain signals. Typical examples are jaw clenching and swallowing. They generate high-frequency
<figure><img src="image_153.png" /></figure>

**Fig. 6** Data analysis in M/EEG: general workflow. QC stands for quality check. Source reconstruction is not compulsory but advisable in specific cases. The code to generate this figure is accessible via the dedicated Github repository
activity that propagates to temporal electrodes. In the specific case of MEG, the device consisting in a helmet, it is strongly sensitive to head motion. A possible way to reduce the muscular artifacts is to instruct the subjects to remain as quiet as possible and to avoid moving their jaws.

### 4.1.2 Environmental Noise

This category refers to the artifacts generated by the environment that surrounds the experimental setup. They can be magnetic (e.g., magnetized devices that can interfere with the MEG sensors), linked with mechanical vibrations (e.g., presence of a tramway nearby), or simply associated with power line (occurring at 50 Hz or 60 Hz; see Fig. 5c). We do not aim at being exhaustive. We simply want the reader to be aware of the possible sources of environmental noise when analyzing M/EEG signals even though the Faraday cage and the shielded room, used respectively, in EEG and MEG, can partly prevent them.

### 4.1.3 System Noise

This category refers to artifacts generated by the sensors themselves. For example, in MEG, one can observe SQUID jumps or saturation. In both MEG and EEG, one can have broken sensors.

## 4.2 Preprocessing

This section aims at presenting the main steps that constitute the preprocessing pipeline, dedicated to artifact removal. This is probably the most crucial part when analyzing M/EEG data. Indeed, the point here is to remove noise without eliminating information of interest or distorting the signal. Attention must be paid to build the pipeline the most suited to the dataset and to the scientific question to be addressed. As such, the first thing to do when working with a new dataset is to extensively study it, in particular, inspecting the M/EEG signals but also the associated broadband power spectra. This preliminary step enables to identify most of the artifacts and, more importantly, if they have a specific temporal and/or frequency signature (e.g., presence of periodic artifacts).

From this point, it is possible to choose a specific strategy to remove the observed noise. In the case of cardiac and ocular artifacts, given their clear pattern, an efficient way to isolate and reduce them consists in applying independent component analysis (ICA) [13]. One can visually identify the components to be removed from both the temporal and the topographies (to avoid removing too many components) and manually select them. Another possibility, more reproducible, consists of using biosignals (e.g., electrocardiogram and electrooculogram) and to compute correlations between time series. This technique enables to ensure the robustness of the decision of removing a component.

In the case of artifacts at a specific frequency (e.g., power line noise at 50 Hz or 60 Hz), one can consider applying notch filters. With the same philosophy, in the case of muscular activity, applying a low-pass filter with a cutoff frequency at 40 Hz can be of interest.
Nevertheless, one objection can be raised: the signal distortion induced by the filtering. As previously explained, here, we aim at finding a trade-off between removing artifacts and preserving the information of interest. That is why the pipeline strongly depends on the scientific question to be addressed. In the case of muscular activity, if someone is interested in the activity in the gamma band (>30 Hz), applying a low-pass filter will be a poor choice, and as such, removing noisy trials can be an option. Regarding head motion, as explained in Subheading 3.2, MEG systems enable to register the head position. Methods relying notably on signal space separation [14] can correct small movements (i.e., less that several centimeters).

Another type of artifacts consists of a broken channel. To avoid having a different number of sensors from one subject to another, the proposed solution depends on the sensor location. If the sensor has four neighbors, strategies relying on the interpolation can be considered. It consists in creating a virtual sensor that is the linear combination of the signals recorded by the broken sensor's neighbors. If the sensor is located on the periphery, the interpolation is no longer reliable. The experimenter may consider removing the channel from the dataset. In the specific case of MEG, after an optional head movement correction step, if SQUID jump artifacts remain, one should consider reapplying the head movement correction on the raw data after having labeled as “bad” the sensors that show jumps. The bad MEG channels will be reconstructed.

Once the pipeline has been chosen and tested, it is important to check that the signals have been correctly preprocessed. This step corresponds to the quality check. There are different possibilities to perform it. The qualitative way would consist in superimposing preprocessed and postprocessed signals (which can be displayed as time series and/or power spectra) and to visualize potential differences. A more reliable way would consist in identifying a judgment criterion to assess to which extent the output signals are noisy. Possible metrics are the variance, the z-score, or the kurtosis. Using one of these metrics on the output may lead to both noisy channels and trials to be discarded. As a rule of thumb, the trial elimination must not exceed 10% of the total number of trials to ensure to have enough data to perform a relevant analysis [15].

## 4.3 Source Reconstruction

It is possible to directly analyze the signals recorded by the sensors. In such a case, one will say that the analysis is performed in the space of the sensors. However, it is also possible to go one step further and estimate the activity within the brain. This processing step is called source reconstruction and consists in estimating the neural correlates of M/EEG signal location. It can be performed when one wants to have access to a higher spatial resolution to provide a more accurate description, and interpretation, of the neurophysiological phenomena occurring. For that purpose, both direct and inverse problems need to be solved [15, 16].
### 4.3.1 Direct Problem
Here, we aim at modeling the electromagnetic field produced by a cerebral source with known characteristics. For that purpose, it is necessary to consider both a physical model of the sources and a model that predicts the way that these sources will generate electromagnetic fields at the scalp level. The simplest model is the spherical model, which considers the head as an ensemble of spheres. Each sphere corresponds to a given tissue (brain, cerebrospinal fluid, skull, or skin) characterized by a given conductivity. Even though it is possible to adjust the spheres to the geometry of the head or restrict them to a limited number of regions of interest, this model is an oversimplification of the head geometry. More realistic models rely on geometrical reconstruction of the different layers that form the head tissues, directly extracted from the anatomical magnetic resonance imaging (MRI) data of, ideally, the participant (the MRI thus needs to be acquired separately) or a dedicated template (e.g., MNI Colin 27). They consist in building meshes of the interfaces between different tissues. We can cite three approaches: the boundary element method (BEM) [17] that is the most widely used, the finite difference method (FDM), and the finite element method (FEM). Another model, called overlapping spheres [18], consists of fitting a given sphere under each sensor.

Even though there are no guidelines regarding the choice of the method, we could provide some elements of recommendations: given the high sensitivity of the EEG toward variations in terms of conductivity, the BEM model can be a tool of choice. As for the MEG, being less sensitive to changes in conductivity, the overlapping spheres can be considered.

### 4.3.2 Inverse Problem
One of the main challenges of the inverse problem lies in the nonuniqueness of its solution. In other words, a large number of brain activity patterns could generate the same signature detected at the sensor level. Therefore, some constraints or assumptions are essential to lead to a unique solution that reflects the best the acquired data [15, 16]. In this section, we aim at providing a short overview of the methods that are the most used in routine.

The dipole modeling methods rely on a source modeling via a reduced number of equivalent dipoles where each of them represents a source activity. As a result, such methods are based on an a priori hypothesis on the required number of sources.

Scanning methods, such as the MUSIC approach [19], consist in estimating the probability of presence of a current dipole inside each voxel. Among them are the beamformer methods [20], which consist in applying a spatial filtering to estimate the source activity at each location. We can cite the linearly constrained minimum variance (LCMV) and the synthetic aperture magnetometry (SAM) [21] as examples of beamformer methods [22].
The approaches relying on distributed source models consist in estimating the amplitudes of dipoles located on the cortical surface. The characteristics of the groups of dipoles are fixed or are estimated via the individual MRI of the participant. The most famous methods relying on distributed sources models are the weighted minimum norm (wMNE) [23, 24] and LORETA [25].

Similar to the preprocessing step, there is no ideal choice of method for the inverse problem, as it depends on the question to be addressed. A general recommendation would be to consider the minimum norm method when expecting distributed sources and the dipole modeling for focal sources.

# 5 Feature Extraction and Selection

When considering M/EEG from the machine learning perspective, an important aspect is the extraction and the selection of the features. This section aims at presenting the main features that can be extracted from M/EEG. As previously mentioned, the selection of the features depends on the scientific question to be addressed but also on the neurophysiological phenomenon underlying the M/EEG experiment. In M/EEG, filtering both in the time domain and in the spatial domain to select the most relevant features is common.

The two main types of features used in the literature rely on the information in the frequency domain and in the time domain. In an effort of completeness, we will see alternative features that reflect the interconnected nature of the brain.

The event-related features consist of chunks of time series concatenated from all the channels, resulting from a low-pass or band-pass filtering and/or from a down-sampling step. This category of features is relevant when considering evoked activity after the presentation of a given stimulus (e.g., visual, auditory, or sensory). They are therefore of interest when one is expecting significant changes in signal amplitudes occurring at a given moment. In the example presented in Subheading 2.2, a positive wave occurred 300 ms after the visual stimulation. One could consider using chunks of time series centered at *t* = 300 ms to detect automatically the P300 wave.

The spectral features are used in the case of the detection of an oscillatory activity (see Subheading 2.2), when changes in M/EEG rhythms amplitudes are expected. The features are associated with the power spectra estimated in a given channel and in a given frequency band for a specific time window. Power spectra can be computed via a plethora of methods; we can notably cite the spectrogram, the Morlet wavelet scalogram, and the auto-regressive models. For a thorough comparison of spectral feature extraction techniques on EEG signals, please refer to [26].
Spatial filtering can be a valuable tool both for event-related and spectral features [27]. It relies on the combination of signals, recorded from different sensors, to obtain a new one, associated with an improved signal-to-noise ratio. We can divide the spatial-filtering methods into three categories. The first one, not data-driven, relies on physical considerations regarding the way the signals propagate through the different brain tissues. The most famous illustration of this category is the Laplacian filter. In its simplest version, the small Laplacian consists, for each electrode location, of a derivation of the EEG waveform via the average signal computed from the four nearest neighbors [28]. The second category of spatial filtering is data-driven and unsupervised. It can rely, for example, on a principal component analysis (PCA) approach (see Chap. 2, Sect. 13.1). The third category is data-driven and supervised. The most famous examples in M/EEG are the common spatial patterns (CSP) for spectral features [29] and xDAWN for event-related features [30]. The CSP consists of a linear combination of EEG signals to maximize the difference between two classes in terms of variance. The xDAWN approach aims at improving the signal-to-noise ratio obtained with evoked potentials via a projection of the raw EEG signals onto an estimated evoked subspace. Recent efforts have been put together to combine approaches to provide ways to optimize simultaneously spectral and spatial filters, with, for example, the filter bank CSP (FBCSP) [31].

Even though spectral and event-related features are the most used in the M/EEG literature, alternative features have been considered in the past years. Firstly, features relying on covariance matrices have recently been extensively used, in particular for Riemannian geometry-based classification [32]. Despite an unclear neurophysiological interpretation, they enabled to reach state-of-the-art performance and to win a large number of competitions. Secondly, new features, which take into account the interconnected nature of brain functioning, have recently emerged [33]. There is a plethora of estimators to assess the intensity of the interactions between brain areas [34]. The most frequent estimators used as features in M/EEG are derived from the coherency, i.e., the normalized cross-spectral density obtained from two signals (e.g., imaginary part of coherence), or rely on the assessment of the phase synchrony between two signals (e.g., phase-locking value (PLV), phase-lag index (PLI)). Here, two challenges need to be dealt with: the volume conduction that can lead to spurious connectivity<fnref n="1" /> and the online implementation. In the first case, even though some estimators, such as the imaginary coherence, are less sensitive to the volume conduction, working in the source space is recommended. In the second case, a large majority of studies that

<footnote n="1">¹ Originating from the mixture of signals engendered by different sources recorded at a given sensor.</footnote>
consider estimators of functional interactions between two brain areas (i.e., functional connectivity estimators) as features are performed offline. Estimating brain interactions in real time is not trivial: it consists in finding the compromise between ensuring the quasi-stationarity of the signals and the statistical reliability of the functional connectivity estimation [33]. Recent studies considered the use of brain network metrics as potential features. Again, there is a plethora of metrics that characterize brain networks [33]. Here, we will cite the most used metrics. At the local scale, the node degree counts the number of connections linking one node to the other. In weighted networks (i.e., without having filtered the connectivity/adjacency matrix), it is referred to as node strength and consists in summing the weights of the connections of the considered node [35]. Another local-scale property of interest is the betweenness centrality defined as the extent to which a node lies “between” other pairs of nodes via the proportion of shortest paths in the network passing through it. This metric enables the identification of the nodes that are crucial for the information transfer between distant regions. At the global scale, we can cite two metrics: the characteristic path length and the clustering coefficient. The characteristic path length indicates the global tendency of the nodes in the network to integrate and exchange information. The clustering coefficient measures the tendency of having nodes’ neighbors mutually interconnected. Lastly, it is worthwhile noting the use of heterogeneous features (e.g., relying on both functional estimators and power spectra) that improves the classification accuracy [27]. Such an approach leads to an increase of the dimension, requiring cautions to select the most relevant features, via dimensional reduction methods.

The feature selection is a crucial step as it prevents redundancy, ensures the reliability of the features, reduces the dimensionality tuned, and helps in providing interpretable results. In this section, we aim at presenting the most popular feature selection methods in the M/EEG domain. For a complete description of the feature selection methods, the reader can refer to [27]. They can be divided into three categories: embedded, filter, and wrapper methods. In filter methods, the feature selection is performed independently and before the evaluation. Different criteria can be chosen to select features. The most popular criterion is the $R^2$ score, which assesses to which extent a given feature is influenced by a task performed by the subject. In wrapper methods, the feature selection utilizes the classification. In other words, in an iterative process, the relevance of each subset of features is assessed via the classification performance until a given criterion is met. The embedded method consists in integrating both the feature selection and the classification in the same process, via a decision tree, for example, or an $\ell_1$ penalty term.
Box 1: Tools for M/EEG analysis

All these tools provide a wide range of tutorials, publicly
available datasets, and codes.

Python-based:

* MNE-Python [36]

* MOABB [37]

MATLAB-based:

* EEGLAB [38]

* Fieldtrip [39]

* Brainstorm [40]

* SPM [41]

6 M/EEG and Brain Disorders

**6.1 Clinical**
**Applications of M/EEG**

The spatial and temporal resolutions of M/EEG enable the obser-
vation of a large number of processes. Notably, they can detect both
evoked responses and oscillatory activity. As such, using these
information could pave the way to biomarkers of brain disorders.
To illustrate this point, we will focus our presentation on two
specific clinical applications: epilepsy and Alzheimer disease. Never-
theless, M/EEG can be useful for a wider range of applications
both in neurological and psychiatric disorders [42, 43].

6.1.1 Epilepsy

Epilepsy is a neurological disorder that presents a high prevalence of 1% [44]. It is established that between 20 and 30% of the patients present a pharmacoresistant form of epilepsy [45]. Among this proportion of patients, only 30% can undergo a surgery [46]. Epilepsy is a distributed disease that induces brain network reorganization and brain rhythm alterations both during ictal and interictal periods [47, 48]. Due to its time resolution compatible with the capture of dynamical changes as well as its wide availability, EEG is a key modality for the evaluation of epilepsy [44]. In addition to scalp EEG, stereotactic-EEG (SEEG) can be used to further localize epileptogenic foci and proven to provide valuable information on epileptogenic networks [48]. MEG can also be used for pre-surgical evaluation and for functional mapping [49], but it is much more costly and less widely available.

The use of network theory in epilepsy provides a useful frame-
work to characterize the seizure (onset and propagation),and its
clinical expression (e.g., comorbidities) [47, 48]. At the local scale,
the node strength or degrees and the betweenness centrality have been used to characterize the epileptic network [48, 50]. At the global scale, two metrics have proven to be of interest in epilepsy [51]: the characteristic path length and the clustering coefficient.

## 6.1.2 Alzheimer Disease

Alzheimer disease is the most common dementia with 60-80% of the cases. The first symptoms are a deficit in short-term memory and concentration, followed later by a decline of linguistic skills, visuospatial orientation, and abstract reasoning judgment. As the pathophysiological process of the disease starts many years before the occurrence of symptoms [52], it is crucial to elicit biomarkers to provide a diagnosis as soon as possible. Efforts have been put together to describe mild cognitive impairment (MCI) and Alzheimer disease (AD) with M/EEG. These studies are essentially focused on oscillatory activity and on interactions between brain areas [53, 54]. In particular, patients present a reduced synchrony [55], and a decrease of the alpha power (i.e., between 8 and 12 Hz) correlates with lower cognitive status and hippocampal atrophy. Studies performed with MEG in preclinical and prodromal stages of AD showed that the effects of amyloid-beta deposition were associated with an increment of the prefrontal alpha power and that altered connectivity in the default mode network was present in normal individuals at risk for AD [56, 57].

A recent EEG work showed that effects of neurodegeneration were focused in frontocentral regions with an increase in high-frequency bands (beta and gamma) and a decrease in lower-frequency bands (delta) [58]. In particular, EEG patterns differ depending on the degree of amyloid burden, suggesting a compensatory mechanism: following a U-shape curve in delta power and an inverted U-shape curve for other tested metrics.

## 6.2 Advanced Uses: The Example of BCI as a Rehabilitation Tool

### 6.2.1 Presentation of the BCI

Brain-computer interfaces (BCIs) consist of acquiring, analyzing, and translating brain signals into commands in real time for control or communication. These systems present a large number of clinical applications and assistive technologies including control of wheelchairs and brain-based communication. BCI devices can be a valuable tool in the treatment of neurological disorders such as stroke [59] and to provide assistive solutions for patients with spinal cord injury [60] or the amyotrophic lateral sclerosis [59]. With regard to the communication, devices such as the P300 Speller, which rely on the evoked response occurring 300 ms after the visual stimulation, allow the users to communicate by selecting letters to form words and even sentences. For an overview of the main steps to be considered when performing a BCI experiment, please refer to Fig. 7.
<figure><img src="image_154.png" /><figcaption>Fig. 7 BCI experiment workflow</figcaption></figure>

6.2.2 BCI as a
Rehabilitation Tool

Stroke is one of the most common neurological conditions. In 2010, stroke was the second leading cause of death worldwide [61]. After a stroke, most patients require rehabilitation and assistance for daily tasks. Motor deficit of the upper limbs affects 70% of the survivors [62], and 85% of those presenting paralysis will have persistent damage [63]. Rapid recovery is observed during the first 3 months (*acute phase*) but can continue for several months after the accident (*chronic phase*) [64]. Motor imagery (MI)-based BCI can constitute a motor substitution in the case of stroke by building alternative pathways from the stimulation to the brain [65]. In this particular case, the system relies on the desynchronization effect associated with a decrease of the power spectra computed within the contralateral sensorimotor area [66]. In a recent meta-analysis [67], the authors observed that rehabilitation to restore upper limb motor function based on BCIs could improve the motricity, assessed via the Fugl-Meyer scale, more than other therapies. A part of the screened studies showed that BCI could induce neuroplasticity.

Brain network changes in stroke patients represent a very promising clinical application of closed-loop systems in rehabilitation strategies. Motor imagery has been proven to be a valuable tool in the study of upper limb recovery after stroke [68]. It enabled observations of changes in ipsilesional intrahemispheric connectivity [69] but also modifications in connectivity in prefrontal areas and correlations between node strengths and motor outcome [70]. Based on previous observations in resting state [71], a recent double-blind study involving ten stroke patients at the chronic stage revealed that node strength, computed from the
### 6.2.3 Current Challenges and Perspectives

ipsilesional primary motor cortex in the alpha band, could be a target for a motor imagery-based neurofeedback and lead to significant improvement on motor performance [72].

Despite being beneficial for patients, controlling a BCI system is a learned skill that 15-30% of the users cannot develop even after several training sessions. This phenomenon, called “BCI inefficiency” [73], has been presented as one of the main limitations to a wider use of BCI. From the machine learning perspective, the main challenges to overcome in current BCI paradigms relying on EEG recordings are the low signal-to-noise ratio of signals, the non-stationarity over time mainly resulting from the difference between calibration and feedback sessions, the reduced amount of available data to train the classifier explained by the number of classes to be discriminated and/or the need to avoid the subject’s tiredness, and the lack of robustness and reliability of the BCI systems, in particular when decoding the users’ mental command.

To tackle these challenges, efforts have been put to improve the classification algorithms. They can be divided into three main groups: the adaptive classifiers, the transfer learning techniques, and the matrix- or tensor-based algorithms. The adaptive classifiers aim at dealing with EEG non-stationarity by taking into account changes in signal properties, and feature distribution, over time. Their parameters are updated when new EEG signals are available [74]. Even though most of the adaptive classifiers can rely on a supervised approach, the unsupervised one has proven to outperform the classifiers that cannot catch temporal dynamics [75]. Besides, it can be a valuable tool to reduce the training duration and potentially to remove the calibration part. Nevertheless, the adaptive classifiers present one main pitfall: their lack of online validation with a user in most of the current literature. This leads to two potential issues: the difficulty to find a trade-off between fully retraining the classifier and updating some key parameters and the adaptation that may not follow the actual user's intent by being too fast or too slow [76].

Transfer learning consists here in exploiting changes in EEG signal properties over time and subjects to extract knowledge. More specifically, it relies on learned classifiers that are trained on one task (called domain here) and are adapted to another task with little or no new training data [77]. For example, it can be applied to a dataset formed by two motor imagery tasks performed by two different subjects. There is plethora of methods to solve the transfer learning problem [78]. The most common in the EEG-based BCI domain consists in learning the transformation to correct the mismatch between the domains, occurring when one domain corresponds to a hand motor imagery and the other to a foot motor imagery, for instance, finding a common feature representation for the domains, or learning a transformation of the data to make their
distribution match [27]. Despite its robustness and recent advances in proposing guidelines [79], there is a lack of online experiments relying on transfer learning to fully validate this approach and assess to which extent it can be beneficial to patients.

Among the classification methods relying on matrices and tensors, the most well-known is the Riemannian geometry-based one. One of the main original characteristics of this approach is that it is able to manipulate and classify the data by representing them as symmetric positive definite matrices, such as covariance matrices, and by mapping them onto a dedicated geometrical space, involving less steps than the classic approaches. This approach relies on the assumption that the sources are specific of a given task encoded via the covariance matrix computed from EEG signals. Here, trials are classified via nearest neighbor methods relying on the Riemannian distance and the geometric mean. With the method relying on the minimum distance to mean (MDM), each class is associated with a geometric mean computed from the training data. Then, the MDM will attribute an unlabeled trial to the class showing the closest mean [80]. The Riemannian approaches present many advantages: they can be applied to all BCI paradigms, no parameter tuning is required, they are robust to noise, and, combined to transfer learning methods, they can lead to calibration-free BCI sessions [81]. In particular, Riemannian geometry-based methods [80, 82] are now the state of the art in terms of performance [27] and have won several data competitions<fnref n="2" /> [83].

## Box: 2 To go further

### Guidelines and books of reference

* Hari, M., and Puce, A. (2017). MEG-EEG Primer. In MEG-EEG Primer. Oxford University Press.

* M. Clerc, L. Bougrain, and F. Lotte. (2016) Brain-Computer Interfaces 1: Methods and Perspectives, Wiley.

* M. Clerc, L. Bougrain, and F. Lotte. (2016) Brain-Computer Interfaces 2: Technology and Applications, Wiley.

* Gross, J., Baillet, S., Barnes, G. R., Henson, R. N., Hillebrand, A., Jensen, O., Jerbi, K., Litvak, V., Maess, B., Oostenveld, R., Parkkonen, L., Taylor, J. R., van Wassenhove, V., Wibral, M., and Schoffelen, J.-M. (2013). Good practice for conducting and reporting MEG research. Neuroimage, 65, 349–363.

(continued)

<footnote n="2">² See, for example, the 6 competitions won by A. Barachant: http://alexandre.barachant.org/challenges/.</footnote>
**Box 2 (continued)**

* Puce, A. and Hämäläinen, M. S. (2017). A Review of Issues Related to Data Acquisition and Analysis in EEG/MEG Studies. Brain Sci, 7(6).

# 7 Conclusion

EEG and MEG are key modalities for the study of brain disorders. In particular, EEG is relatively cheap and widely available and is thus a widely used tool in neurology. When dealing with EEG and MEG data, it is important to understand the origin of the signals as well as the different steps in their preprocessing and feature extraction. Machine learning is increasingly used on EEG and MEG data, in particular for BCI but also for computer-aided diagnosis and prognosis of brain disorders.

# Acknowledgements

The author acknowledges support from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 864729) and from the program "Investissements d'avenir" ANR-10-IAIHU-06. The author thanks D. Schwartz for his suggestions and insightful comments.

# References

1. Seeck M, Koessler L, Bast T, Leijten F, Michel C, Baumgartner C, He B, Beniczky S (2017) The standardized EEG electrode array of the IFCN. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neurophysiology 128(10): 2070-2077. https://doi.org/10.1016/j.clinph.2017.06.254

2. Casson AJ (2019) Wearable EEG and beyond. Biomed Eng Lett 9(1):53-71. https://doi.org/10.1007/s13534-018-00093-6. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6431319/

3. Hämäläinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV (1993) Magnetoencephalography-theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys 65(2):413-497. https://doi.org/10.1103/RevModPhys.65.413. https://link.aps.org/doi/10.1103/RevModPhys.65.413

4. Cohen D (1972) Magnetoencephalography: detection of the brain's electrical activity with a superconducting magnetometer. Science 175(4022):664-666

5. Vrba J, Robinson SE (2001) Signal Processing in Magnetoencephalography. Methods 25(2): 249-271. https://doi.org/10.1006/meth.2001.1238. https://www.sciencedirect.com/science/article/pii/S1046202301912381

6. Corsi MC (2015) Magnétomètres à pompage optique à Hélium 4: développement et preuve de concept en magnétocardiographie et en magnétoencéphalographie. PhD thesis, Grenoble Alpes. http://www.theses.fr/2015GREAT082
7. Tierney TM, Holmes N, Mellor S, López JD, Roberts G, Hill RM, Boto E, Leggett J, Shah V, Brookes MJ, Bowtell R, Barnes GR (2019) Optically pumped magnetometers: From quantum origins to multi-channel magnetoencephalography. NeuroImage 199: 598-608. https://doi.org/10.1016/j.neuroimage.2019.05.063. http://www.sciencedirect.com/science/article/pii/S1053811919304550

8. Boto E, Seedat ZA, Holmes N, Leggett J, Hill RM, Roberts G, Shah V, Fromhold TM, Mullinger KJ, Tierney TM, Barnes GR, Bowtell R, Brookes MJ (2019) Wearable neuroimaging: combining and contrasting magnetoencephalography and electroencephalography. NeuroImage 201:116099. https://doi.org/10.1016/j.neuroimage.2019.116099. http://www.sciencedirect.com/science/article/pii/S1053811919306901

9. Barry DN, Tierney TM, Holmes N, Boto E, Roberts G, Leggett J, Bowtell R, Brookes MJ, Barnes GR, Maguire EA (2019) Imaging the human hippocampus with optically-pumped magnetoencephalography. NeuroImage 203: 116192. https://doi.org/10.1016/j.neuroimage.2019.116192. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6854457/

10. Boto E, Holmes N, Leggett J, Roberts G, Shah V, Meyer SS, Muñoz LD, Mullinger KJ, Tierney TM, Bestmann S, Barnes GR, Bowtell R, Brookes MJ (2018) Moving magnetoencephalography towards real-world applications with a wearable system. Nature 555(7698):657-661. https://doi.org/10.1038/nature26147. https://www.nature.com/articles/nature26147

11. Brookes MJ, Boto E, Rea M, Shah V, Osborne J, Holmes N, Hill RM, Leggett J, Rhodes N, Bowtell R (2021) Theoretical advantages of a triaxial optically pumped magnetometer magnetoencephalography system. NeuroImage 236:118025. https://doi.org/10.1016/j.neuroimage.2021.118025. https://www.sciencedirect.com/science/article/pii/S1053811921003025

12. Labyt E, Corsi MC, Fourcault W, Laloy AP, Bertrand F, Lenouvel F, Cauffet G, Prado ML, Berger F, Morales S (2019) Magnetoencephalography with optically pumped 4He magnetometers at Ambient temperature. IEEE Trans Med Imaging 38(1):90-98. https://doi.org/10.1109/TMI.2018.2856367. Conference name: IEEE Trans Med Imaging

13. Bell AJ, Sejnowski TJ (1995) An information-maximization approach to blind separation and blind deconvolution. Neural Comput 7(6): 1129-1159. https://doi.org/10.1162/neco.1995.7.6.1129

14. Taulu S, Simola J (2006) Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements. Phys Med Biol 51(7):1759-1768. https://doi.org/10.1088/0031-9155/51/7/008

15. Gross J, Baillet S, Barnes GR, Henson RN, Hillebrand A, Jensen O, Jerbi K, Litvak V, Maess B, Oostenveld R, Parkkonen L, Taylor JR, van Wassenhove V, Wibral M, Schoffelen JM (2013) Good practice for conducting and reporting MEG research. Neuroimage 65: 349-363. https://doi.org/10.1016/j.neuroimage.2012.10.001

16. Baillet S, Mosher J, Leahy R (2001) Electromagnetic brain mapping. IEEE Signal Process Mag 18(6):14-30. https://doi.org/10.1109/79.962275

17. Fuchs M, Wagner M, Kastner J (2001) Boundary element method volume conductor models for EEG source reconstruction. Clin Neurophysiol 112(8):1400-1407. https://doi.org/10.1016/S1388-2457(01)00589-2. http://www.sciencedirect.com/science/article/pii/S1388245701005892

18. Huang MX, Mosher JC, Leahy RM (1999) A sensor-weighted overlapping-sphere head model and exhaustive head model comparison for MEG. Phys Med Biol 44(2):423-440. https://doi.org/10.1088/0031-9155/44/2/010

19. Mosher J, Baillet S, Leahy R (1999) EEG source localization and imaging using multiple signal classification approaches. J Clin Neurophysiol 16(3):225-238. https://doi.org/10.1097/00004691-199905000-00004

20. Hillebrand A, Barnes GR (2005) Beamformer Analysis of MEG Data. In: International review of neurobiology, magnetoencephalography, vol 68. Academic Press, New York, pp 149-171. https://doi.org/10.1016/S0074-7742(05)68006-3. https://www.sciencedirect.com/science/article/pii/S0074774205680063

21. Robinson S, Vrba J (1999) Functional neuroimaging by synthetic aperture magnetometry (SAM)—ScienceOpen. Recent advances in Biomagnetism. Tohoku University Press, Sendai, pp 302-305. https://www.scienceopen.com/document?vid=067e8134-c846-4b5f-9ae1-656869ca8727

22. Van Veen B, Van Drongelen W, Yuchtman M, Suzuki A (1997) Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans Biomed Eng 44(9):867-880. https://doi.org/10.1109/10.623056
23. Fuchs M, Wagner M, Köhler T, Wischmann HA (1999) Linear and nonlinear current density reconstructions. Journal of Clinical Neurophysiology: Official Publication of the American Electroencephalographic Society 16(3):267-295

24. Lin FH, Witzel T, Ahlfors SP, Stufflebeam SM, Belliveau JW, Hämäläinen MS (2006) Assessing and improving the spatial accuracy in MEG source localization by depth-weighted minimum-norm estimates. NeuroImage 31(1):160-171. https://doi.org/10.1016/j.neuroimage.2005.11.054

25. Pascual-Marqui RD (2002) Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find Exp Clin Pharmacol 24(Suppl D):5-12

26. Brodu N, Lotte F, Lécuyer A (2011) Comparative study of band-power extraction techniques for Motor Imagery classification. In: 2011 IEEE symposium on computational intelligence, cognitive algorithms, mind, and brain (CCMB), pp 1-6. https://doi.org/10.1109/CCMB.2011.5952105

27. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F (2018) A review of classification algorithms for EEG-based brain-computer interfaces: a 10-year update. J Neural Eng 15(3):031005. https://doi.org/10.1088/1741-2552/aab2f2

28. McFarland DJ, McCane LM, David SV, Wolpaw JR (1997) Spatial filter selection for EEG-based communication. Electroencephalogr Clin Neurophysiol 103(3):386-394. https://doi.org/10.1016/S0013-4694(97)00022-2. http://www.sciencedirect.com/science/article/pii/S0013469497000222

29. Ramoser H, Muller-Gerking J, Pfurtscheller G (2000) Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 8(4):441-446. https://doi.org/10.1109/86.895946

30. Rivet\* B, Souloumiac A, Attina V, Gibert G (2009) xDAWN Algorithm to enhance evoked potentials: application to brain-computer interface. IEEE Trans Biomed Eng 56(8): 2035-2043. https://doi.org/10.1109/TBME.2009.2012869. Conference name: IEEE Trans Biomed Eng

31. Ang KK, Chin ZY, Zhang H, Guan C (2008) Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pp 2390-2397. https://doi.org/10.1109/IJCNN.2008.4634130. ISSN: 2161-4407

32. Yger F, Berar M, Lotte F (2017) Riemannian approaches in brain-computer interfaces: a review. IEEE Trans Neural Syst Rehabil Eng 25(10):1753-1762. https://doi.org/10.1109/TNSRE.2016.2627016

33. Gonzalez-Astudillo J, Cattai T, Bassignana G, Corsi MC, De Vico Fallani F (2020) Network-based brain computer interfaces: principles and applications. J Neural Eng 18(1):011001. https://doi.org/10.1088/1741-2552/abc760. http://iopscience.iop.org/article/10.1088/1741-2552/abc760

34. Bastos AM, Schoffelen JM (2016) A tutorial review of functional connectivity analysis methods and their interpretational pitfalls. Front Syst Neurosci 9:175. https://doi.org/10.3389/fnsys.2015.00175. https://www.frontiersin.org/articles/10.3389/fnsys.2015.00175/full

35. Fornito A, Zalesky A, Bullmore E (2016) Fundamentals of brain network analysis, reprint edizione edn. Academic Press, Amsterdam

36. Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, Parkkonen L, Hämäläinen MS (2014) MNE software for processing MEG and EEG data. NeuroImage 86:446-460. https://doi.org/10.1016/j.neuroimage.2013.10.027

37. Jayaram V, Barachant A (2018) MOABB: trustworthy algorithm benchmarking for BCIs. J Neural Eng 15(6):066011. https://doi.org/10.1088/1741-2552/aadea0. Publisher: IOP Publishing

38. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134(1):9-21. https://doi.org/10.1016/j.jneumeth.2003.10.009

39. Oostenveld R, Fries P, Maris E, Schoffelen JM (2010) FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011:e156869. https://doi.org/10.1155/2011/156869

40. Tadel F, Baillet S, Mosher J, Pantazis D, Leahy R (2011) Brainstorm: a user-firendly application for MEG/EEG analysis. Comput Intell Neurosci 2011:1-13. https://doi.org/10.1155/2011/879716

41. Penny WD, Friston KJ, Ashburner JT, Kiebel SJ, Nichols TE (eds) (2006) Statistical parametric mapping: the analysis of functional brain images, illustrated edn. Academic Press, Amsterdam

42. Hari R, Baillet S, Barnes G, Burgess R, Forss N, Gross J, Hämäläinen M, Jensen O, Kakigi R,
Mauguière F, Nakasato N, Puce A, Romani GL, Schnitzler A, Taulu S (2018) IFCN-endorsed practical guidelines for clinical magnetoencephalography (MEG). Clin Neurophysiol 129(8):1720-1747. https://doi.org/10.1016/j.clinph.2018.03.042. http://www.sciencedirect.com/science/article/pii/S1388245718306576

43. Boutros NN, Galderisi S, Pogarell O, Riggio S (2011) Standard electroencephalography in clinical psychiatry: a practical handbook. Wiley, London

44. des Enseignants de Neurologie C (2016) Épilepsies de l'enfant et de l'adulte. https://www.cen-neurologie.fr/deuxieme-cycle/epilepsies-enfant-l-adulte

45. Kwan P, Brodie MJ (2000) Early identification of refractory epilepsy. N Engl J Med 342(5): 314-319. https://doi.org/10.1056/NEJM200002033420503

46. Micoulaud-Franchi JA, Lanteaume L, Pallanca O, Vion-Dury J, Bartolomei F (2014) Biofeedback et épilepsie pharmacorésistante: le retour d'une thérapeutique ancienne? Rev Neurol 170(3):187-196. https://doi.org/10.1016/j.neurol.2013.10.011

47. Diessen Ev, Diederen SJH, Braun KPJ, Jansen FE, Stam CJ (2013) Functional and structural brain networks in epilepsy: What have we learned? Epilepsia 54(11):1855-1865. https://doi.org/10.1111/epi.12350. https://onlinelibrary.wiley.com/doi/abs/10.1111/epi.12350

48. Bartolomei F, Lagarde S, Wendling F, McGonigal A, Jirsa V, Guye M, Bénar C (2017) Defining epileptogenic networks: contribution of SEEG and signal analysis. Epilepsia 58(7):1131-1147. https://doi.org/10.1111/epi.13791. https://onlinelibrary.wiley.com/doi/abs/10.1111/epi.13791

49. Stefan H, Trinka E (2017) Magnetoencephalography (MEG): past, current and future perspectives for improved differentiation and treatment of epilepsies. Seizure 44:121-124. https://doi.org/10.1016/j.seizure.2017.06.10.028. http://www.sciencedirect.com/science/article/pii/S1059131116302217

50. Wilke C, Worrell G, He B (2011) Graph analysis of epileptogenic networks in human partial epilepsy. Epilepsia 52(1):84-93. https://doi.org/10.1111/j.1528-1167.2010.02785.x

51. Bartolomei F, Bettus G, Stam CJ, Guye M (2013) Interictal network properties in mesial temporal lobe epilepsy: a graph theoretical study from intracerebral recordings. Clinical Neurophysiology: Official Journal of the International Federation of Clinical

Neurophysiology 124(12):2345-2353. https://doi.org/10.1016/j.clinph.2013.06.003

52. Bateman RJ, Xiong C, Benzinger TLS, Fagan AM, Goate A, Fox NC, Marcus DS, Cairns NJ, Xie X, Blazey TM, Holtzman DM, Santacruz A, Buckles V, Oliver A, Moulder K, Aisen PS, Ghetti B, Klunk WE, McDade E, Martins RN, Masters CL, Mayeux R, Ringman JM, Rossor MN, Schofield PR, Sperling RA, Salloway S, Morris JC, Dominantly Inherited Alzheimer Network (2012) Clinical and biomarker changes in dominantly inherited Alzheimer's disease. N Engl J Med 367(9): 795-804. https://doi.org/10.1056/NEJMoa1202753

53. Babiloni C, Lizio R, Marzano N, Capotosto P, Soricelli A, Triggiani AI, Cordone S, Gesualdo L, Del Percio C (2016) Brain neural synchronization and functional coupling in Alzheimer's disease as revealed by resting state EEG rhythms. International Journal of Psychiatry: Official Journal of the International Organization of Psychophysiology 103: 88-102. https://doi.org/10.1016/j.ijpsycho.2015.02.008

54. Engels MMA, van der Flier WM, Stam CJ, Hillebrand A, Scheltens P, van Straaten ECW (2017) Alzheimer's disease: the state of the art in resting-state magnetoencephalography. Clinical Neurophysiology 128(8):1426-1437. https://doi.org/10.1016/j.clinph.2017.05.012. https://www.sciencedirect.com/science/article/pii/S1388245717301979

55. Stam CJ (2010) Use of magnetoencephalography (MEG) to study functional brain networks in neurodegenerative disorders. J Neurol Sci 289(1):128-134. https://doi.org/10.1016/j.jns.2009.08.028. https://www.jns-journal.com/article/S0022-510X(09)00784-9/abstract

56. Nakamura A, Cuesta P, Kato T, Arahata Y, Iwata K, Yamagishi M, Kuratsubo I, Kato K, Bundo M, Diers K, Fernández A, Maestú F, Ito K (2017) Early functional network alterations in asymptomatic elders at risk for Alzheimer's disease. Sci Rep 7(1):6517. https://doi.org/10.1038/s41598-017-06876-8

57. Nakamura A, Cuesta P, Fernández A, Arahata Y, Iwata K, Kuratsubo I, Bundo M, Hattori H, Sakurai T, Fukuda K, Washimi Y, Endo H, Takeda A, Diers K, Bajo R, Maestú F, Ito K, Kato T (2018) Electromagnetic signatures of the preclinical and prodromal stages of Alzheimer's disease. Brain J Neurol 141(5): 1470-1485. https://doi.org/10.1093/brain/awy044
58. Gaubert S, Raimondo F, Houot M, Corsi MC, Naccache L, Diego Sitt J, Hermann B, Oudiette D, Gagliardi G, Habert MO, Dubois B, De Vico Fallani F, Bakardjian H, Epelbaum S (2019) EEG evidence of compensatory mechanisms in preclinical Alzheimer's disease. Brain 142(7):2096-2112. https://doi.org/10.1093/brain/awz150. https://aca.demic.oup.com/brain/article/142/7/2096/5519996

59. Kashihara K (2014) A brain-computer interface for potential non-verbal facial communication based on EEG signals related to specific emotions. Front Neurosci 8:244. https://doi.org/10.3389/fnins.2014.00244

60. King CE, Wang PT, Chui LA, Do AH, Nenadic Z (2013) Operation of a brain-computer interface walking simulator for individuals with spinal cord injury. J Neuroeng Rehabil 10:77. https://doi.org/10.1186/1743-0003-10-77

61. Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, Moran AE, Sacco RL, Anderson L, Truelsen T, O'Donnell M, Venketasubramanian N, Barker-Collo S, Lawes CMM, Wang W, Shinohara Y, Witt E, Ezzati M, Naghavi M, Murray C (2014) Global and regional burden of stroke during 1990-2010: findings from the Global Burden of Disease Study 2010. Lancet 383(9913):245-254. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4181600/

62. Houwink A, Roorda LD, Smits W, Molenaar IW, Geurts AC (2011) Measuring upper limb capacity in patients after stroke: reliability and validity of the stroke upper limb capacity scale. Arch Phys Med Rehabil 92(9):1418-1422. https://doi.org/10.1016/j.apmr.2011.03.028. https://linkinghub.elsevier.com/retrieve/pii/S0003999311002218

63. Kavanagh S, Knapp M, Patel A (1999) Costs and disability among stroke patients. J Public Health 21(4):385-394. https://doi.org/10.1093/pubmed/21.4.385. https://academic.oup.com/jpubhealth/article-lookup/doi/10.1093/pubmed/21.4.385

64. de Santé HA (2012) Accident vasculaire cérébral: méthodes de rééducation de la fonction motrice chez l'adulte. Tech. rep. https://www.has-sante.fr/jcms/c_1334330/fr/accident-vasculaire-cerebral-methodes-de-reéducation-de-la-fonction-motrice-chez-l-adulte

65. Prasad G, Herman P, Coyle D, McDonough S, Crosbie J (2010) Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study. J Neuroeng Rehabil 7:60. https://doi.org/10.1186/1743-0003-7-60

66. Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110(11):1842-1857. https://doi.org/10.1016/S1388-2457(99)00141-8. http://www.sciencedirect.com/science/article/pii/S1388245799001418

67. Cervera MA, Soekadar SR, Ushiba J, Millán JdR, Liu M, Birbaumer N, Garipelli G (2018) Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann Clin Transl Neurol 5(5):651-663. https://doi.org/10.1002/acn3.544. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5945970/

68. Sharma N, Pomeroy VM, Baron JC (2006) Motor imagery: a backdoor to the motor system after stroke? Stroke 37(7):1941-1952. 10.1161/01.STR.0000226902.43357.fc

69. Pichiorri F, Morone G, Petti M, Toppi J, Pisotta I, Molinari M, Paolucci S, Inghilleri M, Astolfi L, Cincotti F, Mattia D (2015) Brain-computer interface boosts motor imagery practice during stroke recovery. Ann Neurol 77(5):851-865. https://doi.org/10.1002/ana.24390. https://www.readcube.com/articles/10.1002/ana.24390

70. Sharma N, Baron JC, Rowe JB (2009) Motor imagery after stroke: relating outcome to motor network connectivity. Ann Neurol 66(5):604-616. https://doi.org/10.1002/ana.21810. https://onlinelibrary.wiley.com/doi/abs/10.1002/ana.21810

71. Dubovik S, Pignat JM, Ptak R, Aboulafia T, Allet L, Gillabert N, Magnin C, Albert F, Momjian-Mayor I, Nahum L, Lascano AM, Michel CM, Schnider A, Guggisberg AG (2012) The behavioral significance of coherent resting-state oscillations after stroke. NeuroImage 61(1):249-257. https://doi.org/10.1016/j.neuroimage.2012.03.024

72. Mottaz A, Corbet T, Doganci N, Magnin C, Nicolo P, Schnider A, Guggisberg AG (2018) Modulating functional connectivity after stroke with neurofeedback: effect on motor deficits in a controlled cross-over study. NeuroImage: Clinical 20:336-346. https://doi.org/10.1016/j.nicl.2018.07.029. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6091229/

73. Allison BZ, Neuper C (2010) Could Anyone Use a BCI? In: Tan DS, Nijholt A (eds) Brain-computer interfaces, human-computer interaction series. Springer London, pp 35-54. http://link.springer.com/chapter/10.1007/978-1-84996-272-8_3

74. Schlögl A, Vidaurre C, Müller KR (2010) Adaptive methods in BCI research—an
introductory tutorial. In: Graimann B, Pfurtscheller G, Allison B (eds) Brain-computer interfaces: revolutionizing human-computer interaction, The Frontiers Collection. Springer, Berlin, pp 331–355. https://doi.org/10.1007/978-3-642-02091-9_18

75. Vidaurre C, Kawanabe M, von Bünau P, Blankertz B, Müller KR (2011) Toward unsupervised adaptation of LDA for brain-computer interfaces. IEEE Trans Biomed Eng 58(3):587–597. https://doi.org/10.1109/TBME.2010.2093133

76. Müller JS, Vidaurre C, Schreuder M, Meinecke FC, von Bünau P, Müller KR (2017) A mathematical model for the two-learners problem. J Neural Eng 14(3):036005. https://doi.org/10.1088/1741-2552/aa620b

77. Waytowich NR, Lawhern VJ, Bohannon AW, Ball KR, Lance BJ (2016) Spectral transfer learning using information geometry for a user-independent brain-computer interface. Front Neurosci 10:430. https://doi.org/10.3389/fnins.2016.00430. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5032911/

78. Pan SJ, Yang Q (2010) A Survey on Transfer Learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/TKDE.2009.191

79. Bougrain L, Rimbert S, Rodrigues PLC, Canron G, Lotte F (2021) Guidelines to use transfer learning for motor imagery detection: an experimental study. In: 2021 10th international IEEE/EMBS conference on neural engineering (NER), pp 5–8. https://doi.org/10.1109/NER49283.2021.9441254. ISSN: 1948-3554

80. Yger F, Berar M, Lotte F (2016) Riemannian approaches in brain-computer interfaces: a review. IEEE Trans Neural Syst Rehabil Eng 25(10):1753–1762

81. Andreev A, Barachant A, Lotte F, Congedo M (2016) Recreational applications of OpenViBE: brain invaders and use-the-force, vol chap. 14. Wiley, London, p 241. https://hal.archives-ouvertes.fr/hal-01366873

82. Congedo M, Barachant A, Bhatia R (2017) Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review. Brain-Computer Interfaces 4(3):155–174

83. Corsi MC, Yger F, Chevallier S, Noûs C (2021) Riemannian geometry on connectivity for clinical BCI. In: ICASSP 2021–2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 980–984. https://doi.org/10.1109/ICASSP39728. 2021.9414790, ISSN: 2379-190X

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
**Working with Omics Data: An Interdisciplinary Challenge at the Crossroads of Biology and Computer Science**

Thibault Poinsignon, Pierre Poulain, Mélina Gallopin, and Gaëlle Lelandais

**Abstract**

Nowadays, generating omics data is a common activity for laboratories in biology. Experimental protocols to prepare biological samples are well described, and technical platforms to generate omics data from these samples are available in most research institutes. Furthermore, manufacturers constantly propose technical improvements, simultaneously decreasing the cost of experiments and increasing the amount of omics data obtained in a single experiment. In this context, biologists are facing the challenge of dealing with large omics datasets, also called “big data” or “data deluge.” Working with omics data raises issues usually handled by computer scientists, and thus cooperation between biologists and computer scientists has become essential to efficiently study cellular mechanisms in their entirety, as omics data promise. In this chapter, we define omics data, explain how they are produced, and, finally, present some of their applications in fundamental and medical research.

**Key words** Genomics, Transcriptomics, Proteomics, Metabolomics, Big data, Computer science, Bioinformatics

# 1 Introduction

There are different types of omics data, each revealing an aspect of cell complexity. To illustrate this complexity, we propose in Fig. 1 an analogy between the functions of a cell and that of a factory. The different omics data types are replaced there, in their specific context. Cells are the building blocks of living organisms. They can be pictured as microscopic, automated factories, made up of thousands of biological molecules (or molecular components) that work together to perform specific functions. Basically, there are four main types of molecular components: DNA, RNA, proteins, and metabolites. The whole population of one type of cellular component is named with the suffix -ome, i.e., *genome* (DNA), *transcriptome* (RNA), *proteome* (proteins), and *metabolome* (metabolites) (*see* Fig. 1). The scientific fields, which aim at studying those respective populations, are named with the suffix -omics,
<figure><img src="image_155.png" /><figcaption>**Fig. 1** The four main -omes and an analogy of their functions. The genome designates all cell's DNA molecules. The transcriptome, the proteome, and the metabolome refer, respectively, to the cell's whole set of RNA, proteins, or metabolites at a given time</figcaption></figure>
i.e., *genomics*, *transcriptomics*, *proteomics*, and *metabolomics*. The common point between the different types of omics data is that they all arise from high-throughput experimental strategies that allow the simultaneous observation of all individual components that constitute either the genome, the transcriptome, the proteome, or the metabolome [1].

The genome is made of DNA molecules, which are the carrier of genetic information. It can be imagined as the blueprint library of the cell (see Fig. 1). From a chemical point of view, DNA molecules are polymers (or sequences) of simpler chemical units called nucleotides. There are four main types of nucleotides: adenosine (A), thymine (T), cytosine (C), and guanine (G). DNA molecules are organized into chromosomes, which are compacted in the cell nucleus. The genome is directly connected to the transcriptome and the proteome (see next sections). The information to synthesize RNA molecules (transcriptome) and proteins (proteome) is encoded in specific regions of the DNA sequence called genes (see Fig. 1). Genes are made of successive nucleotides (clustered into codons), which correspond to amino acids, i.e., the molecules that constitute the proteins. The correspondence between nucleotides, codons, and amino acids is known as the genetic code. To summarize, a genomics dataset thus contains the sequences of DNA molecules present in a cell (or a population of cells) and can be seen as a copy of the cell's blueprint library (its genome) written as a long sequence of A, T, C, and G.

The transcriptome is made of RNA molecules. Multiple types exist, and they can be roughly classified into messenger RNA (mRNA), ribosomal RNA (rRNA), transfer RNA (tRNA), and non-coding RNA (ncRNA). Transcriptomics datasets mainly focus on mRNAs, which are the intermediate messengers between the genome and the proteome (see previous paragraph). The transcriptome is thus intimately connected to the genome and the proteome (see Fig. 1). Notably, the RNA polymerase is required to generate mRNA, reading the genome during transcription. In eukaryotes, mRNAs exit the nucleus to be used as templates by ribosomes (a macromolecular complex made of rRNA and proteins), to synthesize proteins by assembling amino acids (following the genetic code) during translation. Compared to the genome, the transcriptome is much more dynamic. The cell population of mRNA molecule varies according to cell requirement in proteins, and a transcriptomics dataset lists all sequences of mRNA present at a given time. They can be seen as snapshots of which parts of the genome are currently transcribed and in which proportion. Following up on the genome analogy presented in Fig. 1, mRNAs can be seen as active copies of the cell's blueprints that are more or less actively used.
The proteome is made of proteins, i.e., macromolecules made with one or several polymers of amino acids. Proteins are extraordinarily diverse in their three-dimensional (3D) conformations and associated functions. To illustrate this diversity, some proteins constitute the backbone of the cell structure, others detect or transmit external or internal chemical signals, and a large portion of them (enzymes) catalyze chemical reactions of the metabolism (the whole set of chemical reactions sustaining the cell). Proteins are also responsible for the regulation and expression (transcription and translation) of the genetic information (see previous paragraph). Protein functions are closely linked to their 3D spatial conformation, and all processes of the cells are based on protein activities (see Fig. 1). The proteome is as dynamic as the transcriptome because the set of proteins present at a given time in a cell varies accordingly to the current state and function of this cell. Proteomics datasets give a snapshot of which proteins are present at a given moment in the life of the cell. Genomics, transcriptomics, and proteomics resume the classical central dogma of biology, as first stated by Francis Crick in 1957. Even if it has been further detailed since, with, for instance, a better understanding of epigenomics, it still effectively summarizes the principal flow of information between the main molecular components of the cell: DNA is transcribed into RNA which is translated into proteins.

To end this description of omics data types, we believe it is important to mention the metabolome (see Fig. 1). The metabolome is made of metabolites, small molecules that are protein substrates in chemical reactions. Nucleotides and amino acids, cited before, are metabolites, as well as other molecules like lipids (forming bilayer membranes that compartmentalize the cell) or ATP (a molecule used as intracellular energy transfer). To extend, again, the analogy, metabolites can be seen as the raw materials used by the automated microscopic factory (see Fig. 1). Metabolomics datasets peek into the population of metabolites in a cell at a given time. Again, it is important to specify that if each cited “omics” field gives an assessment of its associated “ome” population, it is quite a “blurred” one. Everything is intertwined in a cell. Moreover, most omics studies give only an average observation on a population of cells. Multi-omics and single-cell techniques are trying to overcome these limitations.

In this chapter, we detail the different types of files used for omics data and present examples of databases where they are stored. We introduce different methods for generating omics data and finally provide some applications of omics data in fundamental research, cancer research, and pandemic response.
# 2 What Are Omics Data?

## 2.1 Results from
High-Throughput
Studies Written in
Multiple Binary and
Text Files

To describe the files used to store omics information, it is necessary to consider genomics and transcriptomics on one side and proteomics and metabolomics on the other side. Indeed, these files are generated by different experimental techniques, which are, respectively, sequencing (for genomics and transcriptomics) and mass spectrometry (for proteomics and metabolomics) (see Fig. 2). For each group, two types of files must be distinguished: the ones that are directly obtained after the applications of experimental protocols, i.e., the raw omics data files, and the ones that are generated by downstream informatic analyses, i.e., the processed omics data files (see Fig. 2). Experimental protocols and the informatic treatments applied to raw data files will be detailed in the next section.

Genomics and transcriptomics raw data files are essentially nucleotide sequence files. In that respect, the FASTA and the FASTQ text formats are commonly used. FASTA was created by Lipman and Pearson in 1985 as an input for their software [2] and became a de facto standard, without any clear statement acknowledging it [3]. This probably explains the absence of a common file extension (e.g., .bast, .fna, .faa) even if FASTA is a unified file type. FASTA files contain one or several sequences. A sequence begins with a description line starting with the character ">". NCBI databases (see next sections) have unified rules to write this line.<fnref n="1" />

<figure><img src="image_156.png" /><figcaption>**Fig. 2** Omics data are assessments of -ome populations. Raw omics data are generated through sequencing (for DNA and cDNA) or mass spectrometry (for proteins and metabolites)</figcaption></figure>

<footnote n="1">¹ https://www.ncbi.nlm.nih.gov/genbank/fastafORMAT/</footnote>
Subsequent lines contain the sequence itself split into multiple
blocks of 60 to 80 characters (one per line). With nucleic acid
sequences, the sequence lines are a series of A/T/C/G/U char-
acters, representing the nucleic acids: adenine, thymine, cytosine,
guanine, and uracil (the latter replacing thymine in RNA). FASTQ
is the file format for the raw data generated by the sequencer in
genomics and transcriptomics (see Fig. 2). The first two lines are
similar as with FASTA file: identification line starts with "@" instead
of ">" and the second line contains the nucleic sequence, but a
quality score is associated with each position of the sequence (i.e.,
each letter in the sequence line). This score is called "Phred score,"
and it codes the probability of error in the identification of this
nucleotide [3]. It goes from 0 to 62 and is coded in ASCII symbols.
This allows to code any score using a single symbol, keeping the
same length as the sequence line. FASTA and FASTQ files can be
opened with any text editor software. FASTQ files are mainly lists of
short sequences called "reads" (between 50 and 200 nucleic acids),
which need to be processed (aligned or assembled) to be further
analyzed. Alignment data files are one type of processed data.
Indeed, reads in FASTQ files can be aligned to a reference genome
sequence to allow further analyses (see below for pipeline descrip-
tion and example of applications). The text file format used in this
case is the SAM<fnref n="2" /> (sequence alignment and mapping) format
[4, 5]. It can be further compacted into its binary equivalent,
which are BAM or CRAM formats [6].

The file formats for proteomics and metabolomics data are not
as homogeneous as for genomics and transcriptomics. At least
17 types of formats exist for mass spectrometry files (see below)
[7]. Each machine manufacturer created its own, adapted to pro-
prietary software to read and analyze it, thus multiplying formats.
In an effort to facilitate data exchange and to avoid data loss (in case
of no more readable old file formats), HUPO [8] and PSI<fnref n="3" /> created
the open-source mzML<fnref n="4" /> format (XML text file with specific tag
syntax) in 2011 [9]. In the main databases that host mass spec-
trometry result files, most of the files are in the RAW format,
developed by Thermo Fisher Scientific. These binary files contain
retention time, intensity, and mass-to-charge ratios (see later sec-
tions). Software like Peaks, Mascot, MaxQuant, or Progenesis
[10, 11] use these files to identify proteins present in the sample
and to quantify them. Results from these analyses are shared
through two other text file formats: mzIdentML<fnref n="5" /> and mzTab.<fnref n="6" />

<footnote n="2">² Sequence Alignment/Map Format Specification</footnote>

<footnote n="3">³HUPO Proteomics Standards Initiative</footnote>

<footnote n="4">⁴ mzML 1.1.0 Specification | HUPO Proteomics Standards Initiative|mzML 1.1.0 Specification | HUPO Proteomics Standards Initiative</footnote>

<footnote n="5">⁵ mzIdentML | HUPO Proteomics Standards Initiative|mzIdentML | HUPO Proteomics Standards Initiative</footnote>

<footnote n="6">⁶ mzTab Specifications | HUPO Proteomics Standards Initiative|mzTab Specifications | HUPO Proteomics Standards Initiative</footnote>
Note that many other file formats exist. One of the most critical for omics data analyses concerns the annotations of features on a DNA, RNA, or protein sequence. They are shared through the General Feature Format (GFF<fnref n="7" />) that is a text file with nine tabulated separated fields: sequence, source of the annotation, feature, start of the feature on the sequence, end of the feature, score, strand, phase, and attributes.

## 2.2 Results from High-Throughput Studies Shared Through Multiple Public Databases

The set of public biological databases hosting omics data is large and constantly evolving. Omics terminology started being regularly used in the 2000s. Between 1991 and 2016 (25 years), more than 1500 “molecular biology” databases were presented in publications, with a proliferation rate of more than 100 new databases each year [12]. These numbers are only the visible part of existing databases. How many have been created without being published? Around 500 of those databases are roughly co-occurrent with the apparition of the World Wide Web, the very Internet application allowing the creation of online databases. The availability of molecular biology databases decreased by only 3.8% per year from 2001 to 2016 [12]. This shows a sustained motivation from the community to create and maintain public platforms to share data. But it also highlights that this motivation comes more from a shared need for easy access to data rather than a supervised effort to coordinate approaches and unify sources. Such efforts indeed exist, for example, the ELIXIR project started in 2013 as an effort to unify all European centers and core bioinformatics resources into a single, coordinated infrastructure [13]. This notably produces the ELIXIR Core Data Resources (created in 2017), a set of selected European databases, meeting defined requirements, and the website “bio. tools,” i.e., a comprehensive registry of available software programs and bioinformatics tools. The US National Center for Biotechnology Information (NCBI<fnref n="8" />) databases are also main references.

Given the “raw” nature of omics dataset, they are stored in archive data repositories: raw data from scientific articles, shared on databases easily accessible for reproducibility. Except for the Sequence Read Archive (SRA), the databases cited here are mixed ones: they host raw archive data and knowledge extracted from them. For genomics dataset, NCBI database Genome [14] and EMBL-EBI (member of ELIXIR) database Ensembl [15] are references. They organize genome sequences together with annotations and include sequence comparison and visual exploration tools. Transcriptomics data can be deposited into several databases, like Gene Expression Omnibus (GEO) [16] initially dedicated to microarray datasets, which is structured into samples forming

<footnote n="7">⁷ GFF/GTF File Format</footnote>

<footnote n="8">⁸ NCBI</footnote>