Mobile Devices, Connected Objects, and Sensors

Sirenia Lizbeth Mondragón-González, Eric Burguière, Karim N'diaye

► To cite this version:

Sirenia Lizbeth Mondragón-González, Eric Burguière, Karim N'diaye. Mobile Devices, Connected Objects, and Sensors. Olivier Colliot. Machine Learning for Brain Disorders, 197, Springer US, pp.355-388, 2023, 10.1007/978-1-0716-3195-9_12. hal-04239635

HAL Id: hal-04239635

https://hal.science/hal-04239635v1

Submitted on 12 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
**Mobile Devices, Connected Objects, and Sensors**

**Sirenia Lizbeth Mondragón-González, Eric Burguière, and Karim N'diaye**

**Abstract**

Brain disorders are a leading cause of global disability. With the increasing global proliferation of smart devices and connected objects, the use of these technologies applied to research and clinical trials for brain disorders has the potential to improve their understanding and create applications aimed at preventing, early diagnosing, monitoring, and creating tailored help for patients. This chapter provides an overview of the data these technologies offer, examples of how the same sensors are applied in different applications across different brain disorders, and the limitations and considerations that should be taken into account when designing a solution using smart devices, connected objects, and sensors.

**Key words** Smartphone, Mobile devices, Wearables, Connected objects, Brain disorders, Digital psychiatry, Digital neurology, Digital phenotyping, Machine learning, Human activity recognition

# 1 Introduction

Sensors are devices that detect events or significant changes in their environment and send the information to other electronic devices for signal processing. Since they surround us continuously, we have integrated them so naturally into our lives that we are mostly unaware of their continuous functioning. They exist in everyday objects, from the motion unit installed in your mobile phone that allows you to switch from landscape to portrait view by simply rotating it to the presence detector sensor in your building that switches the light on and off. Indeed, there is a good chance that you are using one or multiple sensors right now without noticing. They provide various means to measure characteristics related to a person's physiology or behavior either in a laboratory/healthcare unit or in their daily life. They have thus raised a major interest in medicine in the past years. They are particularly interesting in the context of brain disorders because they allow monitoring of clinically relevant characteristics such as movement, behavior, cogni-
tions, etc. This chapter provides an introduction to the use of sensors in the context of brain disorders. The remainder of this chapter is organized as follows.

Subheading 2 presents an overview of the various data types collected using mobile devices, connected objects, and sensors that are relevant to brain disorder research and related clinical applications, in particular for machine learning (ML) processing. The relevance of these ubiquitous sensors comes from the possibility of collecting large amounts of data, allowing the continuous documentation of the user's daily life, an often critical issue with ML applications. Subheading 3 describes how these technologies might serve such applications in brain disorder research and clinics. Because of the strategic importance of ML in the on-device experience, mobile manufacturers have recently started to design and include specially designed microprocessors for ML calculations in smartphones and tablets, benefiting the third-party app development community. A different approach consists of cloud offload processing allowing lighter wearables and handheld devices. The main public interest in current applications of ML is to help guess what is expected by the user, eliminating the number of actions and decisions we make each day (facial recognition for security instead of remembering a password, classification in your picture gallery according to names or faces, recommending songs to listen based on your history and ratings, etc.). Although decision support might not necessarily be its first goal, the scholar community interested in brain disorders must be familiarized with this ongoing ML revolution since the technology is already there, opening the way to unprecedented opportunities in research and clinics. Subheading 4 describes limitations, caveats, and challenges that researchers willing to use such technologies and data need to be aware of.

## 2 Data Available from Mobile Device, Sensors, and Connected Objects for Brain Disorders

Far from presenting an extensive list of available sensors and devices, we aim to introduce the type of data one can exploit and sketch possible applications relevant to brain disorder research. The kind of data that we present here comes from sensors that are typically used for human activity recognition (HAR) or that we deemed relevant for the scope of this book. In particular, we have purposely omitted connected technologies that are used by health practitioners or in healthcare units and that require medical or specific training for their use and interpretation and that are therefore not commonly available to the public, such as wireless electroencephalographer (EEG—but *see* Chapter 9 for in-depth coverage of ML applied to EEG). We also set aside mobile technologies that
are not directly aimed at probing brain and behavioral functions, such as blood pressure monitor devices, glucometers, etc. (see [1] for a review).

We present these data types in two groups according to the role that the user (e.g., the patient) takes in acquiring the data: active vs. passive. In this context, we mainly describe typical applications, but we also point the readers to specific applications for which active or passive data can be used. For instance, vocal recordings can be actively collected by instructing the user to self-record (e.g., when completing a survey), but a microphone may also passively and continuously record the sound environment without the user triggering it (e.g., automatic handwashing recognition using the microphone of the Apple watch to detect water sound [2]). To explore the possibilities in data collection, we distinguish three interconnected elements: the person of interest, the device (including its potential interface), and the environment. According to the dimension of interest, we can focus on the data obtained from the interaction between these three elements (see Fig. 1).

<figure><img src="image_2.png" /><figcaption>**Fig. 1** *Active and passive sensing.* Mobile devices and wearable sensors provide metrics on various aspects of the mental and behavioral states through active (requiring an action from the user, often following a prompt) or passive (automatically without intentional action from the user) data collection. This is possible through (a) direct interaction with the device, (b) active use of a device for assessment of internal insight, (c) passive use of inertial and positioning systems, (d) passive interaction with sensors embedded in the environment, and (e) passive or active interaction between users through devices</figcaption></figure>
## 2.1 Active Data Probing

In active data probing, the person of interest must execute a specific action to supply the data, meaning that the quantity and the quality of acquired data directly depend on the user's compliance. These actions usually involve direct interaction with the device. To maximize compliance, the subject needs to spend time and energy collecting the data; therefore, the number of action steps necessary to enter the data must be optimized to avoid user fatigue. It is also essential to care for feature overload by focusing on usability instead of utility and thoughtfully circumscribing the scope of questions or inputs. The amount of information requested and the response frequency are essential aspects to think ahead to maximize the continuous use of the device. If there is an intermediate user interface, following standard UX/UI (user experience/user interface) guidelines is a good starting point for optimization but might not be sufficient according to the target population group. It is crucial to design without making assumptions but by getting patients' early feedback through co-construction or participatory design [3-5]. In summary, there are several considerations that one needs to plan before deploying a solution-using active probing that involves the device itself but also how the user interacts with it.

### 2.1.1 Interaction with the User

Recording the subject's response can provide unique information about the occurrence of experiences and the cognitive processes that unfold over time. We can record the user's feedback at specific points in time or continuously by taking advantage of the interaction between the user and a device (see Fig. 1a).

*Manual devices: response buttons, switches, and touchscreens.*
These devices capture conventional key or screen presses via switches or touchscreens, usually operated by hand. A switch connects or disconnects the conducting path in an electrical circuit, allowing the current to pass through contacts. They allow a subject to send a control or log signal to a system. They have been largely used, for several decades, in computer-based experiments for psychology, psychophysiology, behavioral, and functional magnetic resonance imaging (fMRI) research. The commonly obtained metrics are specific discrete on/off responses (pressed or not) and reaction time [6]. It is usually necessary to measure a person's reaction time to the nearest millisecond which requires dedicated response pads. Indeed, general-purpose commercial keyboards and mice have variable response delays ranging from 20 to 70 ms, a range comparable to or lower than human reaction time in a simple detection task [7]. On the other hand, dedicated computerized testing devices seek to have less variable and smaller response delay. They introduce less variation and biases in timing measurements [7] by addressing problems such as mechanical lags, debouncing, scanning, polling, and event handling. Commercially available response-button boxes (e.g., Psychology Software Tools, Inc., Sharpsburg, PA, USA; Cedrus Corporation, San Pedro, CA, USA; Empirisoft Corporation, New York, NY, USA; Engineering
Solutions, Inc., Hanover, MD, USA; PsyScope Button Box by New
Micros in Dallas, TX, USA) have few options and specific layouts to
collect responses according to standard gamepad layouts while still
being usually customizable for more specific applications.

Alternatively, touchscreens can be used to detect discrete
responses with screen coordinates of the touch or pressing. They
come in many forms, and the most popular type works with capaci-
tive or resistive sensors. Resistive touchscreens are pressure-
sensitive, and capacitive screens are touch-sensitive. Nowadays,
capacitive screens are more used because of their multi-touch cap-
abilities, short response time, and better light transmission. How-
ever, if an application needs the exact coordinates of the contact,
the inductive touchscreens are more suited. This technology is
usually featured in the highest priced tablets along with a special
pen that induces a signature electromagnetic perturbation that
improves its precision compared to finger pointing. The disadvan-
tage of touchscreens is that they lack tactile feedback and have high
energy consumption. For collecting continuous responses, a joy-
stick, computer mouse, or touchscreen may be used to track move-
ment trajectories supposedly reflecting the dynamics of mental
processes [8].

*Connected devices* have been introduced in many domains of everyday life and, more recently, in health and research settings, sometimes with medical-grade applications [9]. Such devices may include sensors of health-relevant physiological parameters (e.g., weight, heart rate, and blood pressure) or health-related behaviors (e.g., treatment compliance). These connected systems make data collection more systematic and readily available to the clinical practitioner. They are automatically integrated into data management systems. For example, on a pre-specified schedule, the patient will measure his/her blood pressure with a so-called smart blood pressure monitor, which may provide reminders and record and transmit these measurements to his/her doctor. Active connected devices (which require the patient to participate in the data collection process) may also track behavior: a connected pillbox would allow monitoring that the patient takes the medication according to the prescribed schedule [10]. In a subsequent part of this chapter, we will refer to passive connected medical devices (which perform measurements without the intervention of the user/patient), such as fall detection systems.

With current knowledge and technologies, data that reflect psycho-
logical states such as emotions and thoughts can only be obtained
by active data probing of the patient or an informer, usually a
partner, family member, or caregiver (see Fig. 1b). The long history
of psychological assessment provides rich conceptual and method-
ological frameworks for collecting valid measures of subjective
states when collected with a traditional semi-directed interview or paper-and-pencil questionnaires. Nevertheless, the novel possibilities allowed by mobile technologies challenge those traditional well-validated assessment tools by renewing the format and the content of questions addressed to the user. In medical care and research, patient-reported outcomes are at the heart of a paradigmatic change in medicine and clinical research, where patient-centric measures tend to be favored over pure biomedical targets.

Subjective assessments may sometime take the form of utterances or text. For machine learning applications, those have to be converted into data usable for feeding mathematical models. Natural language processing (NLP) tools have recently made substantial progress thanks to deep learning techniques, making even complex spontaneous oral or written language amenable to machine processing [11].

## 2.2 Passive Data Probing

In passive data probing, the data is collected without explicitly asking the subject to provide the data. It provides an objective representation of the subject's state in time. In scenarios where the data needs to be acquired multiple times a day, passively collecting the data is a more valuable and ecological way to proceed. It allows objectively measuring the duration and frequency of specific events and their evolution in time. In contrast with active data, probing can provide more samples over a period. Since meaningful events might be embedded in the collected data, this probing type requires reviewing historical loggings or computer applications to extract the information of interest.

### 2.2.1 Inertial and Positioning Systems

Detection of whole-body activities (such as walking, running, and bicycling), as well as fine-grained hand activity (such as smartphone scrolling, typing, and handwashing), can allow the arduous task of studying and monitoring human behavior, which is of great value to understand, prevent, and diagnose brain diseases as well as to provide care and support to the patient. The change in physical activity and its intensity, the detection of sleep disorders, fall detection, and the evolution or detection of a particular behavior are some possibilities that can be assessed with inertial sensors.

Identifying specific activities of a person based on sensor data is the main focus of the broad field of study called human activity recognition (HAR). A widely adapted vehicle for achieving HAR's goal is passive sensor-based systems that use inertial sensors (see Fig. 1c), which transduce inertial force into electrical signals to measure the acceleration, inclination, and vibration of a subject or object (see Fig. 2a). These systems are commonly included in today's portable electronic devices such as mobile phones, smartwatches, videogame controllers, clothes, cameras, and non-portable objects like cars and furniture. Besides offering the advantage, due to their reduced size, of being embeddable in
<figure><img src="image_3.png" /><figcaption>**Fig. 2** *Inertial sensors.* (a) Representation of an inertial measurement unit (IMU) depicting the sensing axes and the corresponding yaw, pitch, and roll rotations. (b) Exemplar accelerometer profiles of two hand gestures (hand rubbing and key locking) for three subjects showing the similar periodic nature of the hand movements. (c) Operating principle of an MEM accelerometer. When a force is detected due to a compressive or extensive movement, it is possible to determine the displacement x and acceleration since the mass and spring constants are known. (d) Representation of a simple gyroscope model. (e) The magnetic field generated by electric currents, magnetic materials, and the Earth's magnetic force exerts a magnetic force detectable by a magnetometer sensor</figcaption></figure>

almost any possible device, they are perceived as less intrusive of personal space than other HAR systems, such as camera and microphone-based systems [12], allowing to sense more naturalistic motion information uninterruptedly. Most prior work on activity detection has focused on detecting whole-body activities that reflect ambulatory states and their degree of locomotion or lack of it, such as running, walking, cycling, lying, climbing stairs, falling, sitting, standing, and monitoring the sleep-wake cycle. Whole-body activities differ from fine-grained human actions, usually undertaken by the hands (see Fig. 2b). These hand activities are often independent of whole-body activity, for instance, sending a text from your smartphone while walking. A sustained sequence of related hand gestures composes a hand activity. Hand gestures like waves, flicks, and snaps tend to have exaggerated motions (used for communications), and hand activities are more subtle, discontinuous, and of varying durations [12]. Examples of complex hand
gestures are writing, typing, painting, searching the Internet, smoking, eating, and drinking. The way one approaches whole-body activity detection differs from fine-grained activity recognition in terms of the analysis approach (e.g., selected features), sensor configuration (e.g., higher sampling frequency for fine-grained activities than for whole-body activities), and location on the body (e.g., wrist vs. hip). In both detection problems, the most common sensors used for HAR applications are accelerometers, gyroscopes, and magnetic sensitive sensors (see Fig. 2c-e).

## Accelerometers

Accelerometers are sensors used to measure linear acceleration, viz., change in velocity or speed per time interval of the object being measured along reference axes. Furthermore, one can obtain velocity information by integrating accelerometry data with respect to time. The measuring acceleration unit in the International System of Units (SI) is a meter per second squared (m/s²). Since we can distinguish a static component in the accelerometer signal as the gravitational acceleration, it is also common to use the unit G-force (g) to distinguish the relative free-fall gravitational acceleration with a conventional standard value of 1 g = 9.81 m/s². A simplistic representation of the accelerometer's operation principle is based on a suspended mass attached to a mechanical suspension system with respect to a reference inside a box, as shown in Fig. 2c. The inertial force due to gravity or acceleration will cause the suspended mass to deflect according to Hooke's law ($F = mk$) and Newton's second law ($F = ma$), where $F$ denotes the force (N), $m$ is the mass of the system (kg), $k$ is the spring constant, $x$ is the displacement (m), and $a$ is the acceleration (m/s²). This acceleration force can then be measured electrically with the changes in mass displacement with respect to the reference. To better understand this working principle, you can think of your experience as a passenger in a car rapidly moving back and forth and how the forces acting on you make you incline backward and forward on your seat. In nowadays-electronic devices, we find mostly miniaturized semiconductor accelerometers (microelectromechanical systems or MEMs), which are small mechanical and electrical devices mounted on a silicon chip. The most common types are piezoresistive, piezoelectric, and differential capacitive accelerometers [13]. Since the accelerometer is usually a built-in component embedded in a mobile device, the data we can obtain is provided in the XYZ coordinate system of the accelerometer component. The XYZ orientation is specific to each device, and its coordinate system is found in the datasheets of the components.

When processing the accelerometer signals, separating the acceleration due to movement from gravitational acceleration and noise sources (e.g., electronic device and measurement conditions) is necessary. A low-pass filter with a cutoff frequency of 0.25–3 Hz is usually applied to raw data to remove noise [14]. Alternatively,
transforming the raw accelerometer data to the vector magnitude (Eq. 1), which measures the instantaneous intensity of the subject's movement at time t can be done before filtering to remove noise and/or gravity from body acceleration. The following processing steps usually include normalization (min-max, division by maximum absolute value, or division by the mean).

$$vm(t) = \sqrt{A_x(t)^2 + A_y(t)^2 + A_z(t)^2} \quad (1)$$

A time-window segmentation is often necessary to retrieve information from the accelerometer time series. The epochs are usually consecutive sliding windows with an overlapping percentage (usually 50% overlap). Different window sizes can be compared to identify the optimal size for HAR analysis.

## Gyroscope

A gyroscope is an inertial sensor that measures the rate of change of the angular position over time with respect to an inertial reference frame, also known as angular velocity or angular rate. The principle of function of MEM's gyroscopes is based on the Coriolis effect, which acts on moving objects within a frame of reference that rotates with respect to an inertial frame. Figure 2d represents a simple gyroscope model where a mass suspended on springs has a driving force on the x-axis and angular velocity ω applied about the z-axis, causing the mass to experience a force in the y-axis as a result of the Coriolis force. In an MEM's gyroscope, the resulting displacement is measured by a capacitive sensing structure. The angular velocity unit is deg./s, but expressing it in radians per second (rad/s) is also common. A gyroscope can provide information about activities that involve rotation around a particular axis. A triaxial gyroscope can provide information from three different angles, pitch (x-axis), roll (y-axis), and yaw (z-axis), to help estimate the movement signature's orientation and rotation.

In human activity recognition, the gyroscope activity helps provide information about activities involving rotation around a particular axis. While a gyroscope has no initial frame of reference like gravity, it can be combined with accelerometer data to measure angular position and help determine an object's orientation within 3D space. To obtain the angular position, we can integrate the angular velocity with Eq. 2, where $p = yaw, pitch, and roll$ and $θ_{p0}$ is the initial angle compared to the Earth's axis coordinates.

$$θ(t) = \int_{0}^{t} \dot{θ}_{p}(t) dt + θ_{p0} \quad (2)$$

When the changes in angular velocity are faster than the sampling frequency, one will not be able to detect them, and the error will continue to increase with time. This error is called drift. Therefore, the sampling rate value should be carefully chosen since gyroscopes are vulnerable to drifting over the long term.
Magnetic Sensitive Sensors (e.g., Hall Sensor)

Magnetic sensors measure the strength and direction of the Earth's magnetic field and are affected by electric currents and magnetic materials (see Fig. 2e). Most MEM's magnetic sensors are based on magnetoresistance to measure the surrounding magnetic field, meaning that the resistance changes due to changes in the Earth's and nearby magnetic fields. They can detect the vector characterized by strength and direction toward the Earth's magnetic north, and with it, one can estimate one's heading. This vector is vertical at the Earth's magnetic pole and has an inclination angle of 0°. When used with accelerometers and gyroscopes, it can help to determine the absolute heading.

IMU Technology

The combination of accelerometers, gyroscopes, and sometimes magnetometers in a single electronic device is referred to as an inertial measuring unit (IMU). Here are some considerations when choosing an IMU system or a device that contains an accelerometer, gyroscope, or magnetic sensor for HAR applications:

1. *Dynamic range.* Dynamic range refers to the range of maximum amplitude that the sensor can measure before distortion. In the case of accelerometers, where the amplitude in locomotion increases in magnitude from cranial toward caudal body parts, they are typically measured in powers of two (±2G, ±4G, ±8G, and so on), with an amplitude range of ±12G for whole-body activities [15]. Gyroscopes are grouped by the angular rotation rate they can quantify (in thousands of degrees/second). The measuring range of magnetometers is in milliTesla (mT).

2. *The number of sensitive axes.* Inertial units that can sense in three orthogonal planes (triaxial) are suitable for most applications since different directions contribute to the total complex movement patterns.

3. *Bandwidth.* The sampling rate determines the frequency range that can be represented in a waveform. Its unit is samples per second or Hertz. For HAR applications, the bandwidth of human accelerations of interest must be covered by the sensor's sampling rate. The sampling rate selection depends on the activity of interest, the measured axes, and the body part to which the sensor is attached. For instance, walking at natural velocity ranges from 0.8 to 5 Hz when measured in the upper body, whereas abrupt accelerations up to 60 Hz have been measured at the foot level [15]. For typical whole-body activities (like lying, sitting, standing, and walking), sampling rates are usually between 50 and 200 Hz. Still, some studies use low ranges 20-40 Hz or as high as 4 kHz [12, 16] with analysis window lengths from 2 to 15 s [14]. A study has reported that frequencies from 0 to 128 Hz best characterize most human activities via hand monitoring [12].
4. *Interface and openness.* In HAR applications, IMUs interface with other systems for signal processing. It is essential to know the communication protocol for data transfer and the degree of openness of the chosen system to allow configuration and extraction of raw signals since not all commercial systems allow raw signal extraction or changes in some parameters, like the sampling frequency.

5. *Sensor biases.* Sensor bias refers to the initial offset in the signal output when there is no movement. In the case of MEM's inertial sensors, it is often indicated as a zero-g offset for accelerometers and a zero-rate offset for gyroscopes. It has been shown that there is a large range of bias variability among different commercial devices and between devices of the same model [17]. Large uncompensated bias in HAR applications can lead to difficulties in detecting states when using different devices. In these cases, oriented data fusion techniques can be used to compensate the biases' effect on the data.

Raw signal periods are further decomposed into a few numbers (in the tens) of features. These are reduced variables of original raw data that represent the main characteristics of the signal. Inertial features are usually a mixture of frequency-domain features and time-domain features, although there are some rare cases of methods that process raw accelerometer data [12]. Table 1 summarizes the most common features applied to human activity recognition using machine learning and groups them into four domain categories: statistical, frequency, time, and time-frequency. Statistical features are descriptive features that summarize and give the variability of the time series. Time-domain features give information on how inertial signals change with time. For instance, zero-crossing is the number of times the signals change from positive to negative values in a window length. Together with frequency-domain features capturing how the signal's energy is distributed over a range of frequencies, they are useful to capture the repetitive nature of a signal that often correlates to the periodic nature of the human

<table><caption>**Table 1**
Accelerometer features for machine learning applied to human activity recognition</caption><thead><tr><th>Features</th><th></th><th></th></tr></thead><tbody><tr><td></td><td>Statistical features</td><td>Kurtosis, skewness, mean, standard deviation, interquartile range, histogram, root mean square, and median absolute deviation</td></tr><tr><td></td><td>Time-domain features</td><td>Magnitude area, zero-crossing rate, pairwise correlation, and autocorrelation</td></tr><tr><td></td><td>Frequency-domain features</td><td>Energy, entropy, dominant frequency (maximum and median frequency) and power of dominant frequency, cepstral coefficients, power bandwidth, power spectral density, and fundamental frequency</td></tr><tr><td></td><td>Time-frequency features</td><td>Spectrogram [12], wavelets, spectral entropy [18]</td></tr></tbody></table>
activity. Their advantage is that they are usually less susceptible to signal quality variations. Time-frequency features such as spectrograms give information about the temporal evolution of the spectral content of the signals. They can represent context information in signal patterns, but they have higher computational costs than other features. Indeed, the low computational cost is a desired characteristic of HAR for their applications, and it is no surprise that most applications with smartphones that use inertial sensors use time-domain features [19].

## Data Fusion

Data fusion is the concept of combining data from multiple sources to create a result with an accuracy that is higher than that obtained from a single source. IMUs can be used to simultaneously provide linear acceleration and angular velocity of the same event, as well as the device's heading. Data fusion techniques provide complementary information to improve human activity recognition. Importantly, they can also be used to correct each other since each IMU sensor has different strengths and weaknesses that can be combined for a better solution. Accelerometers can measure gravity for long terms but are more sensitive to certain scenarios, such as spikes. Gyroscopes can be trusted for a few seconds of relative orientation changes, but the output will drift over longer time intervals, and magnetometers are less stable in environments with magnetic interferences.

Data fusion techniques can be divided into three levels of applications: sensor-level fusion, feature-level fusion, and decision-level fusion [20]. In sensor-level fusion, the raw signals from multiple sensors are combined before feature extraction. For instance, accelerometers are sensitive to sharp jerks, while gyroscopes tend to drift over the long term; thus, sensor-level fusion helps with these problems. This is achieved via signal processing algorithms, where the most popular algorithms are the Kalman filter [21] and the complementary filter [22]. The first, an iterative filter that correlates between current and previous states, consists of low- and high-pass filtering to remove gyroscope drift and accelerometer spikes. Feature fusion refers to the combination of multiple features from different sensors before entering them into a machine learning algorithm through feature selection and reduction methods such as the principal component analysis (PCA) and singular value decomposition (SVD). Feature fusion helps in identifying the correlation between features and working with a smaller set of variables. The models' results (e.g., multiple classifiers) are combined in decision fusion to have a more accurate single decision. The aim is to implement fusion rules to get a consensus that would help in improving the algorithm's accuracy and have a better generalization. These rules include majority voting, boosting, and stacking [23].
**Benchmark Databases**

In the context of HAR, there are several advantages to having access to inertial databases. The most obvious one is that it allows for comparing several solutions. Inertial databases can be used to rapidly focus on the development of signal processing and machine learning solutions before spending time on the development and deployment of hardware sensing solutions. In the past years, several public databases have appeared in the literature for smartphone and wearables studies. The list of the main databases of inertial sensors used in research work is gathered in Lima and colleague's review [19] and in Sprager and Juric's review [24].

**Global Positioning System: Geospatial Activity**

The Global Positioning System (GPS) is a global navigation system based on a network of GPS satellites, ground control stations, and receivers that work together to determine an accurate geographic position at any point on the Earth's surface. The widespread integration of GPS into everyday objects such as smartphones, navigation systems, and wearables (GPS watches) has enabled the objective measurement of a person's location and mobility with minimal retrieval burden and recall bias [25]. At a basic level, raw data from GPS provide latitude, longitude, and time [26]. These data can be further processed to provide objective measurements of location and time, such as measurements of trajectories and locations in specific environments. Newer GPS can provide variables such as elevation, indoor/outdoor states, and speed. GPS devices have proven to be useful tools for studying and monitoring physical activity [27]. When combined with inertial sensors, it is possible to identify activity patterns and their spatial context [28]. The spatial analysis can then be contextualized with environmental attributes (presence of green space, street connectivity, cycling infrastructure, etc.). The data is often analyzed using commercial or open-source geographic information systems (GIS), software for data management, spatial analysis, and cartographic design. According to Krenn and colleagues [28], the main limitation of using GPS in health research is the loss of data quality. Indeed, urban architecture and dense vegetation can lead to signal dropouts.

**2.2.2 Interaction with the Environment**

When a residence uses a controller to integrate various connected objects or home automation systems, we refer to the home as a smart home. In a smart home, the role of the home controller is to integrate the home automation systems and enable them to communicate with each other. In this approach, the subject does not need to carry a device; instead, the environment is equipped with devices that can collect the required data (see Fig. 1d). In smart homes, we can find diverse appliances with some degree of automation. Perhaps the most popular commercial device is the smart speaker, equipped with a virtual assistant that responds to voice commands. More and more common virtual assistant technologies have expanded the use of speech processing, and the so-called vocal
biomarkers are being considered into precision medicine [29]. These technologies can be embedded in what is known as affective signal processing, for example, to monitor the mood states of home residents [30].

Smart plugs are another type of smart device that fits into existing wall outlets. They connect to the Wi-Fi or Bluetooth network and enable the control of various appliances by turning them on and off on pre-programmed schedules. Although they are not sensing devices that collect data per se, by activating the appliances, they allow the interaction between the user and the environment, and they can be used to activate sensing devices. Other smart devices that typically do not collect data from the user but enable interaction with the environment include smart light bulbs that can be turned on at specific times and allow to be controlled to create a colorful ambiance, smart thermostats to control room temperature, and smart showers. Other smart systems that allow data collection and interaction with the user and the environment include smart refrigerators that register the door's opening and the amount of food inside. They also offer the ability to view recipes and videos and adjust the water temperature through a touchscreen. Smart devices that help with sleep are smart mattresses, sleep trackers, and sleep noise machines.

Finally, when installed in strategic places, presence detectors or switches that detect the opening and closing of doors and windows can work together to create a map of presence and displacement activity inside the smart home.

The advent of smart home technology has fostered its development in medicine and human research. One such example is the use of surveillance cameras, which were initially deployed for security monitoring of goods and may now be used to detect falls by elderly persons in everyday life, thanks to advanced image processing techniques [31]. Home automation systems built around dedicated single-board computers (e.g., Raspberry Pi) expand behavioral tracking capabilities to more complex behaviors using off-the-shelf components [32].

The massive adoption of Internet of Things (IoT) devices has made it possible to have a network of interconnected devices that interact to collect and analyze data using an Internet connection for remote computing (see Fig. 1e). Interaction between devices may be used as a proxy to inform about the collective and individual behavior of the user(s) carrying them. This interaction is possible due to numerous wireless technologies that enable communication among devices, such as Wi-Fi and Bluetooth. Moreover, a richer picture of the social world may be obtained from the traces of interactions in cyberspace, such as the analysis of individual devices' communication. Research using the phone call detail records of a sample of elderly participants in France demonstrated that such
passive data could represent a low-cost and noninvasive way to
monitor the fluctuations of mood [33], working as a “social sensor
containing relevant health-related insights.”

Within the wireless technologies available, Bluetooth is widely present in everyday technological devices, and it can be used as a mean to measure the interaction between users. It is based on a radio frequency that allows nearby devices to exchange data wirelessly. Bluetooth devices are paired (established logical link) before transmitting the information for security reasons. Each Bluetooth device is addressable by a unique Bluetooth device address assigned during manufacturing in addition to a textual modifiable identifier [34]. Once the devices are Bluetooth-enabled, they act as passive tools that can be used in the context of interaction monitoring between individuals. The reason why Bluetooth is better fitted to this purpose than Wi-Fi is that the former is mainly used for linking electronic devices for only short communication bouts using relatively small amounts of data and requires less power compared to Wi-Fi, which is designed to shuttle larger amounts of data between computers and the Internet. Another reason is that Bluetooth technology is rapidly evolving, offering simpler connectivity protocols between devices and better security, together with faster communication (Bluetooth V3) and lower energy consumption (Bluetooth Low Energy) with the latest version (Bluetooth 5) offering a more extensive range, speed, and bandwidth.

Implicit Bluetooth encounters can be used to passively detect implicit connections between persons, model and predict social interactions, recognize social patterns, and create networking structures without monitoring physical areas and letting people feel observed. With the COVID-19 pandemic, massive efforts to deploy contact tracing systems to notify for risk of infection used a Bluetooth protocol in smartphones as a way to identify the risk of close contact with infected individuals. In this context, Bluetooth exchanges were considered encounters [35]. This is one remarkable example of Bluetooth technology showing how it can be applied to exploit users’ interactions in real time to help manage an important health issue in modern society.

# 3 Applications to Brain Disorders

A growing number of applications have been developed to collect and exploit sensor data for basic science and clinical applications related to the disorders of the nervous system—as well as in human behavior in general; see [36]. This section presents a selection of original and representative application examples where the previously presented sensors have been put into practice to prevent, early diagnose, monitor, and create tailored help for patients, with what
is referred to as digital phenotyping. The objective of this section,
far from being an extensive review of the sensor-based applications,
is to give an idea to the reader of how the same sensors can be used
with different objectives across a broad range of brain disorders.
The brain disorders mentioned here are Alzheimer's disease
(AD) (see also [37]), Parkinson's disease (PD) [38], epilepsy [39],
multiple sclerosis [40], and some developmental disorders and
psychiatric disorders [41].

**3.1 Prevention**

The blooming market of mobile technologies in the field of well-being and self-quantization, from basic logging to deep personal analytics, represents an opportunity to promote and assist health-enhancing behaviors. For instance, as much as 85% of US adults own a smartphone [42] and 21% an activity tracker [42]. Digital prevention uses these mobile technologies to advise and anticipate a decline in health, the goal being to prevent health threats and predict event aggravation by monitoring continuous patient status and warning indications.

An example of digital prevention in the psychiatric domain
includes specific tools to prevent burnout, depression, and suicide
rates. Web-based and mobile applications have been shown to be
interesting tools for mitigating these severe psychiatric issues. For
instance, a recent study [43] showed how the combination of a
smartphone app with a wearable activity tracker was put into use to
prevent the recurrence of mood disorders. With passive monitoring
of the patient's circadian rhythm behaviors, their ML algorithm was
able to detect irregular life patterns and alert the patients, reducing
by more than 95% the amount and duration of depressive episodes,
maniac or hypomanic episodes, and mood episodes.

In specific contexts known for being risk-prone with respect to
mental health, e.g., high psychological demand jobs, as well as in
more general professional settings, organizations have been start-
ing to deploy workplace prevention campaigns using digital tech-
nologies [44]. In a study by Deady and colleagues [45], the authors
developed a smartphone app designed to reduce and prevent
depressive symptoms among a group of workers. The control
group had a version of the app with a monitoring component,
and the intervention group had the app version that included a
behavioral activation and mindfulness intervention besides the
monitoring component. Their study showed how the smartphone
app helped prevent incident depression in the intervention group
by showing fewer depression symptoms and less prevalence of
depression over 12 months compared to the control group. Both
these examples show how using smartphones and wearable devices
can reduce symptoms and potentially prevent mental health
decline.
## 3.2 Early Diagnosis

Although, in the brain care literature, most applications for diagnosis with ML use anatomical, morphological, or connectivity data derived from neuroimaging [46], there is a growing body of evidence indicating that common sensors could be used in some cases to detect behavioral and/or motor changes preceding clinical manifestations of diverse brain diseases by several years. In contrast, in neurodegenerative diseases like AD [47], PD [48], and motor neuron disease (MND) [49], the symptoms manifest when a substantial loss of neurons has already occurred, making early diagnosis challenging. Because of this, with the increasing adoption of ML in research and clinical trials, directed efforts have been made to diagnose neurodegenerative diseases early. As an example, in PD, a study used IMU in smartphones to characterize gait in the senior population, detecting gait disturbances, an early sign of PD, and showing the feasibility of the approach with a patient who showed step length and frequency disturbances and who was later formally diagnosed with PD [50]. Apathy, conventionally defined as an "absence or lack of feeling, emotion, interest or concern" [51], is one of the most frequent behavioral symptoms in neurological and psychiatric diseases. In the daily life of patients, apathy results in reduced daily activities and social interactions. These behavioral alterations may be detected as a reduction in the second-order moment (variance) of location data (as tracked with GPS measurements [52]) and in the first-order moment (average quantity) of accelerometer measures (e.g., [53] in the context of schizophrenia patients).

Sensors can also be used to differentiate between disorders that have shared symptoms, accelerating diagnosis and treatments. For instance, a study [54] that used wrist-worn devices containing accelerometers analyzed measures of sleep, circadian rhythmicity, and amplitude fluctuations to distinguish with 83% accuracy pediatric bipolar disorder (BD) and attention-deficit hyperactivity disorder (ADHD), two common psychiatric disorders that share clinical features such as hyperactivity.

## 3.3 Symptom and Treatment Monitoring

Monitoring day-to-day activities and the evolution of symptoms is impossible for health providers outside the clinic without automated detection of events of interest and deployment of mobile interventions. Much like apathy, described above, many other psychological constructs may be sensed from continuous monitoring of behavioral parameters, such as agitation or aberrant mobile behavior [55, 56].

Sleeping is one activity that cannot be monitored in any other way than with passive data probing in an ecological manner. Monitoring sleep is relevant when studying sleep disturbance, a core diagnostic feature of depressive disorder, anxiety disorders, bipolar disorder, and schizophrenia spectrum disorder. In this sense, sleep patterns have been scored using light sensors in mobile devices and
usage data, allowing digital phenotyping of the users that, compared to the average, go to bed and wake up later and more often. Disrupted sleep patterns have also been assessed with wrist-worn accelerometers to monitor sleep changes in various psychiatric disorders [57], as well as considered a potential psychiatric diagnostic tool in bipolar disorder, where sleep changes are a warning sign of an affective episode [58].

Given the progressive nature of some diseases, such as Alzheimer's and Parkinson's diseases, the individuals suffering from them must be monitored often or even continuously. In both cases, the patients suffer from functional and cognitive decline, where continuous objective monitoring could help detect the decline in daily capabilities providing opportunities for assistance. In the literature interested in monitoring Alzheimer's disease, the studies mainly focus on the detection of abnormal behavior, the detection of autonomy in activity performance, the provision of assistance with cognitive or memory problems, and the monitoring of functional and cognitive decline [59]. To objectively assess autonomy at home, video cameras and tags on house objects along with a mobile phone application were used in a study [60] with mild cognitively impaired patients, Alzheimer patients, and healthy controls. The activities examined included online payment, preparing a drink, medicine box preparation, and talking on the phone. To monitor cognitive decline, Lyon and colleagues from the Oregon Center for Aging and Technology (ORCATECH) [61] placed a smart sensor platform in 480 homes of an elderly population in an 8-year longitudinal study. The sensors included wireless passive infrared motion sensors, wireless magnetic contact sensors placed outside the door and in the refrigerator, a personal computer that recorded time spent in the computer and the mouse movements, worn actigraphs to measure mobility patterns, and, in some cases, connected objects such as medication trackers, phone monitors, and wireless scales. Using these multimodal data and applying sensor fusion techniques, they could identify decline in cognition, loneliness, and mood anomaly. Finally, as nighttime wanderings and memory loss are common characteristics of Alzheimer's patients, GPS solutions are increasingly used by caregivers to locate missing patients but are also recently being used in various studies [62] as effective noninvasive means of monitoring mobility in these patients. GPS solutions have also been exploited in other areas, such as in monitoring anxiety disorders. For instance, GPS data has helped predict social anxiety scores among college students by analyzing mobility features and detecting that socially anxious students avoid public areas and engage less in leisure activities to spend more time at home after school [63].

An interesting advantage of in-home monitoring of symptoms is collecting ecological data allowing clinicians to contextualize sensor data to guide potential medication changes. For instance,
Chen and colleagues [64] introduced a web-based platform that integrates data from wearable accelerometers and online surveys to estimate clinical scores of tremors, bradykinesia, and dyskinesia. The objective was to facilitate clinicians' decision-making regarding titration and timing of medications in PD patients with later-stage disease. Along the same line, in the aforementioned ORCATECH study [61], specific medication trackers (electronic pillbox) were also used to complement behavioral assessment derived from sensors: they demonstrated a significant impact of early cognitive deficits on medication adherence in everyday life.

Active probing of subjective assessment through the everyday life course of patients (commonly performed by smartphones and now smartwatches) is known as ecological momentary assessment (EMA) [61]. EMA aims at reducing memory bias and increasing the density of longitudinal data available in a single patient while exploring the possible influence of real-life contexts on cognitions and behaviors. EMA may thus capture the dynamic changes seen in psychiatric [65, 66] or neurological [67] conditions across hours, days, or longer periods, delivered according to either a predetermined schedule or in response to some event of interest, as detected by the system. EMA may also be used in combination with other passive measures and can be particularly useful to provide a ground truth concerning subjective states (e.g., mood or apathy [53]).

## 3.4 Tailored Help for Patients and Augmented Therapies

Personalized or precision medicine consists in using collected data to refine the diagnosis and treatment of individual patients. In this sense, connected devices and mobile technologies could contribute to tailoring patients' care. Moreover, personalized or augmented therapies can benefit from using smart devices and connected objects to add additional assistance to classic therapeutic approaches.

An example of this is epilepsy, a central system disorder that causes seizures. Not only the unpredictability of seizure occurrence is distressing for patients and contributes to social isolation, but for unattended patients with recurrent generalized tonic-clonic seizures (GTCS), this may lead to severe injuries and constitute the main risk factor of sudden unexpected death. This is why, in the epilepsy research field, much effort has been put into developing ambulatory monitoring with alarms for automated seizure detection, with most real-time application studies using wrist accelerometers, video monitoring, surface electromyography (sEMG), or under-mattress movement monitors based on electromechanical films [68]. The general purpose of using these sensors is to detect unpredictable changes in motor activity or changes in autonomic parameters characteristic of seizures.

Another illustrative case of the interest in mobile technology for helping patients in their everyday life concerns fall detection in older and/or gait-disabled persons: wireless versions of inertial and
pressure sensors have been used to monitor balance impairments in patients and to trigger an alert system when a fall is detected [69]. Data issued from mobile, wearables, and connected devices may also contribute to adjusting the therapeutic strategy followed by the healthcare provider. Omberg and colleagues [70] demonstrated that in Parkinson's disease patients, remote assessment through smartphones correlated with in-clinic evaluation of disease severity. In the context of rehabilitation following cerebrovascular lesions or neurocognitive training in neuropsychiatric disorders, connected devices may also contribute to making the rehabilitation/training program more engaging for patients and improving its real-life efficacy [71].

Finally, in the context of psychiatric disorders, mobile technologies may also support ecological momentary or just-in-time interventions (EMI), a promising venue for augmenting mental healthcare and psychotherapy through digital technologies [72, 73].

# 4 Considerations and Challenges

When conceptualizing and developing a project involving human behavior recognition, it is essential to anticipate the known challenges and difficulties that can be encountered. We present the general known common challenges for connected devices under three groups: (1) those that are related to sensor function per se, (2) the challenges related to the signal processing and machine learning methods used to exploit the data and that are partly shared with other pattern recognition fields, and (3) the challenges raised by deploying real-life applications.

## 4.1 Related to Sensor Function

### 4.1.1 Sample Rate Stability

We refer to sample rate stability as the homogenous regularity time spans between consecutive samples. In a reliable device, the difference between different time spans between successive measurements is close to zero. When this is not the case, the true measure by the sensor and the timestamp registered by the application differs. Common sources of sample rate instability are the inherent jitter by non-real-time operating systems that cannot guarantee critical execution time or access to resources and the additional communication delay between the devices and applications.

### 4.1.2 The Choice of Technology

Sensors are usually input devices that take part in a bigger system, sending information to a processing unit so that the signals can be analyzed. When choosing a technology to work with, a careful choice of all of the parts must be pre-studied to avoid issues in usability and signal quality since these will have an impact on the difficulty of development and deployment, as well as on the long-term use of the technology. For instance, if we need to record
inertial measurements and the body location is not a major issue,
deciding between a dedicated IMU device, a smartwatch, or a
smartphone would be necessary. Smartwatches, having fewer
resources than smartphones, show larger sampling instabilities,
especially under high CPU load [17], and then the question
would be if a smartwatch would then be appropriate for the appli-
cation, and so, what model would provide the best sampling stabil-
ity over long recordings? Hardware memory usage limitations and
power consumption are critical criteria to consider, especially for
the long-term use of connected devices. Another issue is the open
access to commercial devices. Most commercial devices (smart-
phones, smartwatches, and connected devices) offer the developers
the opportunity to use their integrated sensors to develop applica-
tions using their platforms (i.e., Android, iOS, Tizen, etc.). Usually,
the development of these commercial devices comes with certain
restrictions. For instance, the developers do not have complete
access to the device, and to modifications of the operating system,
the programming language is usually restricted, and some
pre-programmed tasks are usually impossible to modify.

4.1.3 Power
Consumption

One of the main problems preventing the massive expansion and adoption of HAR applications is excessive battery power consumption [75]. Indeed, the major problems that lead to data loss are empty batteries, where the main sources of high power consumption are the high data processing load and the continuous use of sensors. Some strategies can be adopted to minimize energy consumption, although these imply a tradeoff between energy consumption, signal richness, and the accuracy of classification models. The first strategy consists of on-demand activation of sensors only when necessary, in contrast to continuous sampling; this requires a continuous supplementary routine that automatically determines when the timing is appropriate to interrogate the sensor(s). Tech companies have dealt with this problem by integrating “sensor hubs,” i.e., low-power coprocessors that are dedicated to reading, buffering, and processing continuous sensor data for specific functions such as step counting and spoken word detection (for instance, the specific function of detecting the famous popular voice commands “hello google” or “Alexa” for Google’s and Amazon’s vocal assistants). The second strategy consists of choosing the sampling frequency of data collection. The higher the frequency in sampling data, the more energy the sensors, the processor, and the memory unit use. Previous knowledge of the signals and the frequency necessary to capture events is needed to select a sampling frequency which is a good tradeoff between capturing relevant signal information and avoiding an unnecessary battery drop. The third strategy focuses on the applications where the data is processed on the device by strategically selecting lightweight features
to reduce the data processing load. For instance, in inertial data processing, time-domain features have lower computational costs than frequency- and time-frequency-domain features. Considering how sensors' power consumption and applications affect battery life in worn systems with small batteries is essential. Since total power load is hard to estimate, it depends on many external factors such as the main application processor, access to memory by other applications, etc. A good practice is to record battery statistics for several days across different participants to estimate real-life use and average battery life.

## 4.2 Related to the Data

### 4.2.1 Data Collection

Data collection consists of data acquisition, data labeling, and existing database improvement. It is one critical challenge in machine learning and often the most time-consuming step in an end-to-end machine learning application due to the time spent collecting the data, cleaning, labeling (for supervised learning), and visualizing it. The data required by the machine learning models can be experimental, retrospective, observational, and, in some cases, synthetic data. While retrospective data collection methods such as surveys and interviews are easy to deploy, they are subject to recall and to self-selection bias, and they might add tedious collection logistical issues if tools and programs in mobile devices are not deployed. Retrospective data collection is sometimes the only means to capture subjective experiences in daily life. Observation methods such as video-camera surveillance can be impractical for large-scale deployment and are often primarily used in small sample applications. Generation of synthetic data is sometimes necessary to overcome the lack of data in some domains, notably annotated medical data. This kind of data is created to improve AI models through data augmentation from models that simulate outcomes given specific inputs such as bio-inspired data [74], physical simulations, or AI-driven generative models [75]. The issue with this is that there is a lack of regulatory frameworks involving synthetic data and their monitoring. Their evaluation could be done with a Turing test, yet this may be prone to inter- and intra-observer variabilities. Plus, data curation protocols can be as tedious and laborious as collecting and labeling real data.

The availability of large-scale, curated scientific datasets is crucial for developing helpful machine learning benchmarks for scientific problems [76], especially for supervised learning solutions where data volume and modality are relevant [77]. Even though machine learning has been used in many domains, there is still a broad panel of applications and fields, such as neuroscience and psychiatry, with few or even inexistent training databases. This is the case for connected devices' and sensors' derived datasets for brain disorder research. In contrast, there are nowadays larger neuroimaging and biological databases available, e.g., the Alzheimer's Disease Neuroimaging Initiative (ADNI) and the Allen Brain
Atlas. Fortunately, following the moderate adoption of machine
learning in the brain research field, a trend toward increasing shar-
ing of resources has emerged, but for now, it is mainly in the
neuroimaging field. Each year, more scientific open data becomes
available, although their curation, maintenance, and distribution
for public consumption are challenging, especially for large-scale
datasets. Another increasing trend in data collection or human
annotation of data is through crowdsourcing marketplaces, having
the advantage of giving access to diverse profiles from a large
population sample, enabling to find more representative examples
to train the models.

For applications in medicine and healthcare, the datasets used to
train the ML models should undergo detailed examination because
they are central to understanding the model's biases and pitfalls.
Before adopting an openly available dataset or creating one, there
are some considerations that we have to keep in mind. Firstly,
ensure a minimal chance of sample selection biases in the database
(for instance, data acquired with particular equipment or with a
particular setting). Errors from sample selection biases become
evident when the model is deployed in settings different from
those used for training. Secondly, we must be aware of the class
imbalance problem that often occurs in cases where the data is rare
(for instance, in low samples associated with rare diseases), which
could negatively affect models designed for prognosis and early
diagnosis. A few techniques can be adopted to help with class
imbalance, such as resampling, adding synthetic data, or working
directly with the model, such as weighting the cost function of
neural networks.

The data, often obtained from scientific experiments, should be
rich enough to allow different analysis and exploration methods
and carefully labeled when required. For instance, a semantic dis-
crepancy in the labels can dilute the training pool and confuse the
classifier [78, 79]. In contrast with free-form text or audio to mark
the activities, imperfect labeling by the users can occur when scor-
ing the samples with fixed labels. For instance, labeling similar
activities from IMU systems, such as running and jogging under
the fixed label-running, can induce errors in feature extraction
because of their interactivity similarity.

It is also important for signal richness to consider subject
variability and consider differences between gender, age, and any
other characteristic that could lead to improper data representation.
Naïve assumptions can cause actual harm by stigmatizing a popula-
tion subgroup when there is an implicit bias in data collection,
selection, and processing [80]. These can be addressed by expand-
ing the solutions to inclusion at all levels and carefully auditing all
stages of the development pipeline.
## 4.3 Deployment for Real-Life Applications

Real-work applicability requires that the accuracy be tested off laboratory settings, considering real-life factors besides technology function and data collection. Before deploying a solution using connected objects and sensors, these real-life use considerations should be addressed without deprecation. Indeed, factors such as user acceptance and behavior around the devices might even be more important than having a high positive prospect of technology. Here we briefly present three factors to keep in mind. These include thinking ahead of privacy issues and how to handle them, the potential degree of adoption, and wearability and instrumentation unobtrusiveness.

### 4.3.1 Health Privacy

Using mobile devices, connected objects, and sensors to collect data for machine learning for health applications is a process that generates data from human lives. In this sense, privacy is a common concern with health data. The concept of privacy in health refers to the contextual rules around generated data or information: how it flows depending on the actors involved, what is the process by which it is accessed, the frequency of the access, and the purpose of the access [81].

The machine learning community has generally valued and embraced the concept of openness. It is common for code and datasets to be publicly released and paper preprints to be available on dedicated archival services before an article is published (despite rejection). Therefore, regulatory bodies should encourage and enforce data holders to collect and provide data under clear legal protection. To ensure data security, these regulations might suggest adopting different solutions: not transmitting raw data, having an isolated sensor network, transmitting encrypted data, and controlling data access authorizations [82]. While individual countries decide where to draw the line regarding regulations, sometimes, depending on the data type, this is more or less difficult to define. For instance, there might be clearer limits on the exploitation and use of patient video recordings because there is explicit reasoning that the patient's identity is easily accessible with image processing. In contrast, this reasoning is less straightforward with other types of data. For instance, even though inertial sensor data might be sufficient to obtain information about a person based on their biometric movement patterns, these sensors are currently not perceived as particularly sensitive by the public. Part of this is because their privacy implications are less well-understood [83]. Thus, they tend to be much less protected (e.g., in wearable devices and mobile apps) compared to other sensors such as GPS, cameras, and microphones. Therefore, requiring proper permission, conscious advertised participation, and explicit consent from the user is essential, no matter the nature of the data collected.
### 4.3.2 Perception and Adoption

The perception and adoption of mobile devices, connected objects, and sensors refer to the negative or positive way the deployed solutions are regarded, understood, and interpreted by the users. This degree of perception directly affects the adherence to a protocol and the solution's use or adoption over the long term. It is one of the most important factors to evaluate ahead of deployment in real-life scenarios. It implies a conscious effort to understand the patient's situation and point of view. It can be overseen by developers and researchers who could focus more on the technical or scientific challenges to overcome or who, because of naivety or distance to the patient's reality, might unwarily not include these considerations in their designs.

The Technology Acceptance Model (TAM) [84], which can be applied to mobile devices, connected objects, and sensors, postulates that two factors predict technology acceptance. The first one is the perceived usefulness or the degree to which a person believes a particular solution will enhance or improve the performance of a specific task. The second factor is perceived ease of use or the degree to which a person believes the solution proposed will be free of effort. The perceived ease of use and usefulness might vary according to the population target and should be studied carefully before deployment. For instance, the perceived ease of use is essential for the elderly [85], who are not core consumers of mobile wireless healthcare technology. There are, of course, other models and theories [86, 87] that have been published since the TAM was proposed, and they include other essential factors to take into account, such as social influence, performance and effort expectancy, and facilitating conditions, or the perceptions of the resources and support available. Although these models have several limitations [88], the identified factors are a good starting point to consider when designing a solution involving wearable, mobile devices, and connected objects. In addition to those factors, clear limits in the cost and benefit ratio of the technology must be communicated since it is one of the main barriers to their acceptance. In that sense, the scientific and healthcare community is responsible for efficiently approaching the patients and clearly explaining the expected positive outcomes and the advantages and disadvantages of the device's ecosystems.

### 4.3.3 Wearability and Instrumentation Unobtrusiveness

Wearability refers to the locations where the sensors are placed and how they are attached to those locations. Wearable devices are typically attached to the body or embedded in clothes and accessories. They are smartwatches and bracelets for activity trackers, smart jewelry, smart clothing, head-mounted devices, and ear devices [89]. Wearability is an aspect to consider because of its direct impact on data collection, signal richness, and quality. The goal is to ensure the device's prolonged and correct use.
In the 1990s, Gemperle and colleagues [90] proposed the first ergonomic guidelines on wearability. Since then, different “wearability maps” have been proposed to approximate the best unobtrusive locations for sensor placement in the human body. A source of the problem in wearability is that the sensors should be securely attached to the human body to prevent relative motion, signal artifacts, and degraded sensing accuracy. Smartwatches are desired to be worn on the dominant arm to capture most of the hand movement, but it is more comfortable for people to wear them on the passive arm.

Similar to wearable devices, one desired characteristic of deployed sensors and tags in the environment is to ensure unobtrusiveness. Unobtrusive sensing allows continuous recording of the patient's activities, behaviors, and physiological parameters without inconveniences to everyday life [82]. This can be achieved by embedding small objects interacting with the subject into the ambient environment, for which the design and usability [91], especially for long-term monitoring, have been considered. There are some devices that are perceived as more invasive than others. For instance, special measures are taken when using cameras regarding sensor selection and sensor placement [82].

## 4.4 Incorporation into Clinical Care

Although there is great potential for connected devices and sensors to prevent, early diagnose, monitor, and create tailored help for patients suffering from brain diseases, there is still a gap to fill to drive transformational changes in health. Besides the challenges mentioned in this section, significant barriers to clinical adoption include the lack of evidence in support of clinical use, the rapid technological development and obsolescence, and the lack of reimbursement models. These problems are often highlighted in preliminary reports of government proposals [92–94] and publications related to mobile health challenges [95, 96].

There is a need for an extensive collection of real-world patient-generated data to reinforce clinical evidence that will change health-care delivery. To date, there is a limitation due to an underpowered number of available pilot datasets that make the comparability of studies difficult and therefore the adoption of these new technologies into the clinical field. Indeed, sensor datasets come mainly from actigraphy and are not as numerous as available neuroimaging, MEG, or EEG datasets.

Opposite to the few large patient-generated evidence, the number of solutions for connected devices and sensors with added features continues to grow every year. This rapid development of technologies represents a challenge to clinicians who might perceive difficulty in the feasibility and scalability of real-word implementations within the clinical workflow, especially since it is noticeable that devices become obsolete, outdated, or no longer useful very quickly. Another negative impact of the higher number
of alternatives in the market is that too much choice can be over-
whelming. In clinical trials and research, it can be challenging to
choose a technical solution when there is little or no clinical evi-
dence and when the features proposed differ significantly between
solutions. Even with well-established companies, for the consu-
mers, there is no guarantee that a product or its support will not
be discontinued in the short term or that the product will not be
rapidly replaced with a newer model. At the same time, ensuring
that the chosen product will be well integrated with other products
(e.g., compatible bricks between other sensors, software, operating
systems, and processing units) is challenging. These factors add up
to the paradox of choice [97], and it is a known consequence of
choice overload.

With newer connected objects and sensors that appear in the
market every month, there is also a rise in their associated mobile
applications available. Among these mobile applications, the most
popular categories are sports and fitness activity trackers, diet and
nutrition, weight loss coaching, stress reduction and relaxation,
menstrual period and pregnancy tracking, hospital or medical
appointment tracking, patient community, and telemedicine
[98]. Most of these applications are not regulated medical health
solutions that work with certified medical devices. They are dedi-
cated to consumers only (not intended for collaboration between
patients and healthcare professionals) and are usually considered or
displayed as well-being apps. In this sense, while various govern-
ments worldwide have opted for different lines of action regarding
the consideration of connected objects and sensors in their health
programs, the appropriate reimbursement models in place are far
from being well integrated into regulatory norms. Take the exam-
ple of France, where connected objects are rarely reimbursed by
social security. For a product to be prescribed by a physician, it must
be considered a regulatorily approved medical device, i.e., be
registered in an official list of medical services and products. This
list also establishes the proper use of the device, the support cost,
the characteristics of the product, and the number of possible
prescription renewals. The heavy administrative burden required
to get registered discourages potential players from requesting
medical approval. In particular, the product has to meet several
compliance rules of the High Authority of Health (HAS), including
the proven good performance of the connected object, the reliabil-
ity of the medical data transmitted, and the respect and protection
of personal and confidential data.

Even though many available connected objects and their
mobile applications are not regulated medical health solutions,
their rapid spread and adoption among the public are starting to
pave the way for motivating future democratization and integration
of these devices in public health policies.
# 5 Discussion

With the amount of innovation and development of smart devices and connected objects, together with the widespread of ML algorithms implemented in faster processing units, we are now many steps closer to having a better understanding of the underlying neural mechanisms of brain disorders with the hope to better intervene at different stages: by preventing health decline, by early and more accurately diagnosing, and by helping to better treat and monitor patients.

In this chapter, we presented the different types of data that one can gather with these devices according to the passive or active role that the user takes in their collection. Many of them are now widely adopted by modern society and used for self-monitoring (e.g., fitness trackers containing IMUs) or in smart home settings (e.g., virtual assistants and presence detectors). When these devices are used together, they represent an opportunity for data fusion allowing the joint analysis of multiple datasets that provide an enhanced complementary view of the phenomenon of interest (e.g., detecting a compulsive behavior like handwashing by combining inertial and acoustic data from a smartwatch). Without a doubt, some brain disorders are better suited for sensor-based assessments, like PD, because of their prominent motor symptoms, unlike other brain disorders whose symptom assessment requires the combination of close behavior observation and access to mental insight (e.g., mood disorders). In the second case, combining sensor data would reduce uncertainty in monitoring and diagnosing, especially when the samples are taken continuously in an ecological manner.

Despite the promising results obtained with these intelligent systems, several conditions need to be addressed before a lab-made application becomes integrated into the clinical routine and in an unsupervised domestic environment. Indeed, most publications do not reach the final phase to be considered as medical devices. Concerning the use of sensors and devices for data collection, a series of considerations to be regarded was presented in Subheading 4. Even though this list could be extended, overall, the main goal remains to assure reproducibility and unbiased collection of high-quality data since ML models can only go so far as the data they rely on.

An exciting, promising extension of the capabilities of smart devices and connected objects is their integration in a closed-loop setting, where the devices serve as real-time continuous monitoring tools that respond to events of interest to treat or intervene on demand and in real time. Indeed, this is a promising approach because of the advantage of early intervention.
Furthermore, we are currently experiencing a new medical revolution with new sensors. Besides what has been presented here, much effort has been put into developing wearable biosensors. These are sensing devices that recognize biological elements (e.g., enzymes, antibodies, and cell receptors), the most known example being glucose monitoring devices. These bioreceptor units are still in their infancy in terms of use and acceptance by the neuroscientific field and medical community in general, but we anticipate that their use and development will continue to grow in the brain disorder research field as smart devices and connected objects have.

Finally, as data and better processing techniques keep increasing, more collaborations between engineers, researchers, and clinicians are formed to contribute to the field of brain disorders positively. We believe that, in the foreseeable future, the rapid evolution of the presented technologies, their use, and their adoption will be key to revolutionizing and addressing the challenges of the traditional medical approach regarding brain disorders.

## Acknowledgments

This work was supported by the following grants (SLMG, EB): Agence Nationale de la Recherche (ANR-19-ICM-DOPALOOPS, ANR-22-ICM-PREDICTOC) and Fondation de France.
The authors would like to thank Dr. Renaud David for reviewing this chapter and providing insightful comments.

## References

1. Sim I (2019) Mobile devices and health. N Engl J Med 381(10):956-968. https://doi.org/10.1056/NEJMra1806949

2. Laput G et al (2021) Methods and apparatus for detecting individual health related events. US20210063434A1, 04 Mar 2021. Accessed: 04 Oct 2022. [Online]. Available: https://patents.google.com/patent/US20210063434A1/en

3. Merkel S, Kucharski A (2019) Participatory design in gerontechnology: a systematic literature review. The Gerontologist 59(1):e16-e25. https://doi.org/10.1093/geront/gny034

4. SENSE-PARK Consortium et al (2015) Participatory design in Parkinson's research with focus on the symptomatic domains to be measured. J Parkinsons Dis 5(1):187-196. https://doi.org/10.3233/JPD-140472

5. Thabrew H, Fleming T, Hetrick S, Merry S (2018) Co-design of eHealth interventions with children and young people. Front

Psychiatry 9. Accessed: 04 Oct 2022. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpsyt.2018.00481

6. Szul MJ, Bompas A, Sumner P, Zhang J (2020) The validity and consistency of continuous joystick response in perceptual decision-making. Behav Res Methods 52(2):681-693. https://doi.org/10.3758/s13428-019-01269-3

7. Li X, Liang Z, Kleiner M, Lu Z-L (2010) RTbox: a device for highly accurate response time measurements. Behav Res Methods 42(1):212-225. https://doi.org/10.3758/BRM.42.1.212

8. Spivey MJ, Dale R (2006) Continuous dynamics in real-time cognition. Curr Dir Psychol Sci 15(5):207-211. https://doi.org/10.1111/j.1467-8721.2006.00437.x

9. Piwek L, Ellis DA, Andrews S, Joinson A (2016) The Rise of Consumer Health Wearables: Promises and Barriers. PLoS Med 13(2):
e1001953. https://doi.org/10.1371/journal.pmed.1001953

10. Faisal S, Ivo J, Patel T (2021) A review of features and characteristics of smart medication adherence products. *Can Pharm J* (Ott) 154(5):312–323. https://doi.org/10.1177/17151635211034198

11. Laine C, Davidoff F (1996) Patient-centered medicine: a professional evolution. *JAMA* 275(2):152–156. https://doi.org/10.1001/jama.1996.03530260066035

12. Laput G, Harrison C (2019) Sensing fine-grained hand activity with smartwatches. In: Proceedings of the 2019 CHI conference on human factors in computing systems, Glasgow, Scotland UK, May 2019, pp 1–13. https://doi.org/10.1145/3290605.3300568

13. Yang C-C, Hsu Y-L (2010) A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors 10(8):7772–7788. https://doi.org/10.3390/s100807772

14. Jones PJ et al (2021) Feature selection for unsupervised machine learning of accelerometer data physical activity clusters – a systematic review. *Gait Posture* 90:120–128. https://doi.org/10.1016/j.gaitpost.2021.08.007

15. Bouten CVC, Koekkoek KTM, Verduin M, Kodde R, Janssen JD (1997) A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity. *IEEE Trans Biomed Eng* 44(3):136–147. https://doi.org/10.1109/10.554760

16. Laput G, Xiao R, Harrison C (2016) ViBand: high-fidelity bio-acoustic sensing using commodity smartwatch accelerometers. In: Proceedings of the 29th annual symposium on user interface software and technology, New York, NY, USA, Oct 2016, pp 321–333. https://doi.org/10.1145/2984511.2984582

17. Stisen A et al (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of the 13th ACM conference on embedded networked sensor systems, New York, NY, USA, Nov 2015, pp 127–140. https://doi.org/10.1145/2809695.2809718

18. Khan AM, Lee YK, Lee SY (2010) Accelerometer's position free human activity recognition using a hierarchical recognition model. In: The 12th IEEE international conference on e-health networking, applications and services, July 2010, pp 296–301. https://doi.org/10.1109/HEALTH.2010.5556553

19. Sousa Lima W, Souto E, El-Khatib K, Jalali R, Gama J (2019) Human activity recognition

using inertial sensors in a smartphone: an overview. Sensors 19(14):14. https://doi.org/10.3390/s19143213

20. Webber M, Rojas RF (2021) Human activity recognition with accelerometer and gyroscope: a data fusion approach. *IEEE Sensors J* 21(15): 16979–16989. https://doi.org/10.1109/JSEN.2021.3079883

21. Castanedo F (2013) A review of data fusion techniques. *Sci World J* 2013:e704504. https://doi.org/10.1155/2013/704504

22. Islam T, Islam MS, Shajid-Ul-Mahmud M, Hossam-E-Haider M (2017) Comparison of complementary and Kalman filter based data fusion for attitude heading reference system. *AIP Conf Proc* 1919(1):020002. https://doi.org/10.1063/1.5018520

23. Nweke HF, Teh YW, Mujtaba G, Al-garadi MA (2019) Data fusion and multiple classifier systems for human activity detection and health monitoring: review and open research directions. *Inf Fusion* 46:147–170. https://doi.org/10.1016/j.inffus.2018.06.002

24. Sprager S, Juric M (2015) Inertial sensor-based gait recognition: a review. *Sensors* 15(9): 22089–22127. https://doi.org/10.3390/s150922089

25. Breasail MÓ et al (2021) Wearable GPS and accelerometer technologies for monitoring mobility and physical activity in neurodegenerative disorders: a systematic review. *Sensors* 21(24):24. https://doi.org/10.3390/s21248261

26. Jankowska MM, Schipperijn J, Kerr J (2015) A framework for using GPS data in physical activity and sedentary behavior studies. *Exerc Sport Sci Rev* 43(1):48–56. https://doi.org/10.1249/JES.000000000000035

27. Maddison R, Ni Mhurchu C (2009) Global positioning system: a new opportunity in physical activity measurement. *Int J Behav Nutr Phys Act* 6(1):73. https://doi.org/10.1186/1479-5868-6-73

28. Krenn PJ, Titze S, Oja P, Jones A, Ogilvie D (2011) Use of global positioning systems to study physical activity and the environment: a systematic review. *Am J Prev Med* 41(5): 508–515. https://doi.org/10.1016/j.amepre.2011.06.046

29. Fagherazzi G, Fischer A, Ismael M, Despotovic V (2021) Voice for health: the use of vocal biomarkers from research to clinical practice. *Digit Biomark* 5(1):78–88. https://doi.org/10.1159/000515346

30. Fedotov D, Matsuda Y, Minker W (2019) From smart to personal environment: integrating emotion recognition into smart houses. In:
2019 IEEE international conference on pervasive computing and communications workshops (PerCom Workshops), Mar 2019, pp 943-948. https://doi.org/10.1109/PERCOMW.2019.8730876

31. De Miguel K, Brunete A, Hernando M, Gambao E (2017) Home camera-based fall detection system for the elderly. Sensors 17(12):12. https://doi.org/10.3390/s17122864

32. Koutli M, Theologou N, Tryferidis A, Tzovaras D (2019) Abnormal behavior detection for elderly people living alone leveraging IoT sensors. In: 2019 IEEE 19th international conference on Bioinformatics and Bioengineering (BIBE), Oct 2019, pp 922-926. https://doi.org/10.1109/BIBE.2019.00173

33. Aubourg T, Demongeot J, Renard F, Provost H, Vuillerme N (2019) Association between social asymmetry and depression in older adults: a phone Call Detail Records analysis. Sci Rep 9(1):1. https://doi.org/10.1038/s41598-019-49723-8

34. Davies N, Friday A, Newman P, Rutlidge S, Storz O (2009) Using bluetooth device names to support interaction in smart environments. In: Proceedings of the 7th international conference on mobile systems, applications, and services, New York, NY, USA, pp 151-164. https://doi.org/10.1145/1555816.1555832

35. Barthe G et al (2022) Listening to bluetooth beacons for epidemic risk mitigation. Sci Rep 12(1):1. https://doi.org/10.1038/s41598-022-09440-1

36. Box-Steffensmeier JM et al (2022) The future of human behaviour research. Nat Hum Behav 6(1):15-24. https://doi.org/10.1038/s41562-021-01275-6

37. Kourtis LC, Regele OB, Wright JM, Jones GB (2019) Digital biomarkers for Alzheimer's disease: the mobile/wearable devices opportunity. NPJ Digit Med 2(1):9. https://doi.org/10.1038/s41746-019-0084-2

38. Channa A, Popescu N, Ciobanu V (2020) Wearable solutions for patients with Parkinson's disease and neurocognitive disorder: a systematic review. Sensors 20(9):2713. https://doi.org/10.3390/s20092713

39. Asadi-Pooya AA, Mirzaei Damabi N, Rostaminejad M, Shahisavandi M, Asadi-Pooya A (2021) Smart devices/mobile phone in patients with epilepsy? A systematic review. Acta Neurol Scand 144(4):355-365. https://doi.org/10.1111/ane.13492

40. Marziniak M, Brichetto G, Feys P, Meyding-Lamadé U, Vernon K, Meuth SG (2018) The use of digital and remote communication

technologies as a tool for multiple sclerosis management: narrative review. JMIR Rehabil Assist Technol 5(1):e7805. https://doi.org/10.2196/rehab.7805

41. Torous J, Onnela J-P, Keshavan M (2017) New dimensions and new tools to realize the potential of RDoC: digital phenotyping via smartphones and connected devices. Transl Psychiatry 7(3):e1053. https://doi.org/10.1038/tp.2017.25

42. Pew Research Center (2021) Mobile fact sheet. Pew Research Center: Internet, Science & Tech. 07 Apr 2021. https://www.pewresearch.org/internet/fact-sheet/mobile/(accessed 06 Oct 2022).

43. Cho C-H et al (2020) Effectiveness of a smartphone app with a wearable activity tracker in preventing the recurrence of mood disorders: prospective case-control study. JMIR Ment Health 7(8):e21283. https://doi.org/10.2196/21283

44. Brassey J, Güntner A, Isaak K, Silberzahn T (2021) Using digital tech to support employees' mental health and resilience. McKinsey & Company

45. Deady M et al (2022) Preventing depression using a smartphone app: a randomized controlled trial. Psychol Med 52(3):457-466. https://doi.org/10.1017/S0033291720002081

46. Vogels EA (2020) About one-in-five Americans use a smart watch or fitness tracker. Pew Research Center. 09 Jan 2020. https://www.pewresearch.org/fact-tank/2020/01/09/about-one-in-five-americans-use-a-smart-watch-or-fitness-tracker/ (accessed 06 Oct 2022).

47. Donev R, Kolev M, Millet B, Thome J (2009) Neuronal death in Alzheimer's disease and therapeutic opportunities. J Cell Mol Med 13(11-12):4329-4348. https://doi.org/10.1111/j.1582-4934.2009.00889.x

48. Michel PP, Hirsch EC, Hunot S (2016) Understanding dopaminergic cell death pathways in Parkinson disease. Neuron 90(4): 675-691. https://doi.org/10.1016/j.neuron.2016.03.038

49. Boillée S, Vande Velde C, Cleveland DW (2006) ALS: a disease of motor neurons and their nonneuronal neighbors. Neuron 52(1): 39-59. https://doi.org/10.1016/j.neuron.2006.09.018

50. Lan K-C, Shih W-Y (2014) Early diagnosis of Parkinson's disease using a smartphone. Procedia Comput Sci 34:305-312. https://doi.org/10.1016/j.procs.2014.07.028
51. Levy R, Dubois B (2006) Apathy and the func-
tional anatomy of the prefrontal cortex-basal
ganglia circuits. Cereb Cortex 16(7):916-928.
https://doi.org/10.1093/cercor/bhj043

52. Saeb S et al (2015) Mobile phone sensor cor-
relates of depressive symptom severity in daily-
life behavior: an exploratory study. J Med
Internet Res 17(7):e175. https://doi.org/10.
2196/jmir.4273

53. Kluge A et al (2018) Combining actigraphy, ecological momentary assessment and neuro-imaging to study apathy in patients with schizophrenia. Schizophr Res 195:176-182. https://doi.org/10.1016/j.schres.2017.09.034

54. Faedda GL et al (2016) Actigraph measures discriminate pediatric bipolar disorder from attention-deficit/hyperactivity disorder and typically developing controls. J Child Psychol Psychiatry 57(6):706-716. https://doi.org/10.1111/jcpp.12520

55. Favela J, Cruz-Sandoval D, Morales-Tellez A, Lopez-Nava IH (2020) Monitoring behavioral symptoms of dementia using activity trackers. J Biomed Inform 109:103520. https://doi. org/10.1016/j.jbi.2020.103520

56. Manley NA et al (2020) Long-term digital device-enabled monitoring of functional status: implications for management of persons with Alzheimer's disease. Alzheimers Dement Transl Res Clin Interv 6(1):e12017. https://doi.org/10.1002/trc2.12017

57. Wainberg M et al (2021) Association of accelerometer-derived sleep measures with lifetime psychiatric diagnoses: a cross-sectional study of 89,205 participants from the UK Biobank. PLoS Med 18(10):e1003782. https://doi.org/10.1371/journal.pmed.1003782

58. Fellendorf FT et al (2021) Monitoring sleep changes via a smartphone app in bipolar disorder: practical issues and validation of a potential diagnostic tool. Front Psychiatry 12:641241. https://doi.org/10.3389/fpsyt.2021.641241

59. Gillani N, Arslan T (2021) Intelligent sensing technologies for the diagnosis, monitoring and therapy of Alzheimer's disease: a systematic review. Sensors 21(12):4249. https://doi.org/10.3390/s21124249

60. Karakostas A et al (2020) A French-Greek cross-site comparison study of the use of automatic video analyses for the assessment of autonomy in dementia patients. Biosensors 10(9):E103. https://doi.org/10.3390/bios10090103

61. Lyons BE et al (2015) Pervasive computing technologies to continuously assess Alzheimer's disease progression and intervention

efficacy. Front Aging Neurosci 7:102. https://
doi.org/10.3389/fnagi.2015.00102

62. Cullen A, Mazhar MKA, Smith MD, Lithander FE, Breasail MO, Henderson EJ (2022) Wearable and portable GPS solutions for monitoring mobility in dementia: a systematic review. Sensors 22(9):3336. https://doi.org/10.3390/s22093336

63. Boukhechba M, Chow P, Fua K, Teachman BA, Barnes LE (2018) Predicting social anxiety from global positioning system traces of college students: feasibility study. JMIR Ment Health 5(3):e10101. https://doi.org/10.2196/10101

64. Chen B-R et al (2011) A web-based system for home monitoring of patients with Parkinson's disease using wearable sensors. IEEE Trans Biomed Eng 58(3):831-836. https://doi.org/10.1109/TBME.2010.2090044

65. Morgiève M et al (2020) A digital companion, the Emma app, for ecological momentary assessment and prevention of suicide: quantitative case series study. JMIR MHealth UHealth 8(10):e15741. https://doi.org/10.2196/15741

66. Seppälä J et al (2019) Mobile phone and wearable sensor-based mHealth approaches for psychiatric disorders and symptoms: systematic review. JMIR Ment Health 6(2):e9819. https://doi.org/10.2196/mental.9819

67. Cain AE, Depp CA, Jeste DV (2009) Ecological momentary assessment in aging research: a critical review. J Psychiatr Res 43(11): 987-996. https://doi.org/10.1016/j.jpsychires.2009.01.014

68. Rugg-Gunn F (2020) The role of devices in managing risk. Epilepsy Behav 103. https://doi.org/10.1016/j.yebeh.2019.106456

69. Usmani S, Saboor A, Haris M, Khan MA, Park H (2021) Latest research trends in fall detection and prevention using machine learning: a systematic review. Sensors 21(15):5134. https://doi.org/10.3390/s21155134

70. Omberg L et al (2022) Remote smartphone monitoring of Parkinson's disease and individual response to therapy. Nat Biotechnol 40(4): 4. https://doi.org/10.1038/s41587-021-00974-9

71. Robert P et al (2021) Efficacy of serious exergames in improving neuropsychiatric symptoms in neurocognitive disorders: results of the X-TORP cluster randomized trial. Alzheimers Dement Transl Res Clin Interv 7(1). https://doi.org/10.1002/trc2.12149

72. Balaskas A, Schueller SM, Cox AL, Doherty G (2021) Ecological momentary interventions for mental health: a scoping review. PLoS One
16(3):e0248152. https://doi.org/10.1371/journal.pone.0248152

73. Stern E et al (2022) How can digital mental health enhance psychiatry? Neuroscientist 10738584221098604. https://doi.org/10.1177/10738584221098603

74. Mondragón-González SL, Burguière E (2017) Bio-inspired benchmark generator for extracellular multi-unit recordings. Sci Rep 7(1):1. https://doi.org/10.1038/srep43253

75. Chen RJ, Lu MY, Chen TY, Williamson DFK, Mahmood F (2021) Synthetic data in machine learning for medicine and healthcare. Nat Biomed Eng 5(6):6. https://doi.org/10.1038/s41551-021-00751-8

76. Thiyagalingam J, Shankar M, Fox G, Hey T (2022) Scientific machine learning benchmarks. Nat Rev Phys 4(6):6. https://doi.org/10.1038/s42254-022-00441-7

77. Myszczyńska MA et al (2020) Applications of machine learning to diagnosis and treatment of neurodegenerative diseases. Nat Rev Neurol 16(8):440–456. https://doi.org/10.1038/s41582-020-0377-8

78. Abdullah S, Lane N, Choudhury T (2012) Towards population scale activity recognition: a framework for handling data diversity. Proc AAAI Conf Artif Intell 26(1):851–857

79. Peebles D, Lu H, Lane N, Choudhury T, Campbell A (2010) Community-guided learning: Exploiting mobile sensor users to model human behavior. Proc AAAI Conf Artif Intell 24(1):1600–1606. https://doi.org/10.1609/aaai.v24i1.7731

80. Ghassemi M, Mohamed S (2022) Machine learning and health need better values. NPJ Digit Med 5(1):1. https://doi.org/10.1038/s41746-022-00595-9

81. Price WN, Cohen IG (2019) Privacy in the age of medical big data. Nat Med 25(1):37–43. https://doi.org/10.1038/s41591-018-0272-7

82. Wang J, Spicher N, Warnecke JM, Haghi M, Schwartz J, Deserno TM (2021) Unobtrusive health monitoring in private Spaces: the smart home. Sensors 21(3):864. https://doi.org/10.3390/s21030864

83. Kröger JL, Raschke P, Bhuiyan TR (2019) Privacy implications of accelerometer data: a review of possible inferences. In: Proceedings of the third international conference on cryptography, security and privacy – ICCSP ’19, Kuala Lumpur, Malaysia, pp 81–87. https://doi.org/10.1145/3309074.3309076

84. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of

information technology. MIS Q 13(3): 319–340. https://doi.org/10.2307/249008

85. Moore K et al (2021) Older adults' experiences with using wearable devices: qualitative systematic review and meta-synthesis. JMIR MHealth UHealth 9(6):e23832. https://doi.org/10.2196/23832

86. Bagozzi RP (2007) The legacy of the technology acceptance model and a proposal for a paradigm shift. J Assoc Inf Syst 8(4):3

87. Venkatesh T, Xu X (2012) Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q 36(1):157. https://doi.org/10.2307/41410412

88. Legris P, Ingham J, Collerette P (2003) Why do people use information technology? A critical review of the technology acceptance model. Inf Manag 40(3):191–204. https://doi.org/10.1016/S0378-7206(01)00143-4

89. Sarkar S, Chakrabarti D (2021) The perception and acceptance of wearable fitness devices among people and designing interventions for prolonged use. In: Ahram TZ, Falcão CS (eds) Advances in usability, user experience, wearable and assistive technology, vol 275. Springer International Publishing, Cham, pp 94–101. https://doi.org/10.1007/978-3-030-80091-8_12

90. Gemperle F, Kasabach C, Stivoric J, Bauer M, Martin R (1998) Design for wearability. In: Digest of Papers. Second international symposium on wearable computers (Cat. No. 98EX215), pp 116–122. https://doi.org/10.1109/ISWC.1998.729537

91. Zheng Y-L et al (2014) Unobtrusive Sensing and Wearable Devices for Health Informatics. IEEE Trans Biomed Eng 61(5):1538–1554. https://doi.org/10.1109/TBME.2014.2309951

92. European Commission (2014) Green paper on mobile health ('mHealth'). Digit Agenda Eur

93. Haut Autorité de santé (2019) Rapport d'analyse prospective 2019 Numérique: quelle (R)volution?. [Online]. Available: https://www.has-sante.fr/upload/docs/application/pdf/2019-07/rapport_analyse_prospective_20191.pdf

94. U.S. Department of Health and Human Services Food and Drug Administration (2016) Use of real-world evidence to support regulatory decision-making for medical devices. Guidance for Industry and Food and Drug Administration Staff

95. Steinhubl SR, Muse ED, Topol EJ (2015) The emerging field of mobile health. Sci Transl Med
7(283):283rv3. https://doi.org/10.1126/
scitranslmed.aaa3487

96. Torous J et al (2021) The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry 20(3):318–335.
https://doi.org/10.1002/wps.20883

97. Schwartz B (2004) The paradox of choice: Why more is less. HarperCollins Publishers, New York, p xi, 265

98. User engagement and abandonment of mHealth: a cross-sectional survey – PubMed. https://pubmed.ncbi.nlm.nih.gov/3520683 7/ (accessed 07 Dec 2022)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.