# SpECTRUM: Smart ECosystem for sTStroke patient's Upper limbs Monitoring

<figure><img src="image_1.png" /></figure>

CrossMark

Maxence Bobin<sup>a,*</sup>, Franck Bimbard<sup>a</sup>, Mehdi Boukallel<sup>b</sup>,
Margarita Anastassova<sup>b</sup>, Mehdi Ammi<sup>a</sup>

## ARTICLE INFO

**Article history:**

Received 4 July 2018

Received in revised form

20 November 2018

Accepted 18 January 2019

Available online 7 February 2019

**Keywords:**

Stroke

Monitoring

Internet of things

Rehabilitation

Motor activity of the upper Limbs

## ABSTRACT

This paper presents a new ecosystem of smart objects designed to monitor motor functions of stroke patients during rehabilitation sessions at the hospital. The ecosystem has been designed starting from an observational study as well as the Action Research Arm Test. It includes a jack and a cube for hand grasping monitoring and a smart watch for the arm dynamic monitoring. The objects embed various sensors able to monitor the pressure of the fingers, the position of the fingers, their orientation, their movements and the tremors of the patient during the manipulation tasks. The developed objects can connect, via Bluetooth or Wi-Fi technology, to an Android mobile application in order to send collected data during the execution of the manipulation task. Performances achieved during the sessions will be displayed on the tablet. Using the collected data, the therapists could assess the upper arm motor abilities of the patient by accessing qualitative information that is usually evaluated by visual estimations or not reported and adapt the rehabilitation program if necessary. The objects, as well as the visualization interfaces, have been evaluated with health care professionals in terms of design and functionalities. The results from this evaluation show that the objects' design is adapted to bring useful information on the patient's motor activities, while the visualization interfaces are useful, but require new functionalities. Finally, a preliminary study has been carried out with stroke patients in order to assess the usability and acceptability of such an ecosystem during rehabilitation sessions. This study indicated that the patients are willing to use the ecosystem during the sessions thanks to its easy usage.

© 2019 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

## 1. Introduction

Worldwide, stroke affects 15 million people each year according to the World Health Organization (WHO) (Organization, 2002) and is the second leading cause of death and the third leading cause of disability across the world (Organization et al., 2017). Survivors often encounter motor or cognitive disorders requiring rehabilitation. Some impairments are usual such as spasticity (Sommerfeld, Eek, Svensson, Holmqvist, & von Arbin, 2004), muscle weakness (Ada, Canning, & Low, 2003) or visual problems (Rowe et al., 2008); but others are uncommon, such as hemibody tremors and can manifest themselves

<footnote><sup>a</sup> LIMSI-CNRS, Orsay, France</footnote>

<footnote><sup>b</sup> CEA-LIST, Palaiseau, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: maxence.bobin@limsi.fr (M. Bobin), franck.bimbard@limsi.fr (F. Bimbard), mehdi.boukallel@cea.fr (M. Boukallel), margarita.anastassova@cea.fr (M. Anastassova), mehdi.ammi@limsi.fr (M. Ammi).</footnote>
months or years after the stroke (Dethy, Luxen, Bidaut, & Goldman, 1993; Kim, 1992). Rehabilitation is a long process that involves medical staff and costly infrastructures for long periods (Anderson et al., 2000). Indeed, patients have to go to the hospital, often daily, at the early stages of post-stroke recovery, to perform rehabilitation exercises. The patient's recovery is assessed by a therapist either from visual observations during rehabilitation sessions (succeed in moving an object, correct global position of the arm, etc.) (Van der Lee et al., 2001; Carr, Shepherd, Nordholm, & Lynne, 1985) or more formally during checkup sessions using standard protocols (e.g. Action Research Arm Test) and specific tools (e.g. manometer or prehension pears) in order to collect quantifiable and objective information on the patient's motor functions. In addition, demographic trends indicate that world population is aging (Bloom, Boersch-Supan, McGee, & Seike, 2011) and the number of stroke patients over 75 years old will increase from 55% in 2005 to 75% in 2050 (Foerch, Sitzer, Steinmetz, & Neumann-Haefelin, 2009). Proportionally, fewer and fewer therapists will be available; and treating all the patients with the same efficiency will be impossible.

Collecting objective information on the patient's motor functions may enhance the patient's recovery thanks to a per-
sonalized treatment. In fact, several research groups investigated the use of technological devices to monitor stroke patients
during the rehabilitation sessions. These devices focus on motor functions monitoring by using serious games and wearables
(Burke et al., 2009: Uswatte et al., 2005). Serious games provide playful environments involving motor functions, but force
the patient to be in front of a screen. Serious games provide new data on the patients' physical state, but do not constitute a
relevant strategy to assess the patient's recovery evolution with quantifiable and objective information, as the required tasks
do not settle in the reality by manipulating physical objects. On the other hand, wearables are often used for everyday life
monitoring. However, wearing sensors is constraining for stroke patients who already have difficulties getting dressed for
example. Wearables are more adapted for monitoring during rehabilitation sessions where a therapist can help the patient
to place the sensors. New platforms based on self-contained objects embedding sensors and displays allow for conceiving
new approaches for stroke monitoring and rehabilitation by collecting objective data during rehabilitation sessions. These
objects communicate with each other, are connected to the Internet and favor connectivity and interoperability.

This paper presents an ecosystem of smart objects, called SpECTRUM, inspired by the current rehabilitation tools observed during rehabilitation sessions and the Action Research Arm Test protocol. SpECTRUM is composed of three objects: (i) a jack to monitor fingers grasping and the input of each finger during grasping, (ii) a cube to monitor the global hand motor functions during manipulation exercises and (iii) a wrist band to monitor the arm motor functions during the ring tree exercise consisting in moving rings along horizontal shafts. The jack, the cube and the wrist band provide relevant and measurable information on the motor functions of the hand and the arm of the patient. These objects are also able to monitor the appearance and evolution of the patient's tremors during exercises. Moreover, we developed an Android application that ensures data management including real-time data visualization and access to data history. With this monitoring platform, the therapists will have better knowledge of the patient's motor functions level and could assess degradation or an improvement of the patient's state thanks to the data history. It is then possible to either adapt the rehabilitation exercises according to the patient's weaknesses or propose a readmission to the hospital if the patient's independence is decreasing.

The paper is organized as follows: existing platforms for hand and arm motor monitoring and rehabilitation are pre-
sented in Section 2. Then, the results of an observational study that we conducted in order to identify the current tools that
are used by health care professionals during rehabilitation session are presented in Section 3. Section 4 introduces the
concept of the SpECTRUM ecosystem that follows the guidelines brought out from the observational study and the Action
Research Arm Test protocol. Section 5 is devoted to the hardware development presentation of the smart objects. After-
wards, a preliminary study was carried out with health care professionals in order to collect feedback on possible
improvements on the objects' functionalities (Section 6). Then, we present the design choices based on the results of the
previous study of a mobile application to record and to perform data visualization collected by the objects in Section 7. A
second preliminary study has been carried out with health care professionals to collect feedback on possible improvements
on the visualization interfaces (Section 7.2). Section 8 presents the results of tests performed with three stroke patients in
order to assess the technical reliability of the SpECTRUM ecosystem. Finally, we conducted a preliminary study involving
nine stroke patients who are asked to assess the usability and the acceptability of the SpECTRUM ecosystem (Section 9).

## 2. Related work

Many researches already investigated the upper limbs motor assessment by monitoring rehabilitation exercises. First, this section presents the scales and tools for motor assessment based on visual estimations and then focuses on the new technological approaches for monitoring rehabilitation exercises.

2.1. Scales and tools for upper limb motor assessment

Many scales have been developed for assessing upper limb motor recovery of stroke patients. Back in the 1960s, the
Swedish occupational therapist Signe Brunnstrom developed an approach, which is now qualified as the "Brunnstrom
Approach", based on a series of longitudinal observations that allows assessment of the motor recovery of the patient
(Brunnstrom, 1966). The Brunnstrom Recovery Stages (BRS) are divided into two stages: (i) Arm assessment (BRS-A)
including seven stages evaluating basic and complex arm controls (e.g. bending, extension or moving forward without moving the trunk) and (ii) hand assessment (BRS-H) including six stages evaluating the recovery of grasping, lateral prehension or palmar prehension. This approach shows strong positive correlations between recovery at admission and discharge (Shah, Harasymiw, & Stahl, 1986). Based on the BRS approach, the Fugle-Meyer Assessment (FMA) has been created and is the first stroke-specific assessment tool following the natural recovery process of a post-stroke hemiparetic patient (Fugl-Meyer, Jääskö, Leyman, Olsson, & Steglind, 1974). The BRS and FMA tools are useful, but follow subjective observations made by the therapists. Other tools exist for assessing the motor recovery of stroke patients, such as the Wolf Motor Function Test (WMFT) (Woodbury et al., 2010), which focuses on constraint-induced movement therapy, or the Motricity Index (MI) (Bohannon, 2001), which focuses on the UE and LE recovery assessment. Finally, the Action Research Arm Test (ARAT) assesses the UE motor functions with four categories: grasp, grip, pinch and gross movements (van der Lee, Beckerman, Lankhorst, & Bouter, 2001). ARAT includes 19 items scored between 0 (cannot perform any part of the test) and 3 (performs the test normally) and grip and gross movements are more used than grasp and pinch (Kwakkel & Kollen, 2006). Moreover, ARAT is easily reproducible (i.e. the test is performed in standard conditions with a specific chair and table set) and requires less time to administer than the BRS or FMA (De Weerdt & Harrison, 1985).

Although many scales are used to assess the arm and hand functions visually, no quantifiable information is collected
offering the possibility to better estimate the patient's motor functions and perform a more individualized treatment.

## 2.2. New platforms for stroke rehabilitation and monitoring

With the recent development of internet of things, different approaches emerged to perform rehabilitation and monitoring of stroke patients' (Fan, Yin, Da Xu, Zeng, & Wu, 2014; Yang et al., 2014; Xu et al., 2014; Yuehong, Zeng, Chen, & Fan, 2016; Yang et al., 2018). This section first presents various rehabilitation platforms and then focuses on activity monitoring platforms.

### 2.2.1. Rehabilitation platforms

Rehabilitation platforms aim to improve the motor functions of patients with different exercises. In most cases, these exercises are presented in the form of fun, original and interactive games in order to maintain patient engagement during the rehabilitation phase despite a intensive and repetitive training (Burke et al., 2009). These exercises are called "serious games" and are designed for a main purpose other than pure entertainment. These platforms are therefore presented in various technological forms: virtual reality, augmented reality or interactive screens.

First of all, Virtual Reality (VR) allows patients to be immersed in a three-dimensional virtual environment via a screen or VR headset. Patients can interact with this virtual environment via sensors to perform various exercises and rehabilitation. Burke et al. used virtual reality by screen to simulate a vibraphone on a computer (Burke et al., 2009). The patient can then interact with the vibraphone and play music using two Wii remote controls. This game seems interesting for the rehabilitation of the wrist and arm with different melodies to accentuate the movements of the affected arm. Others preferred the use of a data glove, i.e. a glove with sensors to track hand movements, to provide rehabilitation exercises. Jack et al. and Boian et al. developed a hand rehabilitation application using a data glove and a force feedback glove to interact with a virtual world (Jack et al., 2001; Boian et al., 2002). Four rehabilitation exercises are available, each designed to exercise a specific parameter of hand movement: range, speed, fractionation or force. VR can be combined with biofeedback and mirror-neurons (i.e using ElectroEncephaloGram, electromyography and Brain Computer Interfaces) to enhance the patient's engagement and to develop new skills (Yin, Fan, & Xu, 2012; Hou & Sourina, 2013; Thomas, Vinod, & Guan, 2013; Cargnina, d'Ornellasa, & Pradob, 2015; Rincon, Yamasaki, & Shimoda, 2016).

Then, Augmented reality (AR) consists of superposing elements (sounds, 2D images, 3D images, videos, etc.) to real-time reality. Burke et al. proposed two augmented reality games that use a webcam to capture reality (Burke et al., 2009). To this reality comes to be superimposed a game interface that evolves according to patients' actions. The first game is to knock out a rabbit coming out of its hole while the second game consists for the patient to follow two arrows, one pointing to the left and the other to the right, and touch the left arrow with the left hand and the right arrow with the right hand. These games allow to work on motor coordination of the upper limbs as well as the amplitude of the movements. Vogiatzaki et al. also developed AR games that use a Kinect as well as video projectors and real physical objects in order to avoid disconnection between the patient and the reality (Vogiatzaki & Krukowski, 2014). The first game consists in placing a physical cube on a target projected on the table while the second exercise consists of throwing with a real paper ball at a virtual target projected on a wall. These exercises involve the arm and hand motor functions and allow to work on motor coordination, prehension and dexterity.

Finally, interactive screens differ from touch screens in their ability to react to surrounding objects. Patients can then interact with these screens using everyday objects. Jacobs et al. developed an exercise that involves the physical manipulation of everyday objects (Jacobs, Timmermans, Michielsen, Vander Plaetse, & Markopoulos, 2013). These objects can be chosen according to the needs and motor abilities of the patient. The purpose of the exercise is to move any object placed on a screen by avoiding bar-shaped obstacles that move from left to right on the screen. Obstacles can be avoided by moving the object on the screen or by lifting it to pass over an obstacle. This exercise helps to work on the motor coordination of the hand and arm.
Although these platforms offer interesting solutions for stroke recovery monitoring, the patients are disconnected from reality by being immersed in the virtual world or being in front of a screen.

### 2.2.2. Monitoring platforms

Unlike rehabilitation platforms, monitoring platforms are intended to monitor the motor activity of patients with no direct objective of improvement. These platforms are essentially based on portable devices, often called wearables, which incorporate sensors and are worn on the human body to track movement and position information of limbs upper or lower. Patients' motor activity can be monitored with the help of wearables integrating electronic or textile sensors.

On the one hand, the use of portable devices integrating electronic sensors enables the design of small, low-cost platforms to improve monitoring of patient activities (Iosa et al., 2012). Many tools based on accelerometers have been developed for monitoring stroke patients' arm activity during rehabilitation (Bussmann, Tulen, van Herel, & Stam, 1998; Busser, De Korte, Glerum, & Van Lummel, 1998). Patel et al. placed bi-axial accelerometers on the arms, forearm and hand as well as uni-axial accelerometers on the thumb, index and chest (Patel et al., 2010). They then assessed the quality of performance of different tasks such as reaching an object near and far, moving your forearm from the knees to the table and compared it to the score that could be obtained with the Functional Ability Scale (FAS) during the assessment by a health professional. The results show that acceleration data collected during the performance of different motor tasks can be used to characterize patient movement and movement quality. Furthermore, some research also studied the monitoring of upper limb movements in space using a kinematic model and multi-sensor fusion (Kemp, Janssen, & van der Kamp, 1998; Williamson & Andrews, 2001; Zijlstra & Bisseling, 2004). Using an inertial unit, Zhou et al. developed a kinematic model of the arm and forearm and compared the results obtained with a motion capture system (Zhou, Hu, & Harris, 2005). The results show that this kinematic model provides sufficient performance to estimate upper limb movement for post-stroke rehabilitation using a weighted least squares filtering algorithm. Moreover, Yan et al. developed a wearable wireless system based on different sensors (heartbeat, ambient temperature, body temperature) to monitor the patient's health conditions and detect anomalies (Yan et al., 2015). Finally, Rigas et al. proposed a wearable solution based on accelerometers to detect tremors of Parkinson's Disease (Rigas et al., 2012). As tremors can appear after a stroke (Bansil et al., 2012; Handley, Medcalf, Hellier, & Dutta, 2009), it should be interesting to use the same method to detect the post-stroke patients tremors. Indeed, some researchers showed that assessing the evolution of the patient's tremors by monitoring the tremors frequency and magnitude may lead to the detection of a relapse of the motor functions (Kim, 2001; Deuschl, Wenzelburger, Löffler, Raethjen, & Stolze, 2000). In addition, post-stroke patients often have tremors with frequency under 5 Hz and perpendicular to the direction of the movement (Smaga, 2003).

On the other hand, new textile fibers enriched with metal particles and conducting electricity allowed to design textile sensors, called *Smart Textiles*. These sensors are commonly based on the piezo-resistivity principle and offer an interesting alternative to electronic sensors (Tognetti et al., 2005; Lorusso, Rocchia, Scilingo, Tognetti, & De Rossi, 2004). They are flexible, inexpensive and can detect stretching or pressure by measuring the variation in electrical resistance. For example, Taccini et al. developed a garment incorporating textile sensors knitted with wool with piezoresistive properties to capture patient movements (Taccini, Loriga, Dittmar, & Paradiso, 2004). These sensors were placed on the elbow, shoulder and buttocks to measure bending angles. Other research used piezoresistive materials to design pressure sensors. Xu, Huang, Amini, He, and Sarrafzadeh (2013) developed a pressure sensor that can detect when a person is sitting. The structure of this sensor consists of the superposition of rows of horizontal and vertical conductive wool yarns separated by a piezoresistive material.

However, these wearables used for post-stroke monitoring are not based on objects used during rehabilitation sessions and require the therapists to change its work habits.

### 2.3. Contribution

The state-of-the-art demonstrates that the current assessment of patient's recovery is performed with visual observation made by the therapist during rehabilitation sessions or more formally during checkup sessions with standard protocols. Among them, ARAT seems the best compromise between administration time and reproducibility and contains objects that can easily embed sensors (a cube or cup, for example). New approaches such as serious games or wearables provide quantifiable information on the motor functions of stroke patients, but can have a heavy impact on the cognitive load or modify the therapist's practices. Embedding sensors inside objects used during rehabilitation sessions or used in a standard protocol seems to serve as an interesting alternative to provide new qualitative information on the patients' motor functions without modifying therapists' practices. Moreover, less adaptation time or appropriation of the system is, thus, required and the patients can entirely focus on the rehabilitation exercises.

Our approach consists of an ecosystem of self-contained objects designed to monitor hand and arm motor functions of stroke patients. This approach aims to complete the visual observations made by the therapists in order to individualize the patients' treatment by monitoring the patient's motor functions during rehabilitation sessions.
# 3. Iteration 1: Observational study

## 3.1. Aim of the study

This study aims to observe occupational therapists during rehabilitation sessions in order to identify the objects and tasks that will be used to design a smart ecosystem of instrumented objects. The ecosystem aims to provide objective and quantifiable information on the parameters subjectively assessed by the therapists for now. These objects will be based on the current rehabilitation tools as well as the ARAT protocol identified previously in order to avoid disturbing patients' and therapists' habits or practices.

## 3.2. Participants

Three rehabilitation centers in the northern of France (Evry, Le Havre, and Lille) were involved in this study. We followed nine health care professionals - including six occupational therapists (OT), two physiotherapists (PT) and one medical doctor (MD) - during their rehabilitation session with stroke patients. The participants were four females and five males aged between 24 and 60 ($\mu$ = 39, $\sigma$ = 12).

## 3.3. Procedure

We followed health care professionals working in functional and medical rehabilitation centers during rehabilitation session. We observed the tasks performed by the patients as well as the tools used during the exercises. Finally, at the end of the day, therapists showed us the other tools for post-stroke rehabilitation and monitoring that were not used during the day by the patients.

## 3.4. Results

The observations reveal two types of rehabilitation objects used by occupational therapists. The first set of objects is used for rehabilitation exercises, while the other set is used for the assessment of physical recovery. Indeed, rehabilitation and assessment tools are different in order to avoid patients learning by heart the tasks required for assessment during rehabilitation sessions.

### 3.4.1. Rehabilitation tools

Rehabilitation tools are varied in size, weight and shape. Some tools are standardized and manufactured, while others are made from scratch by therapists or based on common objects; and the tools differ from one rehabilitation center to another (Fig. 1).

The standardized tools include a rings tree for arm and hand rehabilitation (See Fig. 2.a), two types of hollow cones for arm rehabilitation (See Fig. 2.b) and a solitaire game for finger prehension rehabilitation (See Fig. 2c). The rings tree consists of horizontal metal shafts at different heights on which the patient threads plastic rings. If the therapist sets up three levels with 10 rings, the patient grasps the rings one by one and move them from level one to level two, then from level two to level three, then from level three to level two and finally comes back down to level one. This exercise involves arm and hand functions and requires coordination and muscle strength. The hollow cones measure 18 cm in height and have two different diameters. The biggest one measures 6.5 cm in diameter at the base and 4 cm in diameter at the top, while the smallest one measures 4.5 cm in diameter at the base and 1.5 cm in diameters at the top. The patient piles cones requiring arm strength

<figure><img src="image_2.png" /><figcaption>Fig. 1. The SpECTRUM ecosystem.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Standardized tools for rehabilitation: (a) rings tree and (b) hollow cones.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Custom tools for rehabilitation: (a) the platter with shapes and (b) the box containing common objects for rehabilitation exercises.</figcaption></figure>

and coordination. Finally, finger prehension exercises are made with a solitaire game where the patient has to grasp a little cylinder (1 cm in diameter) and put them into holes.

The tools made from scratch by the therapists are used for arm and hand rehabilitation and can be very different according to the rehabilitation center. A custom manipulation exercise discovered in the rehabilitation center of Le Havre is composed of a platter with holes of different sizes and shapes (See Fig. 3.a) placed on a table in front of the patient. The shapes include circles, squares, triangles and rectangles. The patient has to grasp a wood object (cylinder, cube, prism or parallelepipoid) and slot it into the corresponding shape. This exercise involves motor and cognitive functions required for daily independence. On the other hand, some exercises are based on common objects. For example, the rehabilitation centers in Evry and Le Havre used tennis balls or screwdrivers to perform rehabilitation exercises (See Fig. 3.b). The tennis balls are used for throwing exercises: The patient throws the ball on the wall and solicits completely the arm and hand. The screwdrivers are used to perform finger and wrist exercises in order to work on pronosupitation of the wrist (i.e. the rotation of the wrist) required to screw. Finally, the rehabilitation center at Lille used a jack (i.e. a rounded cube with cylinder recesses) for finger prehension exercises. The exercise consists in grasping the jack on the recesses in order to work on fingers placement precision and releasing of the jack Fig. 4.

### 3.4.2. Assessment tools

In order to assess the patient's arm and hand abilities, four check-ups are used by occupational therapists: (1) a strength check-up, (2) a grasping check-up, (3) a sensitivity check-up and (4) a transfer check-up Fig. 5.

- The strength check-up aims to assess the strength of the patient while grasping an object. Strength pears (manometers) are used to assess the bi-digital pliers (thumb and forefinger), the tri-digital pliers (thumb, forefinger and middle finger) and the whole hand strength. When the patient faces spasticity problem and pears are too soft for finger strength measurement,

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** The jack prototype: (a) opened with electronic, (b) horizontally put down and (c) vertically put down.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** The cube prototype: (a) opened with electronic, (b) with a scale and (c) horizontally put down.</figcaption></figure>

a Collin or Jamar dynamometer is used, as it is very hard to compress. However, if the patient's strength is too weak to be measured by the pears, a plastic tumbler serves as a tool for strength assessment. Indeed, therapists can easily evaluate the pressure exerted on the tumbler with the noise and the crushing.

- The grasping check-up aims to evaluate the approach, the holding and the releasing of an object. This check-up is based on visual estimations and subjective information collected by the therapist during the test such as the way the patient grasps, holds or manipulates the object (a compensatory strategy to reach the object or motor disorder during manipulation).

- The sensitivity check-up is devoted to assessing the finger sensitivity of the patient (i.e. if the patient is able to feel needles or recognize different textures). The textures (cork, linen rug, carpets, etc.) are placed in a closed box preventing the patient from visually recognizing the texture.

- The transfer check-up is based on the "Box and Blocks" test (Mathiowetz, Volland, Kashman, & Weber, 1985) requiring to move little wood cubes from one box to another. The patient has to move the cubes one by one. If the patient grabs two cubes, only one is considered as moved. If a cube falls, it is not considered in the final score. The final score corresponds to the number of cubes moved from one box to another during a minute, subtracting the fallen cubes or the cubes moved with another one.

### 3.5. Discussion

After a stroke, finger extension is the motor function most likely to be impaired (Radomski & Latham, 2008), while grasping and releasing an object is essential to functional movement of the hand. Monitoring the dexterity of stroke patients with quantifiable information, such as the finger placement on an object during pinching, the pressure applied by each jaw of the digital digital pliers on the object while grasping or the global pressure of the pliers is essential to assess the ability of the patient in terms of movements dexterity of the fingers and hand motor functions. It can also help to detect spasticity of the fingers if the pressure applied by each jaw of the pliers is significantly different.

The observational study and the items scored in the ARAT protocol also show that monitoring the movements of the hand and the arm of stroke patients brings useful information on the recovery of motor functions. Indeed, involuntary abnormal movements can appear after a stroke such a chorea i.e. an sudden, brief and non-repetitive arrhythmic involuntary movement (Alarcon, Zijlmans, Duenas, & Cevallos, 2004; Handley et al., 2009).

As is shown in the state-of-the-art tools and the results of the observational study, patients' monitoring is only based on subjective information collected by therapists during the use of rehabilitation tools. Only checkups allows the collection of accurate measures on the patient's motor functions with assessment tools. Collecting intermediate data by embedding sensors into objects designed for assessing the evolution of the patient's motor recovery over rehabilitation sessions seems a good compromise between totally visual assessment and extremely accurate measurements. Moreover, it can also allow for collection of information currently not assessed at all during rehabilitation sessions or checkups such as tremors.

## 4. The SpECTRUM ecosystem

The SpECTRUM ecosystem is inspired by the results of the observational study performed in three rehabilitation centers as well as the ARAT protocol. Three objects provide reliable and quantifiable information on the patient's hand and arm motor functions during rehabilitation sessions which, until then, were only assessed by visual estimations and subjective measures.

First, we proposed a jack to monitor finger dexterity. Indeed, the rehabilitation center in Lille currently uses a jack for precision grasping and dexterity exercises. Moreover, grasping a jack with the fingers is similar to the ball bearing exercise of the ARAT pinch section. The jack is able to monitor the placement of the fingers and the pressure applied by each jaw of the bi-digital or tri-digital pliers of the patient during grasping. Indeed, the use of compensatory strategies such as the help of the second hand to support the object or the use of other fingers during grasping is sometimes observed. Knowing the pressure applied by each jaw of the pliers thus makes it possible to check if the input task is performed correctly. In addition, the jack is able to monitor its orientation and the tremors of the patient during manipulation.
Second, we proposed a cube to monitor the evolution of the hand prehension. The cube collects information about the pressure applied by the patient during the grasping, its orientation and the tremors of the patient during manipulation. Although the cube presents similar functionalities as the jack, the redundancy of information provides additional data during grasping, while the configuration of the hand is different and the pressure or the hand movements can be very different. In addition, the cube allows for following the evolution of the global pressure of the bi-digital or tri-digital pliers on the object during the manipulation - unlike the jack which is intended for a more precise follow-up of the dexterity.

Third, we decided to use a smart wrist band to monitor the arm activity of stroke patients during the rings tree exercise revealed in the observational study. Indeed, collecting data from the rings by adding sensors on them is not conceivable because of their size and shape. The smart wrist band is able to monitor the movements of the patient's arm as well as the patient's tremors during the exercise.

Finally, we designed a visualization interface in order to present the patient's motor data to the therapists in an easy and fast, understandable way. The visualization tool allows one to easily collect, record and visualize data in real-time on a tablet and visualize previous records. The therapists can thus have access to a review of the patient's state at the end of the sessions and assess the evolution of the motor recovery by comparing previous records.

# 5. The objects implementation

## 5.1. The jack prototype

The jack shape is a rounded parallelepiped (6 cm × 6 cm × 3 cm) with cylindric recess where the patient has to mainly place his/her fingers during the grasping. The jack is based on the Raspberry Pi Zero Wireless (RPi-Z) platform, including CPU and Wi-Fi communication. In order to retrieve information on the fingers' positions and the pressure applied on the jack during grasping, we added "Force-Sensing Linear Potentiometers" (FSLPs) from Pololu<fnref n="1" /> in order to measure the magnitude of the force applied on the sensor as well as the force location. The FSLPs are located on the middle of the jack's sides and follow its shape. The movements of the jack as well as tremors are monitored with an Inertial Measurement Unit (IMU) from InvenSense (MPU-9250²) This IMU embeds a tri-axis accelerometer, gyroscope and compass and its ratio price/performance is very good. The jack is powered by a 3700 mAh battery and embeds a micro-USB connector for charging. The data is transmitted in real-time to an Android application via Wi-Fi and a power switch allows users to turn on/off the device.

## 5.2. The cube prototype

The cube's dimensions are 5 cm × 5 cm × 5 cm. Although one of the custom tools from the observational study includes cubes of different sizes, we decided to design the SpECTRUM cube in order to be easily graspable by the patients. Moreover, a 5 cm side cube is a part of the ARAT protocol and this size perfectly fits the pressure sensors size. The cube is based on the RFDuino platform including a micro-controller integrating Bluetooth Low Energy (BLE) for communication. "Force-Sensitive Resistors" (FSRs) have been placed on each side of the cube in order to monitor the physical pressure applied on the cube during grasping. As for the jack, the cube is equipped with an IMU from InvenSense (MPU-9250) in order to monitor the cube's movements and tremors. The cube is powered by a 340 mAh battery as well as a micro-USB connector for charging. The data is transmitted in real-time to an Android application via BLE since Wi-Fi is not available on the RFDuino platform. The BLE transfer rate is lower than Wi-Fi, but assures data reliability and consumes less energy since the battery capacity is 10 times smaller. A power switch allows users to turn on/off the device.

## 5.3. The Wrist Band Prototype

We decided to use the industrial smart watch from Motorola: the Moto 360. Indeed, all the sensors are already integrated into the wrist band and the data collection can easily be performed with use of the Android API. The Moto 360 embeds a tri-axis accelerometer used to monitor the arm movements of the patient during the rings tree exercise.

# 6. Iteration 2: Preliminary study on the objects functionalities

## 6.1. Aim of the study

The aim of this study is to explore the functionalities and the design of the objects with health care professionals and collect feedback on possible improvements.

<footnote n="1">¹ https://www.pololu.com/product/2730</footnote>

<footnote n="2">² https://www.invensense.com/products/motion-tracking/9-axis/mpu-9250/.</footnote>
## 6.2. Participants

Six health care professionals were involved in this study, including five occupational therapists (OT) and one physiotherapist (PT). Two OT and the PT were interns at the hospital. Participants were four females and two male aged between 20 and 35 (μ = 24.6, σ = 4.9).

## 6.3. Materials and procedure

Semi-structured interviews were conducted and recorded (video and audio) for further analysis. The interviews took place in a quiet office separated from the patients. Participants were informed about the nature and the aims of the study. The interview started with a general presentation of the SpECTRUM ecosystem followed by a detailed presentation of each object including its functionalities, the associated sensors and the type of collected data. The participants were able to manipulate the objects and then we collected feedback, recommendations on possible improvements on the design of the objects. The questions asked during the interviews are detailed in Appendix A. The average duration of each interview was approximately 17 min (min: 10 min, max: 21 min).

## 6.4. Results

A thematic content analysis of the interviews was performed. The results are presented in two categories: (1) the functionalities and (2) the usability of the objects.

### 6.4.1. The functionalities of the objects

Monitoring the pressure applied on the object during grasping has been judged useful by five participants (4 OT including an intern and 1 PT) for the jack and by all the participants for the cube. Indeed, grasping objects with a normal grasping strength is essential for everyday life. An OT and the PT mentioned that monitoring the pressure over time could allow therapists to assess its variations according to different ways of grasping - which is impossible with a dynamometer which displays only one value at a time. In addition, three OT mentioned that monitoring the pressure could enhance the detection of spasticity and could also allow detection of a voluntary motor command disruption, according to one OT. However, one OT mentioned the information collected about the pressure should be paired with strength tests with pears and manometers, as this information cannot be used as a checkup with measures. It can only be used as a monitoring tool.

All the participants judged useful to monitor the position of the fingers on the jack. Indeed, the fingers' positions could be an indication of the motor abilities of the patient. Five participants (4 OT including 2 interns and the PT) proposed to create jacks with different sizes in order to assess the position of the fingers, as well as the pressure on the jack according to the spacing of the fingers.

Three participants (3 OT) assured that monitoring irregular movements with each object would be interesting. However, three participants (2 OT interns and the PT) mentioned that irregular movements monitoring is useless because it can be easily assessed by visual observations and only a few patients faced this kind of disorder. These results being ambiguous, we

<figure><img src="image_7.png" /><figcaption>Fig. 6. The interface displaying the fingers' position.</figcaption></figure>
decided to remove this functionality for the moment as many new data is already available to enhance the therapists' diagnosis and the irregular movement detection is not significant.

Monitoring the orientation has not been judged useful by five participants (4 OT and the PT) for the jack and by three participants (3 OT) for the cube. Indeed, as they are symmetrical, their orientation does not bring information about the nature of the grasping. However, three OT suggested that knowing the orientation for a non-cube shape jack (e.g. parallelepiped) could be interesting to assess the nature of the grasping by coupling the orientation with the pressure and/or position information. Finally, only one OT intern mentioned that monitoring the orientation of the jack or the cube would be useful. Indeed, even if the therapists can assess the orientation visually, a precise quantification of the orientation may bring more relevant information on the pronosupitation of the patient. We followed the majority and decided to remove the orientation monitoring.

All the participants mentioned that monitoring the tremors with each object is very useful in order to follow the evolution of the tremors over the rehabilitation sessions and justify the benefits of the rehabilitation. They all agreed that tremor frequency and magnitude would bring complementary information on the patient's physical state.

All the participants agreed that judging the recognition of different movements with the watch would be useful. Moreover, two OT and the PT interns proposed to use the watch during exercises dedicated to shoulder rehabilitation such as a throwing or recognizing tasks such as walking or drawing shapes in the 3D space. Finally, the watch could be useful at home according to all the participants to monitor the patient's activities and follow the patient's evolution.

### 6.4.2. *The usability of the objects*

Three participants (3 OT) mentioned that the pressure and position sensors are not placed optimally on the jack. Indeed, as the jack is quite thick, the sensors should have been placed at the top of the jack's lateral face in order to help the patients to grasp the jack on the sensors. However, all the participants mentioned that the placement of the pressure sensors on the cube is optimal as they cover its entire surface. Moreover, all the participants mentioned that the watch would be interesting to collect data outside the rehabilitation center (e.g. at home).

### 6.5. *Recommendations*

According to the six health care professionals involved in this study, only the jack requires a design improvement. The pressure and position sensors must be moved to the top of the lateral face in order to help the patient grasping the jack on the sensors. Feedback from the health care professionals also indicates that some information currently collected by the objects is not relevant for the assessment of patient recovery. Monitoring the orientation of the jack and the cube is not necessary since therapists can visually evaluate this information and do not need a precise assessment.

## 7. Iteration 3: Design and preliminary study on the visualization interfaces

After we validated the functionalities of the objects with health care professionals, it is necessary to display the collected data in an easy and efficient way in order to make the monitoring and the diagnosis easier during rehabilitation sessions.

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Example of the two representations: (a) the jack and (b) the cube.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 8. The interface displaying the movements of the watch.</figcaption></figure>

The data has to be rapidly understandable in a nonambiguous way for the therapists. We decided to develop a visualization interface for each functionality.

## 7.1. The visualization interfaces design

The orientation monitoring and the detection of irregular movements not being judged useful by health care professionals, they were not included in the visualization interfaces.

### 7.1.1. The position of the fingers on the jack

The position of the fingers on the jack is represented by displaying the distance of the fingers from each cylindrical recesses (Fig. 6). We used a bar graph representation as the placement of the fingers is not supposed to evolve during handling. The use of a bar graph is therefore more suited to the representation of a fixed value over a given period.

### 7.1.2. The pressure applied on an object

We proposed two representations of the pressure applied on an object. First, we proposed a bar graph for the jack in order to display the value of each pressure sensor (Fig. 7a). The purpose of the jack being to allow a quick comparison of the pressure of each jaws of the bi-digital or tri-digital pliers, it is not necessary to visualize the history of the pressures over the time with a line graph. The interpretation of pressure data using a line graph would be blurred by the overload of the visual channel. Unlike the jack which is intended for a detailed monitoring of the dexterity of the fingers, the cube is mainly interested in the monitoring of the grasping of the hand over time. A line graph representation (Fig. 7b) allows thus a better assessment of the evolution of the pressure over time. As the jack already focuses on comparing the pressure of each jaw of the pliers, we decided to only display the average pressure applied by the patient on two opposite sides of the cube. This design choice allows to maintain a necessary degree of understanding in order to evaluate the grip of the hand and avoids overloading the interface with unnecessary information.

### 7.1.3. The movements of the watch

The movements of the patient's arm are displayed using the accelerometer data (Fig. 8). We used a line graph to allow occupational therapists a better assessment of the patient's movements over time but also to easily detect irregular or abnormal movements. Indeed, a sudden movement, for example, will result in a peak on the curves displayed. It will then be easy for the occupational therapist to compare the amplitude of these peaks to determine if they are significant.

### 7.1.4. The tremors during manipulation

The frequencies and the magnitudes of the patient's tremors are displayed to the therapists after the manipulation of an object (Fig. 9). The magnitudes of the tremors being immutable values for each recording, we used a bar graph representation. The translational tremors are displayed on the left and the rotational tremors on the right. Each bar represents the magnitude of the tremor on the corresponding axis. By clicking on this bar, the corresponding shaking frequency appears at the bottom of the screen.

<figure><img src="image_10.png" /></figure>

Frequency: 1.8072289 Hz

<caption>Fig. 9. The tremor interface displaying magnitude and frequency.</caption>
## 7.2. Preliminary study on the visualization interfaces

### 7.2.1. Aim of the study

This study aims to collect feedback on possible improvements for the visualization interfaces.

### 7.2.2. Participants

All the health care professionals who participated in the preliminary study on the functionalities of objects took part in this second preliminary study.

### 7.2.3. Materials and procedure

Semi-structured interviews were conducted with the participants in the same environment as the previous study. The interview started with a reminder of the SpECTRUM ecosystem objectives and functionalities and continued with a detailed presentation of the visualization interfaces. The participants were able to manipulate the objects and the visualization interfaces in order to discover the entire ecosystem. Then, we collected feedback, recommendations and improvements on the interfaces. The questions asked during the interviews are detailed in Appendix B. The average duration of each interview was approximately 14 min (min: 8 min, max: 18 min).

### 7.2.4. Results

A thematic content analysis has been performed based on the videos recorded during the interviews. Results are presented for each functionality.

All participants mentioned that the use of a bar graph to display the distance of the fingers from the recesses is relevant. However, although three OT suggested that using a line graph for visualizing a record as a way to assess the evolution of the fingers' positions over time could bring complementary information on the grasping, two OT interns did not judge this representation useful.

The two representations of the pressure applied on an object were judged relevant by the majority of the participants. All of them agreed on the use of a bar graph for the jack and five OT agreed to use a line graph for the cube. However, the therapists would like to be able to choose the line graph or the bar graph representation as each one has its own advantages. Indeed, the bar graph representation allows an easier and faster understanding of the data. On the other hand, the line graph representation is better for assessing the evolution of the pressure over the time during the exercise and detect variations. Moreover, offering games involving pressure have been suggested by all participants. The jack's interface could propose a functionality where the therapist can set an horizontal line as a pressure goal where the patient has to approach this value or stay in a close range of this value. Then, the cube's interface could propose to the therapists to display pressure curves that patients have to reproduce in order to work on the grasping strength control and maintaining the patient commitment in rehabilitation.

All participants judged the line graph representation relevant for the movements of the watch. Indeed, movements are space displacements over the time and a line graph allows to assess variations and irregular movements.

Finally, the bar graph representation used to display the tremor has been judged relevant by all participants. They mentioned that using a line graph would be irrelevant as translational and rotational tremors frequencies and magnitudes are computed at the end of the record on the whole collected data.

### 7.2.5. Recommendations

Different improvements have been proposed by health care professionals. First of all, the participants proposed to leave the opportunity to the occupational therapists using the interfaces to choose the type of representation they want to visualize the pressure applied on the jack and the cube. In addition, the occupational therapists asked the possibility to propose the games mentioned previously to the patient in order to work the motor control of the fingers.

# 8. Preliminary tests with patients

## 8.1. Aim of the study

Before planning a large study involving a large number of patients, it is necessary to ensure the reliability of the data collected by the objects during their use. The aim of the study is to collect data during the objects usage in order to detect possible dysfunction. The study has been approved by a National Ethical Committee named "Comité de Protection des Personnes" (ID RCB 2017-A02020-53).

## 8.2. Participants

Three patients were involved in this study, including one female and two males. The women participant was 73 years old and had an ischemic stroke one year ago. She was right-handed and had a left hemiparesis. The two right-handed men
participants were 59 and 75 years old and respectively an ischemic 17 months ago and a hemorrhagic stroke 15 months ago. The 59-year-old patient had dysarthria and ataxia while the 75-year-old patient had light motor command disorder.

## 8.3. Materials and procedure

The experiment took place in a rehabilitation center in Le Havre, France. The patient was welcomed in a quiet room separated from the other patients. We started to present the ecosystem, including the features of each object, as well as the associated sensor. We also presented the visualization interface to the patient. Then, the patient was informed about the nature and the aims of the study. The patient was required to sign a consent form and provide personal information, such as the date of the stroke or the type of stroke. Afterwards, we exposed the experiment protocol.

The protocol is divided in tasks to perform with each object. In the following sections, the thumb, the index, the middle finger, the ring finger and the little finger are respectively noted I, II, III, IV and V. The sides of the jack are noted G, P, R, Pi (Green, Purple, Red, Pink) corresponding to the colored dot of each side. The two opposite sides of the cube are associated to R, Y and B (Red, Yellow and Blue) corresponding to the colored dot of each side. The task required with the jack consists in grasping it with two fingers (I/II, I/III, I/IV, I/V) on two opposite sides (G/P, R/Pi) and grasping it with three fingers (I, II and III) - one per side (G/P/R, G/P/Pi, R/Pi/G, R/Pi/R). The task required with the cube consists in grasping it with two fingers (same as the jack) on two opposite sides (R, Y, B) and with three fingers (same as the jack) on three sides (R/Y, B/Y, B/R, Y/R, Y/B) and finally to grasp the cube with the whole hand. Then, the task required with the watch is to perform the rings tree exercise by moving ten rings along horizontals shafts. The detailed protocol are presented in Appendix C.

Each patient performed the experiment with two among the three objects of the ecosystem in order to minimize the time of the experiment. The patient who had an hemorrhagic stroke performed the tasks with the jack and the cube. The patient with dysarthria and ataxia performed the tasks with the cube and the watch, while the third patient with hemiparesis performed the tasks with the jack and the watch.

The experiment was video and audio recorded in order to compare the real task performed by the patient with the data collected from the sensors. An occupational therapist who already was involved in the preliminary study was present during the experiment in order to help the patient, if necessary.

## 8.4. Results and recommendations

The results of the pretests show that the cube is 100% reliable for a bi-digital grasping and 70% reliable for a tri-digital grasping with one finger per side. The watch is also reliable for the movements of the patient's arm. Tremor detection on the cube and the watch suggests that patients did not experience tremor during exercise, which was confirmed by occupational therapists present during the data collection. The main problem highlighted during these pre-tests is a malfunction of the jack pressure sensors in certain configurations. The jack accuracy only reached 70% during a bi-digital grasping and fell down to 56% during a tri-digital grasping with one finger per side. This can be explained by the nature of the pressure sensors, which are resistive. Their shape has been modified in order to conform to the domed shape of the jack, resulting in a minimum detection threshold – sometimes too high for certain patients presenting a motor deficit of the fingers. However, this constraint is not an obstacle to future experiments because this problem was noticed – mainly during three-finger grasping (one finger per side). Indeed, grasping an object like that is unnatural and occupational therapists mentioned that it would be more interesting for future experimentation to modify the three-finger grasping in a tri-digital grasping with a thumb on one sensor and the index finger and middle finger on the opposite sensor. In addition, occupational therapists suggested to use the jack only with patients with sufficient grip strength or spasticity leading to an unusually high grasping force in order to avoid an immediate modification of the prototype while maintaining a reliable data collection. In the next version of the jack prototype, pressure and position sensors will be placed on the top of the lateral face to facilitate detection in any configuration.

<figure><img src="image_11.png" /><figcaption>Fig. 10. Different patients during the experiment: (a) grasping the jack, (b) moving the cube on the A3 sheet and (c) moving rings on the tree.</figcaption></figure>
# 9. Iteration 4: Preliminary study on usability and acceptability with patients

## 9.1. Aim of the study

This study aims to assess the usability and the acceptability of the three objects of the SpECTRUM ecosystem by stroke patients during rehabilitation exercises at the rehabilitation center. The study has been approved by a National Ethical Committee named "Comité de Protection des Personnes" (ID RCB 2017-A02020-53).

## 9.2. Participants

Nine patients were involved in this study, including four females and five males. The patients were aged between 19 and 86 years old ($\mu$ = 63.2, $\sigma$ = 20.2). In average, they had a stroke 16 months ago ($\mu$ = 16.2, $\sigma$ = 15.9). Five patients had an ischemic stroke, while four patients had an hemorrhagic stroke. Moreover, only one patient was left-handed and six patients had impairments on their non-dominant upper limb. The ischemic stroke patients had different troubles right after the stroke such as hemiplegia, ataxia, dysmetria, hemiparesis, dysarthria or tremors. The hemorrhagic stroke patients had troubles such as hemiparesis, light motor disorders or cognitive troubles. The youngest patient who had an hemorrhagic stroke faced facial paralysis on the right and an impairment of the left upper limb.

## 9.3. Materials and procedure

The experiment took place in the same rehabilitation center in Le Havre, France where the pre-tests took place. The patient was welcomed in the same quiet room separated from the other patients. Contrary to the pre-tests, each patient performed the experiment with the three objects of the ecosystem. The experiment started with a presentation of the ecosystem including the features of each objects, the associated sensor and the visualization interface. Then, the patient was informed about the nature and the aims of the study. The patient was required to sign a consent form and provide personal information such as the date of the stroke or the type of stroke. Afterwards, we displayed the experiment protocol, which is divided in tasks to perform with each object. The task required with the jack is consists in grasping it with two fingers (thumbs and forefinger) on two opposite sides and grasping it with three fingers (thumb, forefinger and middle finger) on two opposite sides (See Fig. 10.a). The task required with the cube consists in moving it following a path of targets on an A3 paper sheet by grasping it with two and three fingers as the jack and to grasp it with the whole hand (See Fig. 10.b). Indeed, previous researches showed that positioning and manipulating an object require good coordination of the upper limbs and appear to be suitable tasks for stroke patients (Timmermans et al., 2009). Moreover, these tasks are generally based on an action-perception loop exploiting different sensory channels (vision, tactile, proprioception, audio). Then, the task required with the watch is the same as the pre-tests (See Fig. 10.c). The required information, as well as the experiment protocol are presented in Appendix D. At the end of the experiment, we conducted a semi-structured interview based on a predefined interview guide (See Appendix E) in order to explore the usability and the acceptability of the SpECTRUM ecosystem by stroke patients. The experiment was video and audio recorded in order to perform a further thematic content analysis. The average duration of the interviews was approximately 11 min (min: 6 min, max: 14 min).

## 9.4. Results

A thematic content analysis of the interviews was performed. Based on the topics explored during the interviews, we present the results according to the following categories: (1) the usability of the SpECTRUM ecosystem and (2) the acceptability of the ecosystem and its applications.

### 9.4.1. The usability of the SpECTRUM ecosystem

All the patients mentioned that the size and weight of the jack and the cube is fine. Four patients including three ischemic and one hemorrhagic stroke patients with light motor disorders, ataxia, dysmetria or dysarthria found the jack very easy to use while three patients including two ischemic and one hemorrhagic stroke patients assured that the cube is very easy to use. In addition, all the patients agreed to say that the watch is very easy to use for the rings tree exercise. Indeed, three patients including two ischemic and one hemorrhagic stroke patients with hemiparesis, dysarthria and facial paralysis mentioned that the watch works as a classic watch. In addition, three patients including two ischemic and one hemorrhagic stroke patients with light motor impairments mentioned that they had forgotten that they were wearing the watch on their wrist.

Moreover, all the participants mentioned that the cube is not sliding through their fingers and one of them who faced dysarthria and upper limb motor impairment found the taking good. On the other hand, two patients including an ischemic and a hemorrhagic patients mentioned that the jack is not sliding through their hands. All the participants agreed that the texture of the jack and the cube is very adapted for rehabilitation object.

Then, six patients did not find any problem to the pressure and position sensors placement on the jack while the youngest patient (19 y.o.) with hemiparesis and facial paralysis suggested that the sensors should be at the top of the jack's side in order to facilitate the grasping.
However, one ischemic stroke patient mentioned that the exercise with the jack requires a lot of concentration as this exercise involves finger movements precision. Furthermore, the oldest patient (86 y.o.) who had an hemorrhagic stroke found the jack too heavy at the end of the exercise due to muscles fatigue. It should be noted that this result cannot be considered significant as the muscles fatigue can be increased due to the patient's age.

On the other hand, only one patient with hemiparesis on its dominant hand mentioned that the edges of the cube are too sharp and one patient who had an ischemic stroke and faced upper limb motor impairment had the hand a little numb at the end of the exercise, which is due to its upper limb motor disorder.

### 9.4.2. The acceptability of the SpECTRUM ecosystem and applications

Seven patients, including four ischemic and three hemorrhagic stroke patients are willing to use the SpECTRUM ecosystem during the rehabilitation sessions. Two of them (ischemic) mentioned that this ecosystem seems useful to assess their evolution over the rehabilitation sessions. The patient who faced facial paralysis and moderate finger motor disorder found the jack and the cube very useful to work on movements precision. One ischemic stroke patient with dysarthria qualified the objects of playful and fun. Finally, among the seven patients, three of them (one ischemic and two hemorrhagic) agreed to use these devices if it helps the occupational therapists for the monitoring.

Five patients, including three ischemic and two hemorrhagic stroke patients mentioned having no preference among these objects. A patient with dysarthria and motor impairment on the right upper limb mentioned that all the objects are useful, especially the watch which is the easiest device to use. The patient who had an hemorrhagic stroke with facial paralysis and moderate finger motor disorder, which is the youngest one (19.o.), preferred the jack in order to perform finger precision exercises. Two ischemic stroke patients mentioned that the watch is their preferred device as it can be used as a common watch and wear during the day to enhance the monitoring. One patient who had an ischemic stroke preferred the cube for its design.

Most of the patients (89%) did not find any other application for the devices of the SpECTRUM ecosystem. Only one patient who had an ischemic stroke proposed an exercise based on its rehabilitation sessions. The exercise is based on the rings tree exercise and consists in moving the rings one by one from the Level 1 to the Level 2 by passing them behind the back and moving the rings from the Level 2 to the Level 3 by passing them behind the head. This exercise solicits the shoulders, the neck and the back and involves proprioception.

No participant mentioned concerns about the transmission of collected data as the data will be transferred from the devices to a tablet or a computer through local wireless network and secure connection. They all mentioned that the collected data is not critically personal such as medical information.

None of the patients involved in this experiment figured out potential problems with the SpECTRUM ecosystem either in terms of design or in terms of functionalities.

## 9.5. Conclusion

As a conclusion to this preliminary study, most of the patients mentioned that this ecosystem is easy to use and can be useful for the therapists in order to assess the evolution of the patient's recovery state. The first feedback on the ecosystem shows a very good acceptance by the patients. The SpECTRUM ecosystem could be used during rehabilitation sessions in hospitals in order to collect more data about usability and acceptability on a long period.

# 10. Conclusion and perspectives

The paper presents the conception and development of the SpECTRUM ecosystem, including a smart jack, a smart cube and a smart watch intended for monitoring the arm and hand motor activity of stroke patients during rehabilitation sessions. The ecosystem design is based on an observational study performed in 3 rehabilitation centers with the aim of instrumenting objects currently used during rehabilitation sessions. The SpECTRUM ecosystem also provides a visualization interface for the data collected to help occupational therapists evaluate the evolution of motor functions of patients during rehabilitation sessions. The arm and hand motor activities are assessed by the therapist based on quantitative information (the grasping force on the jack or the cube, the fingers' position on the jack and the movements of the watch). Moreover, appearance and evolution of tremors can be assessed by the therapist based on the information provided by each instrumented object of the ecosystem. The rehabilitation program can be adapted by the therapists according to the patient's recovery state. After the implementation of the first prototype of the SpECTRUM objects, a preliminary study has been carried out with health care professionals in order to collect feedback on possible improvements on the objects' design and functionalities. It evidences that the objects' design has been validated by the health care professionals and some features need to be removed. Only the sensors' placement on the jack needs to be reviewed (i.e. the sensors have to be aligned with the edges of the jack) in order to facilitate the grasping for the patient. Based on these results, a mobile application for recording and visualizing the collect data has been developed. A second preliminary study has been carried out in order to collect feedback on possible improvements on the visualization interfaces. The results show that the visualization interfaces are relevant, but need several improvements. Functionalities, such as pressure goals or pressure curve reproduction, have to be added to the next update. Afterwards, we performed pre-tests with stroke patients in order to assess the reliability of the
collected data. The results show that the cube and the watch data are reliable. The data collected by the jack are not always reliable when the patients have a weak finger strength. This result implies that the jack has to be used by patients with strong finger strength or spasticity. Finally, a preliminary study on the usability and acceptability of the ecosystem has been carried out with stroke patients. The results show that the majority of the patients agree to use these devices during rehabilitation sessions at the hospital, especially if it helps the therapists during its monitoring. The design of the objects does not raise fundamental issues for the patients except for the sensors placements of the jack, which needs to be placed on the top of the lateral face of the jack.

Future works will address several issues. New technologies will be investigated to provide more reliable information about grasping pressure and fingers positions on the jack as mentioned in the preliminary study. A new prototype of the jack will be implemented and technically tested with stroke patients. In order to provide relevant data from the watch to the therapists, the machine learning approach will be investigated in order to quantify the number of rings moved and detect on which level the ring has been moved. Furthermore, a longer study involving patients and therapists during a year is planned to assess the benefit of the SpECTRUM ecosystem during the rehabilitation process. Finally, information about the way the patients approach the object cannot be retrieved at the moment. In order to overcome this limitation, a smart textile sweater integrating textile bending sensors using conductive threads has been developed for monitoring the elbow flexion of the patient (Bobin, Amroun, Coquillart, Bimbard, & Ammi, 2017). This technique could be applied to monitor the shoulder and chest configuration during grasping in order to provide complementary information on the patient's physical state.

## Conflict of interest

None.

## Acknowledgments

We would like to thank the rehabilitation centers who agreed to participate to these studies as well as all the patients who were kind and volunteers participants.

# Appendix A. Interview guide with health care professionals on the functionalities and the design of the objects

This section presents the questions asked to the therapists about the functionalities and the design of the objects during the interview.

## A.1. The jack

* Is monitoring the pressure exerted on the jack relevant for the patient assessment ? Why ?

* Is monitoring the position of the patient's fingers on the jack relevant for the patient assessment ? Why ?

* Is reporting tremors relevant for the patient assessment ? Why ?

* If yes, do you prefer having tremor frequencies or magnitudes ? Why ?

* Is reporting the orientation of the jack relevant for the patient assessment ? Why ?

* Is monitoring the jack irregular movement relevant for the patient assessment ? Why ?

* What do you think about the sensor placement on the jack ?

## A.2. The cube

* Is monitoring the pressure exerted on the cube relevant for the patient assessment ? Why ?

* Is reporting tremors relevant for the patient assessment ? Why ?

* If yes, do you prefer having tremor frequencies or magnitudes ? Why ?

* Is reporting the orientation of the cube relevant for the patient assessment ? Why ?

* Is monitoring the cube irregular movement relevant for the patient assessment ? Why ?

* What do you think about the sensor placement on the cube ?

## A.3. The watch

* Is reporting tremors relevant for the patient assessment ? Why ?

* If yes, do you prefer having tremor frequencies or magnitudes ? Why ?

* Is monitoring the watch irregular movement relevant for the patient assessment ? Why ?

* Is detecting and recognizing activities relevant for the patient assessment ? Why ?
# Appendix B. Interview guide with health care professionals on the visualization interfaces

This section presents the questions asked to the therapists about the visualization interfaces during the interview.

## B.1. The jack interface

* Is a bar graph relevant for the pressure and position real-time monitoring and further assessment ? Why ?

* Is a bar graph relevant for tremors assessment ? Why ?

* Do you have any comments ?

## B.2. The cube interface

* Is a line graph relevant for the pressure real-time monitoring and further assessment ? Why ?

* Is a bar graph relevant for tremors assessment ? Why ?

* Do you have any comments ?

## B.3. The watch interface

* Is a line graph relevant for the movements real-time monitoring and further assessment ? Why ?

* Is a bar graph relevant for tremors assessment ? Why ?

* Do you have any comments ?

# Appendix C. Experiment protocol for the pre-tests

First, we asked some personal information to the patient. The required information is listed below.

* Name

* Surname

* Date of Birth

* Type of stroke (hemorrhagic or ischemic)

* Date of stroke

* Right-handed or left-handed

* Paralyzed member

* Medical troubles

* Profession

Then, the patient was asked to manipulate the objects by performing specific exercises. The patient is installed so that the conditions of the experiment are equivalent for all. The patient sits in a chair in front of a table whose height is adjusted so that the forearms of the patient are placed on the table and the elbows are at right angles. Once the patient is installed, the tasks presented below are performed by the patient under the control of the experimenter.

## C.1. The tasks required with the jack

* Grasp the jack with a bi-digital take (I/II, I/III, I/IV, I/V) on two opposite sides (G/P, R/Pi) and lift it up. Release the jack on the table (2 repetitions)

* Grasp the jack with a tri-digital take (I, II and III), one per side (G/P/R, G/P/Pi, R/Pi/G, R/Pi/R) and lift it up. Release the jack on the table (2 repetitions)

## C.2. The tasks required with the cube

* Grasp the cube with a bi-digital take (I/II, I/III, I/IV, I/V) on two opposite sides (R, Y, B) and lift it up. Release the cube on the table (2 repetitions)

* Grasp the cube with a tri-digital take (I, II and III) on 3 sides (R/Y, B/Y, B/R, Y/R, Y/B) and lift it up. Release the cube on the table (2 repetitions)

* Grasp the cube with the hand as you can (2 repetitions)

## C.3. The tasks required with the watch

This sequence is performed twice by the patient.

* Move 10 rings one by one from level 1 to level 2.

* Move 10 rings one by one from level 2 to level 3.
* Move 10 rings one by one from level 3 to level 2.

* Move 10 rings one by one from level 2 to level 1.

## Appendix D. Experiment protocol of the patient study

First, the same personal information required during pre-tests was requested from the patient. Then, the patient was asked to manipulate the objects during specific exercises under the same conditions as previously described. The tasks to be performed with each object are presented below.

### D.1. The tasks required with the jack

* Grasp the jack with a bi-digital take (thumb and forefinger) and lift it up. Release the jack on the table (5 repetitions)

* Grasp the jack with a tri-digital take (thumb, forefinger and middle finger) and lift it up. Release the jack on the table (5 repetitions)

### D.2. The tasks required with the cube

Before starting the experiment, the cube is placed by the experimenter on the starting point at the center of a A3 sheet marked with a symbol. The 3 tasks presented below have been repeated 5 times.

The bi-digital take (thumb and forefinger):

* Grasp the cube and move it from the starting point to the first target

* Grasp the cube and move it from the first target to the second target

* Grasp the cube and move it from the second target to the third target

* Grasp the cube and move it from the third target to the fourth target

* Grasp the cube and move it from the fourth target to the starting point target

The tri-digital take (thumb and forefinger + middle finger):

* Grasp the cube and move it from the starting point to the first target

* Grasp the cube and move it from the first target to the second target

* Grasp the cube and move it from the second target to the third target

* Grasp the cube and move it from the third target to the fourth target

* Grasp the cube and move it from the fourth target to the starting point target

The hand grasping:

* Grasp the cube and move it from the starting point to the first target

* Grasp the cube and move it from the first target to the second target

* Grasp the cube and move it from the second target to the third target

* Grasp the cube and move it from the third target to the fourth target

* Grasp the cube and move it from the fourth target to the starting point target

### D.3. The tasks required with the watch

Same protocol as the pre-tests.

## Appendix E. Interview guide for the patient study

The interview was divided into two parts: (1) the usability of the ecosystem and (2) the acceptability of such an ecosystem for monitoring during rehabilitation sessions and its applications.

### E.1. The usability of the SpECTRUM ecosystem

* What do you think about the ease of use of the jack? (e.g. size and weight, texture, sensor placement)

* What do you think about the ease of use of the cube? (e.g. size and weight, texture)

* What do you think about the ease of use of the watch?

* What potential problems do you see in the use? Do you have any fears towards this object?

### E.2. The acceptability of the SpECTRUM ecosystem and applications

* Would you like to use these devices during the rehabilitation sessions? Why?
* Do you have a preference among these objects? Why?

* What kind of application do you imagine for these devices?

* Do you have any concern about the transmission of collected data?

Finally, the last question of the interview was to collect comments on the experiment.

## References

Ada, L., Canning, C. G., & Low, S.-L. (2003). Stroke patients have selective muscle weakness in shortened range. *Brain*, **126**(3), 724-731.

Alarcon, F., Zijlmans, J., Duenas, G., & Cevallos, N. (2004). Post-stroke movement disorders: Report of 56 patients. *Journal of Neurology, Neurosurgery Psychiatry*, **75**(11), 1568-1574.

Anderson, C., Mhurchu, C. N., Rubenach, S., Clark, M., Spencer, C., & Winsor, A. (2000). Home or hospital for stroke rehabilitation? Results of a randomized controlled trial: II: Cost minimization analysis at 6 months. *Stroke*, **31**(5), 1032-1037.

Bansil, S., Prakash, N., Kaye, J., Wrigley, S., Manata, C., Stevens-Haas, C., & Kurlan, R. (2012). Movement disorders after stroke in adults: A review. *Tremor and Other Hyperkinetic Movements*, **2**.

Bloom, D. E., Boersch-Supan, A., McGee, P., Seike, A., et al. (2011). Population aging: Facts, challenges, and responses. *Benefits and Compensation International*, **41**(1), 22.

Bobin, M., Amroun, H., Coquillart, S., Bimbard, F., & Ammi, M. 2017. Dnn based approach for the assessment of elbow flexion with smart textile sensor. In *Proceedings of IEEE international conference on systems, man, and cybernetics*.

Bohannon, R. W. (2001). Motricity index scores are valid indicators of paretic upper extremity strength following stroke. *Journal of Physical Therapy Science*, **11**(2), 59-61.

Boian, R., Sharma, A., Han, C., Merians, A., Burdea, G., Adamovich, S., Recce, M., Tremaine, M., Poizner, H., et al. (2002). Virtual reality-based post-stroke hand rehabilitation. *Studies in Health Technology and Informatics*, **64**-70.

Brunnstrom, S. (1966). Motor testing procedures in hemiplegia: Based on sequential recovery stages. *Physical Therapy*, **46**(4), 357-375.

Burke, J. W., McNeill, M., Charles, D. K., Morrow, P. J., Crosbie, J. H., & McDonough, S. M. (2009). Optimising engagement for stroke rehabilitation using serious games. *The Visual Computer*, **25**(12), 1085.

Burke, J.W., McNeill, M., Charles, D., Morrow, P., Crosbie, J., & McDonough, S., 2009. Serious games for upper limb rehabilitation following stroke. In *Proceedings of conference on games and virtual worlds for serious applications*, 2009. VS-GAMES'09. IEEE, pp. 103-110.

Busser, H., De Korte, W., Glerum, E., & Van Lummel, R. (1998). Method for objective assessment of physical work load at the workplace. *Ergonomics*, **41**(10), 1519-1526.

Bussmann, J. B., Tulen, J. H., van Herel, E. C., & Stam, H. J. (1998). Quantification of physical activities by means of ambulatory accelerometry: A validation study. *Psychophysiology*, **35**(5), 488-496.

Cargnina, D.J., d'Ornellasa, M.C., & Pradob, A.L.C. 2015. A serious game for upper limb stroke rehabilitation using biofeedback and mirror-neurons based training. In *Proceedings of the 15th world congress on health and biomedical informatics MEDINFO 2015: EHealth-enabled Health*, vol. 216, IOS Press, p. 348.

Carr, J. H., Shepherd, R. B., Nordholm, L., & Lynne, D. (1985). Investigation of a new motor assessment scale for stroke patients. *Physical Therapy*, **65**(2), 175-180.

De Weerdt, W., & Harrison, M. (1985). Measuring recovery of arm-hand function in stroke patients: A comparison of the brunnstrom-fugl-meyer test and the action research arm test. *Physiotherapy Canada*, **37**(2), 65-70.

Dethy, S., Luxen, A., Bidaut, L. M., & Goldman, S. (1993). Hemibody tremor related to stroke. *Stroke*, **24**(12), 2094-2096.

Deuschl, G., Wenzelburger, R., Löffler, K., Raethjen, J., & Stolze, H. (2000). Essential tremor and cerebellar dysfunction clinical and kinematic analysis of intention tremor. *Brain*, **123**(8), 1568-1580.

Fan, Y. J., Yin, Y. H., Da Xu, L., Zeng, Y., & Wu, F. (2014). lot-based smart rehabilitation system. *IEEE Transactions on Industrial Informatics*, **10**(2), 1568-1577.

Foerch, C., Sitzer, M., Steinmetz, H., Neumann-Haefelin, T., et al. (2009). Future demographic trends decrease the proportion of ischemic stroke patients receiving thrombolytic therapy a call to set-up therapeutic studies in the very old. *Stroke*, **40**(5), 1900-1902.

Fugl-Meyer, A. R., Jääskö, L., Leyman, I., Olsson, S., & Steglind, S. (1974). The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. *Scandinavian Journal of Rehabilitation Medicine*, **7**(1), 13-31.

Handley, A., Medcalf, P., Hellier, K., & Dutta, D. (2009). Movement disorders after stroke. *Age and Ageing*, **38**(3), 260-266.

Hou, X., & Sourina, O. 2013. Emotion-enabled haptic-based serious game for post stroke rehabilitation. In *Proceedings of the 19th ACM symposium on virtual reality software and technology*. ACM, pp. 31-34.

Iosa, M., Morone, G., Fusco, A., Bragoni, M., Coiro, P., Multari, M., Venturiero, V., De Angelis, D., Pratesi, L., & Paolucci, S. (2012). Seven capital devices for the future of stroke rehabilitation. *Stroke Research and Treatment*, **2012**.

Jack, D., Boian, R., Merians, A. S., Tremaine, M., Burdea, G. C., Adamovich, S. V., Recce, M., & Poizner, H. (2001). Virtual reality-enhanced stroke rehabilitation. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, **9**(3), 308-318.

Jacobs, A., Timmermans, A., Michielsen, M., Vander Plaetse, M. and Markopoulos, P. "Contrast: gamification of arm-hand training for strokesurvivors," In *Proceedings of CHI'13 extended abstracts on human factors in computing systems*. ACM, 2013, pp. 415-420.

Kemp, B., Janssen, A. J., & van der Kamp, B. (1998). Body position can be monitored in 3d using miniature accelerometers and earth-magnetic field sensors. *Clinical Neurophysiology*, **109**(6), 484-488.

Kim, J. S. (1992). Delayed onset hand tremor caused by cerebral infarction. *Stroke*, **23**(2), 292-294.

Kim, J. S. (2001). Delayed onset mixed involuntary movements after thalamic stroke: Clinical, radiological and pathophysiological findings. *Brain*, **124**(2), 299-309.

Kwakkel, G., & Kollen, B. (2006). Predicting improvement in the upper paretic limb after stroke: A longitudinal prospective study. *Restorative Neurology and Neuroscience*, **25**(5-6), 453-460.

Lorussi, F., Rocchia, W., Scilingo, E. P., Tognetti, A., & De Rossi, D. (2004). Wearable, redundant fabric-based sensor arrays for reconstruction of body segment posture. *IEEE Sensors Journal*, **4**(6), 807-818.

Mathiowetz, V., Volland, G., Kashman, N., & Weber, K. (1985). Adult norms for the box and block test of manual dexterity. *American Journal of Occupational Therapy*, **39**(6), 386-391.

Organization, W. H., et al. (2017). Global health estimates 2015: Deaths by cause, age, sex, by country and by region, 2000-2015. Geneva: WHO; 2016. *Annexe 2: Principales Structures de Santé de la Nouvelle-Calédonie*

Organization, W. H. (2002). *The world health report 2002: Reducing risks, promoting healthy life*. World Health Organization.

Patel, S., Hughes, R., Hester, T., Stein, J., Akay, M., Dy, J. G., & Bonato, P. (2010). A novel approach to monitor rehabilitation outcomes in stroke survivors using wearable technology. *Proceedings of the IEEE*, **98**(3), 450-461.

Radomski, M. V., & Latham, C. A. T. (2008). *Occupational therapy for physical dysfunction*. Lippincott Williams & Wilkins.

Rigas, G., Tzallas, A. T., Tsipouras, M. G., Bougia, P., Tripoliti, E. E., Baga, D., Fotiadis, D. I., Tsouli, S. G., & Konitsiotis, S. (2012). Assessment of tremor activity in the parkinson's disease using a set of wearable sensors. *IEEE Transactions on Information Technology in Biomedicine*, **16**(3), 478-487.
Rincon, A.L., Yamasaki, H., & Shimoda, S. 2016. Design of a video game for rehabilitation using motion capture, emg analysis and virtual reality. In *Proceedings of IEEE international conference on electronics, communications and computers (CONIELECOMP)*, 2016 pp. 198-204.

Rowe, F., Brand, D., Jackson, C. A., Price, A., Walker, L., Harrison, S., Eccleston, C., Scott, C., Akerman, N., Dodridge, C., et al. (2008). Visual impairment following stroke: Do stroke patients require vision assessment? *Age and Ageing*, 38(2), 188-193.

Shah, S., Harasymiw, S., & Stahl, P. (1986). Stroke rehabilitation: Outcome based on brunnstrom recovery stages. *The Occupational Therapy Journal of Research*, 6(6), 365-376.

Smaga, S. (2003). *Tremor. American Family Physician*, 68(8), 1545-1552.

Sommerfeld, D. K., Eek, E. U.-B., Svensson, A.-K., Holmqvist, L. W., & von Arbin, M. H. (2004). Spasticity after stroke: Its occurrence and association with motor impairments and activity limitations. *Stroke*, 35(1), 134-139.

Taccini, N., Loriga, G., Dittmar, A., & Paradiso, R. 2004. Knitted includes for health monitoring. In *Proceedings of the 26th annual international conference of the IEEE engineering in medicine and biology society, 2004. IEMBS'04.*, vol. 1. pp. 2165-2168.

Thomas, K.P., Vinod, A., & Guan, C. 2013. Enhancement of attention and cognitive skills using eeg based neurofeedback game. In *Proceedings of the 6th international IEEE/EMBS conference on neural engineering (NER)*, 2013. IEEE, pp. 21-24.

Timmermans, A. A., Seelen, H. A., Willmann, R. D., Bakx, W., De Ruyter, B., Lanfermann, G., & Kingma, H. (2009). Arm and hand skills: Training preferences after stroke. *Disability and Rehabilitation*, 31(16), 1344-1352.

Tognetti, A., Lorussi, F., Bartalesi, R., Quaglini, S., Tesconi, M., Zupone, G., & De Rossi, D. (2005). Wearable kinesthetic system for capturing and classifying upper limb gesture in post-stroke rehabilitation. *Journal of NeuroEngineering and Rehabilitation*, 2(1), 8.

Uswatte, G., Foo, W. L., Olmstead, H., Lopez, K., Holand, A., & Simms, L. B. (2005). Ambulatory monitoring of arm movement using accelerometry: An objective measure of upper-extremity rehabilitation in persons with chronic stroke. *Archives of Physical Medicine and Rehabilitation*, 86(7), 1498-1501.

Van der Lee, J. H., De Groot, V., Beckerman, H., Wagenaar, R. C., Lankhorst, G. J., & Bouter, L. M. (2001). The intra-and interrater reliability of the action research arm test: A practical test of upper extremity function in patients with stroke. *Archives of Physical Medicine and Rehabilitation*, 82(1), 14-19.

van der Lee, J. H., Beckerman, H., Lankhorst, G. J., Bouter, L. M., et al. (2001). The responsiveness of the action research arm test and the fugl-meyer assessment scale in chronic stroke patients. *Journal of Rehabilitation Medicine*, 33(3), 110-113.

Vogiatzaki, E., & Krukowski, A. (2014). Serious games for stroke rehabilitation employing immersive user interfaces in 3d virtual environment. *Journal of Health Informatics*, 6.

Williamson, R., & Andrews, B. (2001). Detecting absolute human knee angle and angular velocity using accelerometers and rate gyroscopes. *Medical and Biological Engineering and Computing*, 39(3), 294-302.

Woodbury, M., Velozo, C. A., Thompson, P. A., Light, K., Uswatte, G., Taub, E., Winstein, C. J., Morris, D., Blanton, S., Nichols-Larsen, D. S., et al. (2010). Measurement structure of the wolf motor function test: Implications for motor control theory. *Neurorehabilitation and Neural Repair*, 24(9), 791-801.

Xu, W., Huang, M.-C., Amini, N., He, L., & Sarrafzadeh, M. (2013). ecushion: A textile pressure sensor array design and calibration for sitting posture analysis. *IEEE Sensors Journal*, 13(10), 3926-3934.

Xu, B., Da Xu, L., Cai, H., Xie, C., Hu, J., Bu, F., et al. (2014). Ubiquitous data accessing method in iot-based information system for emergency medical services. *IEEE Transactions in Industrial Informatics*, 10(2), 1578-1586.

Yan, H., Xu, L. D., Bi, Z., Pang, Z., Zhang, J., & Chen, Y. (2015). An emerging technology-wearable wireless sensor networks with applications in human health condition monitoring. *Journal of Management Analytics*, 2(2), 121-137.

Yang, G., Xie, L., Mäntysalo, M., Zhou, X., Pang, Z., Da Xu, L., Kao-Walter, S., Chen, Q., & Zheng, L-R. (2014). A health-iot platform based on the integration of intelligent packaging, unobtrusive bio-sensor, and intelligent medicine box. *IEEE Transactions on Industrial Informatics*, 10(4), 2180-2191.

Yang, P., Stankevicius, D., Marozas, V., Deng, Z., Liu, E., Lukosevicius, A., Dong, F., Xu, L., & Min, G. (2018). Lifelogging data validation model for internet of things enabled personalized healthcare. *IEEE Transactions on Systems Man, and Cybernetics: Systems*, 48(1), 50-64.

Yin, Y. H., Fan, Y. J., & Xu, L. D. (2012). Emg and epp-integrated human-machine interface between the paralyzed and rehabilitation exoskeleton. *IEEE Transactions on Information Technology in Biomedicine*, 16(4), 542-549.

Yuehong, Y., Zeng, Y., Chen, X., & Fan, Y. (2016). The internet of things in healthcare: An overview. *Journal of Industrial Information Integration*, 1, 3-13.

Zhou, H., Hu, H., & Harris, N., 2005. Application of wearable inertial sensors in stroke rehabilitation, In *Proceedings of the 27th annual international conference of the engineering in medicine and biology society, 2005. IEEE-EMBS 2005*. IEEE, pp. 6825-6828.

Zijlstra, W., & Bisseling, R. (2004). Estimation of hip abduction moment based on body fixed sensors. *Clinical Biomechanics*, 19(8), 819-827.