Machine learning‐based edge‐computing on a multi‐level architecture of WSN and IoT for real‐time fall detection

IET Wireless Sensor SystemsVolume 10, Issue 6 p. 320-332 Research ArticleFree Access Machine learning-based edge-computing on a multi-level architecture of WSN and IoT for real-time fall detection Amina El Attaoui, Corresponding Author Amina El Attaoui amina.elattaoui@um5s.net.ma orcid.org/0000-0003-0619-5606 Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorSalma Largo, Salma Largo Smart, Embedded, Enterprise, and Distributed Systems (SEEDS), STRS Laboratory. INPT of Rabat, MoroccoSearch for more papers by this authorSoufiane Kaissari, Soufiane Kaissari Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAchraf Benba, Achraf Benba Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAbdelilah Jilbab, Abdelilah Jilbab Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAbdennaser Bourouhou, Abdennaser Bourouhou Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this author Amina El Attaoui, Corresponding Author Amina El Attaoui amina.elattaoui@um5s.net.ma orcid.org/0000-0003-0619-5606 Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorSalma Largo, Salma Largo Smart, Embedded, Enterprise, and Distributed Systems (SEEDS), STRS Laboratory. INPT of Rabat, MoroccoSearch for more papers by this authorSoufiane Kaissari, Soufiane Kaissari Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAchraf Benba, Achraf Benba Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAbdelilah Jilbab, Abdelilah Jilbab Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this authorAbdennaser Bourouhou, Abdennaser Bourouhou Electronics Sensors, Systems, and Nanobiotechnologies, STIS, ENSET, Mohammed V University in Rabat, MoroccoSearch for more papers by this author First published: 21 October 2020 https://doi.org/10.1049/iet-wss.2020.0091Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Health telemonitoring systems are constrained by the computational and data transmission load resulting from the large volumes of various measured signals, e.g. in the fall detection application. Nevertheless, the trend of movement and the implementation of computer intelligence in intelligent devices ensure an intelligent and convenient method for continuous real-time telemonitoring of health conditions. In this paper, fall detection is presented while leveraging edge computing integrated on a multi-level architecture combines the Wireless Sensors Network and the Internet of Things. Particularly, we present a complete study and implementation scenarios while investigating the performances of machine learning algorithms to distinguish between different fall patterns and activities of daily living using a set of significant extracted features from measured acceleration and angular velocity signals. For low computational requirements and to improve the classification performances, the Linear Discriminant Analysis is used to reduce the dimensionality of extracted features. The experimental results assess the performances of the proposed approach in fall detection that show the highest accuracy of 99.92% provided using the KNN classifier and accuracy of 97.5% for fall pattern recognition using the SVM classifier. Also, the online classification on the Fog device reached an accuracy of 94.42% using the SVM classifier. 1 Introduction Falling is a serious and dangerous incident in which a person ends up on the ground or other lower level [1]. It can be caused by multiple factors such as illnesses (e.g. Dementia and Parkinson) [2], old age, or hazards cause occurring during the activity of daily living (ADL) [3]. The World Health Organization reported that fall is one of the most traumatic-based causes of death with a number of 646,000 per year [1], especially among the elderly. Also, people over 65 years of age account for 50% of injury-related hospitalisations, where 40% of the injury is caused by a fall incident which leads to death [4]. Fall incidents can occur indoors or outdoors and often leave the victim with health difficulties or elderly disoriented and not conscious enough to get up or act independently [5]. This incident can cause hazardous injuries (e.g. joint sprains, broken bones, head trauma, and tissue injuries) when the person remains lying down for a long time after the fall [6]. Also, because of the expensive cost of treating injuries, the fall incident can create a significant financial burden especially in cases of severe injury [7]. Indeed, the lack of immediate treatment within the first hour of the accident is the main cause of death and chronic infections [8]. Owing to the subject conditions during the fall incident, an automatic, continuous, and reliable monitoring system is indispensable for alerting immediately the nearest emergency service to reduce the chronic effect and save the affected person medically in a short time. Hence, the development of a rescue solution for fall telemonitoring and detection is substantially necessary, while ensuring the accuracy, convenience, and real-time detection to avoid false alarm of health service. This issue has paved the way for many research works to ensuring the control and securing of any person vulnerable to fall. The main objective of the fall monitoring system is to distinguish between a fall incident and a daily movement such as sitting down, jumping, squatting, etc. [9]. Besides, a fall detection system can prevent fractures and other related health abnormalities such as heart attack, hypertension, respiratory difficulties, etc. Indeed, medical assistance has seen marked progress driven by the technological development of computer science, wireless communication, and embedded systems [10, 11]. This technological achievement provides continuous health telemonitoring including fall detection and alerting to ensure an immediate intervention. Consequently, many research studies and developed systems have been carried out on fall detection and prevention. Overall, the existing fall detection methods in the literature can be generally categorised into three types: visionary or camera-based detection sensor, environment or ambient sensors, and wearable sensors. The following section discusses the related research approaches for fall detection. 1.1 Visionary sensors-based fall detection These fall detection systems are generally based on visionary devices such as cameras installed in specific areas to monitor the activity of the concerned person. The captured images or video data are processed using computer vision techniques to identify the fall incident. Merrouche and Baha [12] proposed a fall detection framework using a Kinect sensor-based depth camera to analyse the shape, track head, and recognise the centre of mass of the subject. The fall is discriminated by applying thresholding to the covariance of translated distance and time. Triantafyllou et al. [13] presented a fall detection method consisting of the vertical velocity and area variance features extraction. The fall incident is detected by applying of hidden Markov model description. In [14], Sun et al. presented a depth camera-based fall detection scheme, where a thresholding algorithm is applied to the distance change of pixels in histogram presentation to discriminate fall from non-fall. Furthermore, other researchers have included machine-learning algorithms for fall incident identification. De Miguel et al. [15] used the K-nearest neighbour (KNN) classifier to distinguish fall events based on vision data provided by a camera and processed using the Kalman filter to track the subject and optical flow to eliminates static objects. Using the Kinect camera Kasturi and Jo [16], and Kong et al. [17] have proposed a fall detection technic consisting mainly of subject detection, feature extraction, and classification using support vector machine (SVM) to detect fall incident. The internet of things (IoT) system is used to alert when a fall event occurs. Using the same concept, Zarrouki et al. [18] presented a fall detection system using a camera and accelerometer sensor. The extracted features from the camera data are used in fall classification, where the authors evaluate performances and comparing SVM, KNN, Naive Bayes, and neural network (NN) classifiers. The visionary camera-based detection method enjoys many advantages, including any need for wearable devices, also it is based on intelligent image processing techniques that labour to the efficient recognition of subject shape. However, this approach is constrained in a precise coverage area, in which it is unable to track the subject outside the range of the cameras’ visibility. Moreover, it sometimes creates a privacy violation when monitoring human daily activity and focusing on the subject silhouette using a camera. Besides, for viable fall detection, many cameras are required to be deployed to cover the entire environment of the subject displacement, which imposes an expensive cost related to these systems’ implementation. 1.2 Environmental/ambient sensors-based fall detection The ambient fall detection category relay on environmental metrics for tracking the movement and gait of the subject. The main used sensors in this detection method based on the optical, pressure, audio, and floor vibration sensors, which are usually deployed in the area of the user [19]. Fauziah et al. [20] proposed a system composed of a vibration sensor, a voice recorder module to detect fall events, and a GPS and GSM modules to locate the subject and notify the occurrence of fall by SMS. Another solution is described by Muheidat et al. [21] consisting of a real-time fall detection using a sensor pad placed under the carpet. The system can record walking activity, stored and analysed data over a cloud. Convex hull area, heuristic, multilayer perceptron data maiming algorithms are used to detect accurate fall event. Daher et al. [22] have demonstrated that the fusion of accelerometer and force sensors allows accurate fall detection, particularly when the subject is in a lying or falling posture. The proposed system performs a real-time fall tracking, activity recognition, and locating of the subject. Using the combination of an accelerometer with a sound sensor installed in different locations inside the home to detect the accurate fall and avoid misclassification of daily activities as a fall is proposed by Er and Tan [23]. A fuzzy logic-based approach is employed to process measured signals, and to interpret a valid fall occurrence. Also, the system provides the alerting using a base station connected to the sensor. This technique of fall detection provides high performances in terms of accuracy and reliability for the reason that it uses sensible sensing measurements to the fall event. Nevertheless, this approach may not be conventional outside the coverage area of the sensors which makes it constrained. Besides, the complexity of implementation is imposed using this technique, which manifests in the necessity of a raised floor, the installation of several high-resolution sensors in walls and floor which drives to a very high cost of implementation. 1.3 Wearable sensors-based fall detection Recently, the wearable devices present a big challenge in healthcare technological support. A predicted increase from 235 to 929 million between 2016 and 2021 in the use of wearable sensors [24] demonstrates the flexibility and adaptation of these devices with various applications (e.g. health, environment, industry, military, etc.). This technique relays on deploying wearable sensors in the human body to acquire, process and transmit important health data to the health centres. A recent paper by Hussain et al. [25], proposed a continuous fall detection and activity recognition monitoring system based on a wearable 3D accelerometer and gyroscope sensor. They employed the SVM, KNN, and random forest (RF) for fall event identification and the fall associated activity recognition using a series of experiments data. Another solution is described in [26] concerning the prior detection of fall using angles and angular-velocity measurement gathered by two wearable sensors placed in the human waist and thigh. In this work, the performances of the hierarchical Fisher's discriminant analysis classifier are investigated when identifying non-fall, backward fall, and forward fall to avoid false alarms. The results obtained by Chen et al. in [27] suggest that the combination of ensemble stacked autoencoders with one-class classification based on the convex hull presents an intelligent approach for fall detection. Data collected from an accelerometer of a wrist-worn smartwatch is used on unsupervised features extraction and then in the pattern recognition. The obtained results from a series of varied human behaviours characteristics show high performances and stability using the majority voting strategy and weight adaptive adjustment strategy. The most interesting approach for real-time fall detection has been proposed by Torti et al. [28] concerning an architecture design of embedding of recurrent NN on a wearable device consisting of a microcontroller fed by an accelerometer. The SensorTile® device is employed in run-time detection to investigate and validate the memory occupation, power consumption, and computational load. Kerdjidj et al. [9] proposed an automatic fall detection system that relies on Shimmer wearable device connected wirelessly to a computer to transmit the measured data continuously. The compressing sensing technique is used to minimise the data size and to reduce the energy consumption. They investigated and compared the fall detection performances using an accelerometer or gyroscope sensor or both in three distinct situations: identify the presence of fall, detect static or dynamic movement, and recognise the fall and six activities of daily living ADL. Other research works [29–31] focus on the integration of the IoT technologies using wearable sensors and edge computing or IoT event detection service to detect and alert the fall incident. Overall, recent trends in fall detection systems point out the wearable devices are the most suitable and interesting technique that provide continuous monitoring without the limit in data collection regardless of subject location or position. Besides, compared to ambient systems or camera-based system, the wearable devices are less expensive, more power effective, and less intrusive. In this paper, a complete framework for fall incident telemonitoring is proposed using a multi-level architecture that combines the WSN and IoT technologies. This work aims to study and design a system that can detect multiple fall forms in real-time using machine-learning algorithms-based fog computing environment. Data are gathered continuously using a wearable wireless node consisting of a triaxial accelerometer and gyroscope sensors. The preprocessing of sensors data is performed locally to extract and transmit only the features while avoiding the dramatic increase in power consumption [32], and reducing transmission time. Most of the previous research studies focus on fall occurrence, however, it is important to recognise fall patterns to prevent future fall events. For this reason, supervised learning is employed to distinguish various fall patterns from the ADL. The implementation of machine-learning algorithms for online fall detection requires high performances in processing modules and power consumption [33], therefore, the wearable devices require to be lightweight as possible with minimised tightness, and a long battery lifetime [28]. To address these criteria, the learning of the classifier is done on a laptop using the Sisfall database [6], then, the actual test of obtained models is performed by embedding them in the Fog device for runtime experimentation. The Fog device is an intermediate agent that interfaces WSN and IoT cloud, i.e. it collects and processes data transmitted by the wearable node and aggregates continuously the obtained results in an IoT cloud. Also, it configures to alert and notify fall occurrence. The learning and online fall detection analysis were performed according to two scenarios: (A) Consists of two classes, i.e. fall and non-fall. (B) Aims to identify different fall forms from the most frequent intentional activities, The key contributions of this proposed research work are as follows: Present the complete study and implementation of human fall detection using the edge computing based on machine-learning algorithms over a multi-level architecture that combines the wireless sensor network (WSN) and the IoT. According to previous studies, the most significant features for accurate fall discrimination from other daily activities are presented and used in the supervised learning and run-time classification. Investigate the performances on linear discriminant analysis (LDA) in dimensionality reduction of data used in machine-learning algorithms An evaluation of classification performances in the learning and run-time phases is provided while comparing the SVM, RF, KNN, and multilayer perceptron NN classifiers in detection and fall pattern recognition. 2 Fall telemonitoring system architecture The proposed methodology in this work focuses on online fall detection. It relies mainly on a multi-level architecture composed of WSN, the IoT technology, and machine-learning algorithms. The overall structure of the proposed system is described in Fig. 1, where three layers are adopted to acquire, process, and supervise human activities remotely and continuously. Fig. 1Open in figure viewerPowerPoint Proposed fall telemonitoring system architecture 2.1 Acquisition layer As mentioned earlier, data about subject posture are obtained using a triaxial accelerometer MMA8452Q and the MU6050 gyroscope [34, 35] embedded into a wearable wireless sensor node. This node has been placed on the chest, as it is the best place to detect accurate falls according to [36]. The primary falling process releases on four phases: pre-impact phase, impact phase, unstable phase, and stable phase [37]. To cover these four phases, data acquisition is performed continuously with 200 Hz sampling frequency in the form of 5 s chunks. These chunks are then preprocessed to be appropriate for features extraction used in detecting falls. Meanwhile, for precise fall detection and management of energy consumption required in real-time transmission, the detection process is performed in a two-step. The first is based on thresholding executed locally, i.e. in the wearable sensor as follows. According to the study presented in [18, 38], the typical magnitude acceleration [i.e. (1)] Fig. 2 measured during a stationary position is about 9.81 ms2. During the fall event, this acceleration decreases around 0 g to reach in a short time its maximum value (i.e. >1 g) generated by the reaction force of the ground. Using the acceleration magnitude the first step of fall detection is performed using the comparison operation between and indicators while respecting the following rule: (1) The minimum measured acceleration <1 g and precedes the maximum reached acceleration . Fig. 2Open in figure viewerPowerPoint Magnitude acceleration during impact [9] The difference between and exceeds 1 g, i.e. . The time window between and should be <1 s according to [31]. In experimentation three values, i.e. 0.2, 0.5, and 0.9 s are tested. When fall event is detected, the wearable node sends the extracted features of the previous 5 s chunk to the fog device for the next step that consists of the online classification. Using the ultra-low power consumption radiofrequency transceiver NRF24L01 + [39] data is transmitted continuously to the fog device using the same transceiver forming a short-range wireless network up to 100 m. 2.2 Edge computing and data clouding layer With the aim to reduce the bandwidth and the transmission time of the data to the cloud, the post-processing of collected data is performed using edge computing. In this work, we employed a fog or edge node (i.e. the Raspberry pi 3 B + [40]) connected wirelessly as mentioned above to the wearable sensor node to collect data and perform the complex post-processing step for fall occurrences confirmation. Also, the machine-learning algorithms are trained before being implemented in this node for online fall discrimination. Consequently, a real-time decision can be taken and notified earlier to the health centres. Simultaneously, the used fog device is connected permanently to the Internet and configured to aggregate results about human activity in an IoT cloud continuously for the real-time telemonitoring. The flowchart presented in Fig. 3 explains the data routine between the first and second layers, describing the acquisition and edge computing process, and communication protocols between these layers. Fig. 3Open in figure viewerPowerPoint Flowchart of the functioning and communication protocol between (a) Acquisition layer, (b) Fog computing layer 2.3 Telemonitoring layer The last layer of the proposed architecture provides a visual presentation of the supervised subject health information. In this work, the fall detection information can be available remotely using an IoT platform that retrieves data aggregated in the cloud by the fog device. Thanks to this platform, healthcare professionals can monitor patients in their daily environment as the usual monitoring in the hospital using medical visualisation equipment, e.g. the monitor. When a subject suffers from a fall incident, this platform can alert nurses or doctors using visual indicators or sound alarms. Hence, the historic and actual health information about subjects including fall incidents can be managed and controlled easily without the need for handwritten reports and manual records of health data of each subject. 3 Machine learning for fall classification As mentioned above machine learning approaches are widely used in fall detection. Indeed, the process of classification was divided into two main phases, first the learning and then testing. The details about the supervised learning are provided in the following sections. 3.1 Dataset The proposed approach in this work is addressed to any subject in need of telemonitoring of fall incidents. Therefore, a diversified recording in terms of participants involved and types of activities is essential for robust learning and testing of fall recognition algorithms. In this context, the SisFall data set [6] is used as it includes data from 38 participants, 15 elderly, and 23 young people. It consists of heterogeneous activities data, i.e. 15 different falling activities and 19 various daily living activities in several retries for 12–100 s. SisFall data were recorded at 200 Hz sampling frequency using two 3D accelerometer and gyroscope embedded in a wearable device that was fixed as a belt buckle in the human waist. In this work, this dataset was configured to use only data from a single accelerometer and gyroscope as it is designed in the actual experimentation. Also, as all publicly available datasets, the labelling or temporal annotations are not provided, thus to use SisFall datasets manual labelling was performed for supervised learning. For the real-time telemonitoring of fall occurrence, two scenarios were considered as described before and shown in Fig. 4. The first one (A) consists of two classes, i.e. fall or non-fall, where the ADL is assigned to the non-fall class. The second scenario (B) attempts to discriminate falling activities by defining five classes: (i) Non-fall where the frequent daily activities, e.g. sitting down, getting up, jogging, jumping, laying, or squatting were included. (ii) Forward fall. (iii) Backward fall. (iv) Lateral fall. (v) All other fall patterns such as vertical fall, using hand, or when sitting down activity, etc. Fig. 4Open in figure viewerPowerPoint Adopted scenarios in supervised learning 3.2 Preprocessing To improve the learning and testing performances of the algorithms, the pre-processing is a critical task, especially noise cancellation that reduces the computational cost and provides clean data. The choice of filter should be adapted to accurately keep the relevant information, also to reduce the computational burden during learning and run-time with wearable devices. Therefore, the fourth order infinite impulse response low-pass Butterworth filter is employed for data denoising with a cut-off frequency of 5 Hz. Based on previous works [5, 6] this filter offers similar performance to more sophisticated filters, also, it is easy and simple to be implemented in wearable devices. 3.3 Features extraction Based on related previous studies on fall detection and activity recognition, several commonly used features enhance the separation between falls and ADL [5, 6, 41]. Moreover, in this work, the trained models are intended to be embedded in the Fog device for online classification. Therefore, the features extraction step is indispensable for reducing the bandwidth of the network by only transmitting features instead of the raw data. Besides, to ensure less computational burden for the fog device. The preprocessed dataset was segmented into ten seconds windows used to extract a set of features described in Table 1. Table 1. Extracted features sets FV Feature Mathematical equation Size 3 × 2 axis Se1 F1 minimum value [1 × 6] F2 maximum value [1 × 6] F3 mean [1 × 6] F4 variance [1 × 6] F5 standard deviation [1 × 6] F6 root mean square [1 × 6] F7 maximum peak-to-peak amplitude [1 × 6] F8 Pearson's correlation [1 × 6] Se2 F9 sum magnitude vector [2000 × 2] F10 angle between Z-axis and vertical [2000 × 2] F11 orientation of person's trunk [2000 × 2] Se3 F12 standard deviation magnitude [1 × 1] The extracted features were registered in a feature vector (FV) constructed of three sets referred to Se1, Se2, and Se3 consists of statistical, amplitude, orientation angles, and signal magnitude related features. As shown in Table 1, Se1 comprises eight features (i.e. F1–F8), each one is of size [1 × 6] for accelerometer and gyroscope sensors along three axes [x, y, z, Wx, Wy, Wz]. Therefore, the total size of Se1 is [1 × 6 (axis) × 8 (features)] = [1 × 48]. Se2 comprises three features (i.e. F9–F11), due to the sampling frequency is equal to 200 Hz, for 10 s the number N of rows come up to 2000, hence, the size of F1, F2, F3 reaches [2000 × 2] for both sensors, which is very large. Therefore, the statistical metrics, i.e. max, min, the difference (max−min), the mean, and standard deviation were computed to reduce the size of these features to [5 × 2] for both sensors. Hence, the total size of Se2 is [1 × (5 × 2) × 3 (features)] = [1 × 30]. Se3 comprises one feature F12 of size [1 × 2] for both sensors. Finally, for a 10 s window the entire size of extracted features obtained by concatenation of Se1, Se2, Se3 in FV is equal [1 × (48 + 30 + 2)] = [1 × 80]. Considering the resulting size of the FV, it is important to reduce the dimensionality without disturbing the information. This technique can significantly decrease computation time and the complexity of training of the machine-learning algorithms. The LDA is used to reduce the dimensionality of the extracted features set. 3.4 Linear dimensionality reduction The supervised LDA is very used to dimensionality reduction technique in machine learning applications [42]. In this work, LDA is used to reduce the extracted features dimensionality before the learning step. This technic led to select only relevant data for the training of machine-learning algorithms that provide the following improvements: Avoid overfitting caused by many features sets. Improve the computational efficiency by reducing the computational time and cost when using low dataset dimensions. Expedite the interpretation of data particularly when the features are correlated. The supervised linear data reduction LDA relay on the projection of the high number of datasets into less dimensional space while optimising the class separability and minimising simultaneously the separation of data intra-classes. The main goal of LDA is to calculate the projection matrix U [42, 43]. Algorithm 1 (see Fig. 5) describes the main steps for performing LDA. Fig. 5Open in figure viewerPowerPoint Algorithm 1: linear discriminant analysis 3.5 Fall classification The algorithm-based classification strategy aims to discern between confir