<figure><img src="image_1.png" /></figure>

HAL
open science

# IoT Platform for Real-Time Multichannel ECG Monitoring and Classification with Neural Networks

Jose Granados, Tomi Westerlund, Lirong Zheng, Zhuo Zou

► To cite this version:

Jose Granados, Tomi Westerlund, Lirong Zheng, Zhuo Zou. IoT Platform for Real-Time Multichannel ECG Monitoring and Classification with Neural Networks. 11th International Conference on Research and Practical Issues of Enterprise Information Systems (CONFENIS), Oct 2017, Shanghai, China. pp.181-191, 10.1007/978-3-319-94845-4_16 . hal-01888638

HAL Id: hal-01888638

https://inria.hal.science/hal-01888638

Submitted on 5 Oct 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
IoT Platform for Real-time Multichannel ECG
Monitoring and Classification with Neural
Networks

Jose Granados¹, Tomi Westerlund², Lirong Zheng¹, and Zhuo Zou¹

¹ School of Information Science and Technology, Fudan University, Shanghai, China

² Department of Information Technology, University of Turku, Turku, Finland

**Abstract.** Internet of Things (IoT) platforms applied to health promise to offer solutions to the challenges in healthcare systems by providing tools for lowering costs while increasing efficiency in diagnostics and treatment. Many of the works on this topic focus on explaining the concepts and interfaces between different parts of an IoT platform, including the generation of knowledge based on smart sensors gathering bio-signals from the human body which are processed by data mining and more recently, deep neural networks hosted on cloud computing infrastructure. These techniques are designed to serve as useful intelligent companions to healthcare professionals in their practice. In this work we present details about the implementation of an IoT Platform for real-time analysis and management of a network of bio-sensors and gateways, as well as the use of a cloud deep neural network architecture for the classification of ECG data into multiple cardiovascular conditions.

**Keywords:** IoT, ECG, Healthcare, AI, Neural Networks

# 1 Introduction

The advancement of Internet of Things (IoT) platforms has promised to solve many of the challenges that healthcare systems worldwide face today. The IoT platforms, which refer to a comprehensive system involving the interconnection of smart sensors to cloud computing services, has spawn a new paradigm in the way healthcare services can be administered. The traditional setup in which a patient visits the physician's office, where he or she has to recall the symptoms from memory is being re-imagined from a reactive stand point to a proactive one. This is possible thanks to smart sensors that gather bio-signals from the human body which are transmitted to cloud services via intelligent web-enabled gateways as shown in Fig. 1. The health signals then pass through an analysis phase consisting in applying data mining and machine learning algorithms which can diagnose conditions with human comparable accuracy[1]. This way, intelligent systems can provide continuous secure monitoring of physiological parameters in order to detect patterns that might represent an abnormality. This methodology allows to automatically suggest lifestyle changes learned from previous cases or if the
<figure><img src="image_3.png" /><figcaption>Fig. 1. IoT Health Workflow</figcaption></figure>

condition is critical, alert the corresponding healthcare professionals to act on time before the ailment turns into a dangerous or expensive disease. Considering that most hospital visits are due to over-treatment, redundant, inappropriate, or unnecessary tests and procedures that can be diagnosed and treated at home, these intelligent systems enable the optimization of resource allocation to the more demanding cases.

IoT Platforms for healthcare still face significant obstacles in terms of performance, power consumption and functionality in order to unleash all the possibilities that they can provide. Many of the works in this topic study the implementation of specific cases for monitoring different physiological parameters. These bio signals often require distinctive transmission and processing needs which depend upon a particular solution. However, a comprehensive system that allows the monitoring of a network of sensors through embedded connected devices, as well as the management of a set of gateways from a cloud service in real-time with the hardware and software engineering details has been more or less still a work in progress. Data mining, and more recently Deep Learning has also played the fundamental role of classifying signals for diagnostic purposes with human-like accuracy. The potential of implementing Artificial Intelligence modules to the real-time processing of bio-sensor data in the context of an IoT platform for Healthcare therefore could have beneficial implications which could ameliorate healthcare services and revolutionize the insurance industry.
2 Design Considerations

2.1 Workload Balance

An important aspect to consider is the workload balance between different sys-
tem components, namely the edge node, the gateway and the cloud computing
modules. In order to ensure readiness and reaction time against health risks, the
wearable devices must have up to date detection algorithms available or have
permanent access to them on the cloud in real time. Using the cloud approach,
the process is similar to constantly ask the cloud service provider for the latest
diagnostics and treatments so that they can be presented to users immediately,
where the cloud is the knowledge repository that has been learned so far from
previous cases and is continuously expanding. Eventually and as edge node mi-
croprocessors become more powerful and energy efficient, the devices could train
the neural network locally and then share the trained package with other devices
in a peer to peer, decentralized fashion. In this scenario, there is less need to regu-
larly upload personal bio-signals which can reduce privacy concerns, and instead
only the learning, represented as neural networks weights, is communicated[2].
The goal is then to achieve the optimal balance in terms of power consumption,
performance, scalability and security between the different components of the
system, while providing quality diagnostics and treatment in a constantly grow-
ing knowledge database. With sufficient data, the intelligence repository could
achieve human-like detection accuracy of multiple conditions and propose the
best evidence-based treatment while monitoring health improvement.

## 2.2 Mobile Device Gateway

The incorporation of mobile devices such as smartphones and tablets into peo-
ple's the daily life can be leveraged to manage a network of smart sensors. These
mobile devices are usually equipped with Bluetooth 4.1, 4.2 and 5 Bluetooth Low
Energy (BLE) radios that can interface with low-power smart sensors in Body
Area Networks (BAN). These sensors are generally accessed using the Generic
Attribute Profile (GATT) protocol which presents the device properties as a
database of services and characteristics. This is specially suitable for defining
profiles which are definitions of possible applications and general behaviors that
Bluetooth enabled sensors use to communicate with gateways as well as to char-
acterize what kind of data a Bluetooth module is transmitting. Profiles allow
the standardization of intercommunication between different kinds of sensors
and gateways, even if the type of data is different, as long as they respect the
way to organize use case behaviors. A mobile device gateway can also leverage
established Web technologies such as Websockets, in order to send sensor data
to cloud servers in real time. Websockets can also serve as a base protocol on
which to implement device management frameworks that can be used to con-
trol gateways and end devices. In addition, the gateway can perform other tasks
such as preprocessing for computation offloading from the node devices as well
as serve as a firewall for protection against attackers outside and inside of the
sensor network.
## 2.3 Multichannel ECG

The IoT framework proposed in this work is exemplified in an intelligent multi-channel electrocardiography (ECG) monitoring and analysis system. ECG signals represent the heart activity and are used by cardiologists to diagnose cardiovascular and other diseases. The ECG signal has a particular relative high data rate and serves as a challenge for low-power embedded systems design. ECG also has been subject to different types of data mining and machine learning classification techniques particularly to detect arrhythmia, which once incorporated into a real-time IoT platform, can serve as a valuable companion tool for heart specialists. One of the goals of this work is to use ECG signals to diagnose not only cardiovascular diseases but also other kinds of correlated conditions such as pulmonary disorders. We believe that by using deep correlation and invisible features detection, abnormal signal patterns akin to multiple conditions can be exposed. An additional result from this methodology is the ability to infer context activities from the ECG that can be used to suggest better lifestyle habits for the user.

## 3 Related Work

The IoT for Healthcare research community has been active in offering solutions to the lack of efficiency and high cost of healthcare systems around the world. [3] describes the goal and the components of a complete IoT Platform for healthcare, including sensing, cloud service, data mining and end user's perspective. The focus of the IoT platform is to move the treatment to home care in order to save hospital costs and increase efficiency of physician resources. The authors use data clustering mechanisms for improving classification. However, they use a software emulator to generate the ECG data, not an embedded device capturing bio-signals, also most of the software engineering details are left out. A health monitoring system using a smartphone case with integrated electrodes and an Android application to display the data is presented in [4]. However, no integration with cloud services or analytics are depicted. [5] shows an Android application connected to an ECG device and to a cloud database. The details of the implementations are explained including the use of the Bluetooth Serial profile to transmit data between the device and the Android application. In addition, the authors describe the use of a SD Card to store the data locally on the smartphone. The ECG data is then uploaded to the cloud using a FTP server that saves the data on a *SQL Server Filetable*. This implementation nonetheless lacks real-time communication between the smartphone and the cloud server and uses MATLAB for visualization instead of user-friendly Web application. In addition, this solution does not adopt open source components, and seems to require an external Linux of Windows computer to transfer the data from the smartphone to the cloud server using *FileZilla*. An integrated solution in the form of an ASIC for measurement and analysis of ECG including keying for security purposes is presented in [6]. The authors perform Verilog simulations using *Modelsim* and *Synopsys* tools to verify the functionality of the design. The
end-to-end system is designed using Verilog-HDL, and a test-bench is created to simulate it by modeling the input data. The resulting ASIC achieves a reduction of 62.2% in power consumption and a 16% reduction in area when compared to similar state-of-the-art processors. The work in [7] describes a smart gateway for e-health which serves as a bridge between numerous sensor protocols and to execute preprocessing of health data in order to generate health indicators before uploading to the cloud. Fog computing is introduced at the gateway network layer in order to reduce the latency of the decision making process which is performed locally instead of in the cloud. A wearable sensor node with energy harvesting and Bluetooth interface with HTML5 based smartphone app is presented in [8]. The battery-less device is able to work 24 hours on adequate sunlight conditions and can transmit sensor data to a Web based smartphone application designed to display signals and send emergency notifications.

The current literature makes known the general components and concepts of an IoT platform for Healthcare such as bio-sensors, cloud computing, data analysis and visualization. However, there is still room for empirical studies in building such a system. We consider that it is practically important that the details of the actual software and hardware engineering are explained, as there is a pragmatic need to build a solid infrastructure in a way such that cost, performance and scalability assessments of real world implementations can be achieved. Current research is focused on describing specific use cases like transmitting a physiological parameter variable to the cloud using definite methodologies as well as explaining the main components of an IoT platform such as the edge nodes and cloud services without digging too much into the details of the implementation. We propose to create a general IoT framework that can be mapped to well established technologies such as Bluetooth LE, CoAP, Websockets, NoSQL databases, etc with a solid reasoning behind the selection and development of technologies used in order to obtain a comprehensive, well structured solution.

# 4 Real-time IoT Platform for Healthcare

We propose an IoT Platform for ECG analysis as shown in Fig. 2 consisting of a smart ECG sensor, Web-enabled gateway based on a smartphone and cloud server for monitoring and managing devices integrated with a deep neural network for classification of signals. Emphasis is done on the implementation of real-time capabilities by using Bluetooth LE GATT notifications on sensor nodes and Websockets on the gateway and cloud server link. That way we can achieve not only near real-time data transmission but also assemble a base communication channel on which a device management and control framework can be built. Also, the use of GATT profiles is taken to define and translate sensor resources and services into other Internet protocols. This allows scalability by standardizing access from the gateway to the sensors and easy mapping of sensor profiles to specialized web transfer protocols similar to the Constrained Application Protocol (CoAP), which include RESTful interaction models between application endpoints, support for service and resources discovery, and other Web concepts
such as Uniform Resource Identifiers (URIs) and Web Application Programming Interfaces (APIs). Finally, the ECG classification into different abnormal patterns is implemented on a multi layer Convolutional Neural Network realized using TensorFlow and hosted at the cloud. The supervised and unsupervised learning methods are used to classify the signals into normal, abnormal and as part of a cardiovascular disorder such as particular types of arrhythmia. Also, the platform incorporates a Web tool where expert ECG technicians can annotate ECG recordings which can serve as input to the supervised training network. The set of software stacks used in this implementation is shown in Fig. 3

<figure><img src="image_4.png" /><figcaption>Fig. 2. IoT Platform for ECG analysis</figcaption></figure>

## 4.1 Multichannel ECG Monitoring Device

The ECG embedded device consist of ten electrodes that capture electrical signals from the human body. This data gathering node is shown in Fig. 4. The device was designed and built for this work and is branded as *Horizon Medical IoT Holter*. The 12-lead ECG graph can be realized with this configuration which, contrary to single lead configurations, allows to not only recognize a cardiovascular condition but also helps to pinpoint its location on the heart surface. The signals sensed by the electrodes are amplified and converted to 24-bit, 250 samples per second digital representation by a Texas Instruments ADS1298 analog front end, which is then inputted via Serial Peripheral Interface Bus (SPI) into a Nordic Semiconductor nRF52832 microcontroller unit (MCU) with integrated 2.4 GHz Bluetooth Low Energy compatible radio. The embedded software programmed on the device allows to configure the device as a BLE peripheral with one custom GATT service and nine characteristics, one for the status information and one for each of the eight signal channels. The characteristics are defined with a notification property which allows to push the uncompressed signal data into the central device (smartphone) and makes use of the higher data throughput feature of Bluetooth 5 in order to support the higher bandwidth requirements. The ECG device allows secure connection to the smartphone via Near Field Communications (NFC) using the LE Secure Connections with Out
<figure><img src="image_5.png" /><figcaption>Fig. 3. Software stacks used on the device, smartphone gateway and cloud server.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. 8-Channel Bluetooth 5 Embedded Monitoring ECG Device</figcaption></figure>

of Band (OOB) Pairing. In this method, a large temporary key (128 bits) is exchanged when the device is in close range of the smartphone which makes it difficult for an attacker to perform Man-In-The-Middle (MITM) attacks. If the OOB channel is immune to eavesdropping during the pairing process, then the BLE connection will also be immune from passive eavesdropping. Preliminary results obtained for the device energy consumption are shown in Table 1.
<table><caption>**Table 1.** ECG monitoring device energy consumption</caption><thead><tr><th></th><th>Avg Current (mA)</th></tr></thead><tbody><tr><td>Advertising</td><td>4</td></tr><tr><td>Transmitting</td><td>5.4</td></tr><tr><td>Sleeping</td><td>0.8</td></tr><tr><td>Est battery life (800 mAh)</td><td>4.5 days</td></tr></tbody></table>

## 4.2 Mobile Device Gateway

The IoT gateway is a mobile device running Android or iOS operating system. An hybrid mobile app has been developed for visualization of the multichannel ECG signal as well as to forward the sensor data coming from the embedded device to the cloud server. In order to transmit the ECG signal data, the gateway creates a Websocket connection between itself and a cloud server. The Websocket protocol is secured with an OpenVPN encrypted connection. In addition, a device management framework based on top of the Websocket protocol is used to control the gateway and sensors from the remote server using an event-based API. This allows a remote server to scan for devices near each gateway and collect a list of available resources. In addition, the device management framework grants the possibility to ask specific sensors to change its operating modes such as to go from live streaming to local storage for power saving purposes. The mobile application user interface consisting of 8 live charts of sensor data is shown in Fig. 5. The mobile application is built using the Ionic Framework which allows to integrate a HTML5 based application together with native functionality into an hybrid app. The BLE Native plugin is used to establish the communication between the smartphone app and the peripheral device and allows to enable notifications from it. In the background, the Websocket connection between the app and the cloud server is implemented using the Socket.io client library. The services and characteristics are then translated into messages similar to the ones used by the CoAP protocol in order to leverage the use of URI based access to device resources as well as to perform service discovery. Preliminary results obtained for the gateway performance are shown in Table 2.

<table><caption>**Table 2.** Smartphone gateway performance</caption><tr><td>Avg data rate in/out</td><td>15.35 KB/s</td></tr><tr><td>CPU usage %</td><td>47%</td></tr><tr><td>Est energy use</td><td>66 mAh</td></tr><tr><td>Connection time</td><td>15m 51s</td></tr><tr><td>Data sent</td><td>9.92 MB</td></tr><tr><td>Device:</td><td>Samsung Galaxy S8+</td></tr></table>
<figure><img src="image_7.png" /><figcaption>Fig. 5. Smartphone app user interface showing multichannel sensor data.</figcaption></figure>

4.3 Convolutional Neural Network Classifier

The learning module responsible for classifying incoming ECG signals from the device as well as allowing training of the neural network is implemented using the TensorFlow version 1.3 library running on a GPU accelerated machine. A Socket.io server running on an Ubuntu 16.04 instance receives the signal data coming from the gateway. The server is implemented on Node.js version 8.4.0 and the Socket.io server used has version 2.0.3. The server also creates a connection to a RethinkDB version 2.3.6 database instance running on the same computer and stores the ECG sensor data as JSON documents containing arrays of 28 samples. On parallel, a TensorFlow engine is implemented which connects to the same RethinkDB database and queries the original raw ECG signal data. When the network is being trained, the ECG signal is first displayed on a Web based annotation tool which allows a heart technician to select waveform segments and label them according to different types of arrhythmias. The selected signals serve as inputs to train the network. The neural network is designed as successive convolutional, pooling, reshape, fully connected and dropout layers which once trained, takes the live incoming signal from the device and automatically classifies the ECG patterns into possible arrhythmias. The classification outputs are saved into the same RethinkDB instance in order to be used by a Web application designed to show results to the physician. The Web application allows to forward automatic preprogrammed alerts depending on conditions detected or personalized recommendations prepared by the physician via chat messages displayed on the previously mentioned smartphone app used by the patient.
## 5 Conclusion

In this paper we have presented a practical IoT platform for monitoring and classification of the 12-lead ECG. We have designed a Bluetooth 5 multichannel ECG monitoring device which connects to a cloud service via smartphone gateway using the GATT protocol and enabled notifications. The data is forwarded to a cloud service using a Websocket protocol which also serves as a base for a device management framework. The cloud service receives the data and passes it through a convolutional neural network which classifies the incoming signal into different types of arrhythmias. The same platform is also used to train the network by taking the signal incoming from the device and displaying it in a Web based annotation tool where heart specialists can select segments of the signal and label them. The system takes advantage of the protocols' real-time capabilities and allows scalability and workload balance between the different components of the system. This IoT platform allows to increase the efficiency of healthcare services by conducting home treatment instead of occupying valuable hospital resources. Future work includes optimizing energy efficiency on the device and gateway levels as well as implementing a FPGA accelerated neural network system for coprocessing on the cloud server.

## References

1. Pranav Rajpurkar and Awni Y. Hannun and Masoumeh Haghpanahi and Codie Bourn and Andrew Y. Ng, "Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks, " in CoRR, vol abs/1707.01836, 2017.

2. M. Liu, H. Jiang, J. Chen, A. Badokhon, X. Wei and M. C. Huang, "A Collaborative Privacy-Preserving Deep Learning System in Distributed Mobile Environment," 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, 2016, pp. 192-197. doi: 10.1109/CSCI.2016.0043

3. J. H. Abawajy and M. M. Hassan, "Federated Internet of Things and Cloud Computing Pervasive Patient Health Monitoring System," in IEEE Communications Magazine, vol. 55, no. 1, pp. 48-53, January 2017. doi: 10.1109/MCOM.2017.1600374CM

4. M. S. Mahmud; H. Wang; A. M. Esfar-E-Alam; H. Fang, "A Wireless Health Monitoring System Using Mobile Phone Accessories," in IEEE Internet of Things Journal, vol.PP, no.99, pp.1-1. doi: 10.1109/JIOT.2016.2645125

5. J. Mohammed, C. H. Lung, A. Ocneanu, A. Thakral, C. Jones and A. Adler, "Internet of Things: Remote Patient Monitoring Using Web Services and Cloud Computing," 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom), Taipei, 2014, pp. 256-263. doi: 10.1109/iThings.2014.45

6. M. Yasin; T. Tekeste; H. Saleh; B. Mohammad; O. Sinanoglu; M. Ismail, "Ultra-Low Power, Secure IoT Platform for Predicting Cardiovascular Diseases," in IEEE Transactions on Circuits and Systems I: Regular Papers, vol.PP, no.99, pp.1-14. doi: 10.1109/TCSI.2017.2694968

7. Amir M. Rahmani, Tuan Nguyen Gia, Behailu Negash, Arman Anzanpour, Iman Azimi, Mingzhe Jiang, Pasi Liljeberg, "Exploiting smart e-Health gateways at the
edge of healthcare Internet-of-Things: A fog computing approach," Future Generation Computer Systems, 2017, ISSN 0167-739X. doi: 10.1016/j.future.2017.02.014.

8. T. Wu, F. Wu, J. M. Redouté and M. R. Yuce, "An Autonomous Wireless Body Area Network Implementation Towards IoT Connected Healthcare Applications," in IEEE Access, vol. 5, pp. 11413-11422, 2017. doi: 10.1109/ACCESS.2017.2716344