emoTrack: A Low Power Wireless System for Emotion Recognition

The Internet of Things (IoT) with smartphone technologies has vast applications in solving the problems of mental diseases in patients needing care. With wireless sensors and smart devices, remote monitoring can identify the real-time mental status of patients during normal physical activities. In this research, we designed and developed an embedded sensory system (emoTrack) with a low power Bluetooth communication module to discreetly collect Electrocardiogram (ECG) and Electroencephalography (EEG) data using a smartphone in a common environment. Experimentation and verification is conducted on a number of test subjects with different test scenarios. It is observed that the proposed emoTrack system is able to classify mental health crisis with a high accuracy. 1. SYSTEM OVERVIEW Multiple sensor networks in an Internet of Things (IoT) environment, including wearable and/or mobile devices, can be used to develop a much more detailed view of the behaviors that precede a mental health crisis [1], the moment of crisis itself, and its immediate aftermath. This research proposes an IoT approach designed to predict one specific type of psychological crisis event: depression and lack of independence in the user using ECG and EEG sensors data. In the emoTrack system shown in Figure 1, the sensors are used to collect the raw ECG and EEG patterns while the user is walking. Then, the resultant outputs are processed by the smartphone to identify the user’s emotional crisis state. Figure 1. Overview of IoT-based emoTrack System The IoT system consists of an ECG and EEG sensor [2], an Arduino, and a Low Power (LP) Bluetooth communications system. A smartphone is used to collect and process the sensors data. The Arduino is used as an analog to digital converter (ADC). The system reads the sensors value through smartphone health collector interface. 2. EVALUATION OF THE SYSTEM To evaluate the system, we developed a prototype application and investigated its performance. We evaluated the prototype with an extensive experimentation. To test the effectiveness of our proposed methodology, we collected data from an ECG and EEG sensors connected to the user body as presented in Figure 2. We used multiple subjects and collected data for different events of user daily activity. The smartphone placed was either in the subject’s hand or a pocket. For data collection, we implement applications for heart rate and brain electrical activity monitoring on the smartphone. First, the system reads the analog signal from Body Area Sensor (BAS), then passes it to the acquisition device through a built-in amplifier and filter. The signal is send to the Android mobile application using a LP Bluetooth device and later graphed and analyzed on the smartphone. When a crisis event is identified using BAS data for the identified time frame of the event occurrence will be processed to identify potential clusters of early warning signs [3]. The collected data will be systematically processed to reconstruct the events surrounding each potential early warning sign. (a) Sample ECG signal while walking (b) Sample EEG signal while walking Figure 2. Sample BAS Signals for a Test Subject In this research, we have developed an embedded IoT system to monitor user emotional crisis events using a smartphones. The results from sensors’ data are also presented to show that this approach can detect user emotional crisis. REFERENCES [1] H. James et al, “Development of a Wearable Sensor System for Dynamically Mapping the Behavior of an Energy Storing and Returning Prosthetic Foot” Measurement Science Review. Volume 16, Issue 3, Pages 174–182, ISSN 1335-8871, June 2016. [2] https://www.sparkfun. com/products/14022,https://www.sparkfun. com/products/14022 [ Lat Accessed: 14 January 2018] [3] Y. Lin,” EEG-Based Emotion Recognition in MusicListening”, IEEE Transactions on Biomedical Engineering, July 2010.