# Recurrent Neural Network-Based Multimodal Deep Learning for Estimating Missing Values in Healthcare
## 2 Related Work
### Selection of a Representative Value for Data Duplication Processing  
In a heterogeneous healthcare platform, variable duplication occurs one to n times according to user situations. For instance, a user's step count can be collected by means of a smartphone, health band, or GPS. To calculate a user's step count, smartphones and health bands use a gyro sensor and GPS uses the travel distance [28]. Smartphones and health bands produce different results according to the body region on which they are worn and redundantly collect multiple values of a user's count at the same time. If data duplication occurs and variable \"A\" has multiple values at time t, a user can be confused in his/her decision making. In this case, it is necessary to integrate the redundant values into the most appropriate single value. Such data duplication frequently changes according to the user's surrounding situation. By applying the method of selecting a representative value among collected data, it is possible to use a conventional deep learning model regardless of the user's situation and device type changes. Even if two or more devices measure a user's travel distance or a new device is added, there is no need to relearn the existing neural networks. Figure 2 shows the data duplication of heterogeneous devices. Figure 1: RNN-based multimodal deep learning for estimating missing values. If data redundancy occurs, a representative value selection method is used to choose the value at time t of data variable A. Various methods exist for selecting a representative value, such as calculating the mean, median, mode, quartile, percentile, and trimmed mean [9]. Here, health proof data were collected under the supervision of a health examination center. These are the data from 26 people, including 10 men in their 20s, 5 women in their 20s, 7 men in their 30s, and 3 women in their 30s, who are interested in health care. The data were collected from 10 sleeping mats, 6 smartwatches, 17 health bands, 26 smartphones, and blood glucose meters. These empirical health care data were collected for 300 days using 11 sphygmomanometers, 11 sphygmomanometers, and mobile applications. A variety of PHRs were collected through smartphone applications. In this study, variables that have the time series characteristic were used. In reality, two to four data items are redundant on average. In the preprocessing process, an input of the deep learning is a selected representative value from each method. The variables were sleep data, heart rate, step count, temperature, and humidity. Table 1 shows the devices of the healthcare participants. According to the device type that they used, the participants were classified into five groups; Group A consisted of participants who had a smartphone, a mobile application, a health band, a blood pressure meter, and a blood sugar meter. Table 2 presents the data redundancy count by device type. The sleep data were collected from 10 persons by means of smartphone, smart watch, health band, and sleep mat. The attributes of the sleep data were deep sleep, light sleep, and roll over in sleep. There were redundant sleep data in groups B, D, and E. Both Groups B and D had three \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Consist of Device**} \\ \hline
**Group** & **A** & **B** & **C** & **D** & **E** \\ \hline Smart Phone & O & O & O & O & O \\ \hline Mobile App. & O & O & O & O & O \\ \hline Health Band & O & O & O & - & O \\ \hline Sleep Mat & - & O & - & O & O \\ \hline Smart Watch & - & - & O & O & O \\ \hline Blood Pressure Monitor & O & - & - & - & - \\ \hline Glucose Meter & O & - & - & - & - \\ \hline People & 11 & 5 & 5 & 4 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Devices of healthcare participants. Figure 2: Data duplication of heterogeneous devices. duplicate data, and group E had four. The heart rate data were collected by means of smartphone, smart watch, and health band.