# Recurrent Neural Network-Based Multimodal Deep Learning for Estimating Missing Values in Healthcare
## 2 Related Work
### Estimation of Missing Value Using Multimodal Deep Learning  
In a heterogeneous healthcare platform, missing values can occur because of the condition of the user's device. Missing values are caused mainly by battery charge or discharge. They are also caused by various hardware and software errors, such as sensor failure, equipment replacement, and near-field communication error [29]. The higher the rate of the missing values, the more accurate the analysis or prediction influences the health analysis or decision making. Therefore, to achieve a more stable health platform, missing values need to be taken into consideration. A missing value is handled by replacing it with a mean, median, or estimated value, or by removing a relevant attribute [9]. In the method where it is replaced with a mean or median, if there are many missing values, the overall reliability can decrease, and the item is highly likely to become noise in terms of \begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Representative Value** & **t001** & **t002** & **t003** & **t004** & **t005** & **t006** & **t007** & \\ \hline smartphone & 4011 & 7893 & 10,048 & 3212 & 8772 & 3541 & 4065 & \\ \hline travel range & 3678 & 6980 & 9887 & 3098 & 8322 & 3289 & 3808 & \\ \hline smartband & 4237 & 8012 & 10,832 & 3348 & 8845 & 3743 & 4261 & \\ \hline median & 4011 & 7893 & 10,048 & 3212 & 8772 & 3541 & 4065 & \\ \hline mean & 3975 & 7628 & 10,255 & 3219 & 8646 & 3524 & 4044 & \\ \hline \end{tabular}
\end{table} 
Table 3: Selection of representative values with the uses of the mean, median, and quartile (redundancy value of step count is _’3’_). the time series characteristic. Estimation is a method of finding an approximate value to replace incomplete or uncertain data [11; 12; 13; 14]. If it is difficult to find an accurate value for a target, an optimal result is therefore found by means of estimation. Deep learning is able to predict a missing value through the connection of nodes. Nodes are linked with each other through a weight that is updated through repeated learning and back propagation. Deep learning calculates an output value by using the updated weight and the influence of input variables that are transmitted by nodes. In such a structure, deep learning is able to estimate a missing value by means of an FCN. The deep learning-based estimation of missing values solves the first-rating problem and sparsity problem (from which other estimation methods suffer) through the weight learning of nodes. The first-rating problem means that if a new variable has no data at the same time as other variables, it is impossible to make an estimation [4; 30]. Sparsity occurs when a variable has many missing values. If sparsity occurs, either an estimation is impossible, or a large error occurs [31]. Accordingly, in this study, an approximate value of a missing value was determined through the neural network-based estimation. As described in Section 3.2, RNNs that are learned according to variables are connected with each other through an FCN to estimate a missing value. In an FCN, all the nodes are connected with each other. As the learning data of the FCN, the healthcare proof data that were collected for 100 days were used. The data did not include those used for RNN learning and performance evaluation. In learning, the answer for weight correction has the next time step of each variable. The input layer of the FCN comprised five variables as output values of an individual RNN. As shown in Figure 3, the RNN had 36 outputs: 9, 3, 3, 6, 3, 9, and 3. Accordingly, the input layer had 36 nodes and the output layer also had 36 nodes. As a result of learning by changing the number of hidden layers in the proposed network, when the number of hidden layers is small, it appears sensitive to missing values, and when the number of hidden layers increases, it appears that excessive learning is required. The nodes of the hidden layer of the FCN were the same as those of the output layer; therefore, they also numbered 36.