# Improving RNN Performance by Modelling Informative Missingness with Combined Indicators
## 4 Discussion  
The FCS approach with \(m=5\) imputations underperformed all other methods, both in terms of prediction accuracy and computation time (increased by a factor \(m\)). The most likely explanation for its subpar accuracy is that the missing data were MNAR, and FCS is not suitable for MNAR. This assertion is supported by the increased performance observed when directly specifying missingness through indicators (Table 3). Another contributing factor might be that the amount of missingness (70.1% on filled in surveys; 98.9% in total) is simply too large, causing issues when imputing missing data and then'reusing' the data for the predictive model. While FCS might have benefitted from a considerably larger number of imputations, it was already the most computationally costly method. Since model averaging generally tends to increase performance, the additional computation time--if available--might perhaps be better spent on e.g., bagging other methods, rather than drawing additional imputations with FCS. Interestingly, correctly padding and masking the missing days did not improve accuracy, nor did the forward filling strategy proposed in [PERSON] et al. (2016) [PERSON] et al. (2016). The former could either be attributed to a small effect of time in between survey entries on the outcome, or inability of the model to learn this effect using the number of observations at hand (\(n_{\text{train}}=4776\)) in conjunction with the high amount of missingness. A limitation of this comparison is that we did not further investigate how different regularization techniques might have affected the performance of the padding strategies. The forward filling strategy might be better justifiable for clinical data, where measurements are taken at intervals after which they are expected to change. While ignoring the missing days resulted in the greatest performance (Table 6), reasonable performance could also be achieved by masking and padding the missing days in between measurements. This method might achieve higher accuracy on problems where the time in between observations is of greater relevance to the outcome. These results demonstrate that with appropriate adjustments, a deep learning algorithm can even learn reasonably from a sparse sequence of gappy data. Various simplifications of previously suggested methods greatly reduced computational time, while simultaneously outperforming more costly methods in terms of prediction accuracy. The greatest accuracy was achieved on these data, by including a LR variable and a count of the missing variables, followed by Z-scoring and zero imputation. \begin{table}
\begin{tabular}{l l c c} \hline \hline  & & **10**-Fold CV Mean \(\pm\) Std & **Test Performance** \\ \hline \multirow{2}{*}{F\({}_{1}\)} & **Minimal (2k)** & 0.780 \(\pm\) 0.039 & 0.781 \\  & **Minimal (4k)** & 0.796 \(\pm\) 0.034 & 0.795 \\  & **Minimal (6k)** & 0.796 \(\pm\) 0.034 & **0.796** \\  & **Partial (2k)** & 0.779 \(\pm\) 0.037 & 0.776 \\  & **Partial (4k)** & 0.787 \(\pm\) 0.036 & 0.785 \\  & **Partial (6k)** & 0.788 \(\pm\) 0.038 & 0.788 \\  & **Complete (2k)** & - & - \\  & **Complete (4k)** & - & - \\  & **Complete (6k)** & - & - \\ \hline \multirow{2}{*}{AUROC} & **Minimal (2k)** & 0.751 \(\pm\) 0.040 & 0.740 \\  & **Minimal (4k)** & 0.754 \(\pm\) 0.044 & 0.743 \\  & **Minimal (6k)** & 0.758 \(\pm\) 0.045 & **0.750** \\  & **Partial (2k)** & 0.723 \(\pm\) 0.062 & 0.719 \\  & **Partial (4k)** & 0.736 \(\pm\) 0.051 & 0.731 \\  & **Partial (6k)** & 0.735 \(\pm\) 0.047 & 0.731 \\  & **Complete (2k)** & - & - \\  & **Complete (4k)** & - & - \\  & **Complete (6k)** & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of padding strategies after 2k, 4k, and 6k epochs. The highest test performance is shown in bold. Importantly, our method does not require a subset of the data which is complete, nor does it require the missing data to be MCAR or MAR.