## 4 Discussion and Limitation  
In this study, we proposed a novel entropy-based feature engineering framework for vital signs, based on their frequency of records and clinical availability. Both quantitative analysis of the features and data quality are taken into analysis. In addition, we conducted feature window and time gap experiment in order to determine the best time window to maximize the accuracy of AKI prediction in ICU. The etiology of AKI is usually multi-factorial. Dehydration, infection, renal toxic medications, and contrast medium injection during computed tomography scanning are common risk factors for AKI [19]. Clinicians are usually aware of AKI only when patients' urine output decrease or elevated serum creatinine occur. In 2014, the FDA approved a commercial urine stress biomarker, Nephrocheck(r), to be used for ICU patients for early prediction of AKI with 12-h time gap windows in advance [20]. However, the commercial kit is very expensive and has not been approved by other countries other than the United States. On the other hand, we sought to integrate an AKI prediction algorithm into electronic health information system in our hospital to provide a real time AKI risk probability for ICU patients without spending additional cost and nursing labors. To achieve this goal, we mathematically define the EHR data availability according to different time gaps and feature windows to apply state-of-the-art machine learning techniques. \begin{table}
\begin{tabular}{p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt}} \hline \hline
**Feature Types** & \multicolumn{3}{c}{**AUROC (\%)**} & \multicolumn{3}{c}{**Accuracy (\%)**} & \multicolumn{3}{c}{**NRI (\%)**} \\ \hline
**Feature Window** & **24 h** & **48 h** & **Improvement** & **24 h** & **48 h** & **Improvement** & **24 h** & **48 h** & **Improvement** \\ \hline Shannon entropy & 70.01 & 76.8 & 6.79 & 65.82 & 69.33 & 3.5 & – & 8.32 & 8.32 \\ Mean & 72.48 & 71.31 & \(-\)1.16 & 67.91 & 67.42 & \(-\)0.48 & – & \(-\)0.88 & \(-\)0.88 \\ Variance & 67.08 & 69.51 & 2.42 & 64.7 & 65.96 & 1.26 & – & 3 & 3 \\ \hline Mergo & 77.64 & 81.24 & 3.59 & 71.62 & 73.67 & 2.05 & – & 4.4 & 4.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Comparison of 24 h and 48 h feature windows on AUROC, Accuracy, NRI. \begin{table}
\begin{tabular}{p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt} p{28.5 pt}} \hline \hline
**Feature Types** & \multicolumn{3}{c}{**F1-Score**} & \multicolumn{3}{c}{**NRI (\%)**} & \multicolumn{3}{c}{**Improvement**} \\ \hline
**Feature Window** & **24 h** & **48 h** & **Improvement** & **24 h** & **48 h** & **Improvement** & **24 h** & **48 h** & **Improvement** \\ \hline Shannon entropy & 54.55 & 64.59 & 10.03 & 55.37 & 59.32 & 3.94 & 54.96 & 61.84 & 6.88 \\ Mean & 60.56 & 59.28 & \(-\)1.27 & 57.56 & 57.48 & \(-\)0.08 & 59.02 & 58.37 & – & 0.65 \\ Variance & 52.2 & 54.97 & 2.76 & 53.97 & 55.9 & 1.92 & 53.07 & 55.43 & 2.35 \\ \hline Mergo & 67.43 & 70.88 & 3.45 & 61.48 & 64.37 & 2.52 & 64.51 & 67.47 & 2.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Comparison of 24 h and 48 h feature windows on Precision and Recall. Compared to recent deep learning-based AKI prediction studies [6; 21; 22], we not only conducted AKI prediction, but also proposed an overall framework that covers feature pre-processing, missing value imputation, and proposed a novel entropy-based vital sign feature engineering. Moreover, we conducted a fine-grained analysis considering both time gap variation and feature window variation. In feature window examination, we tested the feature window length from 24 h to 48 h before AKI onset, with a fixed time gap of 24 h. Our results demonstrated that early AKI can be predicted, based on our proposed vital sign entropy-based features, with a feature window of 24 h and a time gap of 24 h. The best-performing model was obtained when using all available features with the longest feature window.