# A Comparison of Deep Learning Methods for ICD Coding of Clinical Records
## 6 Discussion  
A comparison between the results displayed in Tables 3 and 4 shines a light on the value of the multiview component. The micro F1 scores of the five models are in similar relationship to each other in both tables, except for the two multiview models. They outperform CAML in the full label setting of the MIMIC-III Dis dataset, where they show very similar behavior to CAML in a top-50 category setting, where for each of the categories a decent amount of training samples is available. When the target space increases and more categories have fewer training examples, the added granularity of having multiple kernel sizes in the MVC-(R)LDA model pays off. Table 5 shows results for models trained on the Full dataset. The best performing model (MVC-LDA, 58.12%) gets outperformed by the best performing model for all labels on Dis (MVD-LDA, 59, 75%). The addition of the information in other medical documents than just discharge summaries thus seems to complicate instead of facilitate the classification process.  
Furthermore, comparing Tables 3 and 6, where the influence of the hierarchical parameter can be assessed in a top-50 category setting, reveals a shift in the opposite direction. While in general, the modeling of the hierarchical relationships hurts the classification process for all categories, it hinders the multiview models the most. This time, DR-CAML is clearly the best performing model. Adding  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Micro F1**} & **Macro F1** & **Micro AUC** & **P@5** \\ \hline \multirow{4}{*}{**CodiEsp**} & **CNN** & 12.44 & 6.18 & 53.08 & 2.84 \\ \cline{2-6}  & **GRU** & 11.87 & 11.50 & 50.14 & 7.68 \\ \cline{2-6}  & **DR-CAML** & 10.35 & 3.97 & 53.61 & 5.59 \\ \cline{2-6}  & **MVC-LDA** & 13.00 & 2.79 & 60.76 & 11.94 \\ \cline{2-6}  & **MVC-RLDA** & 13.92 & 4.21 & 56.38 & 8.72 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Results of hierarchical models for top 50 codes obtained on the CodiEsp dataset.  
Figure 9: Ablation study on highest performing model (hierarchical MVC-RLDA) for CodiEsp dataset.  
multiview and simultaneously modeling the hierarchical relationships between the target categories tend to make the model overfit on the training data.  
Looking at Tables 7 and 8, it is clear that the lack of a sufficient amount of training data in CodiEsp (about 100 times less than for the Dis dataset) for most categories led to lower performance of all models on this dataset. For the flat variants of the models, a regular CNN even outperforms the more complex models. As the amount of training data is low, the added complexity of the latter models hinders them generalizing well for unseen data. Comparing the results in both tables also leads to the conclusion that in contrast to the results on MIMIC-III, on average the hierarchical component increases the classification performance based on Micro F1 on CodiEsp. Where the information embedded in the ICD taxonomy is redundant and even counteracting the performance for the larger MIMIC-III dataset, it is leveraged when there is a lack of information in the training data itself, which is the case for CodiEsp.  
Last, Figures 8 and 9 display the relative importance of the long description regularization, the multi-view. and the hierarchy for the top performing model on both the Dis and CodiEsp datasets. For the Dis dataset, not using the hierarchy is by far the most important component. The regularization on long descriptions still adds 0.46% and the multi-view almost does not influence the results. For CodiEsp, it shows that the multi-view component has the biggest influence, followed by the hierarchy, whose importance on this smaller dataset is already shown previously.