  International Classification of Disease (ICD) coding procedure which refers
to tagging medical notes with diagnosis codes has been shown to be effective
and crucial to the billing system in medical sector. Currently, ICD codes are
assigned to a clinical note manually which is likely to cause many errors.
Moreover, training skilled coders also requires time and human resources.
Therefore, automating the ICD code determination process is an important task.
With the advancement of artificial intelligence theory and computational
hardware, machine learning approach has emerged as a suitable solution to
automate this process. In this project, we apply a transformer-based
architecture to capture the interdependence among the tokens of a document and
then use a code-wise attention mechanism to learn code-specific representations
of the entire document. Finally, they are fed to separate dense layers for
corresponding code prediction. Furthermore, to handle the imbalance in the code
frequency of clinical datasets, we employ a label distribution aware margin
(LDAM) loss function. The experimental results on the MIMIC-III dataset show
that our proposed model outperforms other baselines by a significant margin. In
particular, our best setting achieves a micro-AUC score of 0.923 compared to
0.868 of bidirectional recurrent neural networks. We also show that by using
the code-wise attention mechanism, the model can provide more insights about
its prediction, and thus it can support clinicians to make reliable decisions.
Our code is available online (https://github.com/biplob1ly/TransICD)
