# Leveraging Language Models for Inpatient Diagnosis Coding
## 5 Conclusions  
This study investigated the use of language models for inpatient diagnosis coding. Using our institutional dataset of over 230,645 admissions, our study highlighted the competitive performance of general-purpose language models, Multilingual BERT and Multilingual E5, and a domain-specific language model, MEDPSU-RoBERTa. Model performance also varied across diseases, with larger sample sizes leading to better results. The rarity of certain diseases poses a challenge for accurate coding, with more than 100 samples per diagnosis label suggested. We also analysed the impact of document length on performance and found that longer documents could pose a challenge to the model. Co-occurrence analysis demonstrated the model's ability to capture relevant associations. The results highlight the need for larger datasets, strategies to deal with class imbalance, and integration of historical data to improve performance. Future research directions include improving the handling of rare diseases, improving the integration of historical data, integrating additional healthcare data sources and focusing on the implementation of coding systems in hospital information systems. These results contribute to the further development of diagnosis coding systems, highlighting their potential to facilitate accurate and efficient workflows.  
Conceptualization, K.S. and D.T.; methodology, K.S. and D.T.; software, K.S. and D.T.; validation, K.S., D.T., S.C. ([PERSON]) and S.C. ([PERSON]); formal analysis, K.S. and D.T.; investigation, K.S. and D.T.; resources, S.C. ([PERSON]); data curation, S.C. ([PERSON]) and D.T.; writing--original draft preparation, K.S., S.C. ([PERSON]) and S.C. ([PERSON]); writing--review and editing, K.S., S.C. ([PERSON]) and S.C. ([PERSON]); visualization, K.S. and S.C. ([PERSON]); supervision, S.C. ([PERSON]); project administration, S.C. ([PERSON]); funding acquisition tion, S.C. ([PERSON]). All authors have read and agreed to the published version of the manuscript.  
This work was supported by the National Science, Research and Innovation Fund (NSRF) and Prince of Songkla University (Grant No. MED6601201S). [PERSON] and [PERSON] received graduate scholarships from the Faculty of Medicine of Prince of Songkla University.  
The study was approved by the Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University (REC.65-120-38-2) and was conducted in accordance with the principles of the Declaration of Helsinki and the International Conference on Harmonization in Good Clinical Practice.  
Informed Consent Statement: Informed consent was waived due to the retrospective nature of the study.  
Due to privacy concerns and the inclusion of sensitive patient information, the dataset used in this study was not published. MEDPSU-RoBERTa is a private clinical language model pre-trained on clinical documents from Songklanagarind Hospital. Due to the risk of data leakage during language generation, this model is not publicly available.  
The authors declare no conflict of interest.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l} AI & Artificial Intelligence \\ BERT & Bidirectional Encoder Representations from Transformers \\ CNN & Convolutional Neural Network \\ HIS & Hospital Information System \\ ICD & International Classification of Diseases \\ ICD-10-AM & International Classification of Diseases, Tenth Revision, Australian Modification \\ ICD-10-TM & International Classification of Diseases, Tenth Revision, Thai Modification \\ NLP & Natural Language Processing \\ RNN & Recurrent Neural Network \\ SVM & Support Vector Machine \\ TF-IDF & Term Frequency-Inverse Document Frequency \\ WHO & World Health Organization \\ XLM & Cross-lingual Language Model \\ \end{tabular}