P0788 Exploring topics of Dutch Inflammatory Bowel Disease patient messages for improving transmural communication efficiency: A natural language processing approach
Authors: Zhang J , Fijten R , Puts S , Hendrix E , Okegunna Y , Mujagic Z , Pierik M 
Published: January 2025

Background

Patients with inflammatory bowel disease (IBD) in the Netherlands included in a remote monitoring care path can use the myIBDcoach app to communicate with their healthcare providers1. Currently, IBD physicians and nurses respond to the messages without triage, leading to limited communication efficiency and increased workload. We aim to identify common topics in Dutch IBD patient messages with natural language processing (NLP) topic models to enable automated triage.

Methods

We analyzed 13,199 text messages from 856 Dutch IBD patients sent via myIBDcoach between September 23, 2014, and October 19, 2022. After anonymization and cleaning, the messages were split into 42,537 sentences and processed using two Dutch text-embedding models: multilingual Sentence-Transformer and BERTje.2,3 Model-generated topics were manually merged and mapped to 16 predefined labels (medical or administrative) developed with expert input (Table 1). Messages were categorized as medical if they contained at least one sentence-level medical topic; otherwise, they were categorized as administrative. To evaluate model performance, we also manually annotated a random sample of 328 sentences from 100 messages. Model performance was assessed using label-matching accuracy, defined as the percentage of matches between computer-generated labels and manual annotations. Descriptive statistics were used to analyze topic distribution and message categories.

Results

The model with multilingual Sentence-Transformer embeddings (average label-matching accuracy = 50%) outperformed BERTje (average label-matching accuracy = 43%) and was chosen. Excluding irrelevant information (n = 8,722) such as greetings, the five most frequently identified sentence-level topics were Test Procedure & Results (n = 5,968), Appointment/Contact (n = 5,784), Symptoms (n = 4,690), Medication (n = 4,361), and General Health Update (n = 3,472). The two most common topics were administrative. The distribution of all identified sentence-level topics can be found in Figure 1. After aggregating the sentence-level topics back to the original 13,199 messages, we found that a significant proportion (41%, n = 5,412) contained exclusively administrative topics. In contrast, 59% of the messages (n = 7,787) concerned at least one medical topic requiring specialist responses.

Conclusion

Using NLP, we identified the common topics in text messages from Dutch IBD patients and evaluated the performance of general Dutch topic models in this specialized context. The findings of our research provide a strong basis for developing automated triage systems to increase response efficiency and reduce provider workload in the future for digital monitoring of IBD.

References

1.de Jong MJ, van der Meulen-de Jong AE, Romberg-Camps MJ, Becx MC, Maljaars JP, Cilissen M, et al. Telemedicine for management of inflammatory bowel disease (myIBDcoach): a pragmatic, multicentre, randomised controlled trial. Lancet. 2017;390(10098):959-68. doi: 10.1016/S0140-6736(17)31327-2

2.Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv. Published online August 27, 2019. doi:10.48550/arXiv.1908.10084

3.de Vries W, van Cranenburgh A, Bisazza A, Caselli T, van Noord G, Nissim M. BERTje: A Dutch BERT Model. arXiv. Published online December 19, 2019. doi:10.48550/arXiv.1912.09582