Towards ML -supported Triage Prediction in Real-World Emergency Room Scenarios (2024)

Abstract: In emergency wards, patients are prioritized by clinical staff according to the urgency of their medical condition. This can be achieved by categorizing patients into different labels of urgency ranging from immediate to not urgent. However, in order to train machine learning models offering support in this regard, there is more than approaching this as a multi-class problem. This work explores the challenges and obstacles of automatic triage using anonymized real-world multi-modal ambulance data in Germany.

## introduction the differentiation of treatment urgency is an important step in clinical emergency medicine. various validated triage systems have been established for this purpose, and in germany, their use is virtually mandatory. in practice, this means that within the first 10 minutes of a patient's arrival, assigning a treatment priority and thus setting a time target until contact with a medical professional is a required process. according to the manchester triage system (mts), the possible triage level ranges from immediate to non-urgent, which is mainly meant as guidance to lead employees in the emergency rooms (er) in making their triage decisions. although only five triage levels exist, the problem is not as straightforward as it seems. the triage model of mts follows a decision tree, where on the first level, a so-called diagram or lead symptom (diagnosis) is determined, and on the second level, indications (discriminator) specific to the selected diagram are identified. the indications translate to predefined triage levels, where the most urgent triage level among them expresses the severity of the case. since the diagrams and indications are not defined by sharp boundaries, it may well happen that a medical professional reaches the same indication through different diagrams. so, the same triage level can be decided on by choosing different, but equally valid indications. the data used in this work is a mixture of multiple text fields describing the situation of the patient and some first diagnosis (in the form of text), and a large set of structured information, i.e. medical measurement of vital signs, age or sex. in particular, vital signs such as temperature, oxygen saturation, etc. are an essential part of the mts model defining an indication. in this work, we have built prototypical machine learning models using retrospective data for automatic triage in the emergency ward and examine the results and obstacles of our approaches. more specifically, we examine to which extent a transformer-based bert model can address the problem of noisy, unbalanced, semi-structured multi-class real-world data. different training strategies are explored, particularly to deal with the different interconnected classes as well as to deal with the varying label frequencies. moreover, we investigate how we can extend a given bert model, which is normally only suitable for text data, by additional structured information. finally, we test an approach to build up a hybrid model, combining machine learning with a rule-based component. ## related work various studies so far have looked at the possibilities of automatic triage but differ in terms of data, models/solutions, target, and results. @xcite provide an overview of different triage use cases strongly related to nlp. however, many approaches target, for instance, text @xcite or a mix of structured and unstructured (text) data @xcite to predict a binary label, such as mortality or hospitalization. some others focus on a larger number of triage labels used in emergency care, such as @xcite ; @xcite . depending on the given data, solutions such as gradient-boosting @xcite or bert-based approaches @xcite , also in a hybrid setup @xcite , are popular. also, with the rise of large language models, llm-based solutions have been tested @xcite @xcite . so far, however, there are no studies that have automatically determined treatment priorities based on data from emergency services, nor are there any that predict such a large number of different classes simultaneously as we are trying to do. in this work, we deal with different types of data -(partly sequential) numerical, categorical, and text data. to handle such data types, many different approaches and architectures exist, to combine different 'modalities', such as for instance mapping all different information into one vector space @xcite , the combination of transformers with linear layers and lstms @xcite , or llms with time series @xcite . however, in this work, we rely on a simple architecture based on transformers, integrating different data types and exploring how far we can get. ## data and methods this work is based on anonymized ambulance reports with triage assignments from a german emergency ward covering two years and including more than 18k cases. the data was recorded electronically and contains a wide variety of different information-overall, about 600 different features exist, ranging from binary to numeric and sequential (e.g., sequences of particular measurements during the ride in the ambulance). the data includes information such as age, sex, blood pressure, pain score, information about consciousness, burns, medications, or motoric skills. in addition to the structured information, the data also includes text fields, describing the emergency situations, an initial diagnosis, injuries, symptoms, as well as the original cause of alarm. an example of a patient case is provided in the appendix. the data represents real-world data and has been labeled with the triage categories, consisting of a diagram (diagnosis) and an indication (discriminator), by the emergency department staff in accordance with the mts. as mentioned, the selected @xcite . for more details see ## conclusion in this work, we presented a challenging real-world problem to support employees in an emergency ward. although the data is multi-modal (numerical and text), we approached the problem with text-based transformer solutions. considering the difficulties with noise, missing labels, the number of different labels, and the long tail problem, the results are promising. however, we foresee that we need to include additional information as extra features to further boost the performance and to provide models with a more substantial benefit in an emergency ward.