Triaging and Referring in Adjacent General and Emergency Departments (the TRIAGE-trial): a Cluster Randomised Controlled Trial
Phase: NA
Conditions: Triage
Sponsor: Universiteit Antwerpen

  
Short Research protocol  
Version: 1.1 
Triaging and Referring In Adjacent General and 
Emergency departments (the TRIAGE-trial): a 
cluster randomised controlled trial 
 

Triage-trial short research protocol 1.1 
 
7 November 2018 
2 
 
 Table of contents 
 
 
Table of contents ....................................................................................................................................... 2 
 
Administrative information ....................................................................................................................... 3 
 
Introduction ............................................................................................................................................... 4 
Dutch Abstract .................................................................................... Fout! Bladwijzer niet gedefinieerd. 
Overview .................................................................................................................................................... 4 
Study sites and partners ............................................................................................................................ 4 
eMTS .......................................................................................................................................................... 5 
 
Research Approach .................................................................................................................................... 5 
A diagnostic Intervention .......................................................................................................................... 5 
Hypotheses and Research questions ......................................................................................................... 6 
Design of the clinical trial .......................................................................................................................... 8 
Methodology for the medical aspect ........................................................................................................ 9 
Methodology for the financial aspects ...................................................................................................... 9 
 
References ............................................................................................................................................... 11 
 
 
 

Triage-trial short research protocol 1.1 
 
7 November 2018 
3 
 
 Administrative information 
Full title: Triaging and Referring In Adjacent General and Emergency departments (the TRIAGE-trial): a cluster 
randomised controlled trial 
Short Title: the TRIAGE-trial: Triaging and Referring In Adjacent General and Emergency departments 
Protocol version: short protocol 1.0 (7 september 2018) 
Financial support: this trial is sponsored by the Research Foundation - Flanders (FWO). Grant Number 
T000718N. The sponsor is not involved in the study-design, data collection, management, analysis and 
interpretation, writing of the report and publication of the results.  
Authors: 
The TRIAGE-trial is a complex healthcare intervention with involvement of many stakeholders: 
• 
The Centre of General Practice of the University of Antwerp (part of research group ELIZA), 
coordination centre 
o Stefan Morreel1,2: main researcher, principal author of this entire document unless stated 
otherwise 
o Hilde Philips1,2: promotor of the entire trial, data protection officer 
o Veronique Verhoeven1,2: promotor of the entire trial, spokesperson 
• 
The Emergency Department of the Antwerp University Hospital (part of research group ASTARC) 
o Koen monsieurs1: promotor of the medical aspect 
o Joo-Ree Melis2: study-nurse 
• 
Department of General Economics of the University of Antwerp 
o Diana De Graeve1: promotor for the financial aspect 
• 
Local clinical committee (“werkgroep triage”): 
o Sander Naeyaert2: chief nurse of the Emergency Department (ED) of the general hospital AZ 
Monica Deurne 
o Arnoud Bonemeyer and Mark Timmermans: emergency physicians at the ED of AZ Monica 
Deurne 
o Edwin Vanbeveren: head of the General Practice Cooperative (GPC) “Antwerpen Oost” 
o Guido Michielsen and Lotte Fivez: Board members of the GPC “Antwerpen Oost” 
1: co-author of this document 
2: receives (indirect) FWO support 
 
 
 

Triage-trial short research protocol 1.1 
 
7 November 2018 
4 
 
 Introduction 
Overview 
The problem: I am feeling ill during the weekend, where should I go? 
When confronted with an unexpected illness during the weekend, patients can go either to the emergency 
department (ED) of a hospital or to the general practitioner (GP) on call. In Flanders, the GPs often organise 
this on call service for a specified region in a central location called General Practice Cooperatives (GPC, 
“Wachtpost” in Dutch). Patients do not know the characteristics of these different out of hours (OOH) 
services and find it hard to estimate the urgency of their own complaints. Therefore, many patients with 
presentations suitable for primary care go directly to the ED. This leads to additional costs for both 
government and patient, long waiting times at the ED and a high workload for the emergency physicians in 
Flanders. 
Aim of this project: we will advise you where to go 
The aim of this project is to deliver the most appropriate care for patients presenting at the ED. To achieve 
this objective a triage nurse will assess all patients presenting at an ED and advise them on the most 
appropriate point of care: GPC or ED. Triage is ‘the sorting out and classification of patients to determine 
priority of need’. Extended triage is triage combined with allocation to either ED or GPC, which is new in 
Flanders.  
Many EDs in Flanders use the Manchester Triage system (MTS) in their regular care for triage within the ED, 
to sort patients according to the urgency of their presentation. We developed an extended MTS (eMTS) which 
foresees allocation to either ED or GPC. Scientific research is crucial to evaluate such triage systems before 
implementation is possible. Our study is a cluster-randomised trial. During one year, we will perform 
extended triage using the eMTS at one ED/GPC collaboration. Every month one weekend will serve as a 
control: the patient does not get an allocation advice. 
Scientific methodology of this project: is this service at this time the most appropriate for me? 
Three aspects determine the most appropriate OOH service for a specific patient at a specific time: 
• 
Medical aspects: is this service capable of handling the medical problem in an efficient and qualitative 
way? Is it safe?  
• 
Financial aspects: is it the most cost effective service for both government and patient?  
• 
Process aspects: can the delivered service satisfy the perceived needs of patients and staff? What are 
the major barriers and facilitators? What can we learn from reported incidents? How can we improve 
safety? This aspect is only briefly described in this protocol, as we will make a separate protocol. 
Study sites and partners 
The TRIAGE-trial is a collaboration between the University of Antwerp, Antwerp University Hospital, clinicians 
at a GPC (Antwerpen Oost) and the ED of a general hospital (AZ Monica Deurne).  

Triage-trial short research protocol 1.1 
 
7 November 2018 
5 
 
eMTS 
The Manchester Triage System (MTS) is one of the 
most commonly used triage systems in Europe(1). It 
enables nurses to assign a clinical priority to 
patients, based on presenting signs and symptoms, 
without making any assumption about the 
underlying diagnosis. The MTS assigns patients to 
one out of five urgency categories, which determine 
the maximum time to first contact with a physician 
(2).  In the TRIAGE-trial, we will use an extended 
MTS (eMTS): we did not change the MTS but added 
some extra questions to allow allocation to ED or 
GPC. Because of legal and practical reasons, we 
have decided to allow allocation to primary care 
only for the two lowest urgency categories. The 
patients in the green category (called standard 
urgency) have to see a doctor within two hours. For 
the blue category (called non-urgent) the maximum 
period is four hours. In Figure 1, we demonstrate 
the eMTS flow chart for shortness of breath in 
children.  
Figure 1: eMTS flow chart for shortness of breath in children. 
The orange arrows and boxes are the extension. 
 
 
 
 
 
 Research Approach 
A diagnostic Intervention 
A nurse will see the patient ideally within ten minutes after arriving at the ED. The nurse will classify a 
patient’s presentation using the MTS, which will enable to make a categorisation of urgency. If the patient’s 
urgency category is either blue or green, the nurse will use the eMTS to assess suitability for primary care. 
This assessment depends on the presentational flow chart used: for some flowcharts, all or none of the 
blue/green presentations will be referred to primary care, for others the nurse has to ask additional 
questions. The result of this extended triage is a shared decision of the patient and the nurse: the patient is 
(or is not) eligible for primary care. Because not all possible questions and signs are incorporated in the eMTS 
the triage nurses will have the possibility to overrule the eMTS advice in both directions. An adapted 
registration system will help the nurse select and follow the correct flowchart and will give an eMTS advice 
once all questions have been assessed. Patients will have the right to refuse the advice of the nurse without 
any further consequences for their treatment at the ED. We will provide all participating nurses and 
physicians with a detailed manual in Dutch about the intervention and the research protocol. Our entire 
eMTS is also available as a separate file. 
 
 

Triage-trial short research protocol 1.1 
 
7 November 2018 
6 
 
Hypotheses and Research questions 
Medical aspect 
Our primary hypothesis: the extended triage intervention will lead to a substantial increase in the proportion 
of patients seen by the GP.  
Without the intervention, all study patients can be divided in three categories: seen by ED, seen by GPC and 
left without being seen (after the triage). By adding the compliance of the patient to the allocation advice we 
get the following table: 
 
Patient goes to GPC 
Patient goes to ED 
Left 
without 
being 
seen 
Advice GPC 
Compliant patient GPC 
Non-compliant 
patient 
GPC 
Left 
without 
being 
seen type 1 
Advice ED 
Non-compliant patient ED 
Compliant patient ED 
Left 
without 
being 
seen type 2 
Table 1: patient categories 
The proportion of study patients seen at the GPC is a combination of patients compliant to a GPC advice and 
patients not compliant to an ED advice. The latter category will probably be small. 
Research question: 
1. What is de the difference in the proportion of study patients seen by the GP between intervention and 
control weekends? 
1.1. Is this proportion associated with the patient’s presentation (chosen eMTS flowchart and urgency)? 
1.2. Is this proportion associated with the occupancy rate of the ED and/or GPC?  
1.3. Is this proportion associated with the operating triage-nurses and/or the operating emergency 
physician? 
1.4. Is this proportion associated with the patient’s background characteristics? (sex, age, community, 
insurance status)? 
1.5. Is there seasonal variation in this proportion? 
Secondary hypothesis: a nurse using the eMTS is able to detect primary care patients. We can assess the 
correctness of the nurse’s allocation advice as a diagnostic instrument. 
 The results of the triage consult is either a GPC or ED advice. Arbitrarily we will call a GPC advice a positive 
diagnosis. The gold standard is the opinion of the physician (either emergency physician or GP) after the 
consultation. This standard is used in almost all of the literature(3), as no better gold standard is available. 
We are aware of some disadvantages: this opinion depends on the personality and other characteristics of 
the treating physician and some patients have presentations suitable for both ED and GPC. On the other 
hand, agreement between nurse and physician is important for the feasibility of extended triage. 
The way we will measure the correctness of the triage advice depends on whether the patient is compliant 
to this advice. This makes Table 1 more complex: 
 
Patient goes to GPC 
Patient goes to ED 
Opinion physician 
GPC correct 
Had to go to ED 
ED correct 
Had to go to GPC 
Advice GPC 
Correct 
decision 1 
False positive 1 
False 
positive 2 
Correct decision 2 
Advice ED 
False 
negative 1 
Correct decision 3 
Correct 
decision 4 
False negative 2 
Table 2: patient categories for the eMTS as a diagnostic instrument 

Triage-trial short research protocol 1.1 
 
7 November 2018 
7 
 
Because our eMTS is risk adverse, we already know we will have a certain amount of false negative 2: patients 
who are compliant to the advice of ED although they had to go to the GPC according to the emergency 
physician. This category is not our primary focus.  
We are mostly interested in false positive 1: patients who complied with the advice GPC but should have 
stayed at the ED. Some of these patients might be subject to a safety risk. For example, a patient with a heart 
attack allocated to the GPC might lose valuable time.  This category is the same as the proportion of study 
patients allocated to the GP but referred back to the ED. We will give this category special attention in our 
process analysis. 
Finally, we are also interested in the category correct decision 2: patients who are not compliant to a correct 
GPC advice. These patients made an inefficient decision so there is room for improvement. 
Research questions: 
2. What is the validity of the eMTS as a preliminary diagnostic instrument to detect a primary care patient? 
2.1. What is the proportion of correct extended triage decisions measured by comparing the triage 
advice with the urcenty evaluation of the physician? 
2.2. What is the proportion of correct extended triage decisions with advice GP measured by the 
proportion of referrals within this category? 
2.3. What is the proportion of patients wrongly assigned to GPC (all false positives)? 
2.4. What is the proportion of patients wrongly assigned to GPC who did go to the GPC (type 1 false 
positives) 
3. What is the proportion of patients who are not compliant to a correct GPC advice (Correct decision 2). 
3.1. What are the determinants for this non-compliance? (same determinants as studied in question 1.1-
1.4) 
4. What is the proportion of patients in which the nurse gives another advice comparing to the eMTS? 
5. Does the intervention influence the number of patients left without being seen? 
6. What is the shift in workload during the different phases of this study for all participating healthcare 
providers (GPs, triage nurses and ED physicians)? 
7. Does the epidemiology in terms of reasons for encounter and diagnoses at both OOH services differ 
between intervention and control weekends? 
8. What is the proportion of patients who return to the ED within two weeks for each of the categories 
described in table 2? 
Financial aspect 
Hypothesis: The extended triage intervention leads to an overall reduction in expenditures for the Health 
Insurance System (HIS) and the patient. The size of this reduction is determined by the presenting complaint 
(eMTS flowchart and urgency category), diagnosis, time of the day and patient characteristics. 
Research questions 
9. What is the difference in average expenditures per patient for the Health Insurance System (HIS) and for 
the patient between intervention and control weekends for ambulatory patients presenting at the ED 
(and seen at the ED or at the GPC)? 
10. What is the difference in average expenditures per patient for the HIS and for the patient between 
intervention and control weekends for different patients groups: 
10.1. 
Patients coded in the two least urgent categories (green and blue) presenting at the ED (and 
seen at the ED or at the GPC?  
10.2. 
Patients coded in the two most urgent categories (red and orange) presenting and treated at 
the ED? 

Triage-trial short research protocol 1.1 
 
7 November 2018 
8 
 
11. For which type of medical treatment expenditures (consultation fees, medical imaging, clinical biology…) 
is there a difference between the control weekends and the intervention weekends (for patients 
described in 9 and 10)? 
12. What are the determinants of treatment costs? 
Design of the clinical trial 
A controlled longitudinal prospective intervention trial 
To answer the above-mentioned research questions we need a longitudinal prospective intervention trial.  
To allow definitive conclusions for our questions we need a control group. Historical controls will have too 
much confounding (not all data is available, the ED was organised in a different way, the GPC was unknown 
for many patients). Individually randomised controls are impossible from a practical point of view and 
because of contamination (patients will talk to each other and will not understand why some get a GPC advice 
and others not). Finally, randomisation by shift is not possible because they overlap in time. Therefore, 
randomisation is only possible by weekend. We will roll out one control weekend out of every four 
intervention weekends.  
During a control weekend, all data registration and collection will be the same as during intervention 
weekends but we will not inform patients about their allocation advice. The emergency physician will see all 
patients deciding to stay at the ED, without influence of the triage advice. As in standard clinical care, patients 
will have the right to change their mind and go spontaneously to the GPC. Another difference is the 
communication to the patient: during intervention weekends, we will inform them about the intervention 
using leaflets and broadcasting in the waiting room of the ED. During control weekends, we will only inform 
about MTS-triage in general but no about the GPC. We will always mention the iCAREdata opting-out 
procedure. 
Inclusion and Exclusion 
We will systematically include all patients presenting at the ED during the weekend (the GPC is only open 
during the weekend starting from Friday 7 pm until Monday 6 am). By presentation at the ED we mean the 
patient is registered by the receptionist of the ED as a patient. Patients who just pass the ED to enter the 
hospital or who go directly to a medical department (e.g. obstetrics) are thus not included. The availability of 
a Belgian citizen national insurance number is the only inclusion criterion. This number is necessary to send 
the data needed for this study to the iCAREdata database. For the vast majority of patients this number is 
available. It will not be available for patients who present for the first time at the hospital and forgot all 
official documents. For patients without a Belgian identity (foreign citizens without Belgian health insurance, 
people without a legal status and babies less than one week old) the number is never available. The EDs own 
software allows us to calculate the number of excluded patients. 
Patients arriving at the ED by an ambulance with a doctor or nurse (in Dutch called MUG or PIT) is the only 
exclusion criterion (these patients have already passed through a sort of triage). We estimate an exclusion 
rate of 15 patients per weekend. 
Sample size 
For this trial we consulted our subcontractor StatUA (core statistical facility of the University of Antwerp). If 
we only look at our primary research question (“What is the difference in the proportion of patients 
presenting at the ED and seen by the GP between intervention and control weekends”), straightforward 
power computations will tell that two weekends (one with, and one without intervention) are sufficient to 
provide empirical evidence of a statistically significant shift. However, as we want to know the determinants 
of this proportion and perform safety monitoring (including incident analysis), we need observations over a 
longer period of time. We do not know the impact size of the determinants that might act as confounders in 
this context. Because the eMTS is risk adverse, safety issues will be rare and a long study period is required 

Triage-trial short research protocol 1.1 
 
7 November 2018 
9 
 
to identify and study possible safety issues.  Finally, we expect nurses will need to build up experience with 
the eMTS before optimal results can be expected. For these reasons, we have chosen to include a large 
sample of patients by studying all weekends during one calendar year.  
Randomisation procedure 
We will assign the control weekends using randomisation software, taking into account possible confounders 
like bank holidays, school holidays, seasonal fluctuations…). We will not be able to blind the participants, the 
staff members nor the researchers. We will inform the staff about the nature of the upcoming weekend on 
Friday morning. Doing so, we aim to prevent anticipation of the staff (e.g. switching shifts or using less staff) 
and thus allocation bias. 
Recruitment strategy and expected dropout 
Unlike most trials, we do not foresee any recruitment problems. In 2016, 8000 patients presented themselves 
during the weekend at the ED.  
For each patient the study period is the time between presentation at the ED and discharge at either the ED 
or the GPC. This is a time span of only a few hours so the dropout rate will be negligible. Only when a patient 
leaves the site before consultation at the ED or GPC he or she is a dropout. This number of patients leaving 
the ED without being seen after triage (i.e. dropouts) is one of the safety indicators we will study. The number 
of these patients is currently less than five per weekend. We are able to measure this number using the triage 
reports through iCAREdata. Patients who leave without being seen before the triage-consultation will not be 
analysed. 
Methodology for the medical aspect 
The data necessary for this work package are part of a routine clinical report. We will use three types of these 
reports: triage-consultation reports, ED consultation reports and GPC consultation reports. We will collect 
these data using the iCAREdata database. This database can link our three types of clinical reports using an 
irreversibly coded national number. Due to data protection rules, the iCAREdata data manager will execute 
all queries and only aggregated results will be transferred to the researchers. For more complex statistical 
analysis, the data manager will transfer an analysis specific table or the analysis will be carried out on the 
computer of the data manager in cooperation between the researcher and the data-manager. A detailed 
data management plan is available. The correctness of a triage decision depends on the patient’s situation: 
• 
At the GPC (patients compliant to a GPC advice or non-compliant to an ED advice): the ratio of 
patients referred to an ED to the total number of patients coming from the ED of AZ Monica. 
• 
At the ED (patients compliant to an ED advice or non-compliant to a GPC advice): the physician will 
answer the same question for all patients: “Should this patient better go to the GPC?”. The 
correctness is defined as the ratio of the patients with a positive answer to this question to the total 
number of patients staying at the ED after triage. 
The parameters of interest in questions 1 through 8 are (conditional) proportions, which calls for analysis by 
a non-linear model (e.g. binary logistic regression). The study design results in the creation of nested data, 
where patients are nested in weekends, and weekends are nested in (multiple) nurses. Appropriate analysis 
of such auto-correlated data requires a nonlinear mixed model. During the first months, data will be used to 
produce point-estimates of these proportions (e.g., patients seen by GP, correct triage decisions, all false 
positives, type 1 false positives, patients left without being seen, etc.). With increasing duration of the study, 
incoming data will be used to explain variability of these probabilities in terms of patient background 
characteristics, operating nurses/emergency physician, season, etc. Statistical assistance will be called for in 
order to conduct these analyses. 
Methodology for the financial aspects  

Triage-trial short research protocol 1.1 
 
7 November 2018 
10 
 
Resources are scarce and therefore it is essential to look at both outcome and resource use when introducing 
new treatments or treatment paths.  This project incorporates both sides.  We will use HIS expenditure data 
(including the share paid by the patient) to get a first insight in the changes in resource use due to the 
extended triage.  Expenditures will be limited to the expenditures of the OOH consultation.  In this way, we 
gain insight in the short-term impact of the new triage system on medical costs from the point of view of the 
patient and the HIS. For this aspect, we only focus on ambulatory patients (not admitted to the hospital). 
Although some inpatients can also present at the ED; admitted patients have far more complicated costs and 
they are not the focus of this project.  
In our research we will make use of a cluster randomized trial to get an unbiased estimate of the difference 
in expenditures due to the extended triage.  Randomization allows to measure the impact of the extended 
triage on average expenditures per patient (research question 9) or specific types of average expenditure per 
patient (research question 11) by comparing these average expenditures per patient between the 
intervention weekends and the control weekends. Information on the expenditures of consultations of the 
GPC are collected in the iCAREdata database. Expenditures generated by the ED are not yet routinely 
collected for research purposes, but billing data are available and these will be used. We will calculate the 
averages and use a t-test to test statistical significance. Medical bills also contain detailed information of the 
nomenclature codes (these are the codes used by the HIS, describing reimbursed care and reimbursement 
tariffs), performed during an ED visit.  We will aggregate these interventions in similar groups such as 
consultation fees, medical imaging, clinical biology and so on.  We will then compare these average 
expenditures per patient for the various aggregates between the intervention weekends and the control 
weekends (research question 10) and use a t-test for statistical significance. 
Currently, the ED’s software package sends all nomenclature codes except those for clinical biology and 
medical imaging to the financial department of the hospital, where the actual bills are made. The financial 
department draws up the bill, taking into account regulations; e.g. sometimes, regulation prohibits to bill two 
nomenclature codes simultaneously; the department then decides which one to charge.  
To gain more insight in differences in expenditures for different patient groups (research questions 9 and 11), 
we need demographic and medical diagnostic information of the patient, which is not available on the 
invoices and is not accessible by the financial department of AZ Monica. We will therefore proceed as follows. 
We will make a table of all included patients with the following columns: time of arrival at the ED, year of 
birth, gender, ZIP-code and a new serial number created by us. We will transfer these data using encrypted 
messages. Because the financial department already has these data to make the invoices this does not raise 
any new privacy issues. The financial department will search for these included patients and deliver us a 
datasheet with all the information of the actual invoices linked to this new serial number but without any 
additional personal data. The nomenclature codes of the interventions performed at the ED can be 
transferred using iCAREdata. 
If the reconstructed bills prove to be valid, research questions 10.1, 10.2. and 12 can be addressed with the 
variables collected in iCAREdata. For questions 10.1 and 10.2, we will calculate expenditures per patient (for 
the HIS and the patient) for urgency categories blue and green separately, and for urgency categories red 
and orange and compare them between the intervention and control weekends, using a t-test to test for 
statistical difference.  
For research question 12 we will perform a multivariate regression analysis, explaining the expenditures 
(dependent variable) with three sets of explanatory variables: general background characteristics of the 
patient (age, sex, insurance status), diagnostic information (urgency code, chosen eMTS flowchart) and 
system characteristics (occupancy rate ED, time of the day, triage nurse, intervention or control weekend). 
The coefficient for the dummy variable indicating an intervention weekend then gives the impact of the 
extended triage. It should confirm the effect found in 8.  The inclusion of control variables, when estimating 
the impact in our regression, however reduces residual standard deviation and thus improves the precision 

Triage-trial short research protocol 1.1 
 
7 November 2018 
11 
 
of the estimate.  The coefficients on the other variables give a further insight in the additional determinants 
of expenditures. The regression analyses will be performed on the computer of the data manager using 
STATA. 
 References 
1. 
Mackway-Jones K, Marsden J, Windle J, Manchester Triage Group. Emergency triage. Third edition. 
ed2014. p. p. 
2. 
Zachariasse JM, Seiger N, Rood PP, Alves CF, Freitas P, Smit FJ, et al. Validity of the Manchester Triage 
System in emergency care: A prospective observational study. PLoS One. 2017;12(2):e0170811. 
3. 
Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: 
a systematic review. Scand J Prim Health Care. 2011;29(4):198-209.