Evaluation of a Clinical Decision-Support App for Emergency Care in a Rural Ugandan Hospital: A Pilot Randomized Crossover Simulation Trial
Phase: NA
Conditions: Diarrhea Infectious, Seizures, Emergency Care, Clinical Decision Support Systems, Rural Health, Dyspnea
Sponsor: University of Milano Bicocca

Evaluation of a Clinical Decision-Support App for Emergency Care in a Rural Ugandan Hospital: A 
Pilot Randomized Crossover Simulation Trial 
 
NCT Number: NCT07257705 
Protocol ID: UNIMIB-972-OASES 
Brief Title: Pilot Trial of a Clinical Decision-Support App for Managing Emergencies Among Clinicians 
in a Rural Ugandan Hospital 
Version: 1.0 
Date: 30/01/2026 
Principal Investigator (PI): Prof. Michele Bombelli, University of Milano-Bicocca, Milan, Italy  
Co-Principal Investigator (Co-PI): Dr. Lucrezia Rovati, University of Milano-Bicocca, Milan, Italy 
Study Site: Dr Ambrosoli Memorial Hospital, Kalongo, Agago District, Uganda 
 
Abstract 
Background: Emergency conditions such as severe diarrhea, dyspnea, and seizures remain major 
causes of morbidity and mortality in low- and middle-income countries. Despite available World 
Health Organization (WHO) and national guidelines, adherence in rural outpatient departments 
(OPDs) is often suboptimal. The Organization for the Advancement and Support of Emergency 
Systems (OASES) App is a clinical decision-support system (CDSS) developed in 2025 by the 
University of Milano-Bicocca as part of a quality improvement collaboration with Dr. Ambrosoli 
Memorial Hospital, Kalongo, Uganda. The App digitizes validated WHO-based triage and emergency 
care algorithms for offline use in resource-limited hospitals, but it has not yet undergone formal 
testing and evaluation by intended end-users. 
Objective: To evaluate the effectiveness, usability, and feasibility of the OASES App among frontline 
clinicians during simulated emergency scenarios. 
Methods: This pilot study will employ a prospective, randomized, stratified crossover simulation 
design at Dr. Ambrosoli Memorial Hospital. Approximately 16 OPD clinicians (nurses, clinical officers, 
nursing assistants) will complete 12 facilitator-led standardized scenarios (diarrhea, dyspnea, 
seizures), randomized between App-assisted and standard practice conditions with a ≥7-day 
washout. Primary outcome is clinicians’ adherence to evidence-based guidelines for diagnostics and 
treatments. Secondary outcomes include triage accuracy, diagnostic accuracy, disposition decision 
appropriateness, process quality, case completion time, and user perceptions of usability, trust, and 
feasibility. Quantitative data will be analyzed using mixed-effects models, while qualitative feedback 
will undergo thematic analysis. 
Expected Outcomes: Findings will provide preliminary evidence on the effectiveness and 
acceptability of the OASES App, inform App refinement, and support the design of a subsequent 
powered clinical evaluation. 
 
 

A. Background and Rationale 
Emergency conditions such as severe diarrhea, dyspnea, and seizures remain leading causes of 
morbidity and mortality in low- and middle-income countries (LMICs)1. Despite the availability of 
World Health Organization (WHO) and national guidelines, the delivery of evidence-based emergency 
care is often suboptimal, particularly in rural and remote areas where limited training, high workload, 
and supply-chain constraints undermine adherence to recommended practices2-4. 
Mobile health (mHealth) innovations, including digital clinical decision-support systems (CDSS) 
delivered via smartphones and tablets, offer a promising means of bridging these implementation 
gaps5 6. However, evidence regarding their effectiveness in rural emergency departments is scarce, 
with most interventions implemented at small scale and rarely subjected to rigorous evaluation7-10. 
Simulation-based studies provide a safe, controlled environment in which to test digital tools, 
allowing systematic assessment of clinical performance, usability, and feasibility before clinical 
deployment11 12. 
The Organization for the Advancement and Support of Emergency Systems (OASES) App is a CDSS 
developed in 2025 by the University of Milano-Bicocca. Its development was undertaken as part of a 
broader quality improvement collaboration between the University of Milano-Bicocca and Dr. 
Ambrosoli Memorial Hospital, aimed at strengthening emergency and acute care services in rural 
Uganda. The App is an intuitive, tablet-based system that digitalizes validated triage and emergency 
care algorithms derived from WHO frameworks. Designed for offline use in hospitals with limited 
infrastructure, it guides frontline clinicians, particularly those with limited training, through systematic 
triage, diagnosis, and treatment steps for common emergency presentations in real time, while 
simultaneously collecting structured clinical data. The current version of the App includes a triage 
algorithm based on the Interagency Integrated Triage Tool (IITT) and three condition-specific modules 
for the management of diarrhea, dyspnea, and seizures in both adults and children13.  To date, the 
OASES App has not been tested in clinical or simulated settings. Prior to implementation in rural 
emergency departments, it therefore requires rigorous evaluation in simulation to determine its 
effectiveness, usability, and feasibility among intended end-users. 
 
B. Study Objectives 
Primary Objective: 
• 
To determine whether the OASES App improves clinicians’ adherence to evidence-based 
guidelines for the management of diarrhea, dyspnea, and seizures during simulated 
emergency scenarios. 
Secondary Objectives: 
• 
To assess the effect of the OASES App on triage accuracy, disposition appropriateness, and 
diagnostic accuracy in simulated emergency scenarios. 
• 
To evaluate whether the OASES App improves clinicians’ adherence to guideline-
recommended history taking and physical examination (process quality). 
• 
To compare time to completion of simulated cases between App-assisted and standard 
practice arms. 
• 
To evaluate usability, perceived usefulness, trust, satisfaction, and feasibility of the App. 
• 
To explore potential learning and carryover effects associated with App use. 
 
C. Methods 
C.1. Study Design 
This study will use a prospective, randomized, stratified crossover simulation design, conducted over 
a two-week period at Dr. Ambrosoli Memorial Hospital, Kalongo. Each participant will complete 
standardized simulation scenarios under two conditions: standard practice (control) and App-
assisted practice (intervention). A washout period of at least seven days will separate the two 
conditions to minimize potential learning and carryover effects. 
 
 

C.2. Setting 
Dr Ambrosoli Memorial Hospital is a private, non-profit general hospital located in Kalongo Town 
Council, Agago District, Northern Uganda. The outpatient department (OPD) is the first point of 
contact for patients seeking care and has a dual role, providing both primary and emergency services. 
Simulations will be conducted in repurposed OPD consultation rooms to maintain ecological validity 
while minimizing disruption to routine clinical activities. 
The study is part of a broader quality improvement collaboration between the University of Milano-
Bicocca and Dr. Ambrosoli Memorial Hospital, which aims to strengthen emergency and acute care 
services in rural Uganda. This collaboration provides an appropriate and relevant context for 
evaluating the OASES App in a setting where it is intended to be implemented. 
 
C.3. Participants 
This study will focus on frontline clinicians responsible for the initial assessment and management of 
patients in the OPD at Dr Ambrosoli Memorial Hospital, where emergency care is primarily delivered 
by clinical officers, nurses, and nursing assistants. 
Inclusion Criteria: 
• 
Nurses, clinical officers, or nursing assistants currently employed in the OPD of Dr Ambrosoli 
Memorial Hospital. 
Exclusion Criteria: 
• 
Individuals who are unable or unwilling to participate in both scheduled simulation sessions. 
 
C.4. Sample Size 
This pilot study will use a feasibility-based sample size, determined by the total number of OPD 
clinicians available at Dr Ambrosoli Memorial Hospital. We anticipate enrolling approximately 16 
participants, representing all nurses, clinical officers, and nursing assistants currently working in the 
OPD of Dr. Ambrosoli Memorial Hospital. Each participant will complete 12 simulated scenarios, 
yielding an estimated 192 scenario-level observations. While not powered to detect small effect sizes, 
this sample is deemed to be sufficient to assess feasibility, refine study procedures, and provide 
preliminary effect estimates to inform the design and power calculations of a subsequent larger trial. 
 
C.5. Recruitment and informed consent 
Recruitment will be conducted among OPD staff at Dr Ambrosoli Memorial Hospital. The study will be 
introduced by the research staff during short informational meetings with staff, and a brief 
announcement will also be shared in an existing WhatsApp group used for communication between 
researchers of the University of Milano-Bicocca and the OPD staff (Appendix A). Interested 
individuals will then be approached privately and provided with detailed study information. Written 
informed consent will be obtained before starting any study procedure. 
 
C.6. Randomization and allocation 
Participants will be randomized to the order of study conditions—App-assisted (intervention) vs. 
standard practice (control)—using computer-generated permuted blocks of size 2, stratified by cadre 
(nurse/nursing assistant vs clinical officer) to ensure balance across professional groups. The 
randomization list will be computer-generated in advance by a member of the research team not 
involved in recruitment or outcome assessment and stored in a password-protected master sheet. 
Participant IDs will be assigned sequentially after written consent in order of recruitment; each ID is 
then linked to a pre-specified condition sequence from the master sheet.  
To control for case content and ordering effects, matched scenario pairs were created for each 
condition (A-set for one period, B-set for the other period), with random assignment of A vs. B to the 
first or second period per participant. Within each period, the case order (1–6) was independently 
randomized. Thus, for every participant the master sheet specifies: (i) intervention sequence, (ii) 
period assignment of A vs. B cases, and (iii) the randomized within-period case order. 
 
 

C.7. Intervention and Control 
Experimental arm (OASES App): Participants will use the OASES tablet-based CDSS, which 
incorporates the Interagency Integrated Triage Tool (IITT) and three condition-specific algorithms 
(diarrhea, dyspnea, and seizures) derived from WHO and Uganda Clinical Guidelines. No formal 
training sessions will be conducted; however, participants will receive a brief orientation walk-through 
of the App immediately prior to its use. 
Control arm (standard practice): Participants will manage simulated cases without the App, relying on 
their usual knowledge and clinical experience. Standard resources available in OPD, including paper-
based versions of the IITT and Uganda Clinical Guidelines, will be available for reference. 
All participants will complete both arms using a crossover design, with a washout period of at least 
one week between conditions. During the washout, participants will return to their normal OPD duties 
without access to the OASES App. No additional clinical teaching or training will be provided during 
the study period. 
 
C.8. Simulation and Scenarios 
Each participant will complete 12 facilitated clinical vignette scenarios, with six scenarios per study 
arm (two diarrhea, two dyspnea, and two seizures). Scenarios will be adapted from WHO training 
materials for emergency care (e.g., Basic Emergency Care course, Emergency Triage Assessment and 
Treatment guidelines) and tailored to reflect common emergency presentations in the OPD of Dr 
Ambrosoli Memorial Hospital. Both adult and pediatric cases will be included, varying in severity and 
presentation to capture the spectrum of conditions typically encountered. Draft cases will undergo 
expert review and pilot testing with representative clinicians to ensure clarity, realism, and contextual 
relevance. Cases will be designed in matched pairs to ensure comparable content and difficulty 
between study periods, with equivalence determined by expert review and pilot testing. Scenarios will 
be balanced for complexity, number of critical actions, and expected completion time. 
Cases will be delivered verbally by a trained facilitator, who will respond only to participant questions 
to simulate real-world information gathering. Facilitators will log which key history and examination 
items are elicited using a structured tick-box script. 
At two predefined points in each vignette, participants will record their diagnostic and treatment plan: 
1. Initial assessment stage – working impression, immediate treatments, and diagnostic tests 
ordered. 
2. Post-investigation stage – final diagnosis, disposition decision, treatment plan and follow-up 
plan. 
Each participant’s simulation session will last approximately one hour. All sessions will be scheduled 
during off-duty hours to ensure voluntary participation and to avoid disruption of patient care. 
 
C.9. Outcomes 
Primary outcome: 
• 
Guideline-adherent management: Proportion of pre-specified, guideline-consistent critical 
management actions (diagnostics and treatments) completed per vignette (proportion 0-1). 
Participant responses recorded at the initial assessment and post-investigation stages will be 
scored against a gold-standard answer key derived from WHO guidelines (Basic Emergency 
Care assessment checklists, ETAT performance checklists, Integrated Management of 
Childhood Illness checklists). Scoring will be performed by two independent assessors, 
blinded to allocations, mapping free text to binary items (performed correctly vs. not 
performed/incorrect). Disagreements will be resolved by consensus. 
Secondary Outcomes: 
• 
Triage accuracy: Correct assignment to Interagency Integrated Triage Tool (IITT) category 
(Red/Emergency, Yellow/Urgent, Green/Non-urgent) compared with the case-specific 
expected category (binary). Scoring will be performed by two independent assessors, blinded 
to allocations. 
• 
Disposition appropriateness: Appropriateness of the disposition decision (hospitalize, 
discharge) compared with case-specific expected disposition in the answer key, based on 

WHO/Uganda guidelines (binary). Scoring will be performed by two independent assessors, 
blinded to allocations. 
• 
Diagnostic accuracy: Correctness of working impression (initial stage) and final diagnosis 
(post-investigation stage), compared with case-specific diagnostic labels in the answer key, 
based on WHO/Uganda guidelines (binary). Scoring will be performed by two independent 
assessors, blinded to allocations, mapping free text to binary items (performed correctly vs. 
not performed/incorrect). 
• 
Guideline-adherent evaluation (process quality): Proportion of critical history and examination 
items elicited by the participant during the vignette, compared with case-specific list of critical 
items derived from WHO guidelines (proportion 0-1). Scoring will be performed by two 
facilitators during each simulation session using the facilitator tick-box logs. 
• 
Time to completion: Duration needed to complete each vignette, measured by facilitators 
(continuous). 
• 
Usability: Total score on System Usability Scale (SUS), a widely validated 10-item tool for 
evaluating software systems in healthcare and other fields (continuous)14. 
• 
Perceived usefulness, trust, satisfaction, and feasibility: assessed via structured Likert-scale 
items, adapted from the Technology Acceptance Model (TAM) and related implementation 
frameworks (categorical), and via brief semi-structured interviews15. 
 
C.10. Data Collection and Management 
Data will be collected electronically using Kobo Toolbox, an open-source, web-based platform 
developed by the Harvard Humanitarian Initiative for data collection in resource-limited and field 
settings (https://www.kobotoolbox.org/). 
• 
Baseline data: Prior to randomization, participants will complete a baseline survey capturing 
demographic and professional characteristics, prior emergency training, and digital literacy 
(Appendix B). 
• 
Simulation data: During each vignette, two facilitators (emergency medicine clinicians from 
the study team) will deliver cases, log process measures using structured tick-box forms 
(Appendix C), and capture start/end times. Any discrepancies will be reconciled immediately 
after the session. Participants will record clinical decisions directly into Kobo Toolbox forms at 
predefined stages (Appendix D). 
• 
Outcome scoring: Responses will be exported in de-identified forms and scored by two 
independent assessors, blinded to allocation, using pre-defined gold-standard answer keys 
(Appendix E). The two assessors will be two clinicians not involved in App or case 
development. Discrepancies will first be discussed by the two reviewers to reach consensus; 
if disagreement persists, a third senior adjudicator (not involved in App or case development) 
will provide the final decision. 
• 
Post-simulation survey and interview: After completing sessions that involve App use, 
participants will complete a structured electronic questionnaire assessing usability (SUS), 
perceived usefulness, trust, satisfaction, and feasibility (Appendix F). To complement these 
quantitative measures, the two facilitators will also conduct brief semi-structured interviews 
with each participant immediately after their simulation session (Appendix G). Interviews will 
not be audio-recorded; instead, facilitators will take contemporaneous field notes, which will 
be expanded into detailed summaries for subsequent qualitative analysis. 
 
C.11. Statistical Analysis  
All analyses will follow the intention-to-treat principle. For vignette-level outcomes, if a participant 
does not complete an entire scenario, that scenario will be excluded from analysis. Within completed 
scenarios, missing or blank responses will be treated as incorrect (i.e., non-adherent), on the 
assumption that an unanswered item reflects lack of knowledge or failure to act. For survey 
outcomes, only available responses will be analyzed, with no imputation. Descriptive statistics will 
therefore be based on the number of participants who provided a response to each item. The number 
of missing scenarios and survey responses will be reported. 

The primary analysis will follow a crossover design, with each participant contributing observations 
under both study conditions (App-assisted vs. control) across distinct but content-equivalent case 
scenarios: 
• 
Guideline-adherent management: Analyzed as a bounded proportion (0-1) using a generalized 
linear mixed-effects model with binomial distribution and logit link. The model will include 
fixed effects for study arm (App vs. control), period (first vs. second session), sequence 
(allocation order), scenario type (diarrhea, dyspnea, seizure), cadre (clinical officers vs 
nurses), years of experience (continuous), baseline comfort in using mobile Apps 
(dichotomous). Random intercepts will be specified for participant to account for repeated 
measures. Results will be reported as adjusted mean differences with 95% CIs. Inter-rater 
reliability will be assessed using Cohen’s κ for individual items and intraclass correlation 
coefficients (ICC) for total checklist scores. 
• 
Triage appropriateness, disposition appropriateness, and diagnostic accuracy: Each will be 
analyzed as binary outcomes (correct vs. incorrect) using mixed-effects logistic regression, 
with the same fixed and random effects structure as above. Results will be reported as 
adjusted odds ratios with 95% CIs. Inter-rater agreement will be evaluated using Cohen’s κ. 
• 
Guideline-adherent evaluation (proportion score): Analyzed as a bounded proportion (0-1) 
using a generalized linear mixed-effects model with binomial distribution and logit link, with 
the same fixed and random effects as above. Results will be reported as adjusted mean 
differences with 95% CIs. Inter-rater reliability will be assessed using Cohen’s κ for individual 
items and intraclass correlation coefficients (ICC) for total checklist scores. 
• 
Time to case completion (minutes): Analyzed as a continuous outcome using a linear mixed-
effects model. Results will be reported as adjusted mean differences with 95% CIs. 
As a sensitivity analysis, a first-exposure parallel analysis will be conducted, in which only the first 
period for each participant is analyzed. This eliminates potential carryover or learning effects, at the 
cost of reduced sample size. Findings from the crossover and first-exposure analyses will be 
compared to assess robustness. Multiplicity across secondary outcomes will be managed by treating 
the crossover analysis of the primary outcome as the only confirmatory analysis, with all secondary 
outcomes considered exploratory. 
Survey outcomes, including SUS scores and additional Likert-scale items (perceived usefulness, 
trust, satisfaction, feasibility), will be summarized descriptively and reported as medians with 
interquartile ranges or proportions, as appropriate. 
Qualitative data will be analyzed by two researchers using inductive thematic analysis to identify key 
themes related to usability and feasibility, and to guide further App refinement. 
 
D. Ethical Considerations 
Risks and benefits: The study poses minimal risk to participants. All simulation scenarios will involve 
fictional, non-identifiable patient cases and will be conducted in a controlled, educational 
environment. No real patients will be involved at any stage. The main potential risk is discomfort or 
anxiety during performance assessment; however, participants will be reassured that the study is for 
research purposes only and not a test of their professional competence, and that their individual 
results will not be shared with supervisors or used in performance evaluations. Participants may gain 
educational value from practicing emergency scenarios in a structured environment and becoming 
familiar with digital decision-support tools. The study may also benefit the hospital by informing the 
refinement of a context-appropriate tool to strengthen emergency and acute care. At a broader level, 
the findings will contribute to evidence on the feasibility and effectiveness of digital clinical decision-
support systems in rural, resource-limited settings. 
Voluntariness, withdrawal, consent: Participation is strictly voluntary. Participants may refuse to take 
part or withdraw from the study at any time without penalty or consequence to their employment or 
professional standing. Written informed consent will be obtained prior to enrollment, using a 
standardized consent form. 

Data protection and confidentiality: All data will be de-identified at entry, with no names or personal 
identifiers collected, and only the University of Milano-Bicocca research team will have access to the 
raw de-identified data. Hospital management will receive only aggregated results. No identifiable 
information will be linked to individual performance results. Individual results will not affect 
employment status, performance evaluation, or professional progression. Data will be stored 
securely on EU-hosted Kobo Toolbox servers and password-protected Bicocca servers. Data will be 
retained for 5 years, after which it will be permanently deleted. 
Compensation: To minimize disruption of clinical services, simulations will be conducted outside of 
OPD duty hours. Each participant will receive a modest token of appreciation of 20,000 UGX 
(corresponding to €5) and a small refreshment upon completion of their sessions. This amount is 
intended to offset inconvenience and time but does not constitute undue inducement. 
Community engagement and dissemination of results: This study is part of a broader collaboration 
between the University of Milano-Bicocca and Dr. Ambrosoli Memorial Hospital, within a quality 
improvement program aimed to strengthen emergency services at Dr. Ambrosoli Memorial Hospital. 
The OASES App is designed to support, rather than replace, frontline clinicians by guiding them 
through evidence-based triage and management steps and collecting structured data. Stakeholder 
engagement has been central to the study design and conduction of the study has been approved by 
the Dr. Ambrosoli Memorial Hospital Medical Director. Hospital leadership and OPD staff were 
consulted on protocol feasibility, workflow fit, and alignment with ongoing initiatives. A key issue 
identified during early discussions was staff concern that digital tools might threaten job security. To 
mitigate this, we will engage staff before enrollment through informational meetings and informal 
discussions, emphasizing that the App is a supportive tool intended to strengthen, not replace, 
clinical roles. During and after the study, we will maintain transparency with OPD staff and hospital 
leadership, providing regular updates and holding dissemination meetings to share findings. Results 
will be shared with hospital staff, hospital leadership, and published in peer-reviewed journals. 
Findings will be used to refine the OASES App before clinical deployment. 
Ethical and Regulatory Approvals: The study protocol, participant information/consent materials, and 
data-protection plan have received ethical approval from the Gulu University Research Ethics 
Committee (GUREC) and the University of Milano-Bicocca Research Ethics Committee, and 
regulatory clearance from the Uganda National Council for Science and Technology (UNCST). In 
addition, Dr. Ambrosoli Memorial Hospital (DAMHK) management has approved the conduct of the 
study on site. All procedures will adhere to the Declaration of Helsinki and applicable Good Clinical 
Practice principles, as well as the EU GDPR and the Uganda Data Protection and Privacy Act (2019). 
 
E. Bibliography 
1. Wimmer S, Sarabu S, Calvello Hynes E, et al. Global burden of emergency and operative conditions: 
an analysis of Global Burden of Disease data, 2011-2019. Bull World Health Organ 
2025;103(3):194-203. doi: 10.2471/blt.24.292412 [published Online First: 2025/03/03] 
2. Abtew S, Negatou M, Wondie T, et al. Poor Adherence to the Integrated Community Case 
Management of Newborn and Child Illness Protocol in Rural Ethiopia. Am J Trop Med Hyg 
2022;107(6):1337-44. doi: 10.4269/ajtmh.21-1018 [published Online First: 2022/11/01] 
3. Irimu GW, Gathara D, Zurovac D, et al. Performance of health workers in the management of 
seriously sick children at a Kenyan tertiary hospital: before and after a training intervention. 
PLoS One 2012;7(7):e39964. doi: 10.1371/journal.pone.0039964 [published Online First: 
2012/08/04] 
4. Mohanan M, Vera-Hernández M, Das V, et al. The know-do gap in quality of health care for 
childhood diarrhea and pneumonia in rural India. JAMA Pediatr 2015;169(4):349-57. doi: 
10.1001/jamapediatrics.2014.3445 [published Online First: 2015/02/17] 
5. Craig A, Lawford H, Miller M, et al. Use of Technology to Support Health Care Providers Delivering 
Care in Low- and Lower-Middle-Income Countries: Systematic Umbrella Review. J Med 
Internet Res 2025;27:e66288. doi: 10.2196/66288 [published Online First: 2025/06/19] 

6. Agarwal S, Glenton C, Tamrat T, et al. Decision-support tools via mobile devices to improve quality 
of care in primary healthcare settings. Cochrane Database Syst Rev 2021;7(7):Cd012944. doi: 
10.1002/14651858.CD012944.pub2 [published Online First: 2021/07/28] 
7. Bilal S, Nelson E, Meisner L, et al. Evaluation of Standard and Mobile Health-Supported Clinical 
Diagnostic Tools for Assessing Dehydration in Patients with Diarrhea in Rural Bangladesh. Am 
J Trop Med Hyg 2018;99(1):171-79. doi: 10.4269/ajtmh.17-0648 [published Online First: 
2018/05/16] 
8. Biswas D, Asadullah A, Khan SH, et al. Qualitative Assessment of a Smartphone-Based Mobile 
Health Tool to Guide Diarrhea Management in Bangladesh. Am J Trop Med Hyg 
2024;110(1):159-64. doi: 10.4269/ajtmh.23-0444 [published Online First: 2023/12/11] 
9. Haque F, Ball RL, Khatun S, et al. Evaluation of a Smartphone Decision-Support Tool for Diarrheal 
Disease Management in a Resource-Limited Setting. PLoS Negl Trop Dis 2017;11(1):e0005290. 
doi: 10.1371/journal.pntd.0005290 [published Online First: 2017/01/20] 
10. Keita AM, Brintz BJ, Khan AI, et al. Introduction of an Electronic Clinical Decision Support Tool to 
Inform Prescribing for Pediatric Diarrhea in Bangladesh and Mali: Do Provider Expectations 
Predict Experiences? Am J Trop Med Hyg 2022;107(1):32-34. doi: 10.4269/ajtmh.21-1248 
[published Online First: 2022/07/28] 
11. Lamy JB, Falcoff H, Dubois S, et al. Simulation Trials for Evaluating Clinical Decision Support 
Systems. Stud Health Technol Inform 2025;327:213-14. doi: 10.3233/shti250304 [published 
Online First: 2025/05/17] 
12. Lawson J, Martin G, Guha P, et al. Effect of Mixed Reality on Delivery of Emergency Medical Care in 
a Simulated Environment: A Pilot Randomized Crossover Trial. JAMA Netw Open 
2023;6(8):e2330338. doi: 10.1001/jamanetworkopen.2023.30338 [published Online First: 
2023/08/28] 
13. World Health Organization (WHO). Interagency Integrated Triage Tool (IITT). 2020. 
https://www.who.int/publications/m/item/IITT  [ 
14. Hyzy M, Bond R, Mulvenna M, et al. System Usability Scale Benchmarking for Digital Health Apps: 
Meta-analysis. JMIR Mhealth Uhealth 2022;10(8):e37290. doi: 10.2196/37290 [published 
Online First: 2022/08/19] 
15. Adnan A, Irvine RE, Williams A, et al. Improving Acceptability of mHealth Apps-The Use of the 
Technology Acceptance Model to Assess the Acceptability of mHealth Apps: Systematic 
Review. J Med Internet Res 2025;27:e66432. doi: 10.2196/66432 [published Online First: 
2025/05/07] 
 
 
 

Appendix A. Draft WhatsApp Recruitment Message for OPD Staff 
 
Dear OPD colleagues, 
We are inviting OPD staff at Dr Ambrosoli Memorial Hospital to take part in a research study 
developed by the University of Milano-Bicocca to evaluate a new emergency care App (OASES App). 
The study involves managing simulated patient cases in a safe training environment to understand 
how the App can support emergency decision-making. 
Participation is completely voluntary and will have no effect on your job, supervision, or evaluation. If 
you decide to participate, you will receive a modest token of appreciation (20,000 UGX) and a small 
refreshment (soda and snack) after completing your sessions. If you are interested in learning more, 
please attend a short information session on February 4, 2025.  
Thank you for considering supporting this research to strengthen emergency care at Dr Ambrosoli 
Memorial Hospital. 
 

Appendix B. Baseline Survey 
 
Participant information 
This survey collects background information to help us understand participants’ experience and 
context. Your answers are confidential and will only be used for research purpose. Please select the 
response that best applies to you. 
 
Participant ID: (assigned after randomization) 
 
1. Cadre: 
o Clinical Officer 
o Nurse 
o Nursing Assistant 
 
2. Highest professional qualification: 
o Bachelor’s Degree 
o Diploma 
o Ordinary Certificate or Uganda Advanced Certificate of Education (UACE) 
o Uganda Certificate of Education (UCE) 
o Primary School Leaving Certificate 
 
3. Years of clinical experience: (numeric entry) 
 
4. Years worked in Kalongo OPD: (numeric entry) 
 
5. Trainings completed in past 24 months: (select all that apply) 
 None 
 Emergency Triage, Assessment and Treatment (ETAT) 
 Basic Life Support (BLS) 
 Advanced Cardiac Life Support (ACLS) 
 Advanced Trauma Life Support (ATLS) 
 Basic Emergency Care (BEC) 
 Integrated Management of Childhood Illness (IMCI) 
 Other (specify) 
 
6. How often do you use national or WHO guidelines in your OPD work? 
o Never 
o Rarely 
o Sometimes 
o Often 
o Always 
 
7. How often do you personally manage patients with the following conditions in the OPD? 
- 
Severe diarrhea 
- 
Dyspnea 
- 
Seizures 
à Response options for each condition: 
o 
Never  
o 
Rarely (less than once per month) 
o 
Occasionally (about once per month) 
o 
Regularly (about once per week) 
o 
Frequently (several times per week) 
 

8. Do you have regular access to an Android smartphone or tablet? 
o Yes 
o No 
 
9. How comfortable are you using mobile apps on a smartphone or tablet? 
o Not at all comfortable 
o Slightly comfortable 
o Somewhat comfortable 
o Moderately comfortable 
o Extremely comfortable 
 
10. Have you ever used a mobile App for clinical decision support? 
o Never 
o Rarely 
o Sometimes 
o Often 
o Always 
 
11. If yes, which ones? List the names here. (free text entry) 
 
11. Have you ever participated in a simulation-based clinical training? 
o Never 
o Rarely 
o Sometimes 
o Often 
o Always 
 
 

Appendix C. Case Script and Facilitator Tick-Box Checklist (Sample: Diarrhea – Pediatric) 
 
Facilitator instructions: 
• 
Provide information only when the participant asks. 
• 
Do not volunteer information unless explicitly prompted. 
• 
Tick each item when it is asked about. 
• 
Critical WHO-required actions are underscored. 
 
Participant instructions (to be read aloud by the facilitator) 
Thank you for joining this simulation session. You will be presented with a patient case, just as if you 
were in the OPD. Please ask me any questions you think are important for history, examination or 
investigations—I will only give you the information you request. At two points, I’ll pause and ask you to 
record your impression, tests, treatments, and final decisions on the tablet form. Each case will take 
about 10 minutes, and you will complete 12 cases in total across two sessions.  
When you are using the OASES App, it will guide you step by step through case evaluation and suggest 
possible treatments and diagnostic tests. However, the App could be wrong or incomplete, so you are 
free to follow its recommendations or decide differently if you think another approach is better for the 
patient. 
There are no right or wrong answers. Your responses are confidential, and you may withdraw any time. 
This is not an exam, but part of a research study to evaluate the OASES App. 
 
Case Scenario (Diarrhea – Pediatric) 
Case ID: A001 
Patient: 2-year-old child, male 
Location: OPD consultation room 
Presenting complaint: Mother brings her child with diarrhea. 
 
History of present illness: 
 Diarrhea duration: 2 days 
 Number of stools: 6–7 stools/day 
 Presence of blood in the stool: watery stools, no blood 
 Oral intake: decreased since yesterday, but still drinks/breastfeeds 
 Associated symptoms: moderate vomiting (3–4 times/day), no convulsions, no fever or other 
symptoms 
 
Past medical history and social history: 
 
Previous illnesses: no chronic illness or known HIV exposure 
 
Prior surgeries: none 
 
Prior transfusions: none 
 
Prior hospitalizations: none 
 
Immunizations: up to date 
 
Allergies: none known 
 
Medications: none, no treatment at home 
 
Feeding history: still breastfeeding occasionally, eats family foods, no recent new foods or 
known exposure to contaminated food/water 
 
Social history: lives with both parents and two siblings, borehole water sources, no other sick 
contacts at home 
 
Physical examination: 
 
Vital signs 
 Temperature: 37.2 °C 
 Heart rate: 128/min 

 Respiratory rate: 28/min 
 Oxygen saturation: 97% (room air) 
 
Danger signs: alert but irritable, no convulsions, no shock signs (capillary refill: <2 seconds, 
warm extremities) 
 
Hydration status: sunken eyes, mucous membranes dry, skin pinch goes back slowly, drinks 
eagerly when offered fluids 
 
Nutrition status: No visible wasting, oedema, or other signs of malnutrition, MUAC 13 cm, 
weight 12 Kg, Length/height: 86 cm 
 
Abdomen: Soft, no distension, no tenderness 
 
Heart/lung/other systems: normal 
 
Patient condition at reassessment: The patient has improved after 4hr of observation and hydration 
status is better after completing ORS administration. 
 
Test results (only if requested): 
 RBS: normal 
 CBC: normal 
 Malaria (MRDT and B/S): negative 
 Stool culture: normal 
 Abdomen x-ray: negative 
 Chest x-ray: negative 
 Other labs/diagnostics if asked: normal 
 
Facilitator Summary (to be entered in Kobo Toolbox at the end of the simulation session): 
• 
Participant ID: (assigned after randomization) 
• 
Case ID: 
• 
Case start time:  
• 
Case end time: 
• 
Guideline-adherent evaluation score (n° of ticked underscored items): _____/8 
• 
Notes/comments: 
 
 

Appendix D. Participant Response Form 
 
• 
Participant ID: (assigned after randomization) 
• 
Case ID: 
 
Stage 1 – Initial Assessment 
• 
Triage Category (according to IITT criteria): 
o Red (Emergency) 
o Yellow (Urgent) 
o Green (non-urgent) 
• 
What is your working impression based on this initial evaluation? (free text) 
• 
Which treatments would you start immediately? (free text) 
• 
Which diagnostic tests would you request at this stage? (free text) 
 
Stage 2 – Post-Investigation Assessment 
• 
What is your final diagnosis after results? (free text) 
• 
Disposition decision 
o Hospitalize 
o Discharge 
• 
What is your treatment and follow-up plan for the patient? (free text) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Appendix E. Scoring Checklist for External Assessor with Gold Standard 
 
External Assessor Instruction: 
Task: evaluate participants’ responses in Kobo Toolbox against the gold-standard checklist for each 
case scenario. Score according to WHO and Uganda Clinical Guidelines. 
How to score: 
• 
Flag each item where the participant’s response matches or is clinically equivalent to the gold 
standard. 
• 
Leave blank if the response is absent, incorrect, or inappropriate. 
• 
Do not infer beyond what is explicitly written by the participant. 
Domains: 
• 
Diagnostics: Correct if all indicated tests are ordered and unnecessary tests are avoided. 
• 
Treatments: Correct if the participant specifies the correct plan, dose/volume, 
delivery/reassessment, and avoids harmful or unnecessary therapies. 
• 
Diagnosis, triage, and disposition: Correct if the participant’s answer matches the gold-
standard label/category for the scenario. 
 
Assessor sheet format (same for all cases): 
• 
Participant ID: (assigned after randomization) 
• 
Case ID: 
• 
Correct guideline-adherent management items 
 Triage Category 
 Immediate Treatments 
 Investigations 
 Diagnosis 
 Disposition 
 Treatment and follow-up plan 
 
Example Case (A001: Pediatric Diarrhea) 
Patient: 12 years, 12 kg, MUAC 13 cm, watery diarrhea, vomiting, some dehydration, no danger signs 
Gold Standards: 
• 
Triage Category: Yellow (Urgent; some dehydration, no danger signs) 
• 
Immediate Treatments: Only Plan B rehydration: ORS ≈75 mL/kg (≈900 mL) over 4 hours, small 
frequent sips, reassess at 4 h; continue breastfeeding 
• 
Investigations: None (child alert, afebrile, no severe dehydration, no red flags), malaria test 
(MRDT or B/S) acceptable in Kalongo due to local epidemiology 
• 
Diagnosis: Diarrhea with some dehydration (acceptable: “acute watery diarrhea,” “AWD,” 
“viral gastroenteritis with some dehydration”) 
• 
Disposition: Discharge 
• 
Treatment Plan: No antibiotics (no blood in stool, no cholera outbreak); supportive therapy 
such as paracetamol if febrile acceptable 
• 
Follow-up Plan: since patient has improved after 4 hr observation, discharge with plan A = 
extra fluids, continue feeding/breastfeeding, return precautions (worsening dehydration, 
blood in stool, persistent vomiting, convulsions, lethargy, inability to drink); optional zinc 
supplements (not available in Kalongo) 
 

Appendix F. Post-Simulation OASES App Evaluation Questionnaire 
 
Instructions 
Please answer the following questions about your experience using the OASES App during today’s 
simulation exercises. There are no right or wrong answers. Your responses are confidential and will 
not be shared with supervisors. All responses will be recorded in a de-identified form, and only de-
identified data will be accessed by the research team for analysis. 
 
Participant ID: (assigned after randomization) 
 
Response options (for all Likert items): 
1 = Strongly disagree · 2 = Disagree · 3 = Neither agree nor disagree · 4 = Agree · 5 = Strongly agree 
 
Usability (System Usability Scale) 
1. I think that I would like to use this app frequently. 
2. I found the app unnecessarily complex. 
3. I thought the app was easy to use. 
4. I think that I would need the support of a technical person to be able to use this app.  
5. I found the various functions in this app were well integrated. 
6. I thought there was too much inconsistency in this app. 
7. I would imagine that most people would learn to use this app very quickly. 
8. I found the app very cumbersome to use. 
9. I felt very confident using the app. 
10. I needed to learn a lot of things before I could get going with this app. 
Perceived Usefulness 
11.  The OASES app helped me adhere to clinical guidelines. 
12.  Using the OASES app helped me make better clinical decisions. 
13. The OASES app would improve the quality of patient care in our OPD. 
Trust/Reliability 
14.  The app provides accurate and reliable information. 
15. I would trust the recommendations provided by the OASES app for patient management in the 
OPD. 
Behavioral Intention and Satisfaction 
16. Overall, I was satisfied with using the OASES app during the simulations. 
17. I would recommend the OASES app to colleagues. 
18.  I intend to use the OASES app in clinical work if it becomes available. 
Feasibility/Burden 
19. The OASES app would be feasible to use during routine patient care in the OPD. 
20. Routinely using the OASES app would add too much effort to patient care in the OPD. 
 
 

Appendix G. Qualitative Debrief Guide 
 
General experience 
1. How was your overall experience using the OASES App in the simulations? Did you find it 
helpful for making clinical decisions or following guidelines? 
Barriers and enablers 
2. What challenges or difficulties might prevent you from using the App in routine care? What 
factors would make it easier for you to use it? 
Suggestions for improvement 
3. What changes or improvements would make the App more useful or easier to use for you and 
your colleagues? 
Priorities for development 
4. Which types of algorithms, conditions, or functions do you think should be prioritized as the 
App is developed further?