Defining, Recognising and Escalating Maternal Early Deterioration (DREaMED): Decreasing Inequality Through Improved Outcomes.
Conditions: Pregnancy Complications, Pregnancy, High Risk
Sponsor: University of Oxford

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 1 of 46 
Study Title:  Defining, Recognising and Escalating Maternal Early Deterioration (DREaMED): Decreasing 
inequality through improved outcomes. 
Internal Reference Number / Short title: DREaMED 
Ethics Ref: X 
CAG Ref: 24/CAG/0060  
IRAS Project ID: 337633 
Joint Research Office Ref: PID17763 
ClinicalTrials.gov: X  
NIHR Project: NIHR131227 
Date and Version No: 3rd June 2024 Version 2.0  
 
Chief Investigator: 
Professor Peter Watkinson, Professor of Intensive Care Medicine, 
University of Oxford co-lead investigator 
Lead Investigator: 
Professor Marian Knight, Professor of Maternal and Child Population 
Health, University of Oxford co-lead investigator  
Investigators:  
Dr Tony Kelly, National Clinical Advisor for Maternal and Neonatal 
Safety Improvement Programme, NHS England and Improvement 
Dr Anita Banerjee, Honorary Reader in Obstetric Medicine, Guy's and St 
Thomas' NHS Foundation Trust 
Professor Jane Sandall, Professor of Social Science & Women’s Health, 
King’s College London 
Professor Gary Collins, Professor of Medical Statistics, University of 
Oxford 
Professor David Clifton, Professor of Clinical Machine Learning, 
University of Oxford 
Dr Stephen Gerry, Senior Medical Statistician, University of Oxford 
Dr Oliver Redfern, Clinical Research Fellow, University of Oxford 
Dr Oliver Rivero-Arias, Associate Professor of Health Economics, 
University of Oxford 
PPI Leads: 
Susanna Stanford, PPI representative 
Zenab Barry, PPI representative 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 2 of 46 
 
Sponsor:  
University of Oxford 
Funder: 
National Institute for Health and Care Research (NIHR) 
Chief Investigator Signature:   
 
Professor Peter Watkinson 
Co-Lead Investigator Signature:  
 
Professor Marian Knight 
Conflicts of interest: 
PW declares significant grants from Wellcome and Sensyne Health (now Arcturis Data). He undertakes 
consultancy for Arcturis Data.  
 
Confidentiality Statement 
This document contains confidential information that must not be disclosed to anyone other than the 
Sponsor, the Investigator Team, HRA, host organisation, and members of the Research Ethics Committee, 
unless authorised to do so. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 3 of 46 
TABLE OF CONTENTS 
 
1 
KEY CONTACTS .................................................................................................................................. 7 
2 
LAY SUMMARY .................................................................................................................................. 8 
3 
SYNOPSIS .......................................................................................................................................... 9 
4 
ABBREVIATIONS .............................................................................................................................. 11 
5 
BACKGROUND AND RATIONALE ..................................................................................................... 12 
5.1 
What is the problem being addressed? .................................................................................. 12 
5.2 
Why is this research important? ............................................................................................. 13 
6 
OBJECTIVES AND OUTCOME MEASURES ........................................................................................ 15 
7 
STUDY DESIGN ................................................................................................................................ 16 
7.1 
Overview ................................................................................................................................. 16 
7.2 
Workstream 1 ......................................................................................................................... 17 
7.2.1 
Objectives ........................................................................................................................... 18 
7.2.2 
Permissions and data flow .................................................................................................. 18 
7.3 
Workstream 2 ......................................................................................................................... 19 
7.3.1 
Identification of existing MOEWS ....................................................................................... 19 
7.3.2 
Workstream 2a ................................................................................................................... 19 
7.3.3 
Workstream 2b ................................................................................................................... 20 
7.4 
Workstream 3 ......................................................................................................................... 20 
7.4.1 
Workstream 3a – developing the candidate variable list ................................................... 20 
7.4.2 
Workstream 3b – developing an optimal vital-signs MOEWS and multivariable eMOEWS 21 
7.5 
Workstream 4 ......................................................................................................................... 21 
7.5.1 
Workstream 4a ................................................................................................................... 21 
7.5.2 
Workstream 4b ................................................................................................................... 22 
7.5.3 
Workstream 4c ................................................................................................................... 23 
8 
PARTICIPANT IDENTIFICATION ........................................................................................................ 24 
8.1 
Study participants ................................................................................................................... 24 
8.1.1 
Patient Retrospective cohort .............................................................................................. 24 
8.1.2 
Patient prospective cohort ................................................................................................. 24 
8.1.3 
MBRRACE-UK cohort ........................................................................................................... 25 
8.1.4 
Patient interviews (escalation pathway) ............................................................................. 25 
8.1.5 
Staff interviews/focus groups (escalation pathway) ........................................................... 25 
8.1.6 
Staff interviews/focus groups (eMOEWS interface development) ..................................... 25 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 4 of 46 
8.1.7 
Staff training (simulation scenarios) ................................................................................... 25 
9 
PROTOCOL PROCEDURES ................................................................................................................ 26 
9.1 
Recruitment ............................................................................................................................ 26 
9.1.1 
Patients ............................................................................................................................... 26 
9.1.2 
Staff and public participants ............................................................................................... 26 
9.2 
Screening and Eligibility Assessment ...................................................................................... 26 
9.2.1 
Patients ............................................................................................................................... 26 
9.2.2 
Staff and public participants ............................................................................................... 26 
9.3 
Informed Consent ................................................................................................................... 26 
9.3.1 
Patients/Public participants ................................................................................................ 26 
9.3.2 
Staff ..................................................................................................................................... 27 
9.4 
Randomisation ........................................................................................................................ 28 
9.5 
Blinding and code-breaking .................................................................................................... 28 
9.6 
Description of study intervention(s), comparators and study procedures (clinical) .............. 28 
9.7 
Baseline Assessments ............................................................................................................. 28 
9.8 
Subsequent Visits .................................................................................................................... 28 
9.9 
Sample Handling ..................................................................................................................... 28 
9.10 
Early Discontinuation/Withdrawal of Participants ................................................................. 28 
9.11 
Definition of End of Study ....................................................................................................... 28 
10 
SAFETY REPORTING ........................................................................................................................ 28 
11 
STATISTICS AND ANALYSIS .............................................................................................................. 29 
11.1 
Statistical Analysis Plan (SAP) ................................................................................................. 29 
11.2 
Description of the Statistical Methods ................................................................................... 29 
11.2.1 
Workstream 1 ..................................................................................................................... 29 
11.2.2 
Workstream 2a ................................................................................................................... 29 
11.2.3 
Workstream 2b ................................................................................................................... 29 
11.2.4 
Workstream 3b ................................................................................................................... 30 
11.3 
Sample Size Determination ..................................................................................................... 32 
11.3.1 
Workstream 2a ................................................................................................................... 32 
11.3.2 
Workstream 3b ................................................................................................................... 32 
11.3.3 
Workstream 4b ................................................................................................................... 32 
11.3.4 
Workstream 4c ................................................................................................................... 33 
11.4 
Analysis populations ............................................................................................................... 33 
11.5 
Decision points ........................................................................................................................ 33 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 5 of 46 
11.6 
Stopping rules ......................................................................................................................... 33 
11.7 
The Level of Statistical Significance ........................................................................................ 33 
11.8 
Procedure for Accounting for Missing, Unused, and Spurious Data. ...................................... 33 
11.9 
Procedures for Reporting any Deviation(s) from the Original Statistical Plan ........................ 33 
11.10 
Health Economics Analysis ..................................................................................................... 33 
12 
DATA MANAGEMENT ..................................................................................................................... 33 
12.1 
Source Data ............................................................................................................................. 33 
12.1.1 
Workstream 1 ..................................................................................................................... 34 
12.1.2 
Workstream 2/3b ................................................................................................................ 34 
12.2 
Access to Data ......................................................................................................................... 34 
12.3 
Data transfer and analysis environment ................................................................................. 35 
12.4 
Data Recording and Record Keeping ...................................................................................... 35 
12.4.1 
Electronic data .................................................................................................................... 35 
12.4.2 
Staff/Patient interviews/focus groups/simulation scenarios ............................................. 35 
13 
QUALITY ASSURANCE PROCEDURES ............................................................................................... 36 
13.1 
Risk assessment ...................................................................................................................... 36 
13.2 
Study monitoring .................................................................................................................... 36 
13.3 
Study Committees .................................................................................................................. 37 
13.3.1 Study management group (SMG) ............................................................................................ 37 
13.3.2 Project Oversight Group (POG) A project oversight group will ............................................... 37 
14 
PROTOCOL DEVIATIONS ................................................................................................................. 37 
15 
SERIOUS BREACHES ........................................................................................................................ 37 
16 
ETHICAL AND REGULATORY CONSIDERATIONS .............................................................................. 37 
16.1 
Declaration of Helsinki ............................................................................................................ 37 
16.2 
Guidelines for Good Clinical Practice ...................................................................................... 38 
16.3 
Approvals ................................................................................................................................ 38 
16.4 
Other Ethical Considerations .................................................................................................. 38 
16.5 
Reporting ................................................................................................................................ 39 
16.6 
Transparency in Research ....................................................................................................... 39 
16.7 
Participant Confidentiality ...................................................................................................... 39 
16.8 
Expenses and Benefits ............................................................................................................ 39 
FINANCE AND INSURANCE .......................................................................................................................... 39 
16.9 
Funding ................................................................................................................................... 39 
16.10 
Insurance ................................................................................................................................ 40 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 6 of 46 
16.11 
Contractual arrangements ...................................................................................................... 40 
17 
PUBLICATION POLICY ...................................................................................................................... 40 
18 
DEVELOPMENT OF A NEW PRODUCT/ PROCESS OR THE GENERATION OF INTELLECTUAL 
PROPERTY ................................................................................................................................................... 40 
19 
ARCHIVING ...................................................................................................................................... 40 
20 
REFERENCES .................................................................................................................................... 40 
21 
APPENDIX A:  AMENDMENT HISTORY ............................................................................................ 45 
 
 
 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 7 of 46 
1 
KEY CONTACTS 
 
Chief Investigator 
Professor Peter Watkinson  
Professor of Intensive Care Medicine  
University of Oxford  
Kadoorie Centre  
John Radcliffe Hospital  
Oxford OX3 9DU 
Email: peter.watkinson@ndcn.ox.ac.uk 
Co-Lead Investigator: 
Professor Marian Knight  
Professor of Maternal and Child Population Health 
University of Oxford 
National Perinatal Epidemiology Unit, 
Nuffield Department of Population Health  
Richard Doll Building 
Old Road Campus  
Oxford OX3 7LF 
Tel: 01865 289727 
Email: marian.knight@npeu.ox.ac.uk 
Sponsor 
University of Oxford   
Research Governance, Ethics and Assurance (RGEA),   
Boundary Brook House,    
Oxford,   
OX3 7GB,   
United Kingdom  
Telephone: +44 1865 616480  
Email: rgea.sponsor@admin.ox.ac.uk  
Funder(s) 
National Institute for Health and Care Research (NIHR) 
Coordinating Centre, Grange House, 15 Church Street, Twickenham 
TW1 3NL 
Tel: 020 8843 8000 
Fax: 020 8843 8001 
Email: pgfar@nihr.ac.uk 
Statistician 
Professor Gary Collins 
Professor of Medical Statistics 
University of Oxford 
Centre for Statistics in Medicine 
Nuffield Orthopaedic Centre 
Windmill Road 
Headington 
Oxford OX3 7HE 
Tel: 01865 223460 
Email: gary.collins@csm.ox.ac.uk 
Committees 
Study Management Group 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 8 of 46 
The study management group will consist of the Chief Investigator, 
Lead Investigator and the named investigators listed under Key 
Contacts, and individuals directly funded by the project.  
Project Oversight Group 
The project oversight group will consist of independent members as 
approved by the funder. Their role will be to provide ongoing oversight 
of the project, to ensure that it achieves the study objectives. 
 
INCLUSIVITY STATEMENT 
We use the term 'women' throughout this protocol to refer to those who are planning to become 
pregnant, are pregnant, and give birth. We acknowledge that not all people who are pregnant and give 
birth identify as women, and it is important that evidence-based care for maternity, perinatal and 
postnatal health is inclusive. 
2 
LAY SUMMARY  
Every year more than 700,000 women give birth in the United Kingdom. Of these at least 8700 nearly die 
– called a “near-miss”, and 70 die. Many more women suffer harm, often with effects lasting for life. 
Women from less wealthy areas and particular ethnic groups are more likely to come to harm. 
"Vital signs" include measurements of blood pressure, heart and breathing rates. Doctors and midwives 
use tools that score vital signs to spot women becoming unwell. These tools are called “Modified 
Obstetric Early Warning Scores” (MOEWS). Despite their use, poor outcomes still occur. This may be 
because MOEWS use only the most recent vital signs. Using extra data like blood tests may help spot 
unwell people earlier. 
We aim to reduce poor outcomes for women giving birth. We will find better ways of describing, 
spotting, and treating women becoming unwell. 
We plan four linked projects to develop an electronic advanced maternal obstetric early warning system 
(eMOEWS). Patient and Public (PPIE) collaborators have developed this work with us. We will work 
closely with them throughout this project. 
Once we have completed these four projects, we plan to carry out a trial to assess whether the new 
eMOEWS leads to better outcomes than the existing tools. This trial will be described in a separate 
protocol. 
Project One 
We will develop new definitions of worsening illness in women giving birth. We will work with our PPIE 
colleagues and other experts, reviewing published work. This will help staff use routinely collected health 
data to spot early illness, before a woman becomes very unwell. We will check that the new definitions 
reliably identify women becoming unwell. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 9 of 46 
Project Two 
Using the new definitions, we will test how well current MOEWS pick up worsening illness. We will use 
data from eight to twelve NHS maternity units serving diverse women, and our national maternal review 
programme. 
Project Three 
We will develop an advanced, electronic MOEWS (eMOEWS) working with our PPIE collaborators and 
other experts. This will use extra information known to affect the risk of poor outcomes. We will test 
how well the eMOEWS spots worsening illness, using our new definitions. 
Project Four 
We will develop a way to digitally display eMOEWS on maternity units. We will work with staff who use 
computers along with experts in NHS computer systems. This will allow staff to understand quickly which 
women are at risk, and why. We will design guidelines for how to use eMOEWS on maternity units with 
women and staff. This will make sure our new system helps give women the right care at the right time. 
3 
SYNOPSIS 
 
Study Title 
Defining, Recognising and Escalating Maternal Early Deterioration (DREaMED) Phase 1: 
Decreasing inequality through improved outcomes 
Internal ref. 
no. / short 
title 
DREaMED1 
Study 
registration 
The study will be registered on clinicaltrials.gov  
Sponsor  
University of Oxford  
Funder  
National Institute for Health and Care Research (NIHR), Programme Grants for Applied 
Research (NIHR204430)   
Study Design 
Longitudinal Cohort study 
Study 
Participants 
Women aged 16 or over who are pregnant 
Sample Size 
Total: 459,160 
Retrospective cohort: 293,200 
Prospective cohort: 164,300 
MBRRACE-UK cohort: 1,300 
Patient interviews (escalation pathway): Up to 60 
Staff interviews/focus groups (escalation pathway): Up to 100 
Staff interviews/focus groups (eMOEWS interface development): Up to 100 
Staff training (simulation scenarios): Up to 100 
Planned Study 
Period 
Funded study period: 02/05/2023 until 01/05/2029 
Total funded study period will be 6 years  
Planned study period: 01/05/2024 until 01/05/2029 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 10 of 46 
Planned 
Recruitment 
period 
Retrospective cohort: 01/01/2021 to 28/02/2025 
Prospective cohort: 01/03/2025 to 30/06/2027 
MBRRACE-UK cohort: 01/01/2009 to 31/12/2022  
Staff interviews/focus groups (eMOEWS interface development, WS4a): 01/04/2025 to 
31/03/2026 
Patient interviews (escalation pathway, WS4b): 1/06/2024 to 30/08/2025 
Staff interviews/focus groups (escalation pathway, WS4b): 01/06/2024 to 30/11/2025 
Staff training (simulation scenarios, WS4c): 01/11/2026 to 30/11/2027 
 
Objectives 
Outcome Measures 
Timepoint(s) 
Primary 
 
To develop, and validate an 
electronically-embedded, data-
enhanced Maternal Early 
Warning Score (eMOEWS) with 
clinical escalation pathways 
Predictive performance of new 
early warning scores, assessed 
by: discrimination, calibration, 
and clinical utility. 
During pregnancy 
or in the 
immediate 
postpartum 
period. 
Secondary: 
 
 
 
 
To define ‘near-miss’ and ‘pre-
near-miss` criteria for use with 
routinely-collected data to 
measure maternal outcomes 
‘Near-miss’ and ‘pre-near-miss’ 
outcome criteria measurable 
using routinely available 
electronic data, assessed in NHS 
hospitals  
During pregnancy 
or in the 
immediate 
postpartum 
period. 
 
To assess performance of the 
new near-miss and pre-near-
miss outcome criteria in 3 
hospitals and MBRRACE-UK in: 
• 
a stratified random 
sample of 100 sets of 
records of near-miss 
and pre-near-miss cases 
across the three sites.  
• 
clinical serious maternal 
events recorded in 
hospital serious event 
systems.  
• 
MBRACCE-UK cases 
Descriptive statistics to describe 
whether the electronic criteria 
used correctly identified the 
conditions. 
 
Descriptive statistics of missed 
and captured events according 
to electronic criteria 
Descriptive comparison to 
published rates 
During pregnancy 
or in the 
immediate 
postpartum 
period. 
 
To develop a large 
representative eight to twelve 
maternity unit cohort for model 
development and assessment 
Cohort developed and ready for 
assessment 
05/2029 
 
To investigate the performance 
of existing MOEWS/MEWS in 
the new maternity cohort and 
in women who have died or had 
a near-miss or pre-near-miss  
event in pregnancy 
Evidence-based assessment of 
existing MOEWS, both in new 
retrospective (assessed by 
discrimination, risk of our key 
events at each MOEWS/MEWS 
level) and nationally recognised 
mortality/morbidity (assessed 
05/2029 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 11 of 46 
by sensitivity and duration of 
prior warning) cohorts 
 
To develop and validate an 
optimal vital-signs-only-based 
MOEWS 
Relative performance of new 
MOEWS, best published 
MOEWS and NHSI national 
MOEWS for prediction of near-
miss or pre-near-miss 
05/2029 
 
To develop and validate an 
eMOEWS to improve detection 
in comparison to existing 
MOEWS/MEWS 
 
Relative performance 
assessment of eMOEWS, new 
validated vital-signs-based 
MOEWS, best published 
MOEWS and NHSI national 
MOEWS for prediction of ‘near-
miss’ or ‘pre-near-miss’ 
05/2029 
 
To design and deliver an 
eMOEWS interface 
User co-designed eMOEWS 
implemented within 4 NHS 
sites. 
System Usability Scale 
performance 
05/2029 
 
To develop treatment 
escalation pathways 
Escalation and response 
pathways protocolised iterated 
and tested in a simulated 
environment 
05/2029 
 
4 
ABBREVIATIONS 
 
CI 
Chief Investigator 
CRF 
Case Report Form 
eMOEWS 
Electronically-embedded, data-enhanced Maternal Obstetric Early Warning Score 
GCP 
Good Clinical Practice 
GP 
General Practitioner 
HRA 
Health Research Authority 
ICF 
Informed Consent Form 
MEWS 
Maternal Early Warning Score 
MOEWS 
Modified Obstetric Early Warning Score 
MBRRACE-UK 
Mothers and Babies: Reducing Risk through Audit and Confidential Enquiries, United 
Kingdom 
NEWS 
National Early Warning Score 
NHSE 
National Health Service England 
NHSI 
National Health Service Improvement 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 12 of 46 
RES 
Research Ethics Service 
PI 
Principal Investigator 
PIERS 
Preeclampsia Integrated Estimate of Risk 
PIL 
Participant/ Patient Information Leaflet 
PPIE 
Patient and Public Involvement and Engagement 
R&D 
NHS Trust R&D Department 
REC 
Research Ethics Committee 
RGEA 
Research Governance, Ethics & Assurance, University of Oxford 
SOP 
Standard Operating Procedure 
WS 
Workstream 
 
5 
BACKGROUND AND RATIONALE 
5.1 
What is the problem being addressed? 
Of over 700,000 women giving birth in the UK annually, one in every 10,000 women dies during or up to 
six weeks after pregnancy (1). For every woman that dies, 120 will have a ‘near-miss’ (8,700 women 
annually) often with lifelong sequelae (2). Recent data from the United States using adapted Centre for 
Disease Control definitions found even higher rates (2%) (3). There are significant disparities in    
maternal and perinatal outcomes between ethnic, age and socioeconomic groups (1,4). Maternal 
mortality has been consistently shown to be more than four-fold higher among women from Black ethnic 
groups, and two-fold higher among women from Asian ethnic groups compared to women from White 
ethnic groups (5). Mortality rates are similarly two-fold higher amongst women who live in the 20% most 
deprived areas compared to those who live in the least deprived areas (5). Women aged over 40 are four 
times more likely to die during or after pregnancy compared to women aged 25-29 (5). Substantial 
disparities also exist in severe maternal morbidity; a recent analysis by our group (manuscript submitted) 
using Hospital Episodes Statistics data has shown that severe maternal outcomes at the time of a 
childbirth event, measured using the English Maternal Morbidity Outcome Indicator (EMMOI) are up to 
70% higher amongst women from Black Caribbean and Black African ethnic groups. This disparity persists 
across all English regions and is driven primarily by sepsis and complications related to haemorrhage or 
pre-eclampsia. This variability, along with the major underlying causes, suggest a high degree of 
avoidability, a suggestion strongly supported by MBRRACE-UK mortality reports (5). 
Current published morbidity criteria (6–9) represent near end stage (near death), rather than early 
deterioration and are not reliably derivable from routine data, preventing both development of early 
warning systems to detect early maternal deterioration and effective routine quality assurance. 
The detection of women at risk of severe maternal morbidity or mortality is currently limited by the use 
of expert opinion-based Modified Obstetric Early Warning Scores (MOEWS). These MOEWS are based 
only on the most recent set of vital signs (heart rate, blood pressure, oxygen saturation, respiratory rate, 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 13 of 46 
and conscious level), ignoring other highly relevant risk information used by clinicians. Current MOEWS 
have very limited assessment of detection performance (10–12). 
5.2 
Why is this research important?  
It is internationally recognised that new definitions for early deterioration (‘pre-near-miss’ definitions) in 
maternity are required to avoid severe adverse maternal outcomes and for effective quality assurance 
(13). Recent work demonstrates that these are also essential for effective prediction model development 
(14). In our programme of research, we will develop these new definitions for early maternal 
deterioration (outside the current ethics/CAG application), ensuring that they can be operationalised to 
allow electronic extraction from routine clinical data. 
Underlying these adverse maternal outcomes is persistent failure to recognise early deterioration. 
Adverse maternal outcomes investigations consistently show the severity of women’s illness is not 
recognised and women are not listened to (15–17). This has been brought to the fore in the recent 
Ockenden report of the investigation into maternity care at Shrewsbury and Telford Hospitals (18). 
Developing robust pathways for managing women with complex pregnancies is an essential action 
recommended by the Ockenden report (18). Recent research examining the care of women from 
different ethnic groups who died identified that current structures were unable to provide care for 
women with multiple complex clinical, social, and cultural conditions whose needs crossed the expertise 
of multiple specialist teams (18,19). Failure to provide timely care was linked to a lack of understanding 
of the altered physiology in pregnancy and an inability to recognise disease severity, emphasising the 
importance of explaining why our proposed advanced eMOEWS identifies a woman as at risk. Women 
are entering pregnancy older, with increasing multi-morbidity, making recognising deterioration even 
more imperative and potentially more challenging. Earlier deterioration recognition should lead to early 
intervention, preventing progression to severe morbidity or mortality (3). 
Significant deterioration is both common and has substantial immediate and long-term outcomes for the 
women affected. A prospective study of women giving birth in Wales, for example, reports that up to 
3.3% of women (23,000 in the UK each year) have a postpartum haemorrhage of 1500ml or greater (20). 
Early detection and escalation through an enhanced MOEWS would prevent haemorrhage reaching 
higher levels, preventing early consequences of anaemia such as need for transfusion, maternal fatigue 
and depression (21,22). Long-term consequences of adverse maternal outcomes are also common. For 
example, a hysterectomy, which leads to consequent lifelong infertility, is used as a lifesaving 
intervention for severe haemorrhage. Severe uncontrolled blood pressure due to pre-eclampsia can lead 
to stroke and long-term disability. Women who have experienced life-threatening complications in 
pregnancy also highlight the mental health consequences, including post-traumatic stress disorder for 
both them and their partner and long-term impacts on their relationship with their baby (23–25). 
The UK government aims to halve maternal mortality by 2025 (26), yet mortality rates have altered little 
across successive MBRRACE reports. Better early detection, escalation and management are essential. 
Current methods for detecting deteriorating women are not fit for purpose. In the general hospital 
population, The National Early Warning Score 2 (NEWS2) (27) is used for deterioration detection. NEWS2 
is not intended for use in pregnant women where physiological changes in pregnancy substantially 
change the normal vital sign ranges underlying such scores (28,29). Despite this, investigations into 
maternal deaths, including women dying in the recent pandemic, have repeatedly identified that NEWS2 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 14 of 46 
is being used in pregnant women, and failing to identify critical illness (30,31). Alternative, expert 
opinion-based Modified Obstetric Early Warning Scores (MOEWS) have been developed, with limited 
assessment of performance (10–12). A wide variety of different MOEWS are in use in the UK, resulting in 
substantial practice variation (32), with little evidence of effective implementation (33). The urgency of 
this problem is illustrated by the care of pregnant and postpartum women who died during the COVID-
19 pandemic (2).  Women who were clearly in extremis were assumed to be hyperventilating, their care 
was not escalated, the maternal medical team were not involved, and women received potentially life-
saving treatments late or not at all. These systemic deficiencies occurred despite current early warning 
tools. 
We will assess current vital-signs-based MOEWS and externally validate an optimised vital-signs-based 
MOEWS. This will provide a reliable, evidence-based MOEWS tool in environments where a more 
advanced, multivariable, electronic MEWS (eMOEWS) is not feasible. 
Outside maternity, advanced, multivariable, electronic MOEWS incorporating predictors beyond fixed 
vital sign thresholds substantially improve detection of deterioration (34–37). The variation in vital signs 
throughout pregnancy and postpartum (28,29,38), the need for non-vital sign measures to define World 
Health Organisation (WHO) near-miss events (9) and prior work in specific maternal conditions such as 
pre-eclampsia (39) and maternal sepsis (40) suggest an electronic advanced, multi-variable score will 
substantially improve early deterioration detection in maternity. We will therefore develop an eMOEWS 
for clinical testing within the programme. 
Though specified response pathways have shown benefit outside the UK(3), evidence of design or 
implementation of escalation pathways and the organisational and behavioural science of successful care 
escalation is limited (41). For our eMOEWS to have clinical impact, a robust implementation strategy, 
with evidenced escalation pathways is required. The system is designed to work within UK healthcare 
systems whilst minimising workload and is required to ensure effective action in the event of early 
deterioration to prevent ‘near-miss’ or ‘severe morbidity’ events. Again, the Ockenden report 
recommendations emphasise the importance of this work, stating that the hospital ‘must strive to 
develop a safe environment and culture where all staff are empowered to escalate to the correct person’ 
and that the hospital’s ‘escalation policy must be adhered to and highlighted on training days to all 
maternity staff’ (18). 
Our research will fill four important evidence gaps: 
1. the lack of validated early deterioration definitions in maternity. 
2. an optimally developed vital-signs-based MOEWS for use where eMOEWS is not feasible. 
3. an optimally developed eMOEWS. 
4. evidenced eMOEWS and escalation pathway implementation methodology. 
Our research is targeted to reflect the Department of Health and Social Care’s priorities, supporting the 
healthcare workforce by using transformative technology (eMOEWS) to target more efficient care at the 
right time, improving healthcare outcomes and reducing the major health disparities seen in adverse 
maternal outcomes. Although the 2018 national Maternity Digital Maturity Assessment report (42) found 
only 29% of organisations used electronic vital signs systems, the accelerated electronic patient record 
implementation set out in the NHS long term plan (43) and NHS Digital’s Digital Maternity Record 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 15 of 46 
Standard Requirements Specification (44) present an unrivalled opportunity to implement a world-
leading, technologically-advanced eMOEWS deterioration detection system NHS-wide. 
Beyond this protocol, our research programme will implement our eMOEWS and escalation pathways 
within a study designed to provide the clinical and health economic evidence to support wide-scale 
implementation. Learning from this implementation will be used to maximise eMOEWS performance for 
future use. We will seek additional funding to establish our electronic databases as a national resource 
for future use, extending the duration of data collection and use in subsequent applications. 
6 
OBJECTIVES AND OUTCOME MEASURES 
Objectives 
Outcome Measures  
Timepoint(s) of 
evaluation of this 
outcome measure (if 
applicable) 
Primary Objective 
To develop and validate an 
electronically embedded, data-
enhanced Maternal Early Warning 
Score (eMOEWS) with clinical 
escalation pathways 
Predictive performance of new early 
warning scores, assessed by: 
discrimination, calibration, and clinical 
utility. 
During pregnancy or in 
the immediate 
postpartum period. 
Secondary Objectives: 
 
 
To define ‘near-miss’ and ‘pre-
near-miss’ criteria for use with 
routinely-collected data to 
measure maternal outcomes. 
 
‘Near-miss’ and ‘pre-near-miss’ 
outcome criteria measurable using 
routinely available electronic data, 
assessed in NHS hospitals  
During pregnancy or in 
the immediate 
postpartum period. 
To assess performance of the new 
near-miss and pre-near-miss 
outcome criteria in 3 hospitals 
and MBRRACE-UK: 
• 
a stratified random 
sample of 100 sets of 
records of near-miss and 
pre-near-miss cases 
across the three sites.  
• 
review hospital serious 
event systems for serious 
maternal events 
• 
Proportions of women 
meeting criteria 
 
 
Descriptive statistics to describe 
whether the electronic criteria used 
correctly identified the conditions. 
 
Descriptive statistics of missed and 
captured events according to electronic 
criteria 
Descriptive comparison to published 
rates 
During pregnancy or in 
the immediate 
postpartum period 
To develop a large representative 
eight-twelve maternity unit cohort 
Cohort developed and ready for 
assessment 
05/2029 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 16 of 46 
for model development and 
assessment  
To investigate the performance of 
existing MOEWS/MEWS scores in 
the new maternity cohort and in 
women who have died or had 
severe morbidity in pregnancy 
Evidence-based assessment of existing 
MOEWS, both in new retrospective 
(assessed by discrimination, risk of our 
key events at each MOEWS/MEWS 
level) and nationally recognised 
mortality/morbidity (assessed by 
sensitivity and duration of prior 
warning) cohorts 
05/2029 
To develop and validate an 
optimal vital-signs-only-based 
MOEWS 
Relative performance of new MOEWS, 
best published MOEWS and NHSI 
national MOEWS for prediction of near-
miss or pre-near-miss 
05/2029 
To develop and validate an 
eMOEWS to improve detection in 
comparison to existing MOEWS 
Relative performance assessment of 
eMOEWS, new validated vital-signs-
based MOEWS, best published MOEWS 
and NHSI national MOEWS for 
prediction of ‘near-miss’ or ‘pre-near-
miss’ 
05/2029 
To design and deliver an eMOEWS 
interface. 
 
User co-designed eMOEWS 
implemented within 4 NHS sites 
05/2029 
To develop treatment escalation 
pathways 
 
Protocolised escalation and response 
pathways iterated and tested in a   
simulated environment 
05/2029 
 
 
7 
STUDY DESIGN 
7.1 
Overview 
This study consists of a programme of work that is centred around a UK-based, multi-centre cohort study 
of women who are pregnant or have recently been pregnant.  
The overall research question is: “Can an electronically-embedded, enhanced Maternal Early Warning 
Score (eMOEWS) with behaviourally-informed escalation pathways prevent maternal ‘near-miss’, ‘severe 
morbidity’ events and mortality?” 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 17 of 46 
The overall research aim is to prevent maternal morbidity and mortality and reduce inequalities in 
outcomes by developing, refining, validating, embedding, and testing an eMOEWS with clinical escalation 
pathways. 
The initial programme of work, covered by this protocol, is divided into four workstreams as below. The 
work described in this protocol is intended (subject to funder agreement) to be immediately followed by 
a prospective study to assess the clinical and health economic effects of implementation of the eMOEWS 
and the associated escalation pathway (for which a separate protocol will be written and separate 
approvals sought). 
7.2 
Workstream 1 
Workstream 1 will develop ‘near-miss’ and ‘pre-near-miss’ (indicating deviation from normal pregnancy 
such that intervention is required to avoid progression to a ‘near-miss’ event) definitions electronically 
derivable from routine data by a literature review, followed by an international Delphi process. The 
identification performance of these definitions will be assessed using data from MBRRACE-UK and three 
NHS hospitals. See Figure 1 for an overview of the workstream 1 workflow. 
We will convene an expert clinical panel to define the pathological pathways to “near-miss” as defined 
by the World Health Organisation near-miss events and other published criteria. We will include multi-
professional expertise (e.g., midwifery, obstetrics, anaesthetics, obstetric medicine, pathology, clinical 
epidemiology, critical care). We will use these pathways to identify candidate specific pre-near-miss 
maternal complications that could lead to “near-miss” events (or death).  
We will identify consensus definitions of “pre-near-miss” criteria where possible from existing core 
outcome sets (such as the HDR UK Phenotype library and others (45,46)), along with methods to identify 
these outcomes from routinely collected data. Where there is any uncertainty regarding the definition of 
“pre-near-miss” criteria, we will conduct targeted literature reviews. Where consensus definitions are 
not available, they will be determined by the expert panel. 
We will map definitions to the Maternity Services Dataset (MSDS), Hospital Episode Statistics and 
routinely available electronic patient record data in our partner hospitals, identifying where and what 
routine data augmentation may be required for specific definitions and where surrogates are available. 
Definitions (consensus or expert panel), methods for their identification and any required routine data 
augmentation or surrogate available data fields will inform a three-round international consensus Delphi 
process, including the patient voice. The Delphi process will follow guidance from the Core Outcome 
Measures in Effectiveness Trials (COMET) Initiative (47). In round 1, participants will also be asked to 
identify missing important definitions. A Delphi facilitator will curate Round 1 results. Delphi participants 
will be asked to rate definitions on a 9-point Likert scale from 1 (not important) to 9 (critically important) 
in round 2. Following further curation, the results from Round 2 (all definitions, along with summary 
statistics of their respective scores (mean, median, range) will be fed back to the Delphi panel, allowing 
each member to see how the panel had voted on each definition. Using this information, each member 
will be asked to re-score each definition using the same 9-point scale as used in Round 2. Definitions 
rated 7-9 by 70% or more of the panel or 1-3 by less than 15% in at least one stakeholder group will be 
included in the final list (47). 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 18 of 46 
The final list will again be assessed by our expert panel and mapped against routine data reviewed to 
achieve ‘near-miss’ and `pre-near-miss’ for identification performance testing. 
We will use retrospective data from three representative collaborator NHS hospitals (who will form part 
of the retrospective cohort) and from MBRRACE-UK, reviewing deaths, near-miss and pre-near-miss 
cases to estimate identification performance using the new definitions. 
7.2.1 
Objectives 
In the three collaborator hospitals we will: 
• Report the proportion of women meeting each of the new criteria and in total in WS2 
retrospective datasets – comparing these proportions relative to the hospital populations with 
published rates for our individual criteria. Where the criteria definitions appear to under or over-
capture events, we will undertake record review to inform definition adjustment where required 
(approximately 10 case records per event of over/under capture).  
• Undertake validation of the coded criteria through examining a stratified random sample of 100 
sets of records of these cases across the three sites to assess whether the codes used in the 
electronic data correctly identified the conditions – again if necessary, allowing criteria definition 
adjustment. 
• Review hospital systems for recording serious events in the time period, reviewing cases identified 
as maternal serious incidents (maternal serious incident cohort) but not identified by our criteria 
to establish whether criteria definition adjustment is required. 
 
From MBRRACE-UK we will report the proportions of women who meet the new definitions. 
• Learning will be fed back to the expert panel to generate final definitions. 
 
Further details of the statistical analyses related to workstream 1 are given in Section 11.2.1. 
7.2.2 
Permissions and data flow 
For both retrospective case record reviews detailed above we will need to review specific (uncommon) 
cases of adverse events in each of three hospitals, precluding individual consent. An application to the 
Confidential Advisory Group will be made (see Section 9.1.2 for further details). 
 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 19 of 46 
 
Figure 1 Overview of Workstream 1 
7.3 
Workstream 2 
Workstream 2 will, in collaboration with the NIHR Health Informatics Collaborative, develop an eight- to 
twelve maternity unit dataset. Current MOEWS’ ability to identify newly defined ‘near-miss’ and ‘pre-
near-miss’ events will be assessed using this data and data from the MBRRACE-UK dataset. 
7.3.1 
Identification of existing MOEWS 
Existing MOEWS will be identified by updating an existing systematic review (see WS3a for methods) 
along with checking with experts in the field (11), and will include Irish, Scottish and American published 
tools along with the new NHSE/I tool (11,48–50). 
7.3.2 
Workstream 2a 
We will curate retrospective datasets from routine NHS maternity unit electronic data to encompass the 
work in section 2b and 3b. Data will encompass variables required for prediction and data required for 
maternal and neonatal outcomes (to allow assessment of the predictive ability of the scores for adverse 
neonatal outcomes). For data types see section 12.1.2.  
The output of this workstream will be the external validation performance of existing MOEWS. Details of 
the methods that will be used to carry this out are reported in Section 11.2.2. 
7.3.2.1 
Data collection and data flow 
Retrospective whole maternal unit data from available electronic patient records for multiple sites is 
required (see statistical analysis) precluding individual patient consent. Databases will be built by 
automated electronic extraction with manual augmentation (including from paper records) where 
necessary. All data will be pseudonymised prior to analysis by the study team. An application to the 
Confidential Advisory Group will therefore be made to undertake this work (see Section 12 for further 
detail). A data flow diagram is included in the study documentation. 
 
Map to available electronic data 
(criteria):
• near miss events
• pre-near-miss events
Delphi review of:
• near miss events/criteria
• pre-near-miss events/criteria
Near-miss deﬁni=ons literature 
review
Expert panel iden=ﬁca=on of 
pre-near-miss events
Literature review of consensus 
deﬁni=ons of pre-near-miss 
events
Deﬁning electronic 
maternal near-miss 
and pre-near-miss 
events
Valida5ng electronic 
maternal near-miss 
and pre-near-miss 
events
Assess propor=ons of women 
mee=ng criteria in 3-hospital 
retrospec=ve datasets
Case record review of 
under/over iden=ﬁca=on
Propor=ons of MBRRACE-UK 
cases mee=ng criteria
3-hospital retrospec=ve case 
record review of 100 iden=ﬁed 
cases (stra=ﬁed sample)
Criteria deﬁni=on adjustment
Workstream 1
Near-miss ≡ severe morbidity
Expert panel ra=ﬁca=on of pre-
near-miss and near-miss criteria

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 20 of 46 
7.3.3 
Workstream 2b 
7.3.3.1 
Participants 
MBRRACE-UK undertakes confidential enquiries into maternal deaths and specific severe morbidities. 
We have already extracted event and vital sign data for 205 women who died or suffered severe 
morbidity (from pulmonary embolism and severe haemorrhage). In this work stream, we will additionally 
extract available re-laparotomy (n=32) and diabetic ketoacidosis cohorts (n=80) to complete 
representation of the available major reversible events. We will also extract additional outcome data to 
define when women meet the new ‘near-miss’ and ‘pre-near-miss’ criteria (WS1). This will require an 
extended output request to the Healthcare Quality Improvement Partnership. All women in the cohort 
are likely to have had one of the near-miss or pre-near-miss events (as defined in WS1) or death. 
7.3.3.2 
Data collection and data flow 
We aim to replicate the data in WS2a. Anonymised data will be collated into research datasets onto 
University of Oxford secure servers where the MBRRACE dataset already resides. In the case of 
thromboembolism and major haemorrhage datasets, these have already been curated, but will require 
addition of pre-near-miss events and times as defined in WP1. 
7.3.3.3 
Outcome measures 
The primary outcome for MOEWS assessment for both data sources (WP2a&b) will be a combination of 
maternal death, ‘near-miss’ and ‘pre-near-miss’ events (as defined in WS1). Secondary outcomes will be 
maternal death, ‘near-miss’ and ‘pre-near-miss’ events, individually. For more details on the analysis 
methods see Section 11.2.3. 
7.4 
Workstream 3 
Workstream 3 will develop an eMOEWS candidate variable list by literature review informing a further 
Delphi process. An optimal vital-signs MOEWS and a multivariable eMOEWS will be developed and 
validated, exploring regression models and machine learning (ML) approaches. 
7.4.1 
Workstream 3a – developing the candidate variable list 
7.4.1.1 
Systematic review 
We will undertake a systematic review and critical appraisal of routinely collected, electronically 
available variables which affect the risk of sustaining a ‘near-miss’ or ‘pre-near-miss’ event as defined in 
WS1. This will include noting key interactions between variables. We will include existing prediction 
models and associated publications. We will follow guidance by the Cochrane Prognosis Methods Group 
(51,52). We will search bibliographic databases including MEDLINE and EMBASE. We will snowball to 
identify any further relevant studies. Searches will be developed with an academic librarian (with whom 
we have previously worked). The literature between 01/01/1993 and 30/06/2023 will be identified 
without language restrictions. Data will be extracted in duplicate. Authors will be contacted by e-mail 
where additional information is required. 
7.4.1.2 
Review strategy 
We will undertake quality assessment of included studies, where possible using the QUIPS tool for 
prognostic factors, recommended by the Cochrane Handbook for the appraisal of studies of risk factors  
(53,54). The piloted data extraction form will follow the prognostic factors adaptation (CHARMS-PF) of 
the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 21 of 46 
(CHARMS) guidance (52). We will further synthesise available data using a semi-quantitative method 
previously described by Zaal et al. (55) and adapted by Dettmer et al (56). 
7.4.1.3 
Delphi strategy 
We will assess the variables identified during our systematic review against those recorded in our 
partner maternity sites to inform availability for inclusion. We will use this information to inform a 3-
round international Delphi process to compile a list of existing, electronically available variables which 
affect the risk of ‘near-miss’ and ‘pre-near-miss’ events (as defined in WS1). This will generate a 
candidate variable list. Our strategy will follow a similar process to that in WS1. 
The Delphi participants will again be identified from author lists of articles included in our review of the 
literature to ensure diverse expertise and selection objectivity. We will also involve our PPI colleagues. In 
round one, variables identified from our literature review will be presented to the Delphi participants 
along with their purported effect on risk (protective vs predisposing). To provide structure, variables will 
be grouped into a small number of domains including but not limited to demographics, vital signs, and 
laboratory variables. Participants will again be asked to identify any variables felt to be important but not 
included, and whether these are felt to represent protective factors or predisposing factors. Round 1 
results will then be collected and curated by a Delphi facilitator. 
Subsequent rounds will follow the processes described in WS1. 
 
The final candidate variable list will then be assessed against availability in the 8-12 sites followed by 
data extraction of available candidate variables as part of WS2a. For likely variable types see section 
12.1.2. 
 
7.4.2 
Workstream 3b – developing an optimal vital-signs MOEWS and multivariable eMOEWS 
We will develop and externally validate (see Section 11.2.4 for further details):  
• 
A vital-signs-based MOEWS for use in environments where connection to an EPR is not available. 
• 
A multivariable electronic maternal early warning score (eMOEWS) for use in digitally mature 
environments 
We will extract a separate, more recent 2-year validation cohort from eight-twelve sites to allow external 
validation (containing relevant variables) of the optimal vital-signs MOEWS and multivariable eMOEWS.  
 
7.5 
Workstream 4 
In Workstream 4 we will: 
• 
Build working eMOEWS web interfaces following user-centred design principles (Workstream 
4a).  
• 
Develop escalation and response pathways for women identified as at risk by the eMOEWS 
(Workstream 4b) 
7.5.1 
Workstream 4a 
7.5.1.1 
Requirements identification 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 22 of 46 
To ensure a bespoke eMOEWS user interface we will undertake evaluation of end-user needs for user 
interfaces using focus groups, with topic guides underpinned by Normalisation Process Theory and 
informed by the NASSS framework to identify barriers and facilitators, local adaptions and unintended 
consequences influencing implementation and hence potential effectiveness across the implementation 
sites. We aim to conduct 2-3 focus groups with up to 20 professionals per site (we anticipate this number 
will allow representation of the views of a range of professionals of different seniority – but will continue 
focus groups until saturation has been achieved).  
 
7.5.1.2 
Interface design 
We will develop the key design elements and functionality for the web-interface through work including 
Applied Cognitive Task Analysis Interviews (57) with multi-disciplinary staff (including obstetricians, 
midwives and anaesthetists) and “card sorting” (58) exercises in implementation sites (59) to prioritise 
categories of information (e.g., demographics, vital signs, laboratory tests, eMOEWS risk score) displayed 
in the interface. We estimate approximately thirty cross-implementation site meetings with clinical staff 
will be required but will continue until no major new information is being obtained.  
 
After installation of the eMOEWS system, we will trial the user interface with key stakeholders. We will 
encourage staff to use the eMOEWS to display information from patients currently under their care. As 
clinicians are likely to be familiar with these patients’ recent clinical data, we will use this to discover 
minor omissions and local adaptions that could be addressed to enhance the system. We will continue 
trials until saturation occurs – but again estimate around 30 meetings with key stakeholders across sites. 
Learning from eMOEWS use within WS4b will also be iteratively incorporated into the final interface. We 
will document the time and equipment consequences of eMOEWS installation and maintenance at each 
site and centrally to inform WS5c. 
 
Testing metrics will include task-based efficiency and effectiveness, user satisfaction and workload. 
7.5.2 
Workstream 4b 
We will develop escalation pathways using: 
• 
Secondary analysis of interviews with women who experienced near-miss morbidity. 
• 
Interviews with women and partners who have recently experienced maternity complications. 
• 
Focus groups 
• 
A response pathway co-design group. 
We will conduct secondary analysis of interviews with 36 women who experienced near-miss morbidity 
and 11 of their partners who were interviewed through the ‘healthtalk’ series (Health Experiences 
Research Group) to capture women and partners’ perspectives relating to the recognition and 
management of acute illness in a maternity setting. 
In addition, we will conduct a further 10-15 interviews or focus groups with women and partners from 
diverse groups who have recently experienced maternity complications to ensure we are capturing any 
specific barriers identified amongst women from diverse ethnic groups and amongst those who are 
vulnerable. Costs for translation services during these interviews (where required) are included in the 
budget. We will identify participants using posters in our partner hospitals and through our contacts with 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 23 of 46 
trusted community support and patient groups. To maximise participation, we will offer a choice of on-
line and face-to-face interviews and accessible community venues.  A topic guide for interviews will be 
developed from previous work and secondary analysis of interviews above. Additional semi-structured 
questions will be asked, using a topic guide. A list of questions will be submitted as a study amendment 
prior to interviews/focus groups commencing. 
Consensus best practice MOEWS responses to the NHSE/I MOEWS have been identified through a Delphi 
process using example clinical scenarios conducted by the co-applicant team with NHSI. These, with 
other published response pathways (3), and findings from our interviews, guided by Normalisation 
Process Theory underpinned by key NASSS domains, will inform topic guide content for focus groups, to 
identify barriers and facilitators, local adaptions and unintended consequences influencing response 
implementation and effectiveness. 
We will apply inter-related social and behavioural science theories and frameworks to address the 
individual, socio-cultural and organisational factors influencing escalation responses. We will particularly 
explore factors influencing decision making and intentions around whether or not to escalate, and 
factors influencing appropriate and timely response as failure to escalate and respond appropriately 
could influence the effectiveness of the eMOEWS to prevent deterioration. Following the Behaviour 
Change Wheel approach to intervention design (60), we will use the findings from this behavioural 
analysis as a basis for systematically co-designing a response pathway to target the key factors 
facilitating or hindering escalation on the basis of an eMOEWS trigger in this context. 
 
Response pathway co-design will then be undertaken with relevant stakeholder group representatives 
(e.g., pregnant women, healthcare professionals, experienced mMOET course leaders – see below – 
again including translation where necessary) paying particular attention to information from women 
whose voices and concerns find it harder to be heard and responded to (61,62). Key staff from 
intervention sites will be involved in escalation pathway development, ensuring a core intervention 
applicable across sites (and across the NHS), and identifying minor adaptions outside the core 
intervention to allow individual site integration. We will focus on minimising the perceived opportunity 
costs of implementation, working with the focus groups and local champions to foster the internal 
(within NHS) view (63,64). We envisage two meetings will be required with preparation work undertaken 
prior to each meeting, with initial pathways designed by the research team between meetings. 
 
7.5.3 
Workstream 4c 
The research team will develop three simulated scenarios informed by the brief clinical scenarios used 
for development of the NHSE/I National MOEWS. These will be designed to refine the clinical escalation 
and response pathways developed in WS4b for representative near-miss and pre-near-miss events. Local 
champions from intervention sites will be involved in scenario development and manualisation of the 
escalation pathway to ensure that the core intervention remains applicable across sites (and across the 
NHS) and that minor adaptions outside the core intervention allow individual site integration. 
We will then run and record scenario simulations at four mMOET training centres, building on pre-
existing scenarios (e.g. post-partum haemorrhage and sepsis). Simulations will include a mixture of 
medical staff and midwives at representative levels of experience (8 in each of the implementation sites). 
Findings will inform escalation protocol review prior to repeat (potentially adjusted) scenario testing. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 24 of 46 
From our prior experience of this process, we estimate two iterations with decreasing participant time 
will be required to finalise recommended escalation protocols.  
8 
PARTICIPANT IDENTIFICATION 
8.1 
Study participants 
8.1.1 
Patient Retrospective cohort 
All women who are treated by maternity services at study sites. 
8.1.1.1 
Overall Inclusion criteria 
• 
All women aged 16 or over who are pregnant  
• 
At any of the study sites  
8.1.1.2 
Overall Exclusion criteria  
• 
Patients who have requested that their data not be used for research (e.g., NHS Opt-out, see 
Section 9.3.1) 
The model development dataset sits within this cohort. 
The case review dataset sits within this cohort. We will extract a stratified sample of 100 records for case 
review identified by our new criteria. The case review data set will comprise records: 
• 
At one of three partner sites 
• 
Electronically coded to meet a pre-near-miss/near-miss criterion. 
The maternal serious incident dataset sits within this cohort. We will review up to 100 maternal serious 
incidents recorded in local serious event systems at the three partner hospitals not identified by our new 
electronic criteria. The maternal serious incident data set will comprise records: 
• 
At one of three partner sites 
• 
Not electronically coded to meet a pre-near-miss criterion. 
• 
Recorded in local serious incident reporting systems. 
• 
With a clinical rather than “operational” serious incident 
8.1.2 
Patient prospective cohort 
All women who are treated by maternity services at study sites. 
The model validation and real-time platform testing datasets sit within this cohort. 
8.1.2.1 
Inclusion criteria 
• 
All women aged 16 or over who are pregnant.  
• 
At any of the study sites  
8.1.2.2 
Exclusion criteria 
• 
Patients who have requested that their data not be used for research (e.g., NHS Opt-out, see 
Section 9.3.1) 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 25 of 46 
8.1.3 
MBRRACE-UK cohort 
All cases of maternal death and morbidity (for the cohorts described above) collected by MBRRACE-UK 
(see (65)). This cohort will be used in WS1 and WS2b. 
8.1.3.1 
Inclusion criteria 
• 
See (65) 
8.1.3.2 
Exclusion criteria 
• 
See (65) 
8.1.4 
Patient interviews/Focus groups(escalation pathway) 
Women and their partners who have experienced near-miss events or other maternal complications.  
8.1.4.1 
Inclusion criteria 
• 
Women and their partners aged 16 or over who have experienced near-miss events or other 
maternal complications. 
8.1.4.2 
Exclusion criteria 
• 
Women or partners who do not consent.  
8.1.5 
Staff interviews/focus groups (escalation pathway) 
Staff members will be involved in focus groups, interviews and testing the eMOEWS user interface. 
8.1.5.1 
Inclusion criteria 
• 
Staff members aged 16 or over in a partner hospital. 
• 
Involved in the care, management and escalation of hospitalised women who deteriorate in 
maternity services. 
8.1.5.2 
Exclusion criteria 
• 
Staff who do not consent 
8.1.6 
Staff interviews/focus groups (eMOEWS interface development) 
Staff members will be involved in focus groups, interviews and testing the eMOEWS user interface. 
8.1.6.1 
Inclusion criteria 
• 
Staff members aged 16 or over in a partner hospital. 
• 
Involved in the care, management and escalation of hospitalised women who deteriorate in 
maternity services. 
8.1.6.2 
Exclusion criteria 
• 
Staff who do not consent 
8.1.7 
Staff training (simulation scenarios) 
Staff will also take part in scenario simulations (MOET). 
8.1.7.1 
Inclusion criteria 
• 
Staff members aged 16 or over in a partner hospital. 
• 
Involved in the care, management and escalation of hospitalised women who deteriorate in 
maternity services. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 26 of 46 
8.1.7.2 
Exclusion criteria 
• 
Staff who do not consent 
9 
PROTOCOL PROCEDURES  
9.1 
Recruitment  
9.1.1 
Patients 
For the retrospective and prospective cohorts, we will apply for Health Research Authority approval 
(HRA), under advice from the Research Ethics Committee and the Confidentiality Advisory Group (CAG) 
to allow access to confidential medical records without specific written consent (Section 251 support). 
9.1.2 
Staff and public participants 
For medical staff we will use purposive sampling to ensure a diverse range of clinical experience, 
backgrounds, and training. For example, we will work with medical and midwifery leads, and local 
champions at each trust to identify and invite eligible staff members to participate, along with key NHS IT 
personnel. We will then snowball from these participants to ensure we are reaching all relevant staff 
groups. 
For public participants we will identify interested women and their partners using posters in our partner 
hospitals and through our contacts with trusted community support and patient groups. 
9.2 
Screening and Eligibility Assessment 
9.2.1 
Patients 
• 
Retrospective and prospective cohorts - there will be no screening in this study. 
• 
Case review cohort – a stratified random sample of cases for review will be computer-generated 
from each of three partner sites using WP2 pre-anonymisation datasets (post application of the 
NHS opt-out and patients who have made requests to study sites for their data to be excluded). 
• 
Maternal serious incident cohort – serious incident databases at partner hospitals will be 
screened for women with clinical maternal serious incident reports. 
 
9.2.2 
Staff and public participants 
• 
No screening will occur for staff or public participants – with recruitment occurring as described 
above. 
9.3 
Informed Consent 
9.3.1 
Patients/Public participants 
For the retrospective and prospective cohorts, the case review cohort, and the maternal serious incident 
cohort we will apply for Health Research Authority approval (HRA), under advice from the Research 
Ethics Committee and the Confidentiality Advisory Group (CAG) to allow access to confidential medical 
records without specific written consent (Section 251 support). 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 27 of 46 
For the MBRRACE cohorts approvals to use anonymous data for research are already in place. 
For public participants in interviews or focus groups, informed consent will also be taken. Women and 
partners who contact us through our posters or through our written communications or presentations to 
trusted community support and patient groups will be approached by telephone or in person by an 
experienced researcher who will explain the purpose of the interviews within the programme of research 
and provide public participant information. Public participant study documentation will detail the 
content of the interviews and what interviews will involve for the participant. It will be clearly stated that 
the participant is free to withdraw from the study at any time for any reason without prejudice to future 
care, without affecting their legal rights, and with no obligation to give the reason for withdrawal. The 
participant(s) will personally sign and date the latest approved version of the Informed Consent form. 
The participant will be given sufficient time up to 28 days to consider the information, and the 
opportunity to question the Researcher, or other independent parties to decide whether they will 
participate in the study. Written Informed Consent will then be obtained by means of participant-dated 
signature and dated signature of the researcher who presented and obtained the Informed Consent 
Declaration. The person who obtained the consent will be suitably trained and will have been authorised 
to do so by the Chief Investigator. A copy of the signed Informed Consent declaration will be given to the 
participant. The original signed form will be retained at the study site. 
Face to face interviews will be held in a quiet meeting room. Videoconferencing (using Microsoft Teams 
or Zoom software) or telephone interviews will be conducted in a private office. The audio from Face to 
face, telephone and video conferencing interviews will be recorded to allow for transcription. 
 
9.3.2 
Staff 
For each of eMOEWS interface development, escalation development and simulation scenarios, staff will 
undergo informed consent. Staff identified through the medical and midwifery leads, and local 
champions will be approached to take part by telephone or in person by an experienced researcher who 
will explain the purpose of the interface development groups focus groups, interviews and trialling, 
escalation focus group or scenario simulation within the programme of research and provide staff 
participant information. Study staff participant study documentation will detail the content of the 
component and what the component will involve for the participant. It will be clearly stated that the 
participant is free to withdraw from the study at any time for any reason without prejudice to future 
care, without affecting their legal rights, and with no obligation to give the reason for withdrawal. The 
participant(s) will personally sign and date the latest approved version of the Informed Consent form or 
Consultant. The participant will be given sufficient time up to 28 days to consider the information, and 
the opportunity to question the Researcher, or other independent parties to decide whether they will 
participate in the study. Written Informed Consent will then be obtained by means of participant-dated 
signature and dated signature of the researcher who presented and obtained the Informed Consent 
Declaration. The person who obtained the consent will be suitably trained and will have been authorised 
to do so by the Chief Investigator. A copy of the signed Informed Consent declaration will be given to the 
participant. The original signed form will be retained at the study site. 
Participants will be interviewed or join focus groups at a convenient time and location to them. 
Participants will privately confirm their identity to the researcher (e.g., full name, date of birth) at the 
start of the interview/focus group.  

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 28 of 46 
Face to face interviews/focus groups will be held either in a meeting room away from the clinical area or 
in a quiet room within the clinical care, as chosen by the participant. Videoconferencing (using Microsoft 
Teams or Zoom software) or telephone interviews will be conducted in a private office. Face to face, 
telephone and video conferencing interviews will be recorded to allow for transcription. 
Audio video recordings of participants may be required to track how participants interact with test user 
interfaces during eMOEWS interface development.  
Audio visually recordings of participants during eMOEWS simulation scenarios will occur in four different 
training centres across the UK. The video recorded from the four centres will then be sent to a central 
location. Recordings will then be analysed by at least two members of the research team.  
9.4 
Randomisation 
There will be no randomisation in this study. 
9.5 
Blinding and code-breaking  
There is no blinding or code-breaking in the study. 
9.6 
Description of study intervention(s), comparators and study procedures (clinical) 
There is no intervention in this study. 
9.7 
Baseline Assessments 
Not applicable. 
9.8 
Subsequent Visits 
Not applicable. 
9.9 
Sample Handling  
No sample will be taken during the course of this study. 
9.10 Early Discontinuation/Withdrawal of Participants 
This is an observational study. Participants can request their data to be deleted at any time in accordance 
with GDPR and the study privacy policy. 
9.11 Definition of End of Study 
The end of study described in this protocol is the point at which all the study data has been entered, the 
eMOEWS built and checked and any post-implementation changes to eMOEWS algorithm have been 
added to the final derived and validated eMOEWS. 
10 SAFETY REPORTING  

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 29 of 46 
This is a non-interventional study, so safety reporting is not applicable.  
11 STATISTICS AND ANALYSIS 
11.1 Statistical Analysis Plan (SAP) 
The statistical aspects of the study are summarised here, with details fully described in a statistical 
analysis plan that will be publicly available from the time the first retrospective dataset is extracted from 
a minimum of 3 hospitals. The SAP will be finalised before any statistical analysis takes place.  
11.2 Description of the Statistical Methods  
11.2.1 Workstream 1 
We will report descriptive statistics describing the proportion of women meeting each of the new criteria 
and in total in WS2 retrospective datasets – comparing these proportions relative to the hospital 
populations with published rates for our individual criteria, 
We will use descriptive statistics to report the proportions of events correctly captured by our new 
electronic definitions in the stratified sample cohort and events missed in our review of hospital systems 
for recording serious events. 
Similarly, from MBRRACE-UK we will report the proportions of women who meet the new definitions. 
 
11.2.2 Workstream 2a 
We will describe the characteristics of the study population and compare to national figures to ensure it 
is appropriately diverse and representative. To assess the predictive performance of each 
MOEWS/MEWS (11), we will report sensitivity and specificity at pre-defined thresholds, and overall score 
discrimination (c-statistic). Calibration, the level of agreement between observed outcomes and a 
model’s predictions, is a key aspect of assessing predictive performance for prognostic models. However, 
a weakness of existing MOEWS is that they generally do not provide an estimate of risk, therefore, 
assessment of calibration will not be possible. We will, however, calculate the risk of our key events 
according to each level of MOEWS. We will also compare how early each MOEWS alerts prior to events. 
We will use the new multi-site dataset to calibrate the centile-based national MOEWS we developed 
with NHSE/I. 
We will also explore whether MOEWS performance varies between key sub-groups (e.g., age, ethnicity), 
to identify where groups could be disadvantaged, or where the MOEWS could be improved. We will 
report the rates of missing vital sign data and use different approaches to account for these missing 
values during analysis (e.g., complete cases, assuming all missing values are normal or abnormal, and 
multiple imputation), to check the robustness of our estimates. 
11.2.3 Workstream 2b 
We will summarise key characteristics of women who experience these events, including demographics, 
clinical characteristics, vital signs measurements, and type of event. We will describe when women in 
MBRRACE-UK meet the new outcome criteria, prior to the extreme outcomes that result in inclusion in 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 30 of 46 
the MBRRACE-UK dataset. For each existing MOEWS/MEWS, we will calculate sensitivity at pre-defined 
thresholds and assess how early women alert prior to the events occurring, if at all. Where sufficient 
data exist, we will examine performance in key subgroups. Missing vital sign data will be handled 
similarly to WS2a. 
11.2.4 Workstream 3b 
11.2.4.1 Model development 
We will explore both regression and machine learning (ML) approaches for model development. We 
anticipate at least 5800 ‘near-miss’ or ‘severe morbidity’ events, with higher rates using our new 
definitions.  
For approaches using  regression models (e.g., logistic or Cox regression if censoring is an issue) we will 
assess pre-defined pairwise interactions and include non-linear relationships for continuous predictors 
(e.g., using fractional polynomials or restricted cubic splines) (66). Pairwise interactions may be used to 
address apparent discrepancies in model performance for specific subgroups, identified in WS2. We will 
investigate whether accounting for trends of repeated measurements within individuals (e.g., blood 
pressure) can be incorporated to improve predictive accuracy. We will additionally explore how 
performance varies when using a composite outcome versus developing different models for individual 
outcomes (e.g., sepsis, pre-eclampsia). This may be informed by the results from WS2. Where 
appropriate, we will consider  using a least angle selection and shrinkage operator (LASSO) penalty (67). 
In parallel we will investigate and compare ML approaches to developing a prediction model, including 
methods such as random forests, deep neural networks, Bayesian Gaussian processes and gradient-
boosted decision tree survival models (37,68). Such models allow consideration of interactions between 
available variables providing “latent variables” (complex, non-linear transformations of the original input 
variables) that may improve prediction performance over the non-transformed input variables. We will 
prioritise principled, probabilistic methods that permit the incorporation of prior clinical knowledge, such 
that results are “interpretable”, avoiding the “black box” nature of much ML-oriented research in this 
area (68,69) . 
MOEWS are typically calculated several times during a woman’s pregnancy, and other candidate 
variables will also be measured repeatedly (for example, blood tests). The best way of accounting for 
these repeated measures in model development (and validation) is not well defined. Therefore, we will 
explore different options for how best to incorporate this. 
Where applicable learning from the regression and machine learning approaches will be combined to 
achieve a final deliverable model. 
11.2.4.2 Missing data 
Missing data will occur in all medical research with not all patients providing data on all predictors of 
interest. To avoid excluding patients and reducing the sample size, multiple imputation will be used to 
impute missing values, under a missing at random assumption. 
Identifying the true underlying missing data mechanism from the available data is rarely possible. 
Assumptions need to be made on the plausible mechanism, and approaches needed to be used. Under a 
missing completely at random mechanism (MCAR), the missingness mechanism does not depend on 
unobserved (unseen) data. Carrying out a complete case analysis will produce unbiased estimates (but 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 31 of 46 
with a loss precision if full data are observed) but will reduce sample size. Under the missing at random 
(MAR) assumption, the missingness after conditioning on the observed data does not depend on the 
unobserved (unseen) data. Under this approach, we can apply approaches such as multiple imputation, 
by fitting a joint model to the observed data and impute the missing data, taking account of the 
uncertainty in the estimated parameters of this joint model. We feel this, MAR, approach makes a less 
strong and more realistic assumption than the MCAR approach. The MAR imputation model will include 
all variables considered for the multivariable model building, the outcome and any auxiliary variables 
that will help explain the missingness. When imputing missing values for predictors that are repeatedly 
measured (for example, vital signs) we will use methods that account for (and take advantage of) this 
structure in the imputation model. The assumption of a missing not at random (MNAR) approach whilst 
not implausible is considerably more complex to investigate – there is a dearth of research investigating 
MNAR in the context of prediction model research.  We will nevertheless explore whether the MAR 
assumption holds by comparing the imputed values (after accounting for the observed values) and the 
missing values to identify if there are any systematic differences to suggest a MNAR assumption. We will 
explore patterns of missing data in key sub-groups (e.g., ethnicity) to detect any potential bias in data 
recording. We will also explore whether patterns of missingness (e.g. clinicians electing to perform 
specific blood tests) could be utilised to aid prediction (70). 
 
11.2.4.3 Assessment of model performance 
The performance of both new models will be assessed through internal and external validation. The 
performance of a model in the development data is usually over-estimated. Internal validation 
techniques will be used to adjust for this ‘optimism’. The internal validity of the final models will also be 
assessed by the bootstrap re-sampling or k-fold cross validation techniques to adjust for over-optimism 
in the estimation of model performance. To ensure a meaningful comparison we will use the same 
internal validation procedures to compare statistical and machine learning models. The internal 
validation will quantify and be used to adjust the performance measures (e.g., discrimination, 
calibration) for any optimism. Heterogeneity in model performance will be explored across different 
hospitals using internal-external cross-validation (71). 
 
We will use the performance metrics used in WS2 (i.e., discrimination, sensitivity, specificity, and the 
degree of advanced warning prior to an event). As discussed above, the performance of a prediction 
model is typically characterised by assessing both its calibration and discrimination. With our new 
models we will be able to assess model calibration as the new development models will provide risk 
estimates. Calibration reflects how close the predictions from the model are to the observed outcome 
frequencies, so represents a major step forward in assessing eMOEWS performance. It is a key 
performance metric recommended in TRIPOD (Transparent reporting of a multivariable prediction model 
for individual prognosis or diagnosis) guidance (72). Calibration will be assessed graphically, using a 
calibration plot, plotting observed outcomes against predictions using smoothing techniques. The plot 
will also be supplemented with results for individuals grouped by similar probabilities (tenths) comparing 
the mean predicted probability to the mean observed outcome. Calibration will also be quantified by 
calculating the calibration slope and intercept. The discrimination of the prediction models will be 
summarised with the concordance statistic (equivalent to the Area Under Receiver Operating 
Characteristic curve) with 95% confidence interval. Validation will include assessing for variation in 
performance between key sub-groups, as per WS2. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 32 of 46 
We will explore risk thresholds to identify high risk (such that we predict poor outcome) and low risk 
(such that we predict good outcome) groups. We will compare performance of our models to those of 
published MOEWSs established in WS2a, including decision curve analysis. We will externally validate our 
models using new data from our collaborator sites collected in the two years following the derivation 
cohort. We will assess the model performance within key pre-defined sub-groups (including maternal 
age and ethnicity). 
11.3 Sample Size Determination  
11.3.1 Workstream 2a 
We estimate the WHO “near-miss” event rate will be at least 10/1000 (2,3)  and the severe morbidity 
event rate will be at least a further 10/1000 maternities (2,73) . Although rates of our less severe new 
‘pre-near-miss’ will be higher we have calculated our sample size based on these severe event rates 
alone. The retrospective cohort will include four years and two months' worth of data from up to 12 
sites. We anticipate that there will be 5875 births on average per site per year. Therefore, we will have 
approximately 293,200 births in the retrospective cohort, of whom 2932 will have a near-miss and at 
least a further 2932 will have pre-near-miss events. Guidance suggests that at least 100 events are 
required for model validation (74). Therefore, our sample size (which is driven both by the need for 
adequate representation in subgroups and the requirements of WS3, see below) should be more than 
sufficient to generate robust performance metric estimates, including in key subgroups and for the 
secondary outcomes (see model performance in key subgroups WS3b below – note that the 
retrospective cohort is larger than the prospective cohort and therefore the number of events per 
subgroup will be even higher). 
11.3.2 Workstream 3b 
We will use the full augmented WS2a data set (the retrospective cohort) for model development, which 
will be approximately 293,200 women. Assuming a conservative event rate of 2% (as in WS2a), a c-
statistic of 0.8, and a shrinkage factor of 0.9, we will be able to include more than 850 parameters in our 
model if using logistic regression (which is more than adequate) (75). 
After development we will assess the performance of our model in the prospective cohort within key 
sub-groups (including maternal age and ethnicity). The prospective cohort will include the births from up 
to 12 sites over a period of two years and four months, and therefore will be approximately 164,300 in 
total. The national MBRRACE report shows the proportion of women in the different ethnic groups 
(White 79.5%, Asian 10.3%, Black 4.3%, Chinese 4.1%, Mixed 1.8%) (17). Working with the NIHR HIC, we 
will ensure that we have at least this degree of representation in our research database. We would 
therefore expect event rates of greater than 100 (the minimum recommended in TRIPOD guidance and 
quoted papers) (74) for all groups other than “Mixed” (White 2611, Asian 338, Black 141, Chinese 134, 
Mixed 59) assuming a flat 2% event rate. If higher event rates are seen in the different groups as 
reported (relative risk Asian 1.8 95% CI (1.2-2.7), Black 4.9 (3.3-7.3), Mixed 2.0 (0.7-4.9)) (5) we will have 
over 200 events in major groups. 
11.3.3 Workstream 4b 
Transcripts from interviews will be read and re-read, a coding frame constructed, and the data coded. 
Anticipated and emergent themes will then be examined across the whole data set as well as in the 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 33 of 46 
context of each person’s interview. A qualitative interpretive approach will be taken, combining thematic 
analysis with constant comparison. NVIVO will be used to facilitate the analysis. 
11.3.4 Workstream 4c 
Recordings will be dual-coded. We will use the TOAsT framework to identify and categorise observable 
inter-professional teamwork behaviours that occur during the simulated clinical scenarios implementing 
eMOEWS and the escalation pathways. Anticipated and emergent themes will be examined across the 
whole data set and a qualitative interpretive approach taken, combining thematic analysis with constant 
comparison. 
11.4 Analysis populations 
The analysis populations will generally include all eligible participants in each study data set. 
11.5 Decision points  
Not applicable. 
11.6 Stopping rules 
Not applicable. 
11.7 The Level of Statistical Significance 
The level of statistical significance will be set at 5%. 
11.8 Procedure for Accounting for Missing, Unused, and Spurious Data. 
Missing data approaches are described as part of the statistical methods in Section 11.2, for each work 
package (where relevant). 
11.9 Procedures for Reporting any Deviation(s) from the Original Statistical Plan 
Any deviation from the statistical analysis plan will be described and justified in all study publications. 
11.10 Health Economics Analysis  
No health economic analyses are planned for in this study. 
12 DATA MANAGEMENT 
The plan for the data management of the study is outlined below. There is not a separate Data 
Management document in use for the study.  
12.1 Source Data 
Source documents are where data are first recorded, and from which participants’ data are obtained. 
These include, but are not limited to, hospital records (from which medical history and previous and 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 34 of 46 
concurrent medication may be summarised into the CRF), clinical and office charts, laboratory and 
pharmacy records, diaries, microfiches, radiographs, and correspondence. 
Creation of the NHS Trust data set requires accessing and linking confidential health care records. Access 
to these data will be subject to approval from a REC and Section 251 support from the CAG. All data will 
be pseudonymised prior to analysis. 
The following source data resources will be accessed without explicit consent: 
12.1.1 Workstream 1 
Hospital patient records will be the source data for record review from which data will be abstracted into 
limited piloted case reports. 
MBRRACE-UK will be the source for MBRRACE data (also in workstream2) 
12.1.2 Workstream 2/3b 
Electronic patient record data for datasets in workstream 2 and 3 will include both inpatient and 
outpatient data, with sources including: 
• 
Patient administrative records (e.g., admission times, medical/surgical speciality, age, gender, 
ethnicity, ward locations/transfers) 
• 
Diagnostic (ICD-10) and procedure coding (OPCS4) 
• 
Vital signs/early warning score recording systems (e.g., blood pressure, heart rate, early warning 
score) 
• 
Intensive care patient record systems 
• 
Laboratory management systems (e.g., haematology, biochemistry, pathology, and microbiology) 
• 
Electronic prescriptions/administrations 
• 
Point of care testing (e.g., blood gas analysis) 
• 
Electrocardiogram systems 
• 
Echo cardiogram systems 
• 
Diagnostic imaging data reports 
• 
Peripartum monitoring (e.g. Cardiotocography) 
• 
Maternal records from maternity electronic record systems (including the digital “red book”) and 
local maternity data sets submitted to NHS England (e.g., Maternity Services Data Set), including 
information on current and previous pregnancies and maternal (e.g., weight/height/BMI, 
smoking status) and neonatal outcomes/characteristics (e.g., APGAR, requirement for additional 
care) 
 
12.2 Access to Data 
Direct access will be granted to authorised representatives from the Sponsor and host institution for 
monitoring and/or audit of the study to ensure compliance with regulations. 
Creation of the NHS Trusts data set requires accessing and linking confidential health care records. All 
members of the data extraction team involved in this process will undergo appropriate local governance 
training. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 35 of 46 
A pseudonymised data set will be accessed by authorised members of the study team at the University 
of Oxford for analysis purposes. Excluding age, sex, and ethnicity (necessary to investigate model bias) 
we will not be accessing data on protected characteristics (e.g., religion, civil status, sexual orientation). 
12.3 Data transfer and analysis environment 
Pseudonymised data will either be analysed within Oxford University Hospitals NHS Foundation Trust 
Secure Data environment (or equivalent) or the Data Safe Haven within the University of Oxford’s Critical 
Care Research Group (DSH-CCRG). The choice of analysis environment will depend on where appropriate 
computing resources are available. All study data transferred to the CCRG-DSH will be encrypted using 
AES-256 before being transferred using either the secure OUH file download service or the OUH Citrix 
environment.  
Whilst stripped of all direct identifiers, all pseudonymised data will still be considered personal, 
controlled sensitive data, as it will still contain indirect identifiers such as date-time stamps. Data-time 
stamps are required to calculate the study’s primary outcome.  
In the case pseudonymised data are transferred to the CCRG-DSH, they will be loaded onto specialist 
secure hardware held and maintained by the CCRG. The CCRG-DSH is a computing environment designed 
to store and analyse complex datasets in a manner that is safe and secure. The CCRG-DSH conforms to 
NHS Digital Security Toolkit and Cyber Essentials Plus accreditation. The environment is designed so that 
patient level data never leaves the analysis environment. 
12.4 Data Recording and Record Keeping  
All study data and electronic correspondence will be stored securely on Trust or University password-
protected systems as appropriate only accessible by study staff and authorised personnel. Any paper-
based documentation will be kept in the Kadoorie Centre in the research area, behind two access-
controlled doors and in locked filing cabinets. All documentation will be archived at the end of the 
project. Any paper correspondence will be archived at an off-site secure archive facility.  
12.4.1 Electronic data 
All records will be subject to quality assurance polices both at the University and research group level. 
These are designed to guarantee the accuracy and validity of the study data. All participants will be 
identified by a unique study number (pseudonymous study key). Participants names and any other direct 
identifiers will NOT be included in any study data electronic file used for analysis.  
For the retrospective and prospective cohorts, the ledger(s) that link the source Clinical Information 
System (CIS) data and the unique study specific number will be held within secure NHS Trust systems and 
destroyed at the end of the study period. All study data will be stored in a dedicated storage facility and 
retained for a minimum of 25 years after the end of the study in keeping with current MRC guidance. 
Personal data such as contact details that could identify a participant will be destroyed as soon as it is 
practical to do so and no later than 12 months after the end of the study. 
 
12.4.2 Staff/Patient interviews/focus groups/simulation scenarios 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 36 of 46 
For each sub-study, an electronic study log of baseline data will be maintained. This data will be entered 
and validated by the researchers. Transcripts (conducted internally by the research team) from 
interviews/focus groups/simulations audio files will be stored and processed in NVIVO (Lumivero.com). 
The name and any other identifying detail will NOT be included in any study data electronic file. Audio 
files will be destroyed once transcription is completed and checked. 
 
All interview/focus group data will be securely stored and password protected. The video footage will be 
held for up to 3 years after publication of the work. 
13 QUALITY ASSURANCE PROCEDURES 
The study may be monitored, or audited in accordance with the current approved protocol, GCP, 
relevant regulations and standard operating procedures.  
13.1 Risk assessment  
We will undertake and document a formal risk assessment of the project for each component of the 
project. Risks and management strategies for all participants in interviews/focus groups/simulations will 
be considered and RAG-rated with mitigation strategies prior to each component of work being 
undertaken. 
In the retrospective and prospective datasets, researchers will not interact directly with patients or 
intervene in their care. 
This study requires access to confidential patient records. Eligible patient records will be identified by the 
participating organisations. Directly identifiable data will only be required for record linkage and will not 
be available to the Sponsor. 
To mitigate the risk of reidentification of participants and the risk of data loss we will undertake the 
following: 
• 
All records will be accessed and de-identified at each participating site, using a dedicated 
computer that will conform to NHS information security standards. 
• 
Only pseudonymous personal data will be transferred via secure/encrypted protocols to the 
coordinating centre (Critical Care Research Group (CCRG), Nuffield Department of Clinical 
Neurosciences, Oxford University). 
• 
Only pseudonymous personal data will be held by the CCRG. 
• 
Pseudonymous personal data will be held inside the Sponsors “Data Safe Haven” which conforms 
to the same NHS standards of information security and cyber security. 
Where clinicians trial the study system using real patient data of patients for whom they are caring they 
will follow standard clinical practice where new insight occurs. 
13.2 Study monitoring  
All research team members will be fully trained in Information Governance, data protection and 
confidentiality. The study may be monitored, or audited in accordance with the current approved 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 37 of 46 
protocol, GCP, relevant regulations and standard operating procedures. 
 
13.3 Study Committees  
 
13.3.1 Study management group (SMG) 
The study management group will consist of the Chief Investigator, Lead Investigator and the named 
investigators listed under Key Contacts, and individuals directly funded by the project. The study 
management group will be primarily responsible for the running and conduct of the study. They will be 
responsible for ensuring that standard operating procedures are followed and that regulations are 
adhered to. Where appropriate public patient involvement will be gained in any changes or amendments 
that are needed during the study. 
13.3.2 Project Oversight Group (POG) 
A project oversight group will include an independent chair, at least two other independent members, a 
PPI representative(s), and the Chief investigator. The POG will review the progress of the research 
programme and report on progress to the funder. 
14 PROTOCOL DEVIATIONS  
A study related deviation is a departure from the ethically approved study protocol or other study 
document or process (e.g., consent process or administration of study intervention) or from Good 
Clinical Practice (GCP) or any applicable regulatory requirements. Any deviations from the protocol will 
be documented in a protocol deviation form and filed in the study master file.  
15 SERIOUS BREACHES 
A “serious breach” is a breach of the protocol or of the conditions or principles of Good Clinical Practice 
which is likely to affect to a significant degree – 
 (a) the safety or physical or mental integrity of the trial subjects; or 
(b) the scientific value of the research. 
If a serious breach is suspected the Sponsor must be contacted within one working day. In collaboration 
with the C.I., the serious breach will be reviewed by the Sponsor and, if appropriate, the Sponsor will 
report it to the approving REC committee and the relevant NHS host organisation within seven calendar 
days.  
16 ETHICAL AND REGULATORY CONSIDERATIONS 
16.1 Declaration of Helsinki 
The Investigator will ensure that this study is conducted in accordance with the principles of the 
Declaration of Helsinki. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 38 of 46 
16.2 Guidelines for Good Clinical Practice 
The Investigator will ensure that this study is conducted in accordance with relevant regulations and with 
Good Clinical Practice. 
16.3 Approvals 
Following Sponsor approval, the protocol, informed consent form, participant information sheet and any 
proposed advertising material will be submitted to an appropriate Research Ethics Committee (REC), and 
HRA (where required) and host institutions for written approval. 
The Investigator will submit and, where necessary, obtain approval from the above parties for all 
substantial amendments to the original approved documents. 
16.4 Other Ethical Considerations 
The main ethical issues occur in a) use of confidential patient data without explicit consent, b) patient 
interviews, c) staff interviews.  
16.4.1 Use of confidential patient data without explicit consent  
16.4.1.1 Issue 
The study requires a large, unbiased representative population to develop and test our predictive 
models. The acceptability of using patient data without consent for this study was discussed with patient 
and public representatives.  
16.4.1.2 Steps taken 
The patient notification statement and the data privacy statement were discussed with a PPI group. 
These documents will help to ensure patients admitted to participating NHS Trusts are aware that their 
data is being used for the purposes of this study and are provided with information on how to opt out. 
An application will be made to the Confidentiality Advisory Group (CAG) to access to relevant patient 
health records of patients that meet the study eligibility criteria.  
16.4.2 Interviews with patients 
16.4.2.1 Issues 
Access, exploring issues which may cause distress to participants, uncovering poor practice that requires 
action.  
16.4.2.2 Steps taken 
We will address access issues by also offering telephone interviews as an alternative to face-to-face 
interviews. Where poor practice is identified, the researcher has a duty to report this. Any concerns 
about individual practice will be discussed with the line manager of that individual, where they can be 
identified, or the manager of the clinical area. 
16.4.3 Interviews with staff  
16.4.3.1 Issues 
Access, uncovering poor practice which requires action, exploring issues which may cause distress to 
participants.  

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 39 of 46 
16.4.3.2 Steps taken 
Staff members may find it difficult to allocate time for interviews. This is particularly difficult for clinical 
staff who struggle to leave their clinical area during shifts and often work unsocial hours. As well as 
offering flexibility in timings of interviews, we will also offer telephone interviews as an alternative to 
face to face. This will allow all staff members to participate if they wish. The project includes funded time 
for staff to participate. Where poor practice is identified, the researcher has a duty to report this. Any 
concerns about individual practice will be discussed with the line manager of that individual, where they 
can be identified, or the manager of the clinical area. Occupational health will be made aware that we 
are conducting this study. Any participant who is distressed by topics discussed in the interviews will be 
signposted to occupational health. They will also be encouraged to discuss their concerns with their line 
manager or seek support from other colleagues where appropriate. 
16.5 Reporting 
The CI shall submit once a year throughout the study, or on request, an Annual Progress report to the 
REC Committee, HRA (where required) host organisation, Sponsor, and funder (where required). In 
addition, an End of Study notification and final report will be submitted to the same parties.  
16.6 Transparency in Research  
The study will be registered on a publicly accessible database.  
Where the study has been registered on multiple public platforms, the study information will be kept up 
to date during the study, and the CI or their delegate will upload results to all those public registries 
within 12 months of the end of the study.  
16.7 Participant Confidentiality 
The study will comply with the General Data Protection Regulation (GDPR) and Data Protection Act 2018, 
which require data to be de-identified as soon as it is practical to do so. The processing of the personal 
data of participants will be minimised by making use of a unique participant study number only on all 
study documents and any electronic database(s). All documents will be stored securely and only 
accessible by study staff and authorised personnel. The study staff will safeguard the privacy of 
participants’ personal data. 
16.8 Expenses and Benefits 
We will send a card (that will also include a £25.00 voucher) thanking patient/public interview or focus 
group participants for taking part in the study at the completion of their involvement. The card will 
include the public-facing study website details to increase opportunities for participants to hear about 
the final project results. The contribution of participants will be recognised in all publications, 
presentations and publicity associated with the study. 
FINANCE AND INSURANCE 
16.9 Funding 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 40 of 46 
This study is funded by the National Institute for Health and Care Research (NIHR), Programme: 
Programme Grants for Applied Research, Call: PGfAR Competition 37 Stage 2 Panel A, Reference number: 
NIHR204430. 
16.10 Insurance 
The University has a specialist insurance policy in place which would operate in the event of any 
participant suffering harm as a result of their involvement in the research (Newline Underwriting 
Management Ltd, at Lloyd’s of London).  
16.11 Contractual arrangements  
Appropriate contractual arrangements will be put in place with all third parties.  
17 PUBLICATION POLICY 
The Investigators will be involved in reviewing drafts of the manuscripts, abstracts, press releases and 
any other publications arising from the study. Authors will acknowledge that the study was funded by 
the National Institute for Health and Care Research (NIHR). Authorship will be determined in accordance 
with the ICMJE guidelines and other contributors will be acknowledged. 
18 DEVELOPMENT OF A NEW PRODUCT/ PROCESS OR THE GENERATION OF INTELLECTUAL 
PROPERTY 
Ownership of IP generated by employees of the University vests in the University. The University will 
ensure appropriate arrangements are in place as regards any new IP arising from the study.  
19 ARCHIVING 
At the end of the study, electronic files containing the anonymised retrospective and prospective study 
datasets will be stored and retained in the CCRG, (University of Oxford) on secure severs (referred to as 
the Data Safe Haven). This environment stores data in a manner that it can be securely analysed without 
removing or copying the data from its systems. The data will remain here for 25 years in keeping with the 
MRC Research Framework for studies involving pregnant participants. After the archiving period has 
ended, the paper documents and electronic files will be confidentially and securely destroyed in line with 
University of Oxford guidelines. 
The video footage from the simulation scenarios will be held for up to 3 years after publication of 
the work. 
20 REFERENCES 
1. 
Knight M, Bunch K, Felker A, Patel R, Kotnis R, Kenyon S, et al. Saving Lives, Improving Mothers’ 
Care Core Report - Lessons learned to inform maternity care from the UK and Ireland Confidential 
Enquiries into Maternal Deaths and Morbidity 2019-21. Oxford: National Perinatal Epidemiology 
Unit, University of Oxford; 2023.  
2. 
Waterstone M, Bewley S, Wolfe C, Bewley S, Wolfe C. Incidence and Predictors of Severe 
Obstetric Morbidity: Case-Control Study. Br Med J. 2002 Mar;57(3):139–40.  

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 41 of 46 
3. 
Shields LE, Wiesner S, Klein C, Pelletreau B, Hedriana HL. Use of Maternal Early Warning Trigger 
tool reduces maternal morbidity. Am J Obstet Gynecol. 2016 Apr;214(4):527.e1-527.e6.  
4. 
Jardine J, Walker K, Gurol-Urganci I, Webster K, Muller P, Hawdon J, et al. Adverse pregnancy 
outcomes attributable to socioeconomic and ethnic inequalities in England: a national cohort 
study. Lancet. 2021 Nov;398(10314):1905–12.  
5. 
Knight M, Bunch K, Tuffnell D, Patel R, Shakespeare J, Kotnis R, et al. Saving Lives, Improving 
Mothers’ Care - Lessons learned to inform maternity care from the UK and Ireland Confidential 
Enquiries into Maternal Deaths and Morbidity 2017-19. Oxford Natl Perinat Epidemiol Unit, Univ 
Oxford 2021. 2021;  
6. 
Pattinson R, Say L, Souza JP, Broek N van den, Rooney C, WHO Working Group on Maternal 
Mortality and Morbidity Classifications. WHO maternal death and near-miss classifications. Bull 
World Health Organ. 2009 Oct;87(10):734.  
7. 
Centers for Disease Control and Prevention. Severe maternal morbidity in the United States 
[Internet]. Available from: 
http://www.cdc.gov/reproductivehealth/maternalinfanthealth/severematernalmorbidity.html 
8. 
Nair M, Kurinczuk JJ, Knight M. Establishing a National Maternal Morbidity Outcome Indicator in 
England: A Population-Based Study Using Routine Hospital Data. Ciccozzi M, editor. PLoS One. 
2016 Apr 7;11(4):e0153370.  
9. 
The WHO Near-Miss approach for Maternal Health. The WHO Near-Miss approach for Maternal 
Health [Internet]. 2011. p. 1–34. Available from: 
https://apps.who.int/iris/bitstream/handle/10665/44692/9789241502221_eng.pdf 
10. 
Smith GB, Isaacs R, Andrews L, Wee MYK, van Teijlingen E, Bick DE, et al. Vital signs and other 
observations used to detect deterioration in pregnant women: an analysis of vital sign charts in 
consultant-led UK maternity units. Int J Obstet Anesth. 2017 May;30:44–51.  
11. 
Smith V, Kenny LC, Sandall J, Devane D. Physiological track-and-trigger/early warning systems for 
use in maternity care. Cochrane Database Syst Rev. 2019 Jun 10;2021(9).  
12. 
Swanton RDJ, Al-Rawi S, Wee MYK. A national survey of obstetric early warning systems in the 
United Kingdom. Int J Obstet Anesth. 2009;18:253–7.  
13. 
Knight M, Joseph KS. Severe maternal morbidity and maternal mortality: A need for consensus on 
concepts and prevention efforts. Paediatr Perinat Epidemiol. 2020;34(4):377–8.  
14. 
Escobar GJ, Soltesz L, Schuler A, Niki H, Malenica I, Lee C. Prediction of obstetrical and fetal 
complications using automated electronic health record data. Am J Obstet Gynecol. 
2021;224(2):137-147.e7.  
15. 
Rance S, McCourt C, Rayment J, MacKintosh N, Carter W, Watson K, et al. Women’s safety alerts 
in maternity care: is speaking up enough? BMJ Qual Saf. 2013 Apr;22(4):348–55.  
16. 
Easter A, Sandall J, Howard LM. Obstetric near misses among women with serious mental illness: 
data linkage cohort study. Br J Psychiatry. 2021;(March 2016):1–7.  
17. 
MBRRACE-UK. Saving lives, improving mother’s care [Internet]. 2020. Available from: 
https://www.npeu.ox.ac.uk/assets/downloads/mbrrace-uk/reports/maternal-report-
2020/MBRRACE-UK_Maternal_Report_Dec_2020_v10_ONLINE_VERSION_1404.pdf 
18. 
Ockenden Report. Independent review of maternity services at Shrewsbury and Telford Hospital 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 42 of 46 
NHS Trust [Internet]. 2022. Available from: https://www.ockendenmaternityreview.org.uk/ 
19. 
Knight M, Bunch K, Vousden N, Banerjee A, Cox P, Cross-Sudworth F, et al. A National Cohort 
Study and Confidential Enquiry to Investigate Ethnic Disparities in Maternal Mortality. SSRN 
Electron J. 2021;  
20. 
Bell SF, Watkins A, John M, Macgillivray E, Kitchen TL, James D, et al. Incidence of postpartum 
haemorrhage defined by quantitative blood loss measurement: a national cohort. BMC Pregnancy 
Childbirth. 2020 Dec 6;20(1):271.  
21. 
NMPA Project Team. National Maternity and Perinatal Audit: Clinical Report 2021. Based on births 
in NHS maternity services in England, Scotland and Wales between 1 April 2017 and 31 March 
2018. London RCOG. 2021;  
22. 
Young MF, Oaks BM, Tandon S, Martorell R, Dewey KG, Wendt AS. Maternal hemoglobin 
concentrations across pregnancy and maternal and child health: a systematic review and meta-
analysis. Ann N Y Acad Sci. 2019 Aug 17;1450(1):47–68.  
23. 
Hinton L, Locock L, Knight M. Experiences of the quality of care of women with near-miss 
maternal morbidities in the UK. BJOG An Int J Obstet Gynaecol. 2014 Sep 18;121(s4):20–3.  
24. 
Hinton L, Locock L, Knight M. Partner Experiences of “Near-Miss” Events in Pregnancy and 
Childbirth in the UK: A Qualitative Study. Harris F, editor. PLoS One. 2014 Apr 9;9(4):e91735.  
25. 
healthtalk.org. Conditions that threaten women’s lives in childbirth & pregnancy [Internet]. [cited 
2022 Mar 17]. Available from: https://healthtalk.org/conditions-threaten-womens-lives-
childbirth-pregnancy/what-is-a-life-threatening-complication-in-pregnancy-and-childbirth 
26. 
New ambition to halve rate of stillbirths and infant deaths. New ambition to halve rate of 
stillbirths and infant deaths [Internet]. 2015. Available from: 
https://www.gov.uk/government/news/new-ambition-to-halve-rate-of-stillbirths-and-infant-
deaths 
27. 
Royal College of Physicians (London). National Early Warning Score (NEWS) 2: Standardising the 
assessment of acute-illness severity in the NHS. 2017.  
28. 
Green LJ, Kennedy SH, Mackillop L, Gerry S, Purwar M, Urias ES, et al. International gestational 
age-specific centiles for blood pressure in pregnancy from the INTERGROWTH-21st Project in 8 
countries: A longitudinal cohort study. PLoS Med. 2021;18(4):1–16.  
29. 
Green LJ, Mackillop LH, Salvi D, Pullon R, Loerup L, Tarassenko L, et al. Gestation-Specific Vital Sign 
Reference Ranges in Pregnancy. Obstet Gynecol. 2020 Mar;135(3):653–64.  
30. 
Knight M, Bunch K, Cairns A, Cantwell R, Cox P, Kenyon S, et al. Saving Lives, Improving Mothers’ 
Care Rapid Report 2021: Learning from SARS-CoV-2-related and associated maternal deaths in the 
UK June 2020- March 2021. Oxford Natl Perinat Epidemiol Unit, Univ Oxford. 2021;  
31. 
Knight M, Bunch K, Cairns A, Cantwell R, Cox P, Kenyon S, et al. Saving Lives, Improving Mothers’ 
Care Rapid Report: Learning from SARS-CoV-2-related and associated maternal deaths in the UK 
March – May 2020. Oxford Natl Perinat Epidemiol Unit, Univ Oxford 2020. 2020;  
32. 
Isaacs RA, Wee MYK, Bick DE, Beake S, Sheppard ZA, Thomas S, et al. A national survey of 
obstetric early warning systems in the United Kingdom: Five years on. Anaesthesia. 
2014;69(7):687–92.  
33. 
Mackintosh N, Watson K, Rance S, Sandall J. Value of a modified early obstetric warning system 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 43 of 46 
(MEOWS) in managing maternal complications in the peripartum period: an ethnographic study. 
BMJ Qual Saf. 2014 Jan;23(1):26–34.  
34. 
Benchimol EII, Smeeth L, Guttmann A, Harron K, Moher D, Peteresen I, et al. The REporting of 
studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. 
PLOS Med. 2015 Oct 6;12(10):e1001885.  
35. 
Churpek MM, Yuen TC, Winslow C, Robicsek AA, Meltzer DO, Gibbons RD, et al. Multicenter 
Development and Validation of a Risk Stratification Tool for Ward Patients. Am J Respir Crit Care 
Med. 2014 Sep 15;190(6):649–55.  
36. 
Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated Identification of Adults at 
Risk for In-Hospital Clinical Deterioration. N Engl J Med. 2020 Nov 12;383(20):1951–60.  
37. 
Pimentel MAF, Redfern OC, Malycha J, Meredith P, Prytherch D, Briggs J, et al. Detecting 
Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System. Am 
J Respir Crit Care Med. 2021 Jul 1;204(1):44–52.  
38. 
Green LJ, Pullon R, Mackillop LH, Gerry S, Birks J, Salvi D, et al. Postpartum-Specific Vital Sign 
Reference Ranges. Obstet Gynecol. 2021 Feb;137(2):295–304.  
39. 
von Dadelszen P, Payne B, Li J, Ansermino JM, Pipkin FB, Côté AM, et al. Prediction of adverse 
maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet. 
2011 Jan;377(9761):219–27.  
40. 
Albright CM, Ali TN, Lopes V, Rouse DJ, Anderson BL. The Sepsis in Obstetrics Score: A model to 
identify risk of morbidity from sepsis in pregnancy. Am J Obstet Gynecol. 2014;211(1):39.e1-
39.e8.  
41. 
Robbins T, Shennan A, Sandall J. Modified early obstetric warning scores: A promising tool but 
more evidence and standardization is required. Acta Obstet Gynecol Scand. 2019;98(1):7–10.  
42. 
Maternity DMA Report. NHS England. 2018. Maternity DMA Report. Available from: 
https://www.england.nhs.uk/publication/maternitydma-%0Areport-digital-maturity-assessment-
of-maternity-services-in-england-2018/ 
43. 
The NHS Long Term Plan. The NHS Long Term Plan [Internet]. Available from: 
https://www.longtermplan.nhs.uk/ 
44. 
NHS Digital. DCB3066 Digital Maternity Record Standard [Internet]. [cited 2022 Apr 5]. Available 
from: https://digital.nhs.uk/data-and-information/information-standards/information-standards-
and-data-collections-including-extractions/publications-and-notifications/standards-and-
collections/dcb3066-digital-maternity-record-standard 
45. 
Callaghan WM, MacKay AP, Berg CJ. Identification of severe maternal morbidity during delivery 
hospitalizations, United States, 1991-2003. Am J Obstet Gynecol. 2008 Aug;199(2):133.e1-133.e8.  
46. 
HDRUK. Phenotype library [Internet]. [cited 2022 Mar 17]. Available from: 
https://phenotypes.healthdatagateway.org/ 
47. 
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET 
Handbook: version 1.0. Trials. 2017 Jun 20;18(S3):280.  
48. 
Health Improvement Scotland. The Scottish Maternity Early Warning System (MEWS) [Internet]. 
2021. Available from: https://ihub.scot/media/7739/scottish-clinical-guide-national-mews.pdf 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 44 of 46 
49. 
Irish Department of Health. IMEWS (NCEC National Clinical Guideline No. 4 Version 2) [Internet]. 
2019 [cited 2022 Apr 5]. Available from: http://health.gov.ie/national-patient-safety-office/ncec/ 
50. 
Mhyre JM, D’Oria R, Hameed AB, Lappen JR, Holley SL, Hunter SK, et al. The maternal early 
warning criteria a proposal from the national partnership for maternal safety. Obstet Gynecol. 
2014;124(4):782–6.  
51. 
Bedford JP, Harford M, Petrinic T, Young JD, Watkinson PJ. Risk factors for new-onset atrial 
fibrillation on the general adult ICU: A systematic review. J Crit Care. 2019 Oct;53:169–75.  
52. 
Riley RD, Moons KGM, Snell KIE, Ensor J, Hooft L, Altman DG, et al. A guide to systematic review 
and meta-analysis of prognostic factor studies. BMJ. 2019 Jan 30;364:k4597.  
53. 
Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: A Tool to 
Assess the Risk of Bias and Applicability of Prediction Model Studies. Ann Intern Med. 2019 Jan 
1;170(1):51.  
54. 
Hayden JA, van der Windt DA, Cartwright JL, Côté P, Bombardier C. Assessing bias in studies of 
prognostic factors. Ann Intern Med. 2013;158(4):280–6.  
55. 
Zaal IJ, Devlin JW, Peelen LM, Slooter AJC. A Systematic Review of Risk Factors for Delirium in the 
ICU*. Crit Care Med. 2014/09/25. 2015;43(1):40–7.  
56. 
Dettmer MR, Damuth E, Zarbiv S, Mitchell JA, Bartock JL, Trzeciak S. Prognostic Factors for Long-
Term Mortality in Critically Ill Patients Treated with Prolonged Mechanical Ventilation: A 
Systematic Review. Crit Care Med. 2017;45(1):69–74.  
57. 
Militello LG, Hutton RJB. Applied cognitive task analysis (ACTA): A practitioner’s toolkit for 
understanding cognitive task demands. Ergonomics. 1998;41(11):1618–41.  
58. 
The Encyclopedia of Human-Computer Interaction, 2nd Ed [Internet]. Card Sorting.  
59. 
Dahella SS, Briggs JS, Coombes P, Farajidavar N, Meredith P, Bonnici T, et al. Implementing a 
system for the real-time risk assessment of patients considered for intensive care. BMC Med 
Inform Decis Mak. 2020 Dec 16;20(1):161.  
60. 
Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide to Designing Interventions. The 
Behavior Change Wheel. 2014. 1–46 p.  
61. 
Gill FJ, Leslie GD, Marshall AP. The Impact of Implementation of Family-Initiated Escalation of Care 
for the Deteriorating Patient in Hospital: A Systematic Review. Worldviews Evidence-Based Nurs. 
2016 Aug;13(4):303–13.  
62. 
Mackintosh N, Davis R, Easter A, Sevdalis N, Wilson S, Adams M, et al. Interventions to increase 
patient and family involvement in escalation of care for acute life-threatening illness in 
community health and hospital settings (Review). Cochrane Database Syst Rev. 2020;  
63. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation 
of health services research findings into practice: A consolidated framework for advancing 
implementation science. Implement Sci. 2009;4(1):1–15.  
64. 
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Innovations in Service 
Organizations: Systematic Review and Recommendations. Milbank Q. 2004 Dec;82(4):581–629.  
65. 
Knight M, Nair M, Tuffnell D, Shakespeare J, Kenyon S, Kurinczuk J. Saving Lives, Improving 
Mothers’ Care. Lessons learned to inform future maternity care from the UK and Ireland. Confid 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 45 of 46 
Enq into Matern Deaths Morb.  
66. 
Park Y, Hu J, Singh M, Sylla I, Dankwa-Mullan I, Koski E, et al. Comparison of Methods to Reduce 
Bias From Clinical Prediction Models of Postpartum Depression. JAMA Netw Open. 2021 Apr 
15;4(4):e213909.  
67. 
Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B. 1996;58:265–88.  
68. 
Shamout FE, Zhu T, Sharma P, Watkinson PJ, Clifton DA. Deep Interpretable Early Warning System 
for the Detection of Clinical Deterioration. IEEE J Biomed Heal Informatics. 2019;PP(Li):1–1.  
69. 
Yang Y, Walker TM, Walker AS, Wilson DJ, Peto TEA, Crook DW, et al. DeepAMR for predicting co-
occurrent resistance of Mycobacterium tuberculosis. Bioinformatics. 2019;35(18):3240–9.  
70. 
Sperrin M, Martin GP, Sisk R, Peek N. Missing data should be handled differently for prediction 
than for description or causal explanation. J Clin Epidemiol. 2020;125:183–7.  
71. 
Riley RD, Ensor J, Snell KIE, Debray TPA, Altman DG, Moons KGM, et al. External validation of 
clinical prediction models using big datasets from e-health records or IPD meta-analysis: 
opportunities and challenges. BMJ. 2016 Jun 22;353:i3140.  
72. 
Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW, et al. Transparent 
Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): 
Explanation and Elaboration. Ann Intern Med. 2015 Jan 6;162(1):W1.  
73. 
Souza JP, Gülmezoglu AM, Vogel J, Carroli G, Lumbiganon P, Qureshi Z, et al. Moving beyond 
essential interventions for reduction of maternal mortality (the WHO Multicountry Survey on 
Maternal and Newborn Health): a cross-sectional study. Lancet. 2013 May;381(9879):1747–55.  
74. 
Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a 
multivariable prognostic model: A resampling study. Stat Med. 2016;35(2):214–26.  
75. 
Riley RD, Snell KIE, Ensor J, Burke DL, Harrell Jr FE, Moons KGM, et al. Minimum sample size for 
developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat 
Med. 2019 Mar 30;38(7):1276–96.  
 
21 APPENDIX A:  AMENDMENT HISTORY 
 
Amendment 
No. 
Protocol 
Version 
No. 
Date 
issued 
Author(s) of changes 
Details of Changes made 
1 
2 
3Jun24 
Peter Watkinson 
CAG reference added and 
new PPI co-applicant, 
Zenab Barry, name added 
to the protocol 
 
List details of all protocol amendments here whenever a new version of the protocol is produced.  This is 
not necessary prior to initial REC / HRA submission. 

Date and version No:     3rd June 2024; v. 2.0 
 
 
Clinical Research Protocol Template version 15.0  
 
 
 
 
 
CONFIDENTIAL 
© Copyright: The University of Oxford and Oxford University Hospitals NHS Foundation Trust 2019 
 
Page 46 of 46 
Protocol amendments must be submitted to the Sponsor for approval prior to submission to the REC 
committee and HRA (where required).