Use of Efficacy Safety Score in Postoperative Management: A Prospective, Randomized Study
Phase: NA
Conditions: Pain, Postoperative, Postoperative Nausea and Vomiting
Sponsor: St. Olavs Hospital

1 
 
Projct Protocol 
Date final revised: 2018-02-09 
Project title: 
 
“Postoperative treatment with Efficacy Safety Score and wireless patient monitoring system 
at a general ward: a randomised controlled study.” 
 
Authors: 
Erlend Skraastad (MD, phd candidate) 
Johan Ræder (MD, prof. dr.med, main supervisor) 
 
Institution: 
Clinic of Anaesthesia and Intensive care, St. Olavs Hospital, Orkdal Department, Trondheim 
University Hospital, Norway.  
 
 
Part of larger project 
«Development and use of novel tools for assessing efficacy, side-effects and safety of 
postoperative treatment»  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Introduction  

2 
 
In modern perioperative care safety is a primary concern, and in-hospital mortality was recently 
reported to be 0.5% [1]. Much more frequent, although less serious, are the problems of patient 
perceived quality in the post-operative period. In a survey of 2252 patients, 55% reported that they 
suffered from unsatisfactory pain treatment postoperatively [2]. A recent review of 183 studies with 
more than 100.000 patients, showed a high incidence of adverse effects of postoperative pain 
treatment. Twenty-five per cent of the patients suffered from nausea, 20 % from vomiting, 14.7 % 
from pruritus and 23 % from urinary retention. Notably, 24 % of the patients received insufficient 
doses of analgesics for their pain [3]. Side effects were most typically induced by opioid analgesics. 
It might be argued that health care personnel should aim at better surveillance of both effects and 
side effects of analgesics. Lack of feedback on the effects of administration of analgesics, especially 
of opioids, together with insufficient monitoring of respiratory and circulatory variables, may lead 
to under-dosing out of fear of side effects. Administration of analgesics may also cause inadvertent 
over-dosing and even fatal outcomes in hospital as well as at home after discharge [4-8]. As long as 
the patients remain in the post-anesthesia care unit (PACU) with continuous monitoring and 
observation, there are abundant tools and measures at hand to ensure both quality and safety of the 
treatment. On the contrary, after discharge to an ordinary ward, there are less staffing and 
monitoring resources, and patients may even be left alone for shorter periods.  
We hypothesized that medical personnel may benefit from the feedback of a simple monitoring 
system, which detects changes in respiration - and circulation variables in combination with the 
patient's subjective experiences of postoperative quality of care. Such a system would be useful, 
both for the treatment of individual patients as well as for assessing the efficacy and safety of 
postoperative care given to different groups of patients. We started with an algorithm-based 
monitoring system with multiple simultaneously recorded variables, as a prototype tool for 
assessment of quality and safety of postoperative treatment [9, 10]. After a successful pilot study on 
this narratively developed tool, called Efficacy – Safety Score (ESS), Table 1, we decided to 
develop this through a documentation and validation process. The results from this are published as 
the 1st article in this project [11]. As described in this paper there is no such clinical tool as ESS 
presented before, and this makes this project genuine and original with a great potential.    
Research questions and goals   
The main research question is whether introducing new tools for optimizing postoperative treatment 
may improve the clinical outcome for the individual patient and patient groups. Until now, there 
have not been any clinical decision tools available for the specific postoperative period at the ward, 
and we do not have sufficient knowledge about patients in this vulnerable phase. Short term goals 
are to determine whether patients would benefit from a tool as ESS for postoperative treatment. 
Long term goals are to support healthcare providers with needed systematic real-time information 
about patients in the postoperative setting. We plan to include a project-arm, in which we will 
integrate telemetrically retrieved physiological data. A digital wireless platform which integrates 
ESS, respiration rate, patient temperature, ECG and blood pressure readings before-, during and 
after the first postoperative night is developed (Patient Status Engine, Isansys Lifecare Ltd., UK).  
This project is a contribution to validate ESS by means of a variety of monitoring tools as the in 
dedicated groups of surgical patients by using criteria set out by Terwee et al [12]. We expect to 
reproduce findings from the pilot project with a score reflecting the postoperative status of the 
patient, and moreover to observe positive effects on early mobilization including a lower incidence 
of side-effects of medication. If we will be able to achieve such results, we can identify the patient 
in need of more thorough care, and provide clinical statistics for improved management of groups 
of patients.  
   
Study design  
We regarded a prospective randomized study with two groups as the best choice for answering the 
question whether patients might benefit from introducing a new clinical decision tool. One group, 
named Standard Care Group (SCG) (n=100) will receive standard postoperative care and 

3 
 
management. The other group, named Efficacy Safety Score (ESS) (n=100) will receive care and 
management according to ESS. In the ESS group, we will assess ESS in parallel with the 
telemetrically retrieved physiological variables. A randomization of the patients to group SCG and 
ESS will be done a priori by using a simple block randomization and sequentially numbered opaque 
sealed envelopes (SNOSE) will be produced. After obtaining patients’ consent, the nurses at the 
ward will pick a sealed opaque envelope with the randomization code enclosed for each patient. 
This takes place at the ward when the patients are prepared for surgery. The nurses including 
patients will not have access to the envelopes before time of enrolment. Because of the logistics in 
this situation, and the fact that it is the ordinary ward staffing which is to perform this, we have 
concluded that the use of SNOSE is the best allocation concealment we can obtain in this situation. 
The Regional Medical Ethics Committee recognized two different letters of consent to the SCG and 
the ESS groups to be the best solution, and to secure correct handling of this, the manual way with 
sealed envelopes is regarded most robust for this study situation. By this method we plan to reduce 
selection bias to a minimum. See also enclosed flow-chart. 
 
Hospital setting  
We plan to perform the study at the surgical ward 3 at St Olav University Hospital, Orkdal, 
Norway. The regular nursing staff will be collecting the postoperative data after supervision and 
education by the PhD-candidate and a dedicated project nurse. The patient population at this ward 
available for this study will be persons planned for scheduled surgery (bariatric surgery, shoulder 
joint replacements, knee/hip joint replacement, major abdominal surgery (hemicolectomy e.g.,), 
robot-assisted radical prostatectomy (RARP)) and acute surgery (orthopaedic ankle and hip surgery, 
acute abdomen) which is at the hospital staying at least 24 hours after surgery. The number of these 
patients at Orkdal is 8-12 patients a week. Limitations in the number of monitoring devices (two 
sets) for the ESS group, will limit the number of patients which can be included each day. Based on 
this we think that a maximum of 6-8 patients per week is a reasonable goal for inclusion. From this 
a number of 25-35 effective weeks with inclusion would be reasonable.  
 
Inclusion/Exclusion criteria  
Inclusion criteria: all subsequent patients undergoing surgery that expectedly will be treated and 
observed in hospital for more than 24 hours postoperatively. Exclusion criteria: patients < 18 years 
of age, refusal of participation, poor communication capabilities and surgery performed that is not 
compatible with anticipated mobilization during the first 24 postoperative hours. 
 
Primary and secondary endpoints 
Primary endpoint of this study will be time to full mobilization of the patient, defined as being able 
to walk more than one step with/without support. Secondary endpoints will be clinically significant 
reduction (more than 23% [16]) in measured pain evaluated by a 0-10 scale (Verbal Numeric Rating 
Scale), significant changes in doses (OMEQ) of postoperative administered opioids, re-admissions 
to the recovery unit, significant increased patient satisfaction of treatment given and reduction in 
length of hospital stay.  
 
Methods of analysis  
Descriptive statistics will be used to compare baseline characteristics between groups, such as type 
of surgery, type of anaesthesia, regular pain medication, age, relevant comorbidity (Charlson 
Comorbidity Index), ASA-classification and body mass index. The effect of ESS on the primary 
outcome (time to mobilization) will be analysed by Kaplan-Meier estimators, and we will also use 
Cox-regression analysis to adjust for baseline covariates. For the secondary outcomes, we will use 
linear and logistic regression to estimate differences in means or odds ratios between study groups, 
depending on the measurement scale. Non-normally distributed variables will be log transformed, 
or analysed using non-parametric tests. The precision of all estimated effects will be given by a 
95% confidence interval. Due to the anticipated positive influence from the ESS group on 

4 
 
postoperative treatment to the SCG, we will also see if there is a progression in eventually 
differences in the endpoints between the SCG and ESS group. 
 
Sample size  
We performed a power analysis based on the sub-group results of a validation study addressing ESS 
in total 207 patients [11]. Estimates shows that for obtaining a power of 0,8 with alpha cut-off at 
0,05 and beta cut-off at 0,2, a sample size of total 130 patients was required to reach a 25% 
reduction of time until full mobilization (being able to walk one or more steps with/without 
support). A total sample size of 130 patients was required to reach a 25% reduction in total hospital 
stay and a total sample size of 506 and 176 patients was required to reach a 25% reduction in pain at 
rest and at mobilization, respectively. (Kane SP. SampleSizeCalculator. http://clincalc.com/Stats/SampleSize.aspx.). 
Except from pain at rest, which is of limited interest in this study, these sample size calculations 
suggest that a total sample size of 200 (100 in each group) will be be sufficient to detect any 
meaningful effects of the intervention on both primary and secondary outcomes.  
 
Data collection 
Patients will be included after giving their informed signed consent. In both groups, a standardized 
questionnaire (see attachment) focused on postoperative management and satisfaction will be 
performed 24 hours postoperatively. In group ESS, we will register ESS for the first 24 hours 
postoperatively. Registration of ESS is to be done hourly for the first four hours, and then every 
second hour after the patient has arrived to the ward. For all patients the Modified Aldrete Score 
and the New Early Warning Score [14-16] will be performed for comparison. If ESS>10 in patients 
of group ESS, the anaesthesiologist on duty might be called upon (not involved in the study) for 
assistance. He/she will afterwards be asked to evaluate the necessity of the consultation for later 
sensitivity/specificity test. All the data will be collected by the nurses with the help of a designed 
program (Patient Status Engine) on tablet PC from Isansys (Isansys Lifecare Ltd., Oxfordshire, UK) 
that incorporates ESS into their wireless monitoring platform, Patient Status Engine (PSE) which is 
a CE-approved device (class IIa) designed for use in hospital settings to collect vital parameters. 
The data is then transferred to a protected data base locally. Statistical analysis will be performed in 
SPSS.  
Patients not going through the scheduled surgery (removed from surgery plan) will be excluded 
from the study. For patients with a not planned incident/complication during surgery, we will 
continue the protocol as long as they are return to the ward at Orkdal during the 24-hour period. If 
they are transferred to another location, they will be excluded from the study.  
We are aware the potential bias in influence to the SCG from simultaneously performing the 
intervention in ESS group and the focus on postoperative treatment. 
  
Organization and collaboration  
The project is supported by a research group of internationally recognized specialists in 
anaesthesiology with specific interest in postoperative care:  
• 
Prof dr med Johan Ræder, UiO and Oslo University Hospital.  
• 
Prof dr med Vegard Dahl, UiO and Akershus University Hospital. 
• 
Prof dr med Lars J. Bjertnæs, Anaesthesia and Critical Care Research Group Tromsø.  
• 
Dr med Vladimir Kuklin, Akershus University Hospital.  
• 
Prof dr med Petter Chr. Borchgrevink, St Olavs Hospital/Faculty of Medicine NTNU 
  
Progress schedule and publishing plan   
The project is part of a present PhD-program at the University of Oslo, which we now will apply for 
transfer to NTNU and St Olavs Hospital, with mandatory courses and schedule set up according to a 
signed contract between the PhD-candidate and the university. A plan for three articles is approved:  
1. Development and validation of the Efficacy Safety Score, a novel tool for postoperative 
patient management: A prospective observational study (published March 2017 [11])   

5 
 
2. Postoperative treatment with Efficacy Safety Score and wireless patient monitoring system 
at a general ward: a prospective randomized controlled study. 
3. Introducing novel clinical tools in postoperative treatment at the ward. Effects on working 
situation for care providers, patient safety and quality of care. (See attached tentative draft 
for project plan for this.) 
The articles will be submitted for publication in well-recognized international journals.  
   
Ethics  
The protocol and the informed consent form of this prospective observational study were approved 
by the Regional Medical Ethics Committee South East (Approval 2017/1903/REK SørØstA). 
Written informed consent will be taken from every patient included in the study. Correct handling 
of medical records and information is of highest importance, and mandatory training will be given 
to all staff involved.  
Reg.nr ClinicalTrials.gov: NCT03438578. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Tables  
Table 1 Description of Efficacy Safety Score (ESS)  
Mental status  
Score  
Awake and alert patient  
0  

6 
 
Awake patient, but influenced by drugs. Difficulties in communication.  
5  
Acutely confused, upset/uneasy, hallucinated or euphoric patient  
10  
Unresponsive patient  
15*  
Postoperative nausea and vomiting (PONV) status  
  
No postoperative nausea or vomiting  
0  
Postoperative nausea only  
5  
Postoperative nausea and vomiting/retching  
10  
Pain status at rest  
  
No postoperative pain  
0  
Low intensity postoperative pain (VNRS 1-3)  
1-3  
Moderate intensity postoperative pain (VNRS 4-6)  
4-6  
Severe intensity postoperative pain (VNRS 7-10)  
7-10  
Pain status during movement  
  
No postoperative pain  
0  
Low intensity postoperative pain (VNRS 1-3)  
1-3  
Moderate intensity postoperative pain (VNRS 4-6)  
4-6  
Severe intensity postoperative pain (VNRS 7-10)  
7-10  
General condition status  
  
No remarks  
0  
Minor discomfort (e.g. light-headedness, minor itching, blurred vision, decreased urination etc.)  
5  
Excessive discomfort (e.g. severe dizziness, itching, restlessness, urine retention, sensation of 
cold/warmth, cold sweating) 
10  
Acute circulatory abnormalities (blood pressure ≤80 or ≥200mmHg, heart rate ≤40 or > 110)  
15*  
Acute respiratory abnormalities (dyspnoea, respiration rate < 9 or >20/min, long pauses in 
breathing, shallow breathing)  
15*  
*Any single score of 15 (on either consciousness, circulation or respiration) should call for IMMEDIATE activation of 
acute assistance with the patient.  
 

7 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Flowchart of phases in Efficacy Safety Score study, St Olavs Hospital HF, Orkdal B3:
 
 
Assessed for eligibility, n=200
Randomized
Allocated to 
Efficacy Safety 
Score (ESS) 
Allocated to 
Standard Care 
Group (SCG) 
Efficacy Safety 
Score
Standard Care
Excluded 
Did not 
receive 
intervention
Did not 
receive 
intervention
Follow Up 
Interview
Follow Up 
Interview
Discontinued 
Discontinued
Lost to Follow-Up
Lost to Follow-up
Analysis
Analysis

8 
 
References   
1. 
Le Manach, Y., et al., Preoperative Score to Predict Postoperative Mortality (POSPOM): Derivation 
and Validation. Anesthesiology, 2016. 124(3): p. 570-9. 
2. 
Maier, C., et al., The quality of pain management in German hospitals. Dtsch Arztebl Int, 2010. 
107(36): p. 607-14. 
3. 
Dolin, S.J. and J.N. Cashman, Tolerability of acute postoperative pain management: nausea, 
vomiting, sedation, pruritus, and urinary retention. Evidence from published data. Br J Anaesth, 
2005. 95(5): p. 584-91. 
4. 
Benjamin (22) died after bone fracture, in VG. 2011: http://www.vg.no/helse/odelagt-i-
helsevesenet/artikkel.php?artid=10093729 (01.12.2015). 
5. 
Got to strong medications - Davoud (23) died, in Nettavisen. 2011: 
http://www.nettavisen.no/nyheter/article3257721.ece (01.12.2015). 
6. 
Nearly died in the emergency room, in NRK. 2011: http://nrk.no/nyheter/norge/1.7629090 
(01.12.2015). 
7. 
AP, Overdose death at Seattle hospital not the first, in The Columbian. 2010: 
http://www.columbian.com/news/2010/sep/29/overdose-death-at-seattle-hospital-not-the-first/ 
(01.12.2015). 
8. 
Warner, M., et al., Drug poisoning deaths in the United States, 1980-2008. NCHS Data Brief, 
2011(81): p. 1-8. 
9. 
Kuklin V, S.E., Ernst G, Berg P, Bjertnaes LJ , Raeder J, Kongsberg Satisfaction Score: A novel system 
for monitoring the status, efficacy and safety of the postoperative pain treatment after surgery, in 
Annual Meeting of the International Anesthesia Research Society. 2013, Anesth Analg San Diego 
California, USA. p. S-295. 
10. 
Skraastad E, B.L., Ræder J, Kuklin V,.  , 2013,26-4:. Kongsberg Satisfaction Score - et verktøy for 
bedre postoperativ overvåkning og behandling, in NAForum, Tidsskrift for norsk anestesiologisk 
forening, Dnlf. 2013. p. 36-41. 
11. 
Skraastad, E., et al., Development and validation of the Efficacy Safety Score (ESS), a novel tool for 
postoperative patient management. BMC Anesthesiol, 2017. 17(1): p. 50. 
12. 
Terwee, C.B., et al., Quality criteria were proposed for measurement properties of health status 
questionnaires. J Clin Epidemiol, 2007. 60(1): p. 34-42. 
14. 
Royse, C.F., et al., Development and feasibility of a scale to assess postoperative recovery: the post-
operative quality recovery scale. Anesthesiology, 2010. 113(4): p. 892-905. 
15. 
Gardner-Thorpe, J., et al., The value of Modified Early Warning Score (MEWS) in surgical in-patients: 
a prospective observational study. Ann R Coll Surg Engl, 2006. 88(6): p. 571-5. 
16. 
White, P.F. and D. Song, New criteria for fast-tracking after outpatient anesthesia: a comparison 
with the modified Aldrete's scoring system. Anesth Analg, 1999. 88(5): p. 1069-72. 
 
 
 
 
 
 
 
 
 
 
 

9 
 
Attachment 1: 
 
 
 
 
 
 

10 
 
Attachment 2: 
“Introducing novel clinical tools in postoperative treatment at the ward. Effects on working 
situation for care providers, patient safety and quality of care” 
Study protocol (Tentative draft) 
The overall objective of the study is to generate new knowledge and strengthen the decision-making 
basis for how new tools and technical innovations are to be introduced in clinical hospital practice.  
Focus is on how postoperative care providers respond to new checklists and patient monitoring in 
everyday work; how novel tools might influence on the healthcare professionals’ clinical work and 
if this has an effect on patient safety and experienced quality of care.  
The basis for this is the project conducted at St Olavs Hospital, Orkdal ward B3 in 2018: 
“Postoperative treatment with Efficacy Safety Score and wireless patient monitoring system at a 
general ward: a prospective randomized controlled study.” 
The study seeks to highlight the following factors related to the use of Efficacy Safety Score (ESS) 
by doctors and nurses: 
 Functionality 
o Which technical functionality is available to respondents – and how this 
affects the clinical postoperative setting? Including:  Right information to the 
right recipient? 
o Valuable information in daily clinical work? 
o Support for holistic postoperative care as part of standardized patient 
pathways? 
 Use 
o To what extent does the staff use the information from this system? Does 
information gathered with ESS provide better patient care as judged by the 
staff?  
o Does use of ESS promote self-confidence and knowledge for healthcare 
professionals in clinical settings? 
o Does use of ESS improve inter-disciplinary communication? 
o To what extent are respondents satisfied with the functionality of the ESS? 
o Technical implementation and technical support – to what extent are these 
barriers for new clinical tools? 
Methods 
As a background we will perform a thorough search in peer-reviewed published academic works 
about identified pit-falls and success actors when introducing new tools/techniques for clinical 
medical settings. This will be a broad approach to identify obstacles to routine changes and limiting 
factors in external and internal education/support among staffing. 
The project will plan and conduct a survey on the use and experience of the integrated ESS 
monitoring among doctors and nurses at St Olavs Hospital, Orkdal ward 3. The study is based 
methodically on questionnaires that will be developed for this purpose.  Questionnaires will include 
research-oriented issues that are thematically linked to the introduction, management and 
implementation of clinical innovations. We will seek assistance to development of questionnaires 

11 
 
from NTNU and other available contributors at St Olavs Hospital with focus on nursing and 
innovation. 
Recruitment of respondents will take place at St. Olav Hospital, Orkdal ward 3. The target group for 
the survey is nurses and doctors in clinical work who have been working there during the project 
started in 2018.  The survey will be available electronically for recruited respondents and answered 
anonymously through Questback. Age, gender, years of employment and clinical specialization will 
be demographic variables to be analyzed with the findings in the questionnaires. 
The study is planned published in international medical journals with focus on overall system 
improvement and quality of care.