A Patient-oriented Risk Communication Tool to Improve Patient Experience, Knowledge and Outcomes After Elective Surgery
Conditions: Elective Surgical Procedures
Sponsor: Ottawa Hospital Research Institute

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 1 of 6 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
A Patient-oriented risk communication tool to improve patient experience, knowledge and 
outcomes after elective surgery 
 
NCT Number: NCT03422133 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 2 of 6 
 
Background  
Patients consistently report wanting more information about their expected outcomes after 
surgery; when patients have greater knowledge about surgery they are less anxious and more satisfied. 
However, most patients report being poorly informed about what to expect after surgery (such as how 
their risk of complications, or how long they will be in hospital). Our team has reviewed over 240 
preoperative consultations for surgery patients at TOH, and in only 19% was there any documentation 
of a patient’s personalized risks. Furthermore, even in cases where documentation was present, patients 
were provided with qualitative summaries, which are typically meaningless to patients.  Following 
major surgery, 40% of patients experience at least one adverse event, and at TOH, 15% of patients 
experience a serious complication. Therefore, patients at TOH are undergoing surgery with the hope of 
experiencing a positive health outcome, but without clear, timely and engaged communication of the 
risks inherent in having surgery. This lack of communication directly contradicts the values and vision 
of TOH, and those articulated by the Elizabeth and Matthew Policy on Communication and Access. 
More than 15 000 people have surgery at TOH every year, and many suffer complications. 
However, most patients are not informed of their personalized risks, despite this being a requirement of 
informed consent.  Patients want more information about their risks and expected recovery from 
surgery, and lack of timely and effective communication of this information is not in keeping with 
TOH’s vision and values. Therefore, our specific objectives are to: (1) develop a patient-oriented, 
personalized preoperative risk communication tablet application; and (2) to evaluate whether the 
application improves patient knowledge of their personalized risk profile, and satisfaction with their 
perioperative care. 
Our own research on provision of individualized preoperative risk (documented for less than 1 
in 5 patients)1 is consistent with the existing literature.2,3 Additionally, a lack of communication of 
possible risks of surgery is the leading complaint in post-surgery medicolegal cases.4 Several studies 
demonstrate that patients feel that the information that they receive before surgery is insufficient, and 
that this is not adequately addressed by clinicians.2,3,5 Furthermore, despite discussion of risks being 
potentially anxiety provoking, better informed patients tend to have less preoperative anxiety and be 
more satisfied.6  We have completed a scoping review, which screened over 700 papers using a peer-
reviewed, structured search strategy. We identified only 2 processes to communicate personalized risk 
to patients none that used technology, and none that had undergone evaluation of their effectiveness. 
Therefore, our proposed process and tablet application represent a novel intervention to address a key 
gap in perioperative patient-centered care.  
We have evidence from our own preoperative clinics that patients can provide their personal 
health data with a high degree of accuracy (moderate to excellent agreement with physician health 
histories for all health conditions included in the NSQIP risk calculator).7 And when applied to our 
TOH patients, the NSQIP risk calculator shows high levels of predictive accuracy (c-statistic >0.9 for 
mortality, >0.8 for serious complications and LOS; excellent calibration (Hosmer-Lemeshow p>0.05 
for all outcomes; overall accuracy Brier score <0.1).  Finally, systematic reviews demonstrate that 
when risk prediction tools and clinical information technology are integrated into care processes in a 
manner that leverages accurate, local data while engaging and involving patients, the likelihood of 
successful implementation is increased substantially.8,9 

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 3 of 6 
 
Methods 
In collaboration with the TOH mHealth team we are developing a tablet-based, patient-oriented, 
personalized risk communication application. The application leverages features of inclusive design, 
meaning that it will be useable even by older patients, who represent the largest demographic of 
surgical patients at TOH. Older individuals are often less tech-familiar, and may have visual and motor 
challenges that can make mobile technology use difficult. The application features simple and 
consistent design, large well-lit buttons that tolerate tremor, and simple language. The process will be 
applied in preoperative clinics at TOH, where patients will be asked to provide their personal health 
history through a series of questions already used on our TOH preoperative health screener. These 
values will then populate the NSQIP Universal Risk Calculator, which we have calibrated to TOH data, 
to generate personalized risks of mortality, serious complications, and hospital length of stay. These 
risk estimates will be communicated directly to the patient using absolute risk estimates represented 
pictorially (best practice for risk communication to patients). Risk estimates will also be provided to the 
patient’s clinician. To facilitate shared discussion between patients and physicians, we will also ask the 
patient to identify up to 3 benefits they hope to achieve from surgery, and patients and clinicians will 
be provided with 3 evidence-based questions known to support Shared Decision Making (What are my 
options? What are the pros and cons of each option? How do I get the support to make the right 
decision for me?). 
 
Study Design 
The application will be evaluated using a controlled before-after study design. Patients will be 
recruited using standardized procedures, and process and outcome measures will be recorded using the 
same tools and methods in both study phases to decrease the risk of measurement and selection bias. 
The knowledge questionnaire will be completed prior to the clinic appointment, and repeated after. The 
primary outcome measure will be the change in knowledge from prior to the clinic appointment to after 
comparing the pre- and post-implementation groups. The satisfaction (patient experience) questionnaire 
will be administered after each patient’s clinic appointment.  
Outcomes 
Primary outcome: The primary outcome will be the change in patient knowledge of their risk profile 
from before their PAU appointment to after. A knowledge questionnaire has been developed using 
recommended standards.10 The questionnaire will not test perceived knowledge, but will test factual 
items specifically related to the patient’s personalized risk profile.11 The pre- and post-appointment 
knowledge scores will be normalized on a 100 point scale [(questions correct/total questions)*100].  
Secondary patient-centered outcomes: Patient experience will be assessed using a likelihood to 
recommend measurement based on a 10-point Likert scale as recommended by the Institute for 
Healthcare Improvement’s Triple Aim measurement guide.12 The value of the intervention will be 
defined using a patient-perceived equation as proposed by Morgan et al.13 This value framework 
derives patient-value by dividing patient outcomes measured in natural units by total costs that borne to 
patients. We will also use a micro-costing technique14 to estimate the total costs of the eHealth 
application that covers the costs of iPads, application development and training. Patient anxiety levels 

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 4 of 6 
 
will be measured using the Short Form State-Trait Anxiety Inventory,15 and acceptability will be 
measured using a modified version of a validated 5-point Likert scale.16 
Secondary clinician-centered outcomes: Likelihood to change management will be measured using a 
5-point scale, acceptability will be measured using a 5-point Likert scale.16 
Feasibility outcomes: Proportion of patients for whom a risk score can be calculated, proportion of 
missing data from patient-entered health questionnaire, proportion of patients with an expected length 
of stay >75th procedure specific percentile where notification of expected prolonged length of stay is 
sent to the most responsible physician. 
System outcomes: Hospital length of stay in patients with an expected length of stay greater than the 
75th procedure specific percentile (captured from linkage to hospital administrative data; the 75th 
percentile is the most commonly used cut off to define prolonged LOS). 17,18 
Study population: We will recruit all consenting adults aged 18 or older seen in the Pre-Admission 
Unit prior to elective inpatient surgery at The Ottawa Hospital. Participants will be excluded if they are 
unable to communicate in English or French, or if they are unable to provide consent on their own due 
to cognitive impairment. Those scheduled for gastric bypass surgery will also be excluded due to 
significant differences in their pre-operative risk discussion.  
During the post-implementation phase, patients will only be invited to participate in the study if the 
physician in clinic has provided consent for the study. 
Sample Size: Sample sizes of 86 patients before and 86 after achieves 90% power to detect a 
minimally important difference in the mean improvement in knowledge test of 5 points (i.e., an 
improvement of 10 points after implementation versus 5 points before implementation) assuming a 
standard deviation for both groups of 10 and using a two-sided two-sample equal variance t-test at the 
5% level of significance. 95 per group will allow for 10% dropout. 
 
Data collection and management: Baseline and outcome data will be measured using various 
questionnaires (see study flow). Responses will be captured and stored via a RedCap data collection 
interface on a secure institutional research server. Length of stay data will be obtained by linking our 
data to our hospital Data Warehouse. All data will be stored in a privacy legislation-compliant manner. 
Statistical analysis: Descriptive statistics will be used to compare recruitment rates and characteristics 
of patients in the pre- and post-implementation phases. Two-sample t-tests or chi-squared tests, as 
appropriate, will be used to examine statistical significance of differences. All outcomes will be 
expressed as biweekly means with graphical interpretation as a first step, using biweekly time series 
plots to detect the presence of any immediate versus gradual effects of intervention, and to examine the 
presence of trends, cyclical patterns, and outliers.  
Primary outcome (change in knowledge from before to after PAU appointment) will be analyzed using 
a linear segmented regression analysis of the biweekly means. The presence of autocorrelation will be 
assessed using Durbin-Watson tests, as well as visual inspection of residual plots. If autocorrelation is 
present, an autocorrelation parameter will be included in the model. The effect of the intervention will 
be reported as level and trend changes after the intervention, together with 95% confidence intervals.  
Patient-level linear regression analysis will be additionally conducted to adjust for potential differences 

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 5 of 6 
 
in patient characteristics from pre- to post-implementation and to examine any predictors of outcomes 
or subgroup effects (e.g., by patient age, sex, multimorbidity status, and procedure). Subgroup analyses 
will be conducted by including the subgroup variables as well as their interaction with the intervention 
indicator into the regression models. All analyses will be conducted using SAS v. 9.4 with α = 0.05 set 
for statistical significance. 
Secondary patient-centered outcomes (patient experience, value, anxiety, acceptability) will be 
analyzed as described for the primary outcome. Normalizing transformations will be considered for 
patient-level analyses in case of substantial skewness, while ordinal logistic regression will be used for 
outcomes measured using a 5 point Likert scale.  
Clinician-centered outcomes (likelihood to change management and acceptability, both measured using 
a 5-point scale) will be analyzed using biweekly summary measures using segmented autoregression as 
described for the primary outcome.  
Feasibility and system outcomes will be described using frequencies and proportions and tabulated 
overall, and by implementation phase. Hospital length of stay will be analyzed using segmented linear 
regression as described for the primary outcome.  
Study Flow: Questionnaires 
 
      Pre-Implementation Phase 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
         Post-Implementation Phase 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Consent – Clerk/RN asks patient if 
they’d be willing to see research. 
RA obtains consent in waiting 
Before PAU visit 
1. Demographic Questionnaire 
2. Knowledge Questionnaire 
3. Anxiety Questionnaire 
4. Pre-operative Assessment Survey 
PAU 
Visit 
After PAU visit 
1. Knowledge Questionnaire 
2. Anxiety Questionnaire 
3. Patient Experience 
Consent – Clerk/RN asks patient if 
they’d be willing to see research. 
RA obtains consent in waiting 
Before PAU visit 
1. Demographic Questionnaire 
2. Knowledge Questionnaire 
2. Anxiety Questionnaire 
3. Patient enters data in app 
PAU 
Visit 
After PAU visit 
1. Knowledge Questionnaire 
2. Anxiety Questionnaire 
3. Patient Experience 
4. Patient Acceptability 
5. Risk Discussion Questionnaire 
1. Standard visit 
2. Review risks from app 
3. MD Acceptability + 
Change in Management 

McIsaac, Daniel I – Research Proposal 
A Patient-oriented risk communication tool to improve patient experience, knowledge and outcomes 
after elective surgery 
 
Version date: July 16, 2018 
Page 6 of 6 
 
References 
1.  
MacDonald D, Boland L, Pelipychenko D, Mcisaac DI. Documentation of Frailty, Capacity and Consent for 
Elderly Patients Having Elective Inpatient Non-Cardiac Surgery. In: Canadian Anesthesiologists Society 
Annual Meeting. Niagara Falls, ON; 2017. 
2.  
Leclercq WK, Keulers BJ, Houterman S, Veerman M, Legemaate J, Scheltinga MR. A survey of the current 
practice of the informed consent process in general surgery in the Netherlands. Patient Saf Surg. 2013;7(1):4. 
doi:10.1186/1754-9493-7-4. 
3.  
Leclercq WKG, Keulers BJ, Scheltinga MRM, Spauwen PHM, van der Wilt G-J. A Review of Surgical 
Informed Consent: Past, Present, and Future. A Quest to Help Patients Make Better Decisions. World J Surg. 
2010;34(7):1406-1415. doi:10.1007/s00268-010-0542-0. 
4.  
Grauberger J, Kerezoudis P, Choudhry AJ, et al. Allegations of Failure to Obtain Informed Consent in Spinal 
Surgery Medical Malpractice Claims. JAMA Surg. April 2017:e170544. doi:10.1001/jamasurg.2017.0544. 
5.  
Keulers BJ, Scheltinga MRM, Houterman S, Van Der Wilt GJ, Spauwen PHM. Surgeons underestimate their 
patients’ desire for preoperative information. World J Surg. 2008;32(6):964-970. doi:10.1007/s00268-008-
9581-1. 
6.  
Kiyohara LY, Kayano LK, Oliveira LM, et al. Surgery information reduces anxiety in the pre-operative 
period. Rev Hosp Clin Fac Med Sao Paulo. 2004;59(2):51-56. doi:10.1590/S0041-87812004000200001. 
7.  
Neufeld J, Wilson J, Gagne S, Yang H, Crossan M, Taljaard M. VALIDATING PATIENT SELF -REPORT 
TOOL IN PREOPERATIVE TRIAGE. CAS Conf. 2014. 
8.  
Huckel Schneider C, Gillespie JA, Wilson A. Implementing system-wide risk stratification approaches: A 
review of critical success and failure factors. Heal Serv Manag Res. January 2017:951484817695738. 
doi:10.1177/0951484817695738. 
9.  
Scheepers-Hoeks A-M, Grouls RJ, Neef C, Ackerman EW, Korsten EH. Success Factors and Barriers for 
Implementation of Advanced Clinical Decision Support Systems. In: Jao C, ed. Efficient Decision Support 
Systems – Practice and Challenges in Biomedical Related Domain. Rijeka, Croatia: InTech Open; 2011. 
http://cdn.intechopen.com/pdfs/18695/InTech-
Success_factors_and_barriers_for_implementation_of_advanced_clinical_decision_support_systems.pdf. 
10.  
O’Connor A. Ottawa Hospital Research Institute: User Manual-Knowledge. 
http://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_Knowledge.pdf%0A. Published 2000. Accessed 
May 12, 2017. 
11.  
Sepucha KR, Borkhoff CM, Lally J, et al. Establishing the effectiveness of patient decision aids: key 
constructs and measurement instruments. BMC Med Inform Decis Mak. 2013;13(Suppl 2):S12. 
doi:10.1186/1472-6947-13-S2-S12. 
12.  
Stiefel M, Nolan K. A Guide to Measuring the Triple Aim : Population Health, Experience of Care, and Per 
Capita Cost. Cambridge, MA; 2012. www.IHI.org. 
13.  
Morgan SR, Smith MA, Pitts SR, et al. Measuring value for low-acuity care across settings. Am J Manag 
Care. 2012;18(9):e356-63. http://www.ncbi.nlm.nih.gov/pubmed/23009334. 
14.  
Gold M, Siegel J, Russell L, Weinstein M. Cost-Effectiveness in Health and Medicine. New York, NY: Oxford 
University Press; 1996. 
15.  
Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State-
Trait Anxiety Inventory (STAI). Br J Clin Psychol. 1992;31 ( Pt 3):301-306. 
http://www.ncbi.nlm.nih.gov/pubmed/1393159. 
16.  
Brehaut JC, Graham ID, Wood TJ, et al. Measuring acceptability of clinical decision rules: validation of the 
Ottawa acceptability of decision rules instrument (OADRI) in four countries. Med Decis Making. 
2010;30(3):398-408. doi:10.1177/0272989X09344747. 
17.  
Wallner LP, Dunn RL, Sarma A V, Campbell DA, Wei JT. Risk factors for prolonged length of stay after 
urologic surgery: the National Surgical Quality Improvement Program. J Am Coll Surg. 2008;207(6):904-913. 
doi:10.1016/j.jamcollsurg.2008.08.015. 
18.  
Collins TC, Daley J, Henderson WH, Khuri SF. Risk factors for prolonged length of stay after major elective 
surgery. Ann Surg. 1999;230(2):251-259. http://www.ncbi.nlm.nih.gov/pubmed/10450740.