Development and Internal Validation of a Multivariable Risk Prediction Model for Severe Rebound Pain After Lower Limb Orthopedic Surgery Involving Single-Shot Popliteal Sciatic Nerve Blocks: A Single Centre Retrospective Cohort Study
Conditions: Orthopedic Procedures, Pain, Postoperative, Surgery, Nerve Block, Ambulatory Surgical Procedures, Pain
Sponsor: University of British Columbia

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 1 of 10 
July 26, 2021 - Version 2.0 
 
Development and Internal Validation of a Multivariable Risk Prediction Model for Severe 
Rebound Pain after Lower Limb Orthopedic Surgery involving Single-Shot Popliteal 
Sciatic Nerve Blocks: A Single Centre Retrospective Cohort Study 
Protocol and Statistical Analysis Plan 
 
Principal Investigator: 
 
Dr. Cynthia Yarnold, Clinical Assistant Professor 
 
 
 
 
 
Department of Anesthesia, St. Paul's Hospital, 
 
 
 
 
 
Department of Anesthesiology, Pharmacology &  
 
 
 
 
 
 
Therapeutics, University of British Columbia 
 
Co-Investigators: 
Dr. Stephan K. W. Schwarz, Professor & Dr. Jean 
Templeton Hugill Chair in Anesthesia, Anesthesiology, 
Pharmacology & Therapeutics, UBC  
 
Department of Anesthesia, St. Paul's Hospital 
                                                      
Dr. Christopher Prabhakar, Clinical Assistant Professor 
Anesthesiology, Pharmacology & Therapeutics, UBC 
Department of Anesthesia, St. Paul's Hospital 
Dr. Ron Ree, Clinical Associate Professor 
Anesthesiology, Pharmacology & Therapeutics, UBC 
Department of Anesthesia, St. Paul's Hospital 
Dr. Tim Ting Han Jen, Clinical Assistant Professor 
Anesthesiology, Pharmacology & Therapeutics, UBC 
Department of Anesthesia, St. Paul’s Hospital 
Dr. Janny Xue Chen Ke, Anesthesiologist,  
Department of Anesthesia, St. Paul’s Hospital 
Dr. Justine Denomme, Clinical Fellow  
Department of Anesthesia, St. Paul’s Hospital 
 
 
 
 
 
Dr. Shih-Chieh Huang, Medical Resident 
Department of Anesthesia, St. Paul's Hospital 
Dr. Daniel I. McIsaac, Associate Professor of 
 
Anesthesiology and Pain Medicine 
Department of Anesthesiology and Pain Medicine,  
University of Ottawa 
 
 
 
 
 
Dr. Kevin Wing, Orthopedic Surgeon, Department of  
 
 
 
 
 
Orthopedics, St. Paul’s Hospital, UBC 
 
 
 
 
Primary Contact: 
Nicola Edwards, Anesthesia Research Manager,  
Department of Anesthesia, St. Paul’s Hospital,  
 
 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 2 of 10 
July 26, 2021 - Version 2.0 
 
Study Description 
Rebound pain is a well-recognized phenomenon after the resolution of peripheral nerve blocks. 
(1–5) Severe rebound pain is prevalent after ambulatory surgery, with potential resultant 
increased health-care utilization and cost. (6) Risk factors for severe rebound pain may include 
younger age, female sex, high preoperative pain score, bone surgeries, and upper limb surgeries. 
(7–10) Use of tourniquet for lower limb procedures is also associated with worse postoperative 
pain compared to without. (11,12) Protective factors may include continuous perineural catheter, 
regional anesthesia adjuncts, and multimodal oral pain management prior to resolution of 
peripheral nerve blockade. (8,9,13–17) However, placement of continuous peripheral nerve 
block catheters utilizes additional health care resources; hence, a targeted approach where 
continuous perineural catheters are offered to patients at the highest risk of severe rebound pain 
could potentially offer the biggest benefit. The aim of the study is to derive and validate a 
multivariable prediction model for severe rebound pain after lower limb surgery involving 
popliteal blockade, to assist with risk stratification and shared decision making. 
 
 
Primary Objectives 
The primary objective of the study is to derive and validate a multivariable clinical prediction 
model to predict the risk of severe pain after lower limb orthopedic surgeries after single-shot 
popliteal sciatic nerve blocks resolution. 
 
Data Source  
The St. Paul’s Hospital Peripheral Nerve Block (PNB) Database (in REDCap) will be used. 
Study Sample 
Inclusion 
Patients undergoing lower limb surgery who received single-shot popliteal sciatic nerve blocks 
(with or without other peripheral nerve blocks) at St. Paul’s Hospital, Vancouver, from January 
4, 2016 to November 1, 2019 will be included. 
 
Exclusion 
For patients who had multiple surgeries during the study period, only the first surgery within the 
study period will be selected. Patients who had uncontrolled pain (defined as NRS >3 and/or 
nursing note documentation of uncontrolled or severe pain and/or undocumented pain status) in 
the recovery room will also be excluded as this consists of a heterogenous group of patients who 
may have had failed block or experience pain from sites away from the blocked area, neither of 
which may be helped by continuous peripheral nerve block catheters. Patients receiving popliteal 
sciatic nerve catheters will be excluded. 
 
Model Endpoints 
The primary endpoint is severe rebound pain, defined by transition from well-controlled pain 
(numerical rating scale [NRS] 3 or less or patient report of satisfactory pain control) in PACU 
while the block is working to severe pain (NRS pain score 7 or greater) within 48 h of block 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 3 of 10 
July 26, 2021 - Version 2.0 
 
performance. This definition is modified from that of Barry et al., as clinically we have noted 
some patients having sensory blocks lasting more than 24 hours with the use of adjuncts. 
 
Lower Limb Candidate Predictors 
As there is currently no agreed method of classifying lower limb surgeries systematically as it 
pertains to postoperative pain, we reviewed the literature and consulted an orthopedic surgeon 
specializing in foot and ankle surgery (Dr. Kevin Wing) to categorize surgeries into a ranked list 
by anatomy, clinical similarity (e.g. fusion), and known risk factors of increased postoperative 
acute pain or rebound pain. If multiple surgeries are present, the surgery in the category that is 
deemed the most involved (i.e. higher ranking) will be selected as the primary surgery.  
The following a-priori predictors were included: 
 
Patient will be assigned one main surgical type (i.e. surgery type is considered one 
class/categorical variable with Group 9 Soft tissue only as the reference): 
o 
1= Fusion/Osteotomy of Ankle/Hindfoot/Midfoot (10,18,19) 
o 
2= Total Ankle Replacement (18) 
o 
3= Acute Bony Trauma (18–22) 
o 
4= Hallux Valgus Surgeries (18) 
o 
5= Hardware Removal (18) 
o 
6= Amputations 
o 
7= Other Ankle/Hindfoot/Midfoot bony procedure (19,23,24) 
o 
8= Other Forefoot bony procedure (19,23,24) 
o 
9= Soft tissue only (8,18) 
 
Other surgical predictors 
o 
Planned admission (Pre- or post-operatively) vs. ambulatory procedure 
 Patient characteristics:  
o Age (continuous);  
o Sex (binary);  
 Anesthesia-related:  
o Local anesthetic type (Long-acting only vs. mixed); 
o Dexamethasone use (Present or absent); 
o Intraoperative Anesthesia (General anesthesia as defined by need for advanced 
airway/Sedation/Spinal). 
 
Additional cohort characteristics to be reported: 
o Pre-PNB neuropathy (Present or absent; Defined as “Yes” for either Pre-PNB 
neuropathy or diabetic neuropathy) 
o Tourniquet location (Leg versus thigh) 
o Tourniquet duration 
o Block performer (Attending/ Fellow/ Resident) 
o Duration of sensory blockade (as defined by time from PNB performance to return of 
normal sensation) (Continuous variable);  
o Duration of motor blockade (As defined by time from PNB performance to return of 
normal movement) (Continuous variable);  

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 4 of 10 
July 26, 2021 - Version 2.0 
 
o Time of day the nerve block wore off (Between 0000- 0700 / Between 0701-2359) 
(25,26) 
o Health care practitioner contact, as defined by answering “yes” to the question of 
“Did you have to visit the emergency department, or call the surgeon or your family 
doctor for pain control after your nerve block wore off?”. 
o Highest pain score (HPS), as defined by the highest NRS pain score upon resolution 
of peripheral nerve blockade within 48 hours of block performance. 
 
Methods 
 
Data Preprocessing:  
Categorization of Lower Limb Surgeries 
In the database, the operations performed are entered as free texts, transposed from dictated 
operative reports. We created a standardized list of keywords and logic protocol to categorize the 
text variables into groups. Keywords (e.g., “total ankle replacement”) will be used to 
electronically identify main surgical types (e.g., Group 2 = Total Ankle Replacement). Please see 
Supplementary Material for a full list of keywords and their corresponding surgical types. For 
patients who fit into multiple surgical types based on keywords, patients will be electronically 
assigned to the first group on the list. For example, if a patient received both bunionectomy and 
hardware removal, the patient will be assigned into Group 4= Hallux Valgus Surgery. All 
surgical assignments will subsequently be manually checked independently by two authors 
(T.T.H.J. and J.D.), and conflicts will be resolved with a third author (K.W.). To complement 
this manual check, Cohen’s Kappa will be computed to identify the level of agreement of the 
classifications between the two authors (T.T.H.J. and J.D.), after their respective corrections to 
the main surgery type classifications. 
Missing values 
Any variables with >10% missing data among observations will be excluded from the modelling 
stage. Any variables with <10% missing data among observations will have their missing values 
imputed using multiple imputation.  
Since the primary variable of interest “Main surgery type” has numerous categorical levels, we 
intend to omit any observations that are missing a “Main surgery type” value, given the greater 
risk of non-convergence when attempting to impute this variable. Also, any observations with 
missing values in the primary outcome (rebound pain scores) will be excluded too. The 
remaining variables are assumed to be missing at random (MAR). 
The planned method of multiple imputation will be by using multivariate imputation by chained 
equations (MICE). The following imputation techniques will be used to impute each variable 
with missing data: 
 “Predictive mean matching” will be used to impute the continuous variables 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 5 of 10 
July 26, 2021 - Version 2.0 
 
 Logistic regression will be used to impute the binary variables 
 Multinomial logistic regression will be used to impute the nominal categorical variables 
All variables (including the outcome) will contribute to imputation of the missing values. The 
plan of the imputation will be to generate 10 multiply imputed datasets (with an anticipated 
maximum of 20 iterations to achieve convergence), which will then be used to model and pool 
results. 
Cohort characteristics 
Summary statistics of outcomes and candidate predictors will be presented. Continuous variables 
will be presented as mean (SD) if normally distributed and median (IQR) if not normally 
distributed, and categorical variables will be presented as frequency (%).  
Model building 
Correlations amongst candidate predictors will be explored to avoid risks of collinearity (for 
example, between dexamethasone use and type of intraoperative anesthesia). Amongst multiple 
highly correlated predictors, only one will be selected for the modeling based on clinical 
reasoning (e.g. one variable is more easily collected, less missing values, etc.). Additionally, 
potential multicollinearity within the final model will be diagnosed using variance inflation 
factors (VIF) of the predictors, with any values exceeding 4 warranting further investigation. 
Multivariable logistic regression will be the statistical method used to construct a prediction 
model, with the above outlined imputation procedure applied to the data prior to modelling. The 
prediction model will contain the complete variable set as predictors (as outlined in the 
Candidate Predictors section). “Age”, the only continuous predictor to be included in the model, 
will be modelled with a natural cubic spline (with 3 interior knots). 
In addition, an exploratory prediction model will be created where a natural cubic spline (with 3 
interior knots) will also be fitted for an additional predictor of “Tourniquet duration”. This serves 
as an exploration of whether tourniquet duration plays a predictive role. However, this 
exploratory model could only be used postoperatively, when this information becomes available. 
Future database with information on predicted tourniquet duration would be needed for model 
update to minimize bias if this predictor were to be applied preoperatively. The primary and 
exploratory models will then be compared in terms of predictive performance and a likelihood 
ratio test will also be conducted to assess differences in goodness-of-fit.  
To assess the performance of the predictive models, we will proceed with a bootstrap validation 
approach with 100 repetitions. At each repetition, the data will be sampled with replacement to 
form a bootstrap sample (with sample size equivalent to the original dataset). Multiple 
imputation will then be conducted on the bootstrap sample prior to modelling (using the same 
procedure outlined in the above “Missing values” section). After the pooled bootstrap model is 
created for the given bootstrap sample, performance measures (as discussed below) will be 
pooled across the 10 multiply imputed datasets of (1) the bootstrap sample and (2) the original 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 6 of 10 
July 26, 2021 - Version 2.0 
 
dataset. These will individually form estimates of the “training” and “test” performance, 
permitting us to compute the optimism of the pooled bootstrap model. 
This process will be repeated for each of the repetitions, and optimism estimates will be averaged 
over the 100 repetitions. Lastly, an estimate of the original model performance will be computed 
by taking the apparent performance of the original model and subtracting the average optimism. 
Model performance measures 
To assess model performance, we will use the area under the Receiver Operating Characteristic 
(ROC) curve (i.e., AUC) and calibration slope to evaluate the discrimination and calibration of 
the models, respectively. Other overall performance measures that will be computed include the 
Scaled Brier score and Nagelkerke’s R2. Decile calibration plots will also be generated for the 
final prediction models to further assess calibration, with Loess-smoothed calibration curves 
created for each imputed dataset. Additionally, a decision curve analysis will be performed on 
the final prediction models. 
Based on the final validated logistic regression models, a points system will be developed to 
approximate the contribution of the risk factors in the estimate of risk of having a severe rebound 
to increase clinical utility of the models. (27)  
Sample size  
Calculations based on estimated sample size of 1270, and event rate of 50% are shown below. 
(8)  [https://www.bmj.com/content/368/bmj.m441] (Excel spreadsheet of table below 
available) 
 
Binary outcomes    
  
Figure 1  
Margin of error <=0.05  
  
Criteria iii in pms Outcome proportion ->  
0.5 
  
Total n needed  
384.16 
Figure 2 
Mean absolute prediction error (MAPE) at 0.05 in predicted 
probabilities when applied to other targeted individuals  
  
  
number of candidate variables -> 
18 
  
Total n needed  
1013.2 
Figure 3 
Target shrinkage of <=10% (i.e. shrinkage factor = .9), Rsquared 
CS (proportion of variation explained) at least .1 
  
Criteria i in pms 
Total n needed  
716.234 
  
See Fig 4 for notes: Rsq often .1-.2 for binary for medical; 
use .5 for direct/explanatory  
  
Figure 5 
n participants required to ensure expected optimisim of <=0.05 
in the apparent Rsq 
  
Criteria ii in pms 
Anticipated Rsq  
0.2 
  
Ln L null  
-693.15 
  
Max Rsq (depends on proportion) 
0.75 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 7 of 10 
July 26, 2021 - Version 2.0 
 
  
n Outcome  
500 
  
n total sample size (arbitrary) 
1000 
  
Shrinkage factor corresponding to optimism of <=0.05 in Rsq 
0.84211 
  
Total n needed  
420.427 
  
  
  
  
  
Estimate the overall outcome proportion with sufficient 
precision (<=0.05) 
384.16 
  
Target a small mean absolute prediction error (0.05) 
1013.2 
  
Target a shrinkage factor of 0.9 
716.234 
  
Target small optimism of 0.05 in the apparent R2Nagelkerke 
420.427 
 
Data will be analyzed using R (version 4.0.3). Any statistical tests will be two-sided, with 
significance levels of 0.05. 
 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 8 of 10 
July 26, 2021 - Version 2.0 
 
References 
1.  Muñoz-Leyva F, Cubillos J, Chin KJ. Managing rebound pain after regional anesthesia. 
Korean J Anesthesiol. 2020 Oct;73(5):372–83.  
2.  Kim J-H, Koh HJ, Kim DK, Lee H-J, Kwon K-H, Lee K-Y, et al. Interscalene brachial 
plexus bolus block versus patient-controlled interscalene indwelling catheter analgesia for 
the first 48 hours after arthroscopic rotator cuff repair. J Shoulder Elbow Surg. 2018 
Jul;27(7):1243–50.  
3.  Ganta A, Ding D, Fisher N, Lavery J, Jain S, Tejwani NC. Continuous Infraclavicular 
Brachial Block Versus Single-Shot Nerve Block for Distal Radius Surgery: A Prospective 
Randomized Control Trial. J Orthop Trauma. 2018 Jan;32(1):22–6.  
4.  Ding DY, Manoli AI, Galos DK, Jain S, Tejwani NC. Continuous Popliteal Sciatic Nerve 
Block Versus Single Injection Nerve Block for Ankle Fracture Surgery: A Prospective 
Randomized Comparative Trial. J Orthop Trauma. 2015 Sep;29(9):393–8.  
5.  Abdallah FW, Halpern SH, Aoyama K, Brull R. Will the Real Benefits of Single-Shot 
Interscalene Block Please Stand Up? A Systematic Review and Meta-Analysis. Anesth 
Analg. 2015 May;120(5):1114–29.  
6.  Sunderland S, Yarnold CH, Head SJ, Osborn JA, Purssell A, Peel JK, et al. Regional Versus 
General Anesthesia and the Incidence of Unplanned Health Care Resource Utilization for 
Postoperative Pain After Wrist Fracture Surgery: Results From a Retrospective Quality 
Improvement Project. Reg Anesth Pain Med. 2016 Feb;41(1):22–7.  
7.  Sort R, Brorson S, Gögenur I, Nielsen JK, Møller AM. Rebound pain following peripheral 
nerve block anaesthesia in acute ankle fracture surgery: An exploratory pilot study. Acta 
Anaesthesiol Scand. 2019;63(3):396–402.  
8.  Barry GS, Bailey JG, Sardinha J, Brousseau P, Uppal V. Factors associated with rebound 
pain after peripheral nerve block for ambulatory surgery. Br J Anaesth [Internet]. 2020 Dec 
31 [cited 2021 Feb 14];0(0). Available from: https://bjanaesthesia.org/article/S0007-
0912(20)30914-4/abstract 
9.  Williams BA, Ibinson JW, Mangione MP, Modrak RT, Tonarelli EJ, Rakesh H, et al. 
Research priorities regarding multimodal peripheral nerve blocks for postoperative analgesia 
and anesthesia based on hospital quality data extracted from over 1,300 cases (2011-2014). 
Pain Med Malden Mass. 2015 Jan;16(1):7–12.  
10.  Gerbershagen HJ, Pogatzki-Zahn E, Aduckathil S, Peelen LM, Kappen TH, van Wijck AJM, 
et al. Procedure-specific risk factor analysis for the development of severe postoperative 
pain. Anesthesiology. 2014 May;120(5):1237–45.  

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 9 of 10 
July 26, 2021 - Version 2.0 
 
11.  Rai M, Mohapatra A, Radhesh, Bhandary B. Role of tourniquet in post-operative pain after 
total knee replacement surgery: a case control study. Int J Res Orthop. 2018 Oct 
24;4(6):924–7.  
12.  Kumar N, Yadav C, Singh S, Kumar A, Vaithlingam A, Yadav S. Evaluation of pain in 
bilateral total knee replacement with and without tourniquet; a prospective randomized 
control trial. J Clin Orthop Trauma. 2015 Jun;6(2):85–8.  
13.  Williams BA, Bottegal MT, Kentor ML, Irrgang JJ, Williams JP. Rebound Pain Scores as a 
Function of Femoral Nerve Block Duration after Anterior Cruciate Ligament Reconstruction: 
Retrospective Analysis of a Prospective, Randomized Clinical Trial. Reg Anesth Pain Med. 
2007;32(3):186–92.  
14.  Williams BA, Kentor ML, Vogt MT, Irrgang JJ, Bottegal MT, West RV, et al. Reduction of 
Verbal Pain Scores after Anterior Cruciate Ligament Reconstruction with 2-Day Continuous 
Femoral Nerve BlockA Randomized Clinical Trial. Anesthesiol J Am Soc Anesthesiol. 2006 
Feb 1;104(2):315–27.  
15.  Ilfeld BM, Vandenborne K, Duncan PW, Sessler DI, Enneking FK, Shuster JJ, et al. 
Ambulatory continuous interscalene nerve blocks decrease the time to discharge readiness 
after total shoulder arthroplasty: a randomized, triple-masked, placebo-controlled study. 
Anesthesiology. 2006 Nov;105(5):999–1007.  
16.  Nicholson T, Maltenfort M, Getz C, Lazarus M, Williams G, Namdari S. Multimodal Pain 
Management Protocol Versus Patient Controlled Narcotic Analgesia for Postoperative Pain 
Control after Shoulder Arthroplasty. Arch Bone Jt Surg. 2018 May;6(3):196–202.  
17.  Holmberg A, Hassellund SS, Drægni T, Nordby A, Ottesen FS, Gulestøl A, et al. Analgesic 
effect of intravenous dexamethasone after volar plate surgery for distal radius fracture with 
brachial plexus block anaesthesia: a prospective, double-blind randomised clinical trial*. 
Anaesthesia. 2020;75(11):1448–60.  
18.  Kvarda P, Hagemeijer NC, Waryasz G, Guss D, DiGiovanni CW, Johnson AH. Opioid 
Consumption Rate Following Foot and Ankle Surgery. Foot Ankle Int. 2019 Aug;40(8):905–
13.  
19.  Saini S, McDonald EL, Shakked R, Nicholson K, Rogero R, Chapter M, et al. Prospective 
Evaluation of Utilization Patterns and Prescribing Guidelines of Opioid Consumption 
Following Orthopedic Foot and Ankle Surgery. Foot Ankle Int. 2018 Nov;39(11):1257–65.  
20.  Won SH, Chung CY, Park MS, Lee SY, Suh YS, Lee KM. Characteristics of and Factors 
Contributing to Immediate Postoperative Pain After Ankle Fracture Surgery. J Foot Ankle 
Surg Off Publ Am Coll Foot Ankle Surg. 2018 Oct;57(5):890–3.  
21.  Womble TN, Comadoll SM, Dugan AJ, Davenport DL, Ali SZ, Srinath A, et al. Is 
supplemental regional anesthesia associated with more complications and readmissions after 

 
Sciatic Nerve Block Rebound Pain Risk Model 
Page 10 of 10 
July 26, 2021 - Version 2.0 
 
ankle fracture surgery in the inpatient and outpatient setting? Foot Ankle Surg Off J Eur Soc 
Foot Ankle Surg. 2020 Aug 14;  
22.  Goldstein RY, Montero N, Jain SK, Egol KA, Tejwani NC. Efficacy of popliteal block in 
postoperative pain control after ankle fracture fixation:  a prospective randomized study. J 
Orthop Trauma. 2012 Oct;26(10):557–61.  
23.  Merrill HM, Dean DM, Mottla JL, Neufeld SK, Cuttica DJ, Buchanan MM. Opioid 
Consumption Following Foot and Ankle Surgery. Foot Ankle Int. 2018 Jun;39(6):649–56.  
24.  Bhashyam AR, Keyser C, Miller CP, Jacobs J, Bluman E, Smith JT, et al. Prospective 
Evaluation of Opioid Use After Adoption of a Prescribing Guideline for Outpatient Foot and 
Ankle Surgery. Foot Ankle Int. 2019 Nov;40(11):1260–6.  
25.  Haimov I, Lavie P. Circadian Characteristics of Sleep Propensity Function in Healthy 
Elderly: A Comparison With Young Adults. Sleep. 1997 Apr 1;20(4):294–300.  
26.  Hirshkowitz M, Whiton K, Albert SM, Alessi C, Bruni O, DonCarlos L, et al. National Sleep 
Foundation’s sleep time duration recommendations: methodology and results summary. 
Sleep Health. 2015 Mar 1;1(1):40–3.  
27.  Sullivan LM, Massaro JM, D’Agostino RB. Presentation of multivariate data for clinical use: 
The Framingham Study risk score functions. Stat Med. 2004 May 30;23(10):1631–60.