A Single Blinded Randomized Controlled Trial Testing the Efficacy of a Pediatric Interactive Relational Agent vs. Standard Preoperative Education
Phase: NA
Conditions: Anesthesia, Anesthesia Emergence Delirium, Procedural Anxiety
Sponsor: Boston Children's Hospital

1 
 
A Pilot Randomized Controlled Trial Testing the Acceptability and Feasibility of a 
Pediatric Interactive Relational Agent (PIRA) vs. Standard Preoperative Education  
NCT#: 04586569  
Document Date: 01/26/2022 
 

2 
 
Title: A Pilot Randomized Controlled Trial Testing the Acceptability and Feasibility of a 
Pediatric Interactive Relational Agent (PIRA) vs. Standard Preoperative Education 
 
A. Specific Aims/Objectives 
The purpose of this study is to determine the feasibility and acceptability of enhanced 
preoperative education using a Pediatric Interactive Relational Agent (PIRA), designed for 
children ages 4 through 10 years and their families, compared to the standard preoperative 
education currently provided. Specifically, the study aims are to evaluate the following: 1. 
Parental/Child PIRA access and utilization, 2. Parental satisfaction and encountered issues with 
the PIRA, and 3. Differences in parental preoperative anxiety, child preoperative anxiety, and 
child experience with emergence from anesthesia between the PIRA and control group. 
 
B. Background and Significance 
Surgery and anesthesia can be extremely stressful experiences for the pediatric 
population. Children, in particular, may respond negatively to the impact of the experience. Kain 
(1999) and Kotiniemi et al. (1997) found that more than 60% of children experienced anxiety 
during preoperative waiting, increasing the risk of numerous psychological stressors such as 
postoperative delirium and agitation. Additionally, they found sleep and eating disorders, 
separation anxiety, temper tantrums, and aggressive behaviors lasting up to 2 weeks, 6 months, 
and 1 year in 50%, 20%, and 7% of pediatric patients, respectively (Yuki & Daaboul, 2011). The 
negative impact of stress in children can result in short and long-term sequelae (examples as 
mentioned above), a regression of the child’s emotional and behavioral development, delay of 
hospital discharge, and decreased overall parental satisfaction. Likewise, Stargatt et al. (2006) 

3 
 
and Bevan et al. (1990) demonstrated that parental anxieties can have a negative influence over 
the child’s postoperative behavioral outcomes.  
Anecdotally, parents frequently report a lack of understanding or a plan for how to 
address the idea of surgery and anesthesia to their child. This has been especially problematic in 
school-age children, who may lack the ability to anticipate and comprehend the necessary issues, 
solutions, and consequences. Evidence supports the benefits of enhanced preoperative education 
for pediatric patients and families. This, coupled with the lack of developed interactive agents 
preparing the pediatric patient and family for the perioperative anesthesia experience, 
demonstrated a need for the creation of such tools. This PIRA was developed by perioperative 
clinical experts, including perioperative Registered Nurses (RNs), Certified Registered Nurse 
Anesthetists (CRNAs), Physician Anesthesiologists, and Child Life Specialists (CLS) all with 
greater than two years’ clinical experience, in addition to nonmedical parents. Piloting this 
intervention in a randomized controlled trial (RCT) to determine its impact on pediatric and 
parental anxiety, satisfaction, likelihood to recommend to others, and emergence from 
anesthesia, is critical for successful implementation and potential future iterations.  
Given the large anesthetic volume at Boston Children’s Hospital (BCH), this PIRA 
intervention has the ability to positively impact tens of thousands of patients and families to 
adjunct the preoperative process to reduce anxiety, provide knowledge in regards to the 
perioperative anesthesia experience (preoperative, intraoperative and postoperative), enhance 
communication between the child and parent (encourage questions, explanations as it relates to 
the perioperative experience), prevent post-operative maladaptive behavior (POMB), and 
increase parental satisfaction with the perioperative experience. Testing these potential benefits 
while adjusting the PIRA based on user feedback is essential for maximizing the impact and 

4 
 
dissemination of the tool. Moving forward, expanding the PIRA to include additional languages, 
a wider developmental age range, different anesthetic options other than general anesthesia, and 
including anesthetic locations outside of the operating room—such as radiology, procedural 
rooms, wound care, gastroenterology, or the emergency department—will further enhance these 
benefits and include more families. 
 
C. Preliminary Studies 
Perioperative programs designed to decrease patient and parental anxieties and set 
expectations are paramount to positive medical and psychosocial interactions, satisfaction, and 
outcomes. Numerous perioperative program publications and randomized controlled trials 
(RCTs) show interactive preoperative information reduces parental and patient anxiety (Batuman 
et al., 2016; Bartik & Toruner, 2017; Cassidy et al. 1999; Chartrand et al., 2016; Felder-Puig et 
al., 2003; Fortier et al., 2015; Hilly et al., 2015; Kain et al., 1996; Kain et al., 1995; Kain et al., 
2007; Kassai et al., 2016; Liguori et al., 2016; Maclaren & Kain, 2007; McEwen et al., 2007; 
O’Conner-Von, 2008; Perry et al., 2012; Vaezzadeh et al., 2011; Wakimizu, 2015; Wantanakorn 
et al., 2018). Others have demonstrated that education and information programs decrease 
emotional trauma and regressive behaviors such as bedwetting, night terrors, and medical anxiety 
(Bartik & Toruner, 2017; Batuman et al., 2016; Hilly et al., 2015; Kain et al., 2007). RCTs have 
even found lower pain scores and opioid requirements in pediatric patients who participated in a 
preoperative information program (Chartrand et al., 2016; Kain et al., 2007).  Furthermore, a 
systematic review of RCTs using media-based interventions for preoperative education found a 
parental desire for more education. This systematic review reported a statistically significant 
reduction in anxiety and an increase in knowledge after the interventions (Lee et al., 2003). 

5 
 
In adults, educational interactive videos, also called conversational or relational agents 
(Trinh et al., 2017), which include user participation and feedback have shown a significant 
reduction in preoperative anxiety (Ryu et al., 2018; Tou et al., 2012; Hoybye et al., 2016), 
increased satisfaction (Narimatsu et al., 2011), increased knowledge, and a shorter anesthetic 
interview, potentially indicating a greater understanding of the anesthetic plan, process, and risks 
(Kakinuma et al., 2011). In pediatrics, there are no such tested educational interactive relational 
agents, avatars, or animated videos for surgery or anesthesia.  
Using an e-Delphi method to determine important topics to include in such a relational 
agent, in conjunction with a multidisciplinary team to develop age appropriate dialogue and 
content, a Pediatric Interactive Relational Agent (PIRA) was created. This PIRA, developed at 
BCH and programmed with a team at Northeastern University, is the first known relational agent 
specifically targeting pediatric patients ages 4 through 10 years old and their families, to educate 
them about anesthesia and the perioperative process prior to their surgical date. 
 
D. Design and Methods 
(1) Study Design 
This study is a pilot 1:1 randomized trial with one control group and one intervention 
group. Children age 4 through 10 years and their parents will be randomized to receive the PIRA 
versus the standard education prior to Otorhinolaryngology (ORL) or Urology surgery. For the 
purpose of this protocol, “parents” refers to the biological or legally adopted parents or legal 
guardians. Using surveys, assessments, and data we collect from the medical record and PIRA, 
we will look at PIRA access and utilization, parental satisfaction and encountered issues with 

6 
 
PIRA, and differences in parental and child preoperative anxiety as well as child emergence 
agitation between the intervention and control groups.    
 (2) Patient Selection and Inclusion/Exclusion Criteria 
We plan to enroll parents and their children, ages 4 through 10 years old, who are 
undergoing general anesthesia and elective, outpatient ORL or Urology surgery at BCH 
Waltham. Children will be American Society of Anesthesiology (ASA) physical status I or II. 
Parents must be at least 18 years of age or older. Since the study PIRA is currently only available 
in the English language, both parents and their children must be fluent in English in order to 
understand and interact with the PIRA. 
Parents without reliable internet access to the Google Chrome browser through a tablet or 
computer, parents of children with diagnosed developmental delays, parents who have children 
who have had previous surgical history under general anesthesia, and parents of children 
involved in other anesthesia-related studies at BCH will be excluded for this study. A prior 
surgery or procedure performed under local anesthesia or without anesthesia (e.g. newborn 
circumcision, frenulotomy, or newborn Achilles tenotomy), would not exclude the participant. 
These exclusion criteria will be assessed during the consent phone call.  
 (3) Description of Study Treatments or Exposures/Predictors 
Families that agree to participate in the study will be assigned a number (either one or 
two) from a random number generator. This will determine whether they are assigned to the 
PIRA (intervention) group or to the standard education (control) group.  

7 
 
Participants (children and their parents) will not be blinded to the study group (they will 
know if they received the PIRA). On the day of surgery, the research team member observing 
enrolled children for signs of anxiety in the preoperative area and the PACU nurse observing the 
children as they awaken from anesthesia post-operatively will be blinded to group assignment; 
they will not know, nor will they ask families, about their preoperative education experience.  
 
Intervention Group: If assigned to the intervention group, families will receive access to 
the PIRA for a minimum of two days in addition to the standard of care pre-op education 
provided for all families (described below for the control group). The child’s first name 
and age will be entered into the PIRA’s enrollment page so that the tool can address the 
child by name and educate them in an age-appropriate manner.  After an account has 
been created, an email containing instructions on the purpose of PIRA and how to use the 
program (Appendix A) will be sent to the email address provided by the family from a 
research team member at a BCH. Included in this email will be a hyperlink to access the 
PIRA, as well as a unique username and password for families to log in.  Once enrolled, 
the PIRA will be accessible to the family as often and for as long as desired up to 10 days 
prior to the scheduled surgery. Parents will receive a reminder email a few days prior to 
their child’s surgical date prompting them to log-in and complete the PIRA with their 
child at least once before the procedure. The parents will be encouraged to reach out to 
the Principal Investigator at any time if they have difficulty accessing the PIRA or if 
questions arise regarding the educational content within the program. 
Control group: Participants assigned to the control group will receive the standard of 
care preoperative education at BCH Waltham. These participants will not receive any 

8 
 
additional preoperative educational materials or resources other than what is currently 
provided by their care team. This usually includes a routine phone call, text, or email 
from a pre-operative staff member within a few days of the child’s scheduled procedure 
to provide basic information about fasting, when to arrive at the hospital, and anything 
additional that needs to be done prior to the procedure (e.g. COVID-19 testing), when 
applicable. Any specific information about the anesthetic will be deferred to the 
anesthesia team for the surgical day, per usual.  
 
In both the intervention and control groups, if parents specifically request additional 
information to prepare for surgery, pre-op staff may send them the “Social Story” developed by 
the Autism Spectrum Center, Child Life Services, and Waltham Surgical Services. This 
document provides an age-appropriate step-by-step guide as to what to do and what to expect on 
the day of surgery.    
On the day of surgery, all parents participating in the study, regardless of group 
assignment, will receive surveys to complete in the pre-operative holding area via REDCap®. 
The surveys will be given to the parent who consented to the study. If both parents are present, 
and both consented to the study, they will both receive surveys to complete. A research staff 
member trained for reliability and blinded to the study arm will assess the child for signs of 
anxiety in the pre-op holding area.  
For both groups, once the surgical procedure is complete, and the child has gone to the 
PACU, Emergence Agitation/Delirium will be assessed per standard of care at Waltham by the 
bedside PACU nurse using the Pediatric Agitation Emergence Delirium Scale (PAED). The 

9 
 
research team will collect this PAED information retrospectively from the child’s electronic 
medical record. 
(4) Definition of Primary and Secondary Outcomes/Endpoints:  
See Table 1 for Summary of Outcomes/Measures 
Patient Demographics 
 
Parental age, gender, ethnicity, and highest education level achieved will be collected as 
part of the REDCap®  preoperative survey to better understand the pilot group and educational 
needs (Appendix B). 
Primary Outcomes 
Parental Utilization:  
• When provided with the opportunity to obtain more information as it relates to the 
perioperative experience, do parents and their children use the tool?  
• How many times did they access the tool and for how long?  
• Were there any barriers to accessing the PIRA tool?  
If there were problems surrounding the accessibility, content, language, or any other 
system issues, this will affect utilization and the PIRAs future potential impact. 
Regardless of the medium, materials, or content provided, no preoperative educational 
program can be successful and sustainable unless readily adopted and utilized by parents 
and children. As such, a primary outcome will evaluate whether families actually viewed 
the materials, with whom, and how much time they spent with the PIRA. 
Parental Perceived Impact and Impressions:  

10 
 
• Did parents find the tool subjectively helpful?  
• What improvements or changes in content or language would they consider? 
•  Would parents recommend the PIRA to future families?  
Identifying parental perceived benefits or drawbacks to the PIRA may elucidate the 
familial interactions and how they feel about their educational experience in a different 
way than anxiety or other objective data. Similarly, given the novel nature of the PIRA, it 
is important to identify any educational gaps and potential for future topics and 
educational needs. If parents believe the PIRA improved their preoperative experience, it 
could be useful for future families.  
Secondary Outcomes 
Child Anxiety in Preoperative Waiting:  
As discussed, more than 60% of pediatric patients have been found to suffer from anxiety 
in the preoperative period. The effects of anxiety may have behavioral sequelae, or 
Postoperative Maladaptive Behaviors (POMB), which can last up to a year or longer. 
Therefore, reducing anxiety levels should be a priority of any educational tool for 
pediatrics. Numerous randomized control trials have demonstrated decreased child 
anxiety in preoperative waiting for those patients who participated in preoperative 
educational programs including videos, tours, apps, games, and books. Is there a similar 
difference in the control and intervention groups’ preoperative anxiety with this PIRA 
intervention? 
Parental Anxiety in Preoperative Waiting:  

11 
 
In several cited studies, parental anxiety directly correlates with the pediatric patient’s 
anxiety. Thus limiting or reducing parental anxiety is an important part of a preoperative 
education intervention. Does improved communication and education decrease parental 
anxiety?  An intervention which serves to improve communication between the 
parent/child as well as the parent/anesthesia provider AND reduce parental anxiety would 
perhaps provide significant benefits. 
Emergence from Anesthesia: Presence or absence of emergence agitation/delirium 
(ED):  
Depending on the study, ED has an incidence between 10-80%, with most studies 
showing 30-50% of children having ED after a general anesthetic (Przybylo et al., 2003; 
Ringblom et al., 2018). ED can be extremely distressing for parents, delay discharge from 
post-anesthesia care unit (PACU), can result in injury to the patient or staff (due to 
uncontrolled flailing, kicking, punching), and can also potentially result in the 
unintentional dislodgement of intravenous catheters or other tubes (e.g. nasogastric, 
gastrostomy, chest, endotracheal tubes). Data have shown a reduction in ED incidence 
with the use of preoperative education programs (Kain et al., 2004). Is there any effect on 
the incidence of ED for children who have more interactive preoperative education, such 
as PIRA, compared to standard information? 
 
(5) Data Collection Methods, Assessments, Interventions and Schedule 
On the day of the surgical procedure, participating parents in both the PIRA and the 
control groups will receive the same in-person surveys in the preoperative holding area. Surveys 
will be administered via REDCap® and completed on iPads which have been given to the 

12 
 
parents by a member of the research team (Appendix B). In both groups, the parent(s) who 
consented to the study and there on the day of surgery will be given the surveys.  
The surveys will have no identifying questions and will be automatically uploaded to 
REDCap®. They will be accessible only to members of the research team. Children will not be 
asked to complete any surveys but will be observed in the pre-op holding area by a member of 
the research team and in the PACU by a PACU nurse, as per standard of care at BCH Waltham. 
Both will have been trained for reliability in conducting the study-specific assessment tools. 
Pediatric Anesthesia Emergence Delirium (PAED) scores will be collected 
retrospectively from the child’s electronic medical record.  All data collected will be de-
identified and presented or published in aggregate. The research team will provide a finished 
copy of the manuscript upon request. 
Primary Outcomes Data 
Parental Utilization:  
The amount of time spent using the PIRA and the number of times the PIRA was 
accessed will be able to be collected via the program itself, stored on a secure 
Northeastern University Server, and will be accessible only to the study team. Each click 
of the mouse a user makes in the system will be timestamped and saved on the database. 
This information will be de-identified. Additionally, the parental survey administered by 
the research team member in the preoperative holding area will obtain self-reported data 
on PIRA access and time spent using the tool (Appendix B). 
Parental perceived impact and impressions:  

13 
 
This outcome will be assessed in the same survey (Appendix B) as Parental Utilization. 
The survey will have no identifying questions and will be automatically uploaded to 
REDCap®, accessible only to members of the research team. Any data will be de-
identified and presented in aggregate. 
Secondary Outcomes Data 
Child Anxiety in Preoperative Waiting:  
Child anxiety will be assessed using the modified Yale Preoperative Anxiety Scale 
(mYPAS), an observational state anxiety measure for children containing 27 items in five 
categories (activity, emotional expressivity, state of arousal, vocalization, and use of 
parents). See Figure 1. The mYPAS has been validated for measuring childrens’ anxiety 
(Kain et al., 1995; Kain et al., 1997). Members of the research team who will administer 
the mYPAS for this study will be trained for reliability. mYPAS scores range from 22.5 
to 100, with higher scores indicating more anxiety.  
Parental Anxiety in Preoperative Waiting:  
Parental anxiety will be assessed using the State-Trait Anxiety Inventory (STAI). See 
Figure 2. This self-report anxiety instrument contains two separate 20-item subscales that 
measure trait (baseline) and state (situational) anxiety (Spielberger, 1973; Spielberger, 
1983; Spielberger, 1989) This survey will take place in the preoperative holding area and 
will be administered in conjunction with the previous utilization/recommendation survey 
(Appendix B). The STAI is scored from 20-80, with higher scores indicating more 
anxiety. 
 

14 
 
Emergence from Anesthesia: Presence or absence of emergence agitation/delirium 
(ED):  
To determine any effect this PIRA may have on ED, the PACU RN assigned to care for 
the given patient will assess emergence upon patient arrival to PACU using the PAED 
scale. See Figure 3 for PAED Scale. The PAED is a routine assessment conducted by 
PACU RNs at Waltham for all post-operative pediatric patients upon emergence from 
anesthesia. A member of the research team will retrospectively extract the PAED score 
from the patient’s chart. The PAED scale measures five behaviors, each rated on a five-
point scale of zero to four. The scores are summed for total scale score up to a maximum 
value of 20 (Sikich & Lerman, 2004).  An assessment of the internal consistency and 
reliability of the PAED scale validated it for assessing ED in children recovering from 
general anesthesia (Ringblom et al., 2018). A PAED ≥10 yields 64 percent sensitivity and 
86 percent specificity (Sickich & Lerman, 2004), and a score of >12 yields 100 percent 
sensitivity and 94.5 percent specificity for the diagnosis of ED (Bajwa et al., 2010). For 
use in this study, ED will be considered present with a score > 12.  
(6) Study Timeline 
 
This study will take approximately one year to complete after IRB approval. 
E. Adverse Event Criteria and Reporting Procedures 
 
This is a minimal risk study. Any adverse effects associated with this study will be 
reported to the BCH IRB by the Principle Investigator (PI). If participants experience any 
adverse event related to this study and need follow-up for any concerns, they may be referred to 
the Patient Relations Department or Social Work Department at BCH as indicated. 
F. Data Management Methods 

15 
 
 
All collected survey data as well as pre-op and PACU observation data will be de-
identified and presented in aggregate, stored in REDCap® and kept in password-protected 
folders on BCH computers, accessible only by the research team members. The data collected 
directly from the PIRA program itself will be stored on a secure Northeastern University Server, 
and will be accessible only to the study team. Upon completion of data collection, the PIRA data 
will be transmitted back to BCH electronically and stored on the BCH server. All research team 
members will have completed CITI training. 
G. Quality Control Method 
 
Qualitative, free-text data will be analyzed by two or more members of the research team 
to ensure validity and concurrence of identified themes. 
H. Data Analysis Plan 
 
A Student’s t-test will be used to compare the mean anxiety scores, mYPAS, between the 
groups. Free text entries from parents will be clarified, consolidated, and summarized by the 
research team for themes. Quantitative data from the parental preoperative survey will be 
summarized for mean, median, mode, and standard deviation and presented both numerically and 
graphically using Microsoft Excel. 
I. Statistical Power and Sample Considerations 
This pilot protocol is powered to see an effect on pediatric anxiety, as measured by the 
mYPAS. A sample size of 30 per group will provide 80% power to detect a difference of 6 in 
mean mYPAS scores between the two groups (assuming a standard deviation for the mYPAS of 
8 based on published data in preoperative holding) (Kain et al., 1997), based on Student’s t-test 
and assuming a two-tailed alpha level of 0.05. With a range of 22.5-100, prior studies (Kain et 
al., 1997) have used a reference score of 37 as consideration for high anxiety cases, with the 

16 
 
prevalence of such high-anxiety cases at 24%. Based on anticipated attrition of 20% due to 
failure to interact with the PIRA, potential for surgical or anesthesia cancellations before or on 
the day of surgery, or withdrawal from the study, a total of 76 patients (38 in each group) will be 
recruited. 
J. Study Organization 
This study will be directed by Linda Bulich, MD and Mark Blazey, DNP, CRNA with the 
support of other research team members at Boston Children’s Hospital Waltham. The entire 
research team will be responsible for overseeing patient screening, recruitment and consent. 
Members of the research team will also be responsible for data collection, data analysis, as well 
as identifying and reporting adverse events. 
Study team members at Northeastern University (Teresa O’Leary, BA, PhD Student and 
Timothy Bickmore, BSE, MS, PhD, Associate Dean of Research) were responsible for 
programming the PIRA and will be monitoring the tool for any technical issues and updating as 
needed. 
 
Table 1. Outcomes 
Outcome 
Measurement 
Scoring 
Timing 
Participant 
Demographics 
Age of parent, Gender 
of parent who 
completed PIRA, 
Ethnicity 
See Appendix B 
Preoperative Waiting, 
self-reported. 
Parental Utilization of 
PIRA (Primary) 
Parental Survey 
See Appendix B 
Preoperative Waiting, 
self-reported. 

17 
 
Recommendation to 
Future Families 
(Primary) 
Parental Survey 
See Appendix B 
Sliding Scale (0 = 
Definitely NOT 
Recommended - 10 = 
Definitely 
Recommend)  
Preoperative Waiting, 
self-reported.  
Potential Gaps or 
Recommendations for 
Future Iterations 
(Primary) 
Parental Survey, free 
text entry 
See Appendix B 
Summarized for 
themes 
Preoperative Waiting, 
self-reported.  
Parental Anxiety 
(Secondary) 
STAI  
See Appendix B and 
Figure 2. Range 20-
80, higher = greater 
anxiety 
Preoperative Waiting, 
self-reported 
Child Anxiety 
(Secondary) 
mYPAS 
(Observational) 
See Figure 1. Range 
22.5-100 
Preoperative Waiting, 
observed by a 
Research Assistant 
(RA) 
Emergence from 
Anesthesia (ED) 
(Secondary) 
PAED Scale 
(Observational) 
See Figure 3. Range 
0-20; Threshold for 
ED >12  
PACU, observed by a 
Registered Nurse 
(RN) as per BCH 
standard of care. 
 
 
 

18 
 
Appendix A. Email Instructions for use of the Interactive Anesthesia Education Program 
 
To the Parent/Guardian of ______: 
Thank you for your participation in this study. Please refer to the below instructions for using the 
PIRA program. 
Access to the education program is by a computer or tablet device through the hyperlink 
included below on the Google Chrome browser ONLY. Please note, a mobile device and other 
internet browsers will not work with this online tool. 
Along with the hyperlink, we have also provided you with a unique username and password. 
Once you click on the hyperlink, you will be prompted to enter this information to access the 
program. 
Link: 
Username: 
Password: 
When you are ready to review the education materials, click, “start at the beginning” You or your 
child will be required to periodically answer questions or respond by clicking the mouse to move 
it forward. Either you or your child can click, but please make sure that both you and your child 
are able to see and hear the teaching the entire time. 
If you need to stop for a prolonged period of time for whatever reason, you can start at the 
beginning again to ensure no content is missed. Refreshing the page will reset the program. It 
will not save where you left off. 
You will receive one email reminder a few days prior to your child’s surgical date reminding you 
to complete the PIRA at least once with your child. If you have any problems accessing the 
education program, or any issues during its use, please contact the study PI, Linda Bulich, at 
617-355-7737 or linda.bulich@childrens.harvard.edu. 
 
Thank you, 
Linda Bulich, MD 
 
  
 

19 
 
Appendix B. Parental PIRA Survey 
Please complete the survey, below. Your responses to the following questions will better help us 
to understand how to improve our preoperative education and communication for parents and 
their children. As a study participant, you were randomly assigned to either receive an online 
interactive tool focused on educating children, named PIRA, or you received our standard 
preoperative information. If any of the below questions make you uncomfortable, please leave 
them blank. 
Demographics: 
Q1. Your current age (in years):  
Q2. Gender: 
Q3. Ethnicity: 
Q4. Highest level of education achieved:  
Pre-Operative Education Experience: 
Q1. What of the following statements describes your experience with the anesthesia education 
agent, Ira? (Select all that apply) 
❏ I did not receive the link to Ira 
❏ I received the link to Ira, but did not watch it 
❏ I received the link to Ira, but a different care provider or family member watched Ira. 
❏ I watched Ira WITHOUT my child 
❏ I watched Ira WITH my child 
❏ I had a hard time using Ira 
❏ My child liked using or watching Ira 
❏ My child seemed LESS stressed about surgery or anesthesia because they watched 
Ira 
❏ My child seemed MORE stressed about surgery or anesthesia because they watched 
Ira 
❏ I understood anesthesia better after watching Ira 
❏ Ira helped me talk to my child about anesthesia and surgery 
❏ Ira helped answer questions about anesthesia 
❏ I felt less stressed about surgery or anesthesia because I watched Ira 
❏ Ira made my surgery experience BETTER 
❏ Ira made my surgery experience WORSE 

20 
 
❏ Ira was difficult to use or understand 
 
Q2. In total, about how long did you spend with the interactive education agent, Ira? 
❏ Less than 5 minutes 
❏ 5-30 minutes 
❏ 30-60 minutes 
❏ More than 60 minutes 
❏ I did not view the agent 
 
Q3. How likely are you to recommend the interactive education agent, Ira, to future 
families? (Skip if you did not receive or view Ira) 
**This is a sliding scale** 
0 = Not at all Likely to Recommend------------------100 = Extremely likely to recommend 
 
Q4. Write any topics that were missing, unclear, or absent from the interactive education 
agent, Ira. (Skip if you did not receive or view Ira) 
**Free Text Response_____________________________________________ 
Q5. Write any suggestions for future improvements to the interactive education agent, Ira. 
This may include any difficulties in accessing or using it. (Skip if you did not receive Ira) 
**Free Text Response_____________________________________________ 
Q6. Self-evaluation questionnaire. Rate the following statements how they apply to you 
from 1 = Not at all to 4 = Very Much So 
 
 
 
1 = 
Not 
at 
all 
2 = 
So
me
wh
at 
3 = 
Mo
der
atel
y 
so 
4 = 
Ver
y 
mu
ch 
so 
I feel 
calm 
 
 
 
 
 
 
I feel 
secure 
 
 
 
 
 
 
I am 
tense 
 
 
 
 
 
 
I feel 
strained 
 
 
 
 
 
 

21 
 
I feel at 
ease 
 
 
 
 
 
 
I feel 
upset 
 
 
 
 
 
 
I am 
presently 
worrying 
over 
possible 
misfortu
nes 
 
 
 
 
 
 
I feel 
satisfied 
 
 
 
 
 
 
I feel 
frightene
d 
 
 
 
 
 
 
I feel 
comforta
ble 
 
 
 
 
 
 
I feel 
self-
confiden
t 
 
 
 
 
 
 
I feel 
nervous 
 
 
 
 
 
 
I am 
jittery 
 
 
 
 
 
 
I feel 
indecisiv
e 
 
 
 
 
 
 
I am 
relaxed 
 
 
 
 
 
 
I feel 
content 
 
 
 
 
 
 
I am 
worried 
 
 
 
 
 
 
I feel 
confused 
 
 
 
 
 
 

22 
 
I feel 
steady 
 
 
 
 
 
 
I feel 
pleasant 
 
 
 
 
 
 
 
 
 
Figure 1. Modified Yale Preoperative Anxiety Scale (mYPAS) for Child Anxiety 
 
 
 

23 
 
Figure 2. Parental State-Trait Anxiety Inventory (STAI) for Parental Anxiety 
 

24 
 
Figure 3. PACU Pediatric Anesthesia Emergence Delirium (PAED) for Anesthesia 
Emergence Delirium
 
 
 

25 
 
K. References 
  
Akdağ, M. (2014). A multi-centric prospective study: Anxiety and associated factors among  
parents of children undergoing mild surgery in ENT. Journal of Clinical and 
Experimental Investigations, 5(2), 206–210. 
Aytekin, A., Doru, Ö., & Kucukoglu, S. (2016). The effects of distraction on 
preoperative anxiety level in children. Journal of Perianesthesia Nursing: Official 
Journal of the American Society of Paranesthesia Nurses / American Society of 
Paranesthesia Nurses, 31(1), 56–62. 
Bartik, K., & Toruner, E.K. (2017). Effectiveness of a preoperative preparation program on 
children’s emotional states and parental anxiety. Journal of Perianesthesia Nursing: 
Official Journal of the American Society of PeriAnesthesia Nurses / American Society of 
PeriAnesthesia Nurses. https://doi.org/10.1016/j.jopan.2017.09.008 
Batuman, A., Gulec, E., Turktan, M., Gunes, Y., & Ozcengiz, D. (2016). Preoperative 
informational video reduces preoperative anxiety and postoperative negative behavioral 
changes in children. Minerva Anestesiologica, 82(5), 534–542. 
Beringer, R. M., Greenwood, R., & Kilpatrick, N. (2013). Development and validation of the 
Pediatric anesthesia behavior score - an objective measure of behavior during induction 
of anesthesia. Pediatric Anesthesia, 24(2), 196–200. 
Bevan, J.C., Johnston, C., Haig, M.J., Tousignant, G., Lucy, S., Kirnon, V. et al. (1990). 
Preoperative parental anxiety predicts behavioural and emotional responses to induction 
of anaesthesia in children. Canadian Journal Anaesthesia, 37(2), 177-182. 

26 
 
Bajwa SA, Costi D, Cyna AM. A comparison of emergence delirium scales following general 
anesthesia in children. Paediatr Anaesth 2010; 20:704. 
Buckley, A., & Savage, E. (2010). Preoperative information needs of children undergoing 
tonsillectomy. Journal of Clinical Nursing, 19(19-20), 2879–2887. 
Cassady, J. F., Jr, Wysocki, T. T., Miller, K. M., Cancel, D. D., & Izenberg, N. (1999). Use of a 
preanesthetic video for facilitation of parental education and anxiolysis before pediatric 
ambulatory surgery. Anesthesia and Analgesia, 88(2), 246–250. 
Chartrand, J., Tourigny, J., & MacCormick, J. (2016). The effect of an educational DVD on 
parents’ and children’s outcomes after a same day surgery: A randomized controlled trial. 
Journal of Advanced Nursing, (John Wiley & Sons Ltd), 73(3) 599-611. 
Copanitsanou, P., & Valkeapaa, K. (2013). Effects of education of paediatric patients undergoing 
elective surgical procedures on their anxiety – a systematic review. Journal of Clinical 
Nursing, 23(7-8), 940-954. doi:10.1111/jocn.12187 Brazilian Journal of Anesthesiology 
(Elsevier), 63(6), 473–482. 
Drake, J., Johnson, N., Stoneck, A. V., Martinez, D. M., & Massey, M. (2012). Evaluation of a 
coping kit for children with challenging behaviors in a pediatric hospital. Pediatric 
Nursing, 38(4), 215–221. 
Donnohoe, H., Stellefson, M., & Tennant, B. (2011). Advantages and limitations of the e-Delphi 
technique: Implications for health education researchers. American Journal of Health 
Education, 43(1), 38-46. 
Erlingsson, C. & Brysiewicz, P. (2017). A hands-on guide to doing content analysis. African 
Journal of Emergency Medicine, 7, 93-99. 

27 
 
Felder-Puig, R., Maksys, A., Noestlinger, C., Gadner, H., Stark, H., Pfluegler, A., & Topf, R. 
(2003). Using a children’s book to prepare children and parents for elective ENT surgery: 
Results of a randomized clinical trial. International Journal of Pediatric 
Otorhinolaryngology, 67(1), 35–41. 
Fortier, M. A., Bunzli, E., Walthall, J., Olshansky, E., Saadat, H., Santistevan, R. … Kain, Z. N. 
(2015). Web-based tailored intervention for preparation of parents and children for 
outpatient surgery (WebTIPS). Anesthesia & Analgesia, 120(4), 915–922. 
Hilly, J., Hörlin, A.-L., Kinderf, J., Ghez, C., Menrath, S., Delivet, H. … Dahmani, S. (2015). 
Preoperative preparation workshop reduces postoperative maladaptive behavior in 
children. Paediatric Anaesthesia, 25(10), 990–998. 
Hoybye, M.T., Vesterby, M., & Jorgensen, L.B. (2016). Producing patient-avatar identification 
in animation video information on spinal anesthesia by different narrative strategies. 
Health Informatics Journal, 22(2), 370-382. 
Jenkins, B. N., Fortier, M. A., Kaplan, S. H., Mayes, L. C., & Kain, Z. N. (2014). Development 
of a short version of the modified Yale preoperative anxiety scale. Anesthesia & 
Analgesia, 119(3), 643–650. 
Justus, R., Wyles, D., Wilson, J., Rode, D., Walther, V., & Lim-Sulit, N. (2006). Preparing 
children and families for surgery: Mount Sinai’s multidisciplinary perspective. Pediatric 
Nursing, 32(1), 35–43. 
Kakinuma, A., Nagatani, H., Otake, H., Mizuno, J., Nakata, Y. (2011). The effects of short 
interactive animation video information on preanesthetic anxiety, knowledge, and 
interview time: A randomized control trial. Anesthesia & Analgesia, 112(6), 1314-1318. 

28 
 
Kain, Z. N., Caldwell-Andrews, A. A., Mayes, L. C., Weinberg, M. E., Wang, S.-M., MacLaren 
J. E., & Blount, R. L. (2007a). Family-centered preparation for surgery improves 
perioperative outcomes in children. Anesthesiology, 106(1), 65–74. 
Kain, Z. N., Caldwell-Andrews, A. A., Mayes, L. C., Weinberg, M. E., Wang, S.-M., MacLaren, 
J. E., & Blount, R. L. (2007b). Family-centered preparation for surgery improves 
perioperative outcomes in children: a randomized controlled trial. Anesthesiology, 106(1), 
65–74. 
Kain, Z. N., Mayes, L. C., & Caramico, L. A. (1996). Preoperative preparation in children: A 
cross-sectional study. Journal of Clinical Anesthesia, 8(6), 508–514. 
Kain, Z. N., Mayes, L. C., Cicchetti, D. V., Bagnall, A. L., Finley, J. D., & Hofstadter, M. B. 
(1997). The Yale preoperative anxiety scale. Anesthesia & Analgesia, 85(4), 783–788. 
Kain, Z. N., Mayes, L. C., Cicchetti, D. V., Caramico, L. A., Spieker, M., Nygren, M. M., & 
Rimar, S. (1995). Measurement tool for preoperative anxiety in young children: The Yale 
preoperative anxiety scale. Child Neuropsychology: A Journal on Normal and Abnormal 
Development in Childhood and Adolescence, 1(3), 203–210. 
Kain, Z. (1999). Perioperative information and parental anxiety: The next generation. Anesthesia 
& Analgesia, 88, 237-239. 
Kain, Z., Caldwell-Andrews, A.A., Maranets, I., McClain, B.,  Gaal, D., Mayes, L., Feng, R., & 
Zheng, H. (2004). Preoperative anxiety and emergence delirium and postoperative 
maladaptive behaviors. Anesthesia & Analgesia, 99(6), 1648-1654. 
Kassai, B., Rabilloud, M., Dantony, E., Grousson, S., Revol, O., Malik, S…de Souza Neto, E. P. 
(2016). Introduction of a paediatric anaesthesia comic information leaflet reduced 

29 
 
preoperative anxiety in children. British Journal of Anaesthesia, 117(1), 95–102. JAMA 
Pediatrics, 170(8), e160533. 
Kotiniemi L.H., Ryhanen P.T., Valanne J., Jokela, R., Mustonen, A., & Poukkula, E. (1997). 
Postoperative symptoms at home following day-case surgery in children: A multicenter 
survey of 551 children. Anaesthesia, 52, 963–969. 
Kotiniemi L.H., Ryhanen P.T., & Moilanen I.K. (1997). Behavioral changes in children 
following day case surgery: A 4-week follow-up of 551 children. Anaesthesia, 52, 970 
976. 
Lee, A., Chui, P.T., & Gin, T. (2003). Educating patients about anesthesia: A systematic review 
of randomized control trials of media-based interventions. Anesthesia & Analgesia, 96, 
1424-1431. 
Lerman, J.L., Davidson, A., & Crowley, M. (2018). Emergence delirium and agitation in 
children. UpToDate, Post, TW (Ed), UpToDate, Waltham, MA. Retrieved, December 26, 
2019 from https://www.uptodate.com/contents/emergence-delirium-and-agitation-in-
children. 
Maclaren, J., & Kain, Z. N. (2007). Pediatric preoperative preparation: A call for evidence-based 
practice. Pediatric Anesthesia, 17(11), 1019–1020. 
McEwen, A., Moorthy, C., Quantock, C., Rose, H., & Kavanagh, R. (2007). The effect of 
videotaped preoperative information on parental anxiety during anesthesia induction for 
elective pediatric procedures. Paediatric Anaesthesia, 17(6), 534–539. 
Melnyk, B.M. & Fineout-Overholt, E. (2018). Evidence-Based practice in cursing & healthcare, 
a guide to best practice, 4th edition. Philadelphia: Woltners Kluwer/Lippincott Williams 
& Wilkins. 

30 
 
Milliken-Glabe, S.J., Zuk, J., Ziniel, S.I., Bjur, K.A., Álvarez, M.S., Szolnoki, J., & Janosy, N.R. 
(2017). First steps in validating the pediatric anesthesia parent satisfaction (PAPS) 
survey. Pediatric Anesthesia, 27, 153–161. 
Narimatsu, H., Kakinuma, A., Sawa, T., Komatsu, T., Matsumura, T., Kami, M., & Nakata, Y. 
(2011). Usefulness of a bidirectional e-learning material for explaining surgical 
anesthesia to cancer patients. Annals of Oncology, 22, 2121-2128. 
Nilsson, U., Ericsson, E., Eriksson, M., Idvall, E., & Bramhagen, A.-C. (2019). Psychometric 
evaluation of the Post Hospitalization Behavior Questionnaire for Ambulatory Surgery 
and postoperative behavior and recovery in children undergoing tonsil surgery. Journal of 
Perioperative Practice, 29(4), 94–101. https://doi.org/10.1177/1750458918782878 
O’Conner-Von, S. (2008). Preparation of adolescents for outpatient surgery: Using an 
internet program. AORN Journal, 87(2), 374–398. 
Perry, J. N., Hooper, V. D., & Masiongale, J. (2012). Reduction of preoperative anxiety in 
pediatric surgery patients using age-appropriate teaching interventions. Journal of 
Perianesthesia Nursing: Official Journal of the American Society of Paranesthesia 
Nurses, 27(2), 69–81. 
Pinto, R.P. & Hollandsworth, J.G. Jr. (1989). Using videotape modeling to prepare children 
psychologically for surgery: Influence of parents and costs versus benefits of providing 
preparation services. Health Psychology, 8(1), 79-95. 
Przybylo, H. J., Martini, D. R., Mazurek, A. J., Bracey, E., Johnsen, L., & Coté, C. J. (2003). 
Assessing behaviour in children emerging from anaesthesia: can we apply psychiatric 
diagnostic techniques? Pediatric Anaesthesia, 13(7), Retrieved from 
https://onlinelibrary.wiley.com/doi/full/10.1046/j.1460-9592.2003.01099.x. 

31 
 
Ringblom J, Wåhlin I, Proczkowska M. (2018). A psychometric evaluation of the Pediatric 
Anesthesia Emergence Delirium scale. Paediatric Anaesthesia, 28(4). 
Shafer, J. S., Jenkins, B. N., Fortier, M. A., Stevenson, R. S., Hikita, N., Zuk, J. et. al. (2018). 
Parental satisfaction of child’s perioperative care. Pediatric Anesthesia, 28(11), 955–962. 
Sheer, B. (2006). Commentary on Smith L and Callery P (2005) Children’s accounts of their 
preoperative information needs. Journal of Clinical Nursing, 14, 230-238. Journal of 
Clinical Nursing, 15(2), 244–245. Journal of Clinical Nursing, 14(2), 230–238. 
Sikich N, Lerman J. Development and psychometric evaluation of the pediatric anesthesia 
emergence delirium scale. Anesthesiology 2004; 100:1138. 
Skulmoski, G.J., Hartman, F.T., & Krahn, J. (2007). The Delphi method for graduate research. 
The Journal for Information Technology Education, 6, 1-21.  
Sorensen, H. L., Card, C. A., Malley, M. T., & Strzelecki, J. M. (2009). Using a collaborative 
Child life approach for continuous surgical preparation. AORN Journal. 
https://doi.org/10.1016/j.aorn.2009.06.033 
Stargatt, R., Davidson, A.J., Huang, G.H., Czarnecki, C., Gibson, M.A., Stewart, S.A., & 
Jamson, K. (2006). A cohort study of the incidence and risk factors for negative behavior 
changes in children after general anesthesia. Paediatric Anaesthesia, 16(8), 846-59. 
Toronto, C. (2017). Considerations when conducting e-Delphi research: A case study. Nurse 
Researcher, 1, 10-15. 
Tou, S., Tou, W., Mah, D., Karatassas, T., & Hewett, P. (2012). Effect of preoperative two-
dimensional animation information on perioperative anxiety and knowledge retention in 
patients undergoing bowel surgery: A randomized pilot study. Colorectal disease: The 

32 
 
Official Journal of the Association of Coloproctology of Great Britain and Ireland, 15(5), 
e256-265. 
Trinh, H., Asadi, R., & Bickmore, T. (2017). Designing Health Conversations with Relational 
Agents. Retrieved from https://researcher.watson.ibm.com/researcher/files/us-
rjmoore/Trinh.pdf 
Vaezzadeh, N., Douki, Z. E., Hadipour, A., Osia, S., Shahmohammadi, S., & Sadeghi, R. (2011). 
The effect of performing preoperative preparation program on school age children’s 
anxiety. Iranian Journal of Pediatrics, 21(4), 461–466. 
Wakimizu, R. (2015). Effects of preoperative at-home preparation on children’s behavioral 
outcomes in Japan. International Journal of Surgery Open, 1, 5–9. 
Wakimizu, R., Kamagata, S., Kuwabara, T., & Kamibeppu, K. (2009). A randomized controlled 
trial of an at-home preparation programme for Japanese preschool children: effects on 
children’s and caregivers' anxiety associated with surgery. Journal of Evaluation in 
Clinical Practice, 15(2), 393–401. 
Wantanakorn, P., Harintajinda, S., Chuthapisith, J., Anurathapan, U., & Rattanatamrong, P. 
(2018). A new mobile application to reduce anxiety in pediatric patients before bone 
marrow aspiration procedures. Hospital Pediatrics, 8(10), 643–650. 
Wisselo, T.L., Stuart, C., & Muris, P. (2004). Providing parents with information before 
anesthesia: What do they really want to know? Pediatric Anesthesia, 14, 299-307. 
Wolfer, J. A., & Visintainer, M. A. (1980). Prehospital psychological preparation for 
tonsillectomy patients: Effects on children’s and parents' adjustment. Journal of the 
American Academy of Child Psychiatry, 19(3), 540. 

33 
 
Yuki, K. & Daaboul, D. (2011). Postoperative maladaptive changes in children. Middle East 
Journal of Anesthesiology, 21(2), 183-189.