Developing Interoperable Tools for Anxiety and Depression Screening in Epilepsy
Phase: NA
Conditions: Epilepsy
Sponsor: Wake Forest University Health Sciences

 
 
 
 
Developing Interoperable Tools for Anxiety and 
Depression Screening in Epilepsy 
  
National Clinical Trial (NCT) Identified Number:  NCT05864612 
Principal Investigator: Heidi M. Munger Clary, MD, MPH 
 
Grant Title: Developing interoperable tools for anxiety and depression screening 
Grant Number: R03 TR004251 
Version Number:  v4.0 
March 15, 2024 
 
 

 
CONFIDENTIALITY STATEMENT 
This document is confidential communication. Acceptance of this document constitutes agreement by the 
recipient that no unpublished information contained herein will be published or disclosed without prior 
approval of the Principal Investigator or other participating study leadership and as consistent with the 
NIH terms of award.  
 

 
 
15 March 2024 
 
 
Table of Contents 
STATEMENT OF COMPLIANCE ............................................................................................................................. 1 
1 
PROTOCOL SUMMARY ............................................................................................................................... 2 
1.1 
Synopsis ................................................................................................................................................ 2 
1.2 
Schema ................................................................................................................................................. 5 
1.3 
Schedule of Activities .......................................................................................................................... 6 
2 
INTRODUCTION ............................................................................................................................................ 6 
2.1 
Study Rationale .................................................................................................................................... 6 
2.2 
Background........................................................................................................................................... 7 
2.3 
Risk/Benefit Assessment .................................................................................................................. 10 
2.3.1 
Known Potential Risks ................................................................................................... 10 
2.3.2 
Known Potential Benefits .............................................................................................. 11 
2.3.3 
Assessment of Potential Risks and Benefits .............................................................. 11 
3 
OBJECTIVES AND ENDPOINTS .............................................................................................................. 12 
4 
STUDY DESIGN ........................................................................................................................................... 13 
4.1 
Overall Design .................................................................................................................................... 13 
4.2 
Scientific Rationale for Study Design.............................................................................................. 14 
4.3 
Justification for Intervention ............................................................................................................. 14 
4.4 
End-of-Study Definition ..................................................................................................................... 15 
5 
STUDY POPULATION ................................................................................................................................ 15 
5.1 
Inclusion Criteria ................................................................................................................................ 15 
5.2 
Exclusion Criteria ............................................................................................................................... 15 
5.3 
Lifestyle Considerations .................................................................................................................... 15 
5.4 
Screen Failures .................................................................................................................................. 15 
5.5 
Strategies for Recruitment and Retention ...................................................................................... 15 
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) ............................................ 16 
6.1 
Study Intervention(s) or Experimental Manipulation(s) Administration ...................................... 16 
6.1.1 
Study Intervention or Experimental Manipulation Description ................................. 17 
6.1.2 
Administration and/or Dosing ....................................................................................... 18 
6.2 
Fidelity ................................................................................................................................................. 18 
6.2.1 
Interventionist Training and Tracking .......................................................................... 18 
6.3 
Measures to Minimize Bias: Randomization and Blinding ........................................................... 18 
6.4 
Study Intervention/Experimental Manipulation Adherence.......................................................... 18 
6.5 
Concomitant Therapy ........................................................................................................................ 18 
6.5.1 
Rescue Therapy ............................................................................................................. 19 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND 
PARTICIPANT DISCONTINUATION/WITHDRAWAL ......................................................................................... 19 
7.1 
Discontinuation of Study Intervention/Experimental Manipulation ............................................. 19 
7.2 
Participant Discontinuation/Withdrawal from the Study ............................................................... 19 
7.3 
Lost to Follow-Up ............................................................................................................................... 19 
8 
STUDY ASSESSMENTS AND PROCEDURES ..................................................................................... 19 
8.1 
Endpoint and Other Non-Safety Assessments .............................................................................. 19 
8.2 
Safety Assessments & SafETY PLAN ............................................................................................ 20 
8.3 
Adverse Events and Serious Adverse Events ............................................................................... 21 
8.4 
Unanticipated Problems .................................................................................................................... 21 
8.4.1 
Definition of Unanticipated Problems .......................................................................... 21 
8.4.2 
Unanticipated Problems Reporting .............................................................................. 22 

 
 
15 March 2024 
 
 
8.4.3 
Reporting Unanticipated Problems to Participants ................................................... 22 
9 
STATISTICAL CONSIDERATIONS .......................................................................................................... 22 
9.1 
Statistical Hypotheses ....................................................................................................................... 22 
9.2 
Sample Size Determination .............................................................................................................. 23 
9.3 
Populations for Analyses .................................................................................................................. 23 
9.4 
Statistical Analyses............................................................................................................................ 23 
9.4.1 
General Approach .......................................................................................................... 23 
9.4.2 
Analysis of the Primary Endpoint(s) ............................................................................ 23 
9.4.3 
Analysis of the Secondary Endpoint(s) ....................................................................... 23 
9.4.4 
Safety Analyses .............................................................................................................. 23 
9.4.5 
Baseline Descriptive Statistics ..................................................................................... 24 
9.4.6 
Planned Interim Analyses ............................................................................................. 24 
9.4.7 
Sub-Group Analyses ..................................................................................................... 24 
9.4.8 
Tabulation of Individual Participant Data .................................................................... 24 
9.4.9 
Exploratory Analyses ..................................................................................................... 24 
10 
SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS .............................. 24 
10.1 
Regulatory, Ethical, and Study Oversight Considerations ........................................................... 24 
10.1.1 
Informed Consent Process ........................................................................................... 25 
10.1.2 
Study Discontinuation and Closure ............................................................................. 26 
10.1.3 
Confidentiality and Privacy ........................................................................................... 26 
10.1.4 
Future Use of Stored Specimens and Data ............................................................... 27 
10.1.5 
Key Roles and Study Governance .............................................................................. 27 
10.1.6 
Safety Oversight ............................................................................................................. 28 
10.1.7 
Clinical Monitoring .......................................................................................................... 28 
10.1.8 
Quality Assurance and Quality Control ....................................................................... 28 
10.1.9 
Data Handling and Record Keeping ............................................................................ 29 
10.1.10 
Protocol Deviations ........................................................................................................ 29 
10.1.11 
Publication and Data Sharing Policy ........................................................................... 30 
10.1.12 
Conflict of Interest Policy .............................................................................................. 30 
10.2 
Additional Considerations ................................................................................................................. 30 
10.3 
Abbreviations and Special Terms ................................................................................................... 31 
10.4 
Protocol Amendment History ........................................................................................................... 33 
11 
REFERENCES ............................................................................................................................................. 35 
 

 
 
 
15 March 2024 
 
 
 
1 
STATEMENT OF COMPLIANCE 
 
(1) The trial will be carried out in accordance with International Council on Harmonisation Good 
Clinical Practice (ICH GCP) and the following:  
 
• 
United States (US) Code of Federal Regulations (CFR) applicable to clinical studies (45 CFR Part 
46, 21 CFR Part 50, 21 CFR Part 56, 21 CFR Part 312, and/or 21 CFR Part 812).  
 
National Institutes of Health (NIH)-funded investigators and clinical trial site staff who are 
responsible for the conduct, management, or oversight of NIH-funded clinical trials have 
completed Human Subjects Protection and ICH GCP Training. 
 
The protocol, informed consent form(s), recruitment materials, and all participant materials will 
be submitted to the IRB for review and approval. Approval of both the protocol and the consent 
form(s) must be obtained before any participant is consented. Any amendment to the protocol 
will require review and approval by the IRB before the changes are implemented to the study. All 
changes to the consent form(s) will be IRB approved; a determination will be made regarding 
whether a new consent needs to be obtained from participants who provided consent, using a 
previously approved consent form. 
 
 

 
 
 
15 March 2024 
 
 
 
2 
1 
PROTOCOL SUMMARY 
1.1 
SYNOPSIS  
 
Title: 
Developing Interoperable Tools for Anxiety and Depression Screening 
in Epilepsy 
Grant Number: 
R03 TR004251 
Study Description: 
This study will develop evaluate four (4) different screening methods to 
screen for anxiety and depression in patients with epilepsy.  
 
Adult patients scheduled an epilepsy clinical visit at either site will be 
randomized to one of the four groups - to complete standard care 
screening questionnaires on anxiety and depression, delivered by one 
of four methods:  
1. Twilio text message 
2. REDCap email survey link 
3. EHR portal (with reminder) 
4. EHR portal (no reminder) 
 
Hypothesis: the screening completion proportion will vary across the 4 
modalities tested.  
 
Research team will evaluate which of the screening methods were 
completed.  In addition, the study will implement and evaluate a 
reproducible approach to EHR and research system integration via 
Epic’s Kit API integration with REDCap for Epic EHR flowsheet data, 
using flowsheet-stored anxiety and depression instruments from 
epilepsy clinics as a demonstration case. 

 
 
 
15 March 2024 
 
 
 
3 
Objectives*: 
 
Aim 1: To evaluate screening completion (primary outcome) and 
process measures comparing interoperable, REDCap-based methods to 
EHR patient portal-based methods for delivering standard care 
validated anxiety and depression instruments to epilepsy clinic 
patients. In a randomized pragmatic study of four modalities with 
N=220 individuals per arm and primary outcome proportion fully 
screened prior to clinic visit, we will compare Twilio text message 
delivery via REDCap vs. REDCap email survey links vs. EHR portal 
questionnaires with reminder message vs. standard EHR portal 
questionnaires without reminder. 
   
Aim 2a: To implement and evaluate a reproducible approach to EHR 
and research system integration via Epic’s Kit database views and 
integration with a REDCap plug-in for Epic EHR flowsheet data, using 
flowsheet-stored anxiety and depression instruments from epilepsy 
clinics as a demonstration case. We will evaluate this approach by 
measuring accuracy and staff time for data collection via Kit REDCap 
plug-in versus manual data collection. 
Exploratory Sub-aim 2b [NOT HUMAN SUBJECTS RESEARCH]: To 
develop and disseminate a governance process for integration of 
REDCap-originated data into EHR flowsheet discrete data fields via 
Epic’s AddFlowsheetValue API.  
Endpoints*: 
Primary Endpoint: screening completion prior to scheduled clinic visit 
Study Population: 
Sample: 880 adult patients scheduled for an epilepsy clinic 
appointment at Atrium Health Wake Forest Baptist or High Point 
locations 
Facilities Enrolling 
Participants: 
(1) Atrium Health Wake Forest Baptist (2) Atrium Health High Point   
880 adults scheduled for epilepsy clinic visits 
 
Description of Study 
Intervention/Experimental 
Manipulation: 
Participants are not exposed to any interventions or procedures that 
are not a part of their usual standard of care, but the delivery of 
standard care screening instruments will vary as below.   
 
Adult patients scheduled for a clinical visit at either site will be 
randomized to one of four methods for standard care screening 
instrument delivery:  
1. Twilio text message 
2. REDCap email survey link 
3. EHR portal (with reminder) 
4. EHR portal (no reminder; current delivery practice) 
 
Study Duration*: 
Total study duration anticipated 2 years, up to 3 years if needed 

 
 
 
15 March 2024 
 
 
 
4 
Participant Duration: 
Participants will be asked to complete a self-reported standard care 
screening for their clinic visit.  Participants will not be contacted for 
additional follow-up.  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
15 March 2024 
 
 
 
5 
1.2 
SCHEMA  
 
 
 
 
 

 
 
 
15 March 2024 
 
 
 
6 
1.3 
SCHEDULE OF ACTIVITIES  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
*Review of information from day 7 
**These procedures only occur among individuals from EHR-delivery method arms who completed the 
anxiety and depression instruments 
 
2 
 
INTRODUCTION 
2.1 
STUDY RATIONALE  
Anxiety and depression are prevalent yet under-recognized and undertreated among 
people with chronic conditions requiring specialty care including neurological disorders.1-4 In some 
conditions, anxiety and depression impact quality of life and other outcomes more than the primary 
condition targeted by neurology or other subspecialty care.5-8 Epilepsy is a key example, with a third or 
more of screened patients in clinical epilepsy settings having anxiety or depression symptoms.9-13 
However, clinician screening uptake is low despite a quality measure to screen with validated anxiety and 
depression instruments at every visit,14-16 and most epilepsy patients with anxiety or depression do not 
receive treatment,17-19 despite treatment being a patient priority.20 Further, our prior work indicates 
epilepsy patients prefer neurologist prescribing for anxiety and depression over psychiatry referral or 
primary care prescribing.20 Specialty providers including epileptologists report time to conduct screening 
as a major barrier,15,16,21  yet repeated symptom monitoring using anxiety and depression instruments is a 
key component of successful anxiety and depression management by non-psychiatrists,22-24 including 
 
Baseline 
Day 0 to 7 
Scheduled 
clinic visit 
Day 7 
Post-visit 
Chart 
Review*   
Secondary 
Data 
Collection 
(Aim 2a) **  
Baseline Data Collection from the 
EHR 
X 
 
 
 
Randomization  
X 
 
 
 
Send instruments, information sheet 
consent via randomized method 
X 
 
 
 
Patient instrument 
completion/consent 
X 
 
 
 
Research staff enter REDCap-
completed screening results into EHR 
X 
 
 
 
Research staff notify epilepsy clinician 
if high risk response received on 
depression instrument 
X 
 
 
 
Instrument completed? 
 
X 
 
 
Clinic visit completed? 
 
 
X 
 
Process Measures 
X 
X 
 
X 
Kit Flowsheet collection 
 
 
 
X 
Manual instrument result collection 
 
 
 
X 

 
 
 
15 March 2024 
 
 
 
7 
collaborative care, an evidence-based intervention25 meriting investigation in neurology clinics to close 
treatment and outcome gaps. 
Thus, tools and strategies are needed reduce the burden of screening for anxiety and 
depression in subspecialty clinics, to close care gaps and to conduct multicenter implementation 
and effectiveness studies of evidence-based interventions such as collaborative care in neurology 
settings.  Our group successfully implemented electronic health record (EHR)-based screening in a 
tertiary epilepsy center using a theory-informed implementation strategy and EHR-based tools enabling 
patient self-completion of screeners following brief nursing staff activation. This more than quadrupled 
screening, but a substantial gap remained due to the nursing activation step in this model, and barriers to 
scaling included lack of EHR/research database integration and need for custom EHR build at each 
potential site for future multicenter studies.  Interoperable tools facilitating independent patient self-
completion are needed to scale and refine our initial implementation strategy and support recruitment and 
symptom monitoring in multisite intervention trials. Thus, to overcome roadblocks to scaling our 
research program and to disseminate for broader use in implementation and pragmatic research, we 
aim to develop and evaluate interoperable tools for anxiety and depression screening in neurology 
clinics serving epilepsy patients using REDCap (Research Electronic Data Capture)26 and Epic EHR 
Application Programming Interfaces (APIs).  
Our Specific Aims are: 
Aim 1: To evaluate screening completion (primary outcome) and process measures 
comparing interoperable, REDCap-based methods to EHR patient portal-based methods for 
delivering validated anxiety and depression instruments to epilepsy clinic patients. In a randomized 
pragmatic trial of four modalities with N=220 individuals per arm and primary outcome proportion fully 
screened prior to clinic visit, we will compare Twilio text message delivery via REDCap vs. REDCap email 
survey links vs. EHR portal questionnaires with reminder message vs. standard EHR portal 
questionnaires without reminder. 
  Hypothesis: the screening completion proportion will vary across the 4 modalities tested.  
Aim 2a: To implement and evaluate a reproducible approach to EHR and research system 
integration via Epic’s Kit integration with REDCap for Epic EHR flowsheet data, using flowsheet-
stored anxiety and depression instruments from epilepsy clinics as a demonstration case. We will 
evaluate this approach by measuring accuracy and staff time for data collection via a Kit Flowsheet 
REDCap plug-in versus manual data collection. 
Exploratory Sub-aim 2b: [NOT HUMAN SUBJECTS RESEARCH] To develop and disseminate a 
governance process for integration of REDCap-originated data into EHR flowsheet discrete data fields via 
Epic’s AddFlowsheetValue API. 
2.2 
BACKGROUND  
 
Depression and anxiety are prevalent and impactful across various neurological and 
medical conditions, and multiple practice guidelines recommend screening. Numerous chronic 
conditions treated by subspecialists are associated with high prevalence of anxiety and/or depression, 
and these comorbidities are associated with poor outcomes including worsened primary chronic condition 
outcome, reduced treatment adherence, and worse quality of life.1-3,27-29 Guidelines and quality measures 

 
 
 
15 March 2024 
 
 
 
8 
targeting primary care, specialty care, or neurology care settings recommend screening and management 
of depression and/or anxiety.14,30-34  
Depression and anxiety are especially impactful in epilepsy, a high-risk condition with 
well-documented gaps in screening and management. Anxiety and depression are highly prevalent 
and major contributors to poor quality of life and other poor outcomes in epilepsy, including increased 
mortality, health care cost, cognitive dysfunction, medication adverse effects, and worsened seizures.5,9-
12,35-39 Multiple consensus statements for epilepsy recommend routine screening using standardized 
instruments to detect depression and/or anxiety, and anxiety is under-recognized without structured 
assessment.9,14,40,41 Yet significant gaps in both screening and treatment exist, with US and international 
surveys of epilepsy clinicians indicating only 10-30% use validated screening instruments, and multiple 
studies and our own preliminary data demonstrate <50% of symptomatic patients receive treatment.17-19,42 
The top barrier to screening was clinician time required to screen.15,16 Improving the anxiety and 
depression screening rate from 15%15 to 60%, with action to initiate management in 50% of untreated 
symptomatic individuals would result in an additional 100,000 people with epilepsy receiving treatment for 
anxiety and/or depression in the United States.18,19 
Depression and anxiety instruments have important roles in both detection and treatment 
monitoring in evidence-based interventions to manage anxiety and depression across various 
conditions. Collaborative care models for managing anxiety and depression are highly effective in 
primary care and some medical subspecialty settings to improve anxiety and depression symptoms, 
quality of life and other outcomes including some chronic disease specific outcomes.24,25,43-47 These 
models typically involve initial symptom detection via validated screeners and care coordination involving 
measurement-based care with repeated symptom monitoring using anxiety and depression 
instruments.24,25,48  Other care models involving repeated use of screening instruments for symptom 
monitoring during treatment by non-psychiatrists have demonstrated effectiveness, with clinical outcomes 
similar to treatment in a psychiatry specialty practice.22,23 While these models have strong evidence 
supporting their use in primary care and some subspecialty settings, they have not been investigated in 
neurology or epilepsy clinic settings, nor epilepsy patient samples other than a home-visit oriented 
program.49,50 Thus, to close gaps in anxiety and depression screening in subspecialty care settings and 
implement and evaluate evidence-based interventions such as collaborative care in neurology and other 
subspecialty settings, streamlined tools for anxiety and depression instrument delivery to promote patient 
self-completion are needed. 
Electronic solutions promoting patient self-completion of depression and anxiety 
instruments may overcome barriers to screening and symptom monitoring and facilitate research 
recruitment for anxiety and depression effectiveness trials in routine care settings. Not only does 
prior research suggest patients are more likely to report psychiatric symptoms via electronic instruments 
than interview,13,51 but electronic patient reported outcome instruments are acceptable, feasible, and they 
facilitate patient reported outcome completion52 especially when accompanied by reminders.53 Further, 
electronic patient completion of anxiety and depression instruments reduces clinical staff burden to 
conduct screening and facilitates screening completion.54 Use of EHR-based or external survey 
application-based instruments have different advantages, such as better integration in clinical workflow 
(EHR instruments) and better usability (external applications that are more easily tailored for usability). 
Text-message delivery was also recently demonstrated to facilitate access, care engagement, and 
outcomes.55,56 However, these various modalities have not been investigated in a rigorous head-to head 
comparison despite the potential advantage for implementation scaling to offer more than one modality to 
adopting sites. Further, research among patients with neurological conditions is needed to evaluate any 
potential impact of cognitive limitations or other disease features on instrument completion. 
Implementation and use is also limited by roadblocks to scalability and integration of external systems 
with the EHR, and overcoming these barriers would facilitate screening implementation research, efforts 
for research recruitment embedded in routine care, and symptom monitoring components of 
interventional studies. 

 
 
 
15 March 2024 
 
 
 
9 
Scalability of electronic tools is hampered by lack of EHR-research database integration 
and limited interoperability, including across institutional versions of the same EHR platform. 
While the use of EHR data for research is still impaired by a disconnect between the data standards used 
in patient care and those used in clinical research, efforts to bridge this gap are underway. Substitutable 
Medical Applications and Reusable Technologies (SMART) on Fast Health Interoperability Resources 
(FHIR) is a web based standard platform that enables solutions to enhance interoperability with EHRs 
and other systems including clinical research data capture (e.g., REDCap). REDCap has implemented 
SMART on FHIR to bridge the gap between clinical care and data capture systems to improve clinical 
research execution by reducing redundant workflows.  SMART on FHIR implementations can support 
FDAs guidance on Electronic Source (eSource) Data and align with Electronic Source Data Interchange 
(eSDI),57 which aims to encourage community to use electronic source data and available data 
standards.58,59 However, various types of EHR-stored discrete data reside in flowsheets, including anxiety 
and depression instruments used by the study team and other patient reported outcomes, yet these are 
not supported by FHIR and thus require new solutions to support scalability. Further, published EHR-
REDCap integration projects to date have significant limitations to scalability or functionality, including 
designs with multiple intermediary platforms or other complexity that would be difficult to scale across 
systems,60,61 or internally built, clinically dedicated REDCap systems.62 Some integrations are limited to 
presenting a link to REDCap from the EHR, rather than data flow between systems.63 
Our research team developed local EHR-based methods for anxiety and depression screening 
and EHR and REDCap-based preliminary recruitment in randomized trials of anxiety and depression in 
epilepsy.64,65 However, for next step multi-site recruitment for individually randomized collaborative care 
trials for anxiety and depression in epilepsy and screening implementation trials, REDCap/EHR 
integration is needed for seamless data transfer and flexibility of use across sites (REDCap vs. EHR-
based, depending upon site capabilities and preferences). Further, tools to facilitate better patient self-
completion of screening instruments are needed. Thus, in this projectl we aim to evaluate success of 
patient self-completion of instruments in a pragmatic randomized trial comparing two REDCap-based 
methods to two EHR-based methods (Aim 1), and to demonstrate REDCap-Epic integration for flowsheet-
stored EHR instrument data via a generalizable Epic Kit based approach, and support preliminary steps 
to bidirectional flowsheet integration (Aims 2a-2b).   
Relevant Preliminary Data 
The investigators conducted a depression and anxiety screening implementation targeting 
epilepsy center nursing staff, to mitigate provider time-related barriers to screening. The result was 
quadrupling of screening, but with substantial provider-level variability, and a key barrier was the nursing 
activation step (see Summary of Research for details). The PI and investigator team have demonstrated 
success using EHR- and REDCap based tools including anxiety and depression instruments for 
preliminary recruitment in pragmatic randomized trials64,65 and collecting outcomes using Epic EHR-portal 
methods.42,65 The next phase is to develop scalable tools and preliminary data for multi-site trials in two 
areas: 1. effectiveness-implementation trials of collaborative care (tools from this proposal for recruitment 
& symptom monitoring) and 2. refined implementation proposals to close gaps in anxiety and depression 
screening and provider action. Specifically, Aim 1 addresses key next steps by investigating instrument 
delivery methods for patient self-completion to overcome nursing activation barriers, and generating key 
preliminary data (completion rates by modality & patient characteristics) for future multisite study 
planning. 
Novel strategies for capturing, monitoring, and displaying metrics data. Our team developed several 
external modules for REDCap currently in use (Table 1). Streamlining clinical research execution and 
data capture with implementation of available standards could have a large impact on quality issues 
within studies.  
While this work demonstrates our team’s expertise in integration and successful tool development, none 
of the current tools transfer discrete, flowsheet-based data including EHR-based epilepsy anxiety and 

 
 
 
15 March 2024 
 
 
 
10 
depression instruments from EHR to REDCap (or REDCap to EHR) in a generalizable way easily scaled 
across centers. 
Drs. Munger Clary, Topaloglu, and consultant computer scientist Cody Hudson conducted a successful 
proof- of-concept data transfer from Epic Test platform to REDCap for flowsheet-based anxiety & 
depression instruments via the GetFlowsheetRows Epic webservice during Dr. Munger Clary’s KL-2 
project. This was not fully implemented due to a 72-hour lockout parameter that Epic imposes; this 
roadblock prevented collection of flowsheet data after 72 hours. Aim 2 directly addresses this translational 
roadblock via Kit flowsheet integration with REDCap. Due to performance degradations, Epic has started 
to encourage data access to be done through its Caboodle data warehousing solution.  Caboodle has 
capability to import external data generated outside of Epic to provide a fully functional data warehouse. 
The Kit API is the recommended interface for accessing data in Caboodle, which has 
granular access control (i.e. each API has to be added to the app) in Epic’s App Orchard, similar to FHIR 
apps.66 Integrating Kit Flowsheets with REDCap is the next phase in this aspect of the research program. 
 
 
2.3 
RISK/BENEFIT ASSESSMENT  
2.3.1 KNOWN POTENTIAL RISKS  
This is not a treatment/intervention study and risks to the study participants are low.  The only risk would 
be in loss or mismanagement of sensitive data. 
 
Participants are not exposed to any interventions or procedures that are not a part of their usual standard 
of care.  Patients will be prospectively randomized to one of the four methods of delivery for standard 
care anxiety and depression screening instruments - (responses to the screening questionnaire will be 
used in the standard of care clinic) by either:  
1. Twilio text message via REDCap 
2. REDCap email survey link 
3. EHR portal (with reminder) 
4. EHR portal (no reminder; current standard clinical delivery method) 
 
Standard care anxiety and depression screening instrument results received via the REDCap methods will 
be entered into the EHR by study staff prior to the scheduled clinic visit, and clinicians will be notified if 
high-risk results to depression screening occur, to further minimize risk. (This notification represents an 
additional risk-reduction measure beyond what is currently in place for these screeners in routine care in 
Table 1: Wake Forest School of Medicine developed REDCap tools for data coordination 
REDCap Modules 
Function 
Stratification & 
randomization hooks 
Hooks written to stratify & randomize participants based on various 
randomization models 
Caps 
Hooks written to cap number of participants 
Cancer Trials Support Unit 
(CTSU) Open RandoNode 
Implemented  to randomize and register subjects to REDCap via CTSU 
Open for 800+  NCI Community Oncology Research Program (NCORP) 
Sites utilizing APIs 
COMPASS Care Planning 
Bidirectional API with COMPASS (Comprehensive Post-Acute Stroke 
Services) Care Planning middleware between Epic and REDCap. 
Schema Resolver 
Python application to compare schema differences between REDCap 
projects utilizing APIs  

 
 
 
15 March 2024 
 
 
 
11 
this setting). Robust methods to maintain privacy of clinical and research data will be used, as described 
in section 10.3.1.   
2.3.2 KNOWN POTENTIAL BENEFITS  
There are no direct benefits to the patient, but this research will benefit the larger community and 
population if it is able to successfully contribute to improving screening and treatment of depression and 
anxiety in this patient population, and potentially other populations for whom depression and/or anxiety 
screening is recommended. 
2.3.3 ASSESSMENT OF POTENTIAL RISKS AND BENEFITS  
While there is not direct benefit to the patient, this is considered a minimal-risk study that may provide 
benefit to the larger community and population.  
 
Minimizing risks 
Maximal efforts to maintain privacy including use of HIPAA-compliant secure tools will be in place. 
Measures to minimize risk by ensuring data privacy and security and ensuring timely communication of 
sensitive results to clinicians when identified prior to the clinical visit will be taken as described in more 
detail in section 8.2. If a response to the depression instrument indicates potential passive suicidality, 
research staff will notify the treating clinician. This notification represents an additional risk-reduction 
measure beyond what is currently in place for these screeners in routine care in this setting, where pop-
up alerts do not occur until an alert-activating action in the EHR occurs during the scheduled clinic visit. 
Standard clinical care privacy regulations will be followed for each delivery method of the standard care 
screeners. 
 
• 
The study is classified as minimal risk because the anxiety and depression screening instruments 
are a best practice clinical and quality tool currently in use at the center without consent. The 
screener questions in the proposed research are identical to those in current clinical use, with 
the research involving only minor changes in the delivery method (while adhering to clinical 
communication policies and privacy regulations). Randomizing delivery method does not 
increase the study risk level. Otherwise, the research involves collecting a small amount of pre-
existing clinical EHR-based information and data on screener and clinic visit completion. Strict 
efforts to maintain confidentiality/privacy will be followed, and all study team members will 
have rigorous training in maintaining confidentiality. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
15 March 2024 
 
 
 
12 
 
 
 
 
3 
OBJECTIVES AND ENDPOINTS  
 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR ENDPOINTS 
Primary 
 
 
To evaluate screening completion 
(primary outcome) comparing 
interoperable, REDCap-based 
methods to EHR patient portal-
based methods for delivering 
validated standard care anxiety and 
depression instruments to epilepsy 
clinic patients (Aim 1).  
Screening completion 
– Yes or No 
  Proportion of screening instruments 
completed by each of the 4 methods 
prior to clinic visit will be used to 
compare the effect of the different 
methods on patient self-completion, and 
to estimate completion rates by different 
methods for use in future screening 
implementation trials. 
Secondary 
 
 
To evaluate process measures 
comparing interoperable, REDCap-
based methods to EHR patient 
portal-based methods for delivering 
validated anxiety and depression 
instruments to epilepsy clinic 
patients (Aim 1).  
 
 
 
 
 
To evaluate accuracy of a 
reproducible approach to EHR and 
research system integration via 
Epic’s Kitintegration with REDCap 
for Epic EHR flowsheet data, using 
flowsheet-stored anxiety and 
depression instruments from 
epilepsy clinics as a demonstration 
case (Aim2a). 
Research team time: 
-sending instruments 
-instrument data 
entry 
 
Participant time from 
instrument delivery 
to completion 
 
 
 
 
Accuracy of Kit 
flowsheet REDCap 
plug-in data entry vs. 
manual data entry 
   This data will provide information to 
calculate staff resources needed to 
prepare for screener delivery and data 
entry in future studies. 
 
This data will provide additional potential 
information about whether the different 
screener delivery methods vary in 
promptness of patient response to 
screeners. 
 
 
This will provide key information about 
whether a Kit flowsheet REDCap plug-in 
enabled data collection may improve 
accuracy of EHR-based data collection 
compared to manual collection. 
Tertiary/Exploratory  
 
 
To explore visit attendance among 
individuals who completed or did 
not complete anxiety and 
depression screeners (Aim 1). 
 
To explore research team time 
required to collect anxiety and 
Clinic visit 
attendance: 
Visit completed vs. 
canceled vs. no-show 
 
To explore for association between visit 
completion and screener completion for 
the different delivery methods. 
 
 

 
 
 
15 March 2024 
 
 
 
13 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR ENDPOINTS 
depression screener results from the 
EHR via Epic’s Kit integration with 
REDCap for Epic EHR flowsheet data 
(Aim 2a). 
 
Time for data 
collection (Kit vs. 
manual entry) 
 
To explore potential research staff time 
savings resulting from data collection via 
Kit flowsheet integration with REDCap. 
 
4 
STUDY DESIGN 
4.1 
OVERALL DESIGN 
   Aim 1: In a randomized trial, to compare 
patient-completion success of REDCap- and 
EHR-based anxiety & depression instrument 
delivery methods (1-4, Figure 1). 
In this randomized study of four modalities with 
N=220 individuals per arm and primary 
outcome proportion fully screened prior to 
clinic visit, we will compare Twilio text message 
delivery via REDCap vs. REDCap email survey 
links vs. EHR portal questionnaires with 
reminder message vs. current standard delivery 
EHR portal questionnaires without reminder 
(Figure 2). Screening instruments will be 
delivered by randomized method 7 days prior to 
scheduled clinic visit and outcomes occur by the time of the scheduled clinic visit. The study setting is 
the Wake Forest Comprehensive Epilepsy Center, with 2 clinic sites where adult patients are served 
(Atrium Health Wake Forest Baptist main campus, Atrium Health High Point Medical Center). EHR portal 
questionnaires without reminder is the current standard care screener delivery method. 
Hypothesis: the screening completion proportion will vary across the 4 modalities tested.  
Process measures will also be compared 
among the modalities. 
Trial Phase: Non-FDA regulated pragmatic 
trial. 
   Aim 2a: To demonstrate Epic’s Kit 
integration with REDCap for Epic EHR 
flowsheet data (anxiety & depression 
instrument test case in epilepsy). We will 
evaluate this approach by measuring 
accuracy and staff time for data collection via 
Kit flowsheet REDCap plug-inversus manual 
data collection.      
Following build and testing, Kit Flowsheet 
REDCap plug-in will be launched in a production REDCap project and activated for double data entry of 
the flowsheet data generated via participant screener completion in the two EHR allocated arms of the 
randomized screening instrument delivery study (EHR with message and standard EHR). Entry #1 will be 

 
 
 
15 March 2024 
 
 
 
14 
manual and data entry #2 will be the REDCap Kit plug-in collected by a study team member without 
access to the manually collected data. A user who did not collect either the manual or Kit plug-in data 
(the reviewer) will review discrepancies between entries using a REDCap tool. Accuracy will be assessed 
by comparing Kit REDCap plug-in collected values to manually-collected values for the EHR-allocated 
visits.  Reproducibility/generalizability of the Kit REDCap plug-in data collection method will be 
evaluated in a 2nd instance of Epic (Encompass) via use of the Kit flowsheet plug-in to pull the same 
anxiety and depression screening data from Encompass instance flowsheets (generated during clinical 
data conversion of the original WakeOne Epic instance based flowsheets to the new health system-wide 
Encompass Epic instance). 
Exploratory sub-aim 2b: (this portion is NOT HUMAN SUBJECTS RESEARCH) To develop a governance 
process for integration of REDCap data into EHR discrete data fields via Epic AddFlowsheetValue API. 
4.2 
SCIENTIFIC RATIONALE FOR STUDY DESIGN 
 
This project is designed to overcome key roadblocks to future research to close screening, action, and 
outcome gaps for anxiety and depression in epilepsy, specifically: 1. multi-site implementation-
effectiveness trials to evaluate collaborative care in epilepsy, and 2. Implementation-focused trials to 
close the screening and treatment gap for anxiety and depression in epilepsy.  
This will be accomplished by developing tools that can be used flexibly to facilitate multi-site research 
while generating key preliminary data to estimate recruitment potential for collaborative care studies and 
evaluate potential impact of an instrument-delivery focused component of a refined, theory-informed 
implementation strategy for anxiety and depression screening.  
 
Each of the 4 arms of the randomized trial represents a screener delivery method that could be used at 
sites in future trials and in routine clinical care, with the standard EHR arm representing the current 
standard practice at Wake Forest Comprehensive Epilepsy Center. This project will generate the 
preliminary data on screener completion for each modality to support options for screening and symptom 
measurement in future research that could be used flexibly at different sites in future implementation 
research.  This project will also generate valuable quality improvement-related clinical data that could 
result in immediate change in practice for screener delivery in routine care to enhance care and improve 
performance on epilepsy quality measures.14 
4.3 
JUSTIFICATION FOR INTERVENTION 
 
Patient completion of instruments. The patient-focused screening delivery methods are informed by a 
Capability, Opportunity, Motivation-Behavior (COM-B)-Behavior Change Wheel framework-informed 
implementation strategy model.67 Patient-level barriers were identified and mapped to COM-B constructs 
(physical opportunity and automatic motivation), then intervention functions (enablement, environment 
restructuring), then a corresponding behavior change technique (prompts & cues).67  
The four instrument delivery methods to be used in this pragmatic trial represent distinct prompts & cues 
for screening instrument completion [1.text message, 2.customized email prompt, 3.generic EHR portal 
email prompt with customized EHR portal message after login to EHR portal, or 4.no message but 
presence of previsit questionnaires associated with a visit in the EHR portal]. Examining these multiple 
instrument delivery methods will generate outputs that will support flexibility in instrument delivery 
modalities across multiple sites in future multicenter studies in this area, as mentioned above. 
These 4 distinct delivery methods are used to deliver identical standard care validated anxiety and 
depression screening instruments.   

 
 
 
15 March 2024 
 
 
 
15 
The specific wording and appearance of the instrument delivery methods will be further informed during 
study planning by input from five patient stakeholders (with whom the research team has a prior 
relationship) who will participate in a semi-structured interview on barriers and facilitators to pre-visit 
screening completion, informed by the COM-B-Behavior Change Wheel framework and will provide 
usability feedback on the screener delivery methods and reminder message wording. (Additional 
stakeholders may potentially be engaged, to provide five individuals the opportunity to participate in both 
feedback sessions). 
4.4 
END-OF-STUDY DEFINITION 
 
The only participant interaction in this study is completion of the standard care anxiety/depression 
screeners prior to the epilepsy clinic visit after viewing the brief consent information. Delivery of screeners 
occurs 7 days prior to scheduled clinic visit and outcomes occur by the time of the clinic visit (instrument 
completion, visit completion). All data for Aim 1 is generated by completion of the epilepsy clinic 
encounter. Aim2a involves secondary data collection among some participants who completed EHR based 
screening instruments and thus will have consented for research use of the screener data. 
 
The end of the study is defined as completion of all data collection for all participants as shown in the 
Schedule of Activities (SoA), Section 1.3. 
5 
STUDY POPULATION 
5.1 
INCLUSION CRITERIA 
AIM 1: All visits scheduled at least 7 days in advance at two adult epilepsy-focused clinics of the 
Wake Forest Comprehensive Epilepsy Center will be included. 
 
AIM 2a: All visits with completed anxiety and depression screening questionnaires from the EHR-
allocated arms of the randomized study. 
5.2 
EXCLUSION CRITERIA 
 
         AIM 1: Age<18 years at baseline (7 days prior to the scheduled epilepsy clinic visit) 
AIM 2a: no exclusions. 
5.3 
LIFESTYLE CONSIDERATIONS 
N/A 
5.4 
SCREEN FAILURES 
Individuals not included in the study due to age<18 years at one point in the study period who later meet 
inclusion criteria for a later visit (by attaining the age of 18 years prior to a later epilepsy clinic visit) may 
then be included. 
5.5 
STRATEGIES FOR RECRUITMENT AND RETENTION 
 

 
 
 
15 March 2024 
 
 
 
16 
Recruitment.  Eligible patient visits will be identified via EHR clinic schedules. Baseline collection of the 
EHR-based existing clinical and demographic data will occur under a HIPPAA waiver. 
Due to the minimal risk classification for this study involving modified delivery methods for otherwise 
identical standard care screening instruments, a modified informed consent process is acceptable as 
outlined below.  
Eligible participants will be randomized to method for delivery of pre-visit standard care screeners, and 
all screeners will be delivered with adherence to clinical communication and privacy policies, with an 
identical brief consent/information sheet prior to the screening questionnaires. The same wording will be 
used for all study arms to reduce the chance of group allocation bias that would undermine the research 
results: 
 
o 
Your epilepsy doctor has included questionnaires about your mood and anxiety as part of 
your clinic visit. Taking part in this survey may involve providing information that you 
consider confidential or private. Depending upon how you answer these questions, the 
responses may be shared with your neurology provider. Some information you provide 
may be used for research. Careful efforts will be made to keep your information 
confidential. You will not lose any services, benefits, or rights if you choose not to 
participate in this voluntary survey. 
 
This brief consent format is acceptable because the study is minimal risk (see section 2.3.3 for more 
information about the minimal risk classification). 
Thereby, those who complete anxiety/depression screening instruments will provide consent for use of 
research data and potential epilepsy provider notification of passive suicidality screen if needed to 
minimize risk/enhance safety (see section 8.2 for details of safety plan). 
 
For those who do not complete the anxiety/depression screeners, epilepsy provider notification is not 
applicable, and it is impractical to obtain informed consent: 
o 1. It is impractical to approach individuals who canceled or missed the clinic visit. 
o 2. For individuals who do not complete the anxiety/depression screeners but do attend 
the epilepsy clinic visit, it is impractical for appropriately trained staff to approach them 
for consent at the clinic visit, since these visits may occur at any time on any day, 
including simultaneously at disparate clinic sites. 
o Thus, among those who do not complete the screening instruments, waiver of informed 
consent and HIPAA waiver are acceptable because the study is classified as minimal risk 
(see section 2.3.3 for more explanation of this) and because obtaining consent is 
impractical.  
 
Our N=880 sample size compares favorably with the typical >1500 visits at Wake Forest Comprehensive 
Epilepsy Center adult clinics over 6 months, and with our experience during COVID-19-related volume 
disruptions, when 1097 visits were scheduled and 866 were completed over a 5-month period. Finally, 
based on our clinic data we expect only a modest decrease in numbers from visits to individuals, and we 
estimate 923 unique individuals will complete visits over 6 months and thus the sample size goal will be 
achieved within the 6-8 months of recruitment planned for this study. 
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) 
6.1 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) ADMINISTRATION 

 
 
 
15 March 2024 
 
 
 
17 
6.1.1 STUDY INTERVENTION OR EXPERIMENTAL MANIPULATION DESCRIPTION 
While this is not a treatment study, below we describe the different methods of delivering standard care 
screening instruments that are examined in the study. 
AIM 1: Methods to deliver screening instruments.  
The anxiety and depression screening instruments are a best practice clinical and quality tool currently 
in use at the center without consent. The screener questions in the proposed research are identical to 
those in current clinical use, with the research involving only minor changes in the delivery method 
(while adhering to clinical communication policies).  
The delivery methods are described below. 
• Standard EHR: visit-attached instruments in patient portal available ≤7 days before visit (current 
practice) 
• EHR with message: portal message encouraging completion sent and visit-attached instruments 
available ≤7 days prior (system sends generic email; specific portal message visible after patient login 
to EHR portal) 
• REDCap survey: custom email with instrument link sent ≤7 days before visit encouraging completion  
• Twilio® text via REDCap: secure text message delivery of questionnaires ≤7 days prior to visit  
 
 
AIM 2a: THIS ONLY INVOLVES COLLECTION OF DATA FROM THE EHR for EHR-arm participants who 
completed the anxiety & depression instruments in the EHR, and thus consented for research use of 
this data.   
There are no participant interactions.  
The study methods for this secondary data collection and analysis are described below, but these 
methods do not constitute an intervention. 
Implementation and evaluation of EHR/research database integration via Kit Flowsheet REDCap plug-
in 
Diverse patient data are stored in the Epic EHR as flowsheet rows built for clinical care or research as 
discrete data collection fields. In addition to custom data collection forms, flowsheets are how standard 
instruments are programmed into Epic including Promis® measures, GAD-7, NDDI-E and other validated 
patient-reported outcome measures. However, integrating flowsheet discrete data with other research 
systems such as REDCap is a roadblock to translational research, and Epic’s flowsheet webservices 
enforce a 72-hour lockout timeframe due to potential performance risk. Therefore, the 
GetFlowsheetRows webservice is not feasible for research data collection occurring >72 hours after 
entry. Epic’s Kit is a set of database views available in all Epic systems with Caboodle data warehouse 
implementation.66 It is designed to perform large data extracts and analytical reporting in an accessible 
manner. Furthermore, it requires less training for developers and enables quicker runtimes and quicker 
report writing compared to the Clarity relational database.66 Similar to Clarity, the data accessible by Kit 
could be 24 hours old, but there is no lockout time, which makes Kit ideal for research applications 
involving retrospective data extraction for analysis. Thus, Kit integration is more generalizable for 
REDCap research database integration than native Epic webservice approaches, and it overcomes the 
prior roadblocks our research team encountered with the GetFlowsheetRows webservice. In this aim, an 
experienced consultant computer scientist will collaborate with Drs. Munger Clary and Topaloglu to 
integrate REDCap API functionality with Epic’s Kit Flowsheet API by developing a REDCap plug-in. As 
mentioned in preliminary data, the team has prior experience developing REDCap APIs and the 
necessary tools. Epic provides a sandbox development environment for all Kit APIs,66 including the 
Flowsheets API to be integrated with REDCap. Akin to other Epic webservices including FHIR, Epic’s App 
Orchard platform controls the allowed APIs and other respective authentication details. Atrium Health 
Wake Forest Baptist is equipped and licensed with the Kit functionality and is populating flowsheets 
now. 

 
 
 
15 March 2024 
 
 
 
18 
Kit Flowsheet API REDCap Integration & Data Collection. Following build and testing, the REDCap plug-
in for Kit Flowsheet integration will be launched in a production REDCap project and activated for data 
entry of the flowsheet data generated in the EHR allocated arms of the randomized instrument delivery 
study. Entry #1 will be manual and data entry #2 will be the via the REDCap Kit flowsheet plug-in by a 
user without concomitant access to the manual data entry fields. A user (the reviewer with rights to 
review discrepancies between entries and the source data will use a REDCap instrument to review for 
discrepancies and reconcile discrepancies. 
6.1.2 ADMINISTRATION AND/OR DOSING 
 
REDCap anxiety and depression instruments (Generalized Anxiety Disorder-7, GAD-7 and Neurological 
Disorders Depression Inventory-Epilepsy, NDDI-E) currently in use by the PI will be adapted and enabled 
for Twilio text functionality and custom survey link email messages via REDCap developer services. EHR 
patient portal messages to encourage instrument completion will be prepared with support as needed by 
Epic analyst team, similar to the study team’s prior work. 
 
6.2 
FIDELITY 
6.2.1 INTERVENTIONIST TRAINING AND TRACKING 
N/A.   
6.3 
MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING 
Randomization and blinding. Visits will be randomized 1:1:1:1 to the 4 conditions using blocked 
randomization to reduce the chance of imbalances, and to help protect against temporal trends. 
Randomization will be stratified by presence of an existing antidepressant prescription. The 
randomization table will be prepared with consideration of our previously observed rates of 
antidepressant prescribing in our epilepsy clinic setting (152 of 536 consecutive screened individuals 
with epilepsy prescribed an antidepressant). Prior to the start of enrollment, the randomization table 
will be prepared by the study statistician and submitted to the REDCap developer to put into 
production. For each visit, the participant's baseline data collection will be completed; activation of the 
random assignment may then occur. Until that time, the assignment will be concealed to the PI and all 
other study team members to enhance rigor and reproducibility. The nature of outcome collection 
prevents study team blinding to allocation for primary outcome ascertainment. Study team will enter 
instrument results obtained via REDCap into the EHR flowsheets prior to clinic visit (identical EHR 
location as with EHR allocation). Timely entry is feasible using REDCap email alerts to the study team 
(method used by PI in prior projects).  
The brief consent will have identical wording for all delivery arms to minimize potential bias that could 
be introduced by the informed consent process related to the different study arm allocations (see 
sections 5.5 and 10.1.1.1 and 10.1.1.2). 
6.4 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION ADHERENCE 
 
N/A 
6.5 
CONCOMITANT THERAPY 
 

 
 
 
15 March 2024 
 
 
 
19 
N/A 
6.5.1 RESCUE THERAPY 
 
N/A 
 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND 
PARTICIPANT DISCONTINUATION/WITHDRAWAL 
7.1 
DISCONTINUATION OF STUDY INTERVENTION/EXPERIMENTAL MANIPULATION 
 
N/A as there are no ongoing study interactions/this is not a treatment study. 
7.2 
PARTICIPANT DISCONTINUATION/WITHDRAWAL FROM THE STUDY 
 
N/A as the only participant interaction is a one-time completion of standard care screeners delivered by 
one of the allocated methods. 
7.3 
LOST TO FOLLOW-UP 
 
N/A 
 
8 
STUDY ASSESSMENTS AND PROCEDURES 
8.1 
ENDPOINT AND OTHER NON-SAFETY ASSESSMENTS 
Baseline demographics and clinical information (including whether there is evidence of an existing 
antidepressant prescription) will be collected from existing EHR based information using SMART on FHIR 
REDCap integration. These also include: sex, race, ethnicity, age, diagnoses, zip code, and prescriptions. 
 
Anxiety and depression instruments. 
Generalized anxiety disorder-7 (GAD-7)68,69 & Neurological Disorders Depression Inventory-Epilepsy 
(NDDI-E)70,71 are well-validated, freely available anxiety and depression instruments that satisfy the 
American Academy of Neurology Epilepsy Quality Measure for anxiety and depression screening at 
every visit14 and are National Institute of Neurological Disorders and Stroke(NINDS) Common Data 
Elements. Automated EHR-based procedures are already in place for these instruments to be attached 
as EHR-portal previsit questionnaires and available prior to a scheduled visit for all active adult epilepsy 
specialty providers at the Wake Forest Comprehensive Epilepsy Center. 
 
AIM 1 Outcomes. Outcomes are determined by the scheduled clinic visit (Table 2), with primary 
outcome defined as instrument completion prior to visit date/start time. Technical reasons for lack of 
instrument completion (eg. no patient portal account, no email address available in EHR system, text 
messages cannot be sent via clinical protocols) will be recorded, but the primary analysis will follow 
intention to treat principles and include all randomized individuals. 
Table 2: Aim 1 Outcomes 
Outcome 
Source 
How obtained? 

 
 
 
15 March 2024 
 
 
 
20 
Instrument completion (primary) 
EHR or REDCap 
(allocated method) 
Review REDCap (REDCap arms) 
Manual EHR review (EHR arms) 
Research team time: sending instruments 
REDCap 
Start stop timing fields REDCap 
(seconds)  
Research team time: instrument entry (into 
EHR or REDCap, based on allocation) 
REDCap 
Start stop timing fields REDCap 
(seconds) 
Patient time from instrument delivery to 
completion among screening completers 
REDCap  
(and EHR if 
randomized to EHR) 
Send time: study team-
activated REDCap timing field 
at instrument delivery  
Completion time: review of 
instrument completion 
timestamp in EHR or REDCap 
Clinic visit completion: visit completed, 
canceled or patient no-show? 
EHR 
Manual chart review 
 
Baseline instrument completion proportion over a few month timeframe prior to pragmatic trial 
initiation may be collected via a CLARITY data pull to estimate pre-trial completion rate using the 
standard EHR method.  
 
AIM 2a: Accuracy of Kit Flowsheet REDCap plug-in data entry will be assessed by comparing Kit REDCap 
plug-in-collected values to manually-collected values for the EHR-allocated visits, with review of original-
source EHR entries to determine true values for all discrepancies identified 
Staff time for data collection will be measured to the nearest second using REDCap timestamps as in 
Aim 1. 
8.2 
SAFETY ASSESSMENTS & SAFETY PLAN 
 
The NDDI-E standard care depression screening instrument contains a passive suicidality item that has 
been validated as a suicidality screener.72  Tools to notify providers of a positive passive suicidality screen 
have been implemented in the EHR as part of the study team’s prior work and are active at present in the 
EHR in routine care; these include a pop-up alert to clinicians that typically appears during the clinic visit.73 
Tools to evaluate and manage passive suicidality were also developed as part of the study team’s prior 
work and are currently available as well.73  These tools will continue to be available to epilepsy clinicians 
for all participants since NDDI-E instruments will be completed in the EHR for the EHR arms, and they will 
be entered into the EHR by study team for REDCap completers prior to the scheduled clinic visit. 
 
To ensure additional safety measures, since study staff will review REDCap-based screening results prior 
to clinicians and study staff may view EHR based screening results prior to the clinicians, an additional 
clinician notification process for possible passive suicidality will be in place and will be tracked by the study 
team. 
 
For screeners completed via the two REDCap-based methods, high urgency email alerts indicating a 
positive passive suicidality screen will be triggered to the PI and coordinator team members from REDCap 
(in addition to alerts indicating completion to facilitate entry of results into the EHR).  In response to the 
alerts, the epilepsy clinician will be notified by the study team and these notifications will be tracked. 
REDCap email alerts based on NDDI-E passive suicidality positive screen are feasible and currently already 
used by the PI in another collaborative study (IRB00051120). 
For EHR-arms, automated alerts with NDDI-E results and critical value flag will be sent to the PI and study 
team as inbasket messages, and study team will notify the epilepsy clinician in a similar manner to the 

 
 
 
15 March 2024 
 
 
 
21 
REDCap-based methods.  These additional measures enhance safety by alerting clinicians to high-risk 
results earlier than occurs at present in standard care practice. 
 
Safety Action Plan Summary:  
• 
Automated alerts to the PI and study coordinator from the screener delivery system (REDCap 
or the EHR) for any positive NDDI-E passive suicidality screens.  
• 
In response to the automated alerts, PI/study coordinator will notify the treating epilepsy 
clinician of the NDDI-E results. 
• 
Next step in management/response to results will be allocated to the treating clinician. 
• 
The study team will track these actions to ensure adherence to this notification plan. 
This plan is in addition to actions already implemented during our prior work: 
• 
Pop-up clinician notification in EHR during clinic visits if NDDI-E with potential suicidality [active 
for all screeners during project, including the REDCap screens when entered in EHR by study 
team]. 
• 
Resources for clinician action to evaluate and manage suicidality: EHR-built smart phrase tool, 
handouts on suicidality protocol and resources to clinicians posted in clinic and provided prior to 
study start. 
As mentioned above, we have prior experience using automated alerts for screening results in both 
REDCap and the EHR. Thus, the additional action plan is highly feasible. 
Based on our prior work screening >500 consecutive individual patients, we anticipate <5% will require 
action, and very few if any of these will have active suicidal ideation. 
 
 
All study procedures will be conducted by appropriately qualified personnel. 
 
Health Insurance Portability and Accountability Act (HIPAA) rules, other relevant federal or state laws, 
and local institutional requirements will be followed, as applicable.   
 
8.3 
ADVERSE EVENTS AND SERIOUS ADVERSE EVENTS 
N/A as this is not a treatment study.  
8.4 
UNANTICIPATED PROBLEMS 
8.4.1 DEFINITION OF UNANTICIPATED PROBLEMS 
This protocol uses the definition of Unanticipated Problems as defined by the Office for Human Research 
Protections (OHRP).  OHRP considers unanticipated problems involving risks to participants or others to 
include, in general, any incident, experience, or outcome that meets all of the following criteria: 
 
• 
Unexpected in terms of nature, severity, or frequency given (a) the research procedures that are 
described in the protocol-related documents, such as the Institutional Review Board (IRB)-
approved research protocol and informed consent document; and (b) the characteristics of the 
participant population being studied; 

 
 
 
15 March 2024 
 
 
 
22 
• 
Related or possibly related to participation in the research (“possibly related” means there is a 
reasonable possibility that the incident, experience, or outcome may have been caused by the 
procedures involved in the research); and 
• 
Suggests that the research places participants or others at a greater risk of harm (including 
physical, psychological, economic, or social harm) than was previously known or recognized. 
 
8.4.2  UNANTICIPATED PROBLEMS REPORTING  
The investigator will report unanticipated problems (UPs) to the reviewing Institutional Review Board 
(IRB). The UP report will include the following information: 
 
• 
Protocol identifying information: protocol title and number, PI’s name, and the IRB project 
number 
• 
A detailed description of the event, incident, experience, or outcome  
• 
An explanation of the basis for determining that the event, incident, experience, or outcome 
represents an UP 
• 
A description of any changes to the protocol or other corrective actions that have been taken or 
are proposed in response to the UP 
 
To satisfy the requirement for prompt reporting, UPs will be reported using the following timeline:   
 
• 
UPs that are serious adverse events (SAEs: life-threatening, require hospitalization, cause 
persistent or significant incapacity or substantial disruption of ability to conduct normal life 
functions for >48 hours) will be reported to the IRB and to the study sponsor/funding agency 
within 24 hours of the investigator becoming aware of the event  
• 
Any other UP will be reported to the IRB and to the funding agency within 7 days of the 
investigator becoming aware of the problem  
• 
All UPs should be reported to appropriate institutional officials (as required by an institution’s 
written reporting procedures), the supporting agency head (or designee), and the Office for 
Human Research Protections (OHRP) within 1 week of the IRB’s receipt of the report of the 
problem from the investigator 
8.4.3 REPORTING UNANTICIPATED PROBLEMS TO PARTICIPANTS  
N/A 
 
 
 
 
 
 
9 
STATISTICAL CONSIDERATIONS  
9.1 
STATISTICAL HYPOTHESES 
 
Primary Endpoint: AIM 1 

 
 
 
15 March 2024 
 
 
 
23 
Hypothesis: the screening completion proportion will vary across the four modalities tested.  
9.2 
SAMPLE SIZE DETERMINATION 
 
Sample size and Power. We based our sample size calculation on our primary outcome and hypothesis 
of a difference in proportions for instrument completion among the 4 groups. With N=220 in each 
group, we will have statistical power equal to at least 80% to detect an overall difference (two-sided chi-
square testing, alpha = 0.05), given a true range among the proportions equal to 0.16 (16%). We believe 
this range represents a clinically meaningful change in screening rate and would be relevant to sample 
size calculations for future larger studies.  
 
Our N=880 sample size compares favorably with the typical >1500 visits at Wake Forest Comprehensive 
Epilepsy Center adult clinics over 6 months, and with our experience during COVID-19-related volume 
disruptions, when 1097 visits were scheduled and 866 were completed over a 5-month period. Finally, 
based on our clinic data we expect only a modest decrease in numbers from visits to individuals, and we 
estimate 923 unique individuals will complete visits over 6 months. 
9.3 
POPULATIONS FOR ANALYSES 
 
The analysis will follow intention-to treat principles. 
9.4 
STATISTICAL ANALYSES 
9.4.1 GENERAL APPROACH 
 
A pre-specified analysis plan (SAP) will be developed for the primary outcome with additional details 
on computational specifications, potential covariates, diagnostics, alternative approaches if 
assumptions are not met, and supportive analyses. A p value of 0.05 will be considered statistically 
significant. 
9.4.2 ANALYSIS OF THE PRIMARY ENDPOINT(S) 
 
We will test our primary outcome and hypothesis of a difference among the 4 proportions for instrument 
completion using chi-square testing (two-sided, alpha = 0.05). Point estimates for the proportions and 
corresponding 95% confidence intervals will be used for descriptive purposes. These analyses will follow 
intention-to-treat principles, and will include all first visits in our target N=880 individuals. 
9.4.3 ANALYSIS OF THE SECONDARY ENDPOINT(S) 
AIM 1: 
Secondary process measure outcomes will be analyzed similarly to the description above for the primary 
outcome. 
 
AIM 2a: 
Point estimates and 95% confidence intervals (e.g. Wilson or score intervals) will be calculated for each 
modality and differences between modalities. (Kit REDCap plug-in-collected values vs. manually-
collected). 
9.4.4 SAFETY ANALYSES 

 
 
 
15 March 2024 
 
 
 
24 
 
The proportion of NDDI-E screening completers who have a positive passive suicidality screen will be 
calculated, as will the proportion of those who demonstrated any evidence of active suicidality on clinical 
follow-up. 
 
9.4.5 BASELINE DESCRIPTIVE STATISTICS 
 
Baseline demographic and clinical characteristics will be reported for each group using descriptive 
statistics. 
 
9.4.6 PLANNED INTERIM ANALYSES  
N/A 
9.4.7 SUB-GROUP ANALYSES 
Summaries of demographic, geographic, technical, and clinical characteristics (e.g., technical barriers to 
instrument delivery, missed or canceled visits) by instrument completion status will be used for hypothesis 
generation and future study planning. Pre-specified subgroup analysis are not planned. 
9.4.8 TABULATION OF INDIVIDUAL PARTICIPANT DATA 
There are no plans for tabulation of individual participant data. 
9.4.9 EXPLORATORY ANALYSES 
 
AIM 1: Summaries of demographic, geographic, technical, and clinical characteristics (e.g., technical 
barriers to instrument delivery, missed or canceled visits) by instrument completion status will be used 
for hypothesis generation and future study planning. 
 
Analyses based on visits, i.e., including all randomized visits through randomization of the 880th individual, 
are planned as potentially supporting the individual-based analyses, though we acknowledge the 
possibility of real differences between visit- versus individual-based population characteristics due to 
differences in behaviors associated with visit frequency, for example. 
 
AIM 2: Staff time for data collection: mean time per visit will be compared between modalities.(Kit REDCap 
plug-in-collected values vs. manually-collected) 
 
 
 
 
 
 
 
10 SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS 
10.1 REGULATORY, ETHICAL, AND STUDY OVERSIGHT CONSIDERATIONS 

 
 
 
15 March 2024 
 
 
 
25 
10.1.1 INFORMED CONSENT PROCESS 
10.1.1.1 CONSENT/ASSENT 
AND 
OTHER 
INFORMATIONAL 
DOCUMENTS 
PROVIDED 
TO 
PARTICIPANTS 
 
 
As this is a minimal risk non-treatment study, consent will involve a brief consent process, with screening 
instrument completion being the indicator of consent. The wording of the brief consent that will appear 
before screening instruments delivered by all 4 methods is below and has been submitted with this 
protocol. 
-brief consent/study information 
10.1.1.2 CONSENT PROCEDURES AND DOCUMENTATION 
The study is classified as minimal risk because the anxiety and depression screening instruments are a 
best practice clinical and quality tool currently in use at the center without consent. The screener 
questions in the proposed research are identical to those in current clinical use, with the research 
involving only minor changes in the delivery method (while adhering to clinical communication and 
privacy policies). Randomizing delivery method does not increase the study risk level. Otherwise, the 
research involves collecting a small amount of pre-existing clinical EHR-based information and data on 
screener and clinic visit completion. Strict efforts to maintain confidentiality/privacy will be followed, and 
all study team members will have rigorous training in maintaining confidentiality. 
 
Because of the minimal risk classification of this study, the informed consent process has been modified 
in the following ways:  
• 
The following informed consent will appear at the start of the anxiety/depression screening 
questions for all delivery methods:  
o Your epilepsy doctor has included questionnaires about your mood and anxiety as part of 
your clinic visit. Taking part in this survey may involve providing information that you 
consider confidential or private. Depending upon how you answer these questions, the 
responses may be shared with your neurology provider. Some information you provide 
may be used for research. Careful efforts will be made to keep your information 
confidential. You will not lose any services, benefits, or rights if you choose not to 
participate in this voluntary survey. 
o This brief consent format is acceptable because the study is minimal risk. 
o Thereby, those who complete anxiety/depression screening instruments will provide 
consent for use of research data and potential epilepsy provider notification of passive 
suicidality screen. 
• 
For those who do not complete the anxiety/depression screeners, epilepsy provider notification 
is not applicable, and it is impractical to obtain informed consent: 
o 1. It is impractical to approach individuals who canceled or missed the clinic visit. 
o 2. For individuals who do not complete the anxiety/depression screeners but do attend 
the epilepsy clinic visit, it is impractical for appropriately trained staff to approach them 
for consent at the clinic visit, since these visits may occur at any time on any day, 
including simultaneously at disparate clinic sites. 
o Thus, among those who do not complete the screening instruments, waiver of 
informed consent and HIPAA waiver are acceptable because the study is classified as 
minimal risk and because obtaining consent is impractical.  

 
 
 
15 March 2024 
 
 
 
26 
 
10.1.2 STUDY DISCONTINUATION AND CLOSURE 
This study may be temporarily suspended or prematurely terminated if there is sufficient reasonable 
cause. Written notification, documenting the reason for study suspension or termination, will be provided 
by the suspending or terminating party to investigator, funding agency, and regulatory authorities. If the 
study is prematurely terminated or suspended, the Principal Investigator (PI) will promptly inform the 
Institutional Review Board (IRB), and sponsor/funding agency and will provide the reason(s) for the 
termination or suspension.  
 
Circumstances that may warrant termination or suspension include, but are not limited to: 
• 
Determination of unexpected, significant, or unacceptable risk to participants 
• 
Insufficient compliance of study staff to the protocol   (ie, significant protocol violations) 
• 
Data that are not sufficiently complete and/or evaluable 
• 
Determination of futility 
 
The study may resume once concerns about safety, protocol compliance, and data quality are addressed, 
and satisfy the funding agency, sponsor, IRB, Food and Drug Administration (FDA), or other relevant 
regulatory or oversight bodies (OHRP). 
 
 
10.1.3 CONFIDENTIALITY AND PRIVACY  
 
Data for analysis will be housed in REDCap26 and data quality will be monitored by our team throughout 
the study. 
 
Participant confidentiality and privacy is strictly held in trust by the participating investigators, their staff, 
and funding agency. This confidentiality is extended to the data being collected as part of this study. Data 
that could be used to identify a specific study participant will be held in strict confidence within the 
research team. No personally-identifiable information from the study will be released to any unauthorized 
third party without prior written approval of the sponsor/funding agency.  
 
All research activities will be conducted in as private a setting as possible. 
 
Authorized representatives of the sponsor or funding agency, representatives of the Institutional Review 
Board (IRB), or regulatory agencies may inspect all documents and records required to be maintained by 
the investigator, including but not limited to, medical records (office, clinic, or hospital) for the 
participants in this study. The clinical study site will permit access to such records. 
 
The only contact information accessed for participants will be the minimal information necessary for 
screener delivery by the allocated method, and this will not be recorded in study records other than as 
required for screener delivery.  

 
 
 
15 March 2024 
 
 
 
27 
At the end of the study, all records will continue to be kept in a secure location for as long a period as 
dictated by the reviewing IRB, Institutional policies, or sponsor/funding agency requirements. 
 
Study participant research data, which is for purposes of statistical analysis and scientific reporting, will 
be stored at Wake Forest University Health Sciences. This will not include the participant’s identifying 
information. Rather, individual participants and their research data will be identified by a unique study 
identification number. The study data entry and study management systems used by research staff will 
be secured and password protected. At the end of the study, all study databases will be de-identified and 
archived at Wake Forest University Health Sciences. 
 
Measures Taken to Ensure Confidentiality of Data Shared per the NIH Data Sharing Policies  
It is NIH policy that the results and accomplishments of the activities that it funds should be made available 
to the public (see https://grants.nih.gov/policy/sharing.htm). The PI will ensure all mechanisms used to 
share data will include proper plans and safeguards for the protection of privacy, confidentiality, and 
security for data dissemination and reuse (e.g., all data will be thoroughly de-identified and will not be 
traceable to a specific study participant). Plans for archiving and long-term preservation of the data will 
be implemented, as appropriate.  
 
Certificate of Confidentiality  
To further protect the privacy of study participants, the Secretary, Health and Human Services (HHS), has 
issued a Certificate of Confidentiality (CoC) to all researchers engaged in biomedical, behavioral, clinical 
or other human subjects research funded wholly or in part by the federal government.  Recipients of NIH 
funding for human subjects research are required to protect identifiable research information from forced 
disclosure per the terms of the NIH Policy (see https://humansubjects.nih.gov/coc/index). As set forth 
in 45 CFR Part 75.303(a) and NIHGPS Chapter 8.3, recipients conducting NIH-supported research covered 
by this Policy are required to establish and maintain effective internal controls (e.g., policies and 
procedures) that provide reasonable assurance that the award is managed in compliance with Federal 
statutes, regulations, and the terms and conditions of award. It is the NIH policy that investigators and 
others who have access to research records will not disclose identifying information except when the 
participant consents or in certain instances when federal, state, or local law or regulation requires 
disclosure. NIH expects investigators to inform research participants of the protections and the limits to 
protections provided by a Certificate issued by this Policy. 
10.1.4 FUTURE USE OF STORED SPECIMENS AND DATA  
Data collected for this study will be analyzed and stored at Wake Forest University Health Sciences. After 
the study is completed and results published by primary study team, de-identified, archived data will be 
stored for potential use if requested by other qualified researchers including those outside of the study.  
 
When the study is completed and results published by primary research team, access to study data may 
be provided by the PI if requested by qualified investigators. 
 
10.1.5 KEY ROLES AND STUDY GOVERNANCE 

 
 
 
15 March 2024 
 
 
 
28 
Principal Investigator: 
Heidi M. Munger Clary, MD, MPH 
Wake Forest University School of 
Medicine  
1 Medical Center Blvd. 
336-716-7110 
hmungerc@wakehealth.edu 
 
Study team workgroups:  
-bioinformatics 
-data management/statistical analysis 
-operations 
10.1.6 SAFETY OVERSIGHT 
 
Safety oversight for this minimal risk, non-treatment study will be conducted via study team self-
assessments guided by the Quality Management Plan, supervised by the Principal Investigator. Actions to 
notify clinicians of potential suicidality will be tracked and monitored for timely adherence to this 
procedure (see section 10.1.8). Any safety concerns will be promptly reported to the IRB. 
10.1.7 CLINICAL MONITORING 
 
N/A 
10.1.8 QUALITY ASSURANCE AND QUALITY CONTROL 
The study team will perform internal quality management of study conduct, data collection, 
documentation and completion, according to the quality management plan. 
 
Quality control (QC) procedures will be implemented as follows: 
 
Source documents and the electronic data --- Some data will be initially captured in the electronic health 
record or source documents (see Section 10.1.9, Data Handling and Record Keeping) and will ultimately 
be entered into the study database.  To ensure accuracy site staff will compare a representative sample 
of source data (where applicable) against the database, targeting key data points in that review. 
 
Protocol Deviations – The study team will review protocol deviations on an ongoing basis and will 
implement corrective actions when the quantity or nature of deviations are deemed to be at a level of 
concern. 
 
Safety/clinical alerts – the study team will track and review timeliness and completion of clinician 
notification in response to screening instruments for all results requiring alerts and corrective action will 
be carried out immediately if any notification delays are identified. The study team will track and review 
timeliness of EHR entry of REDCap-origin screening results on a weekly basis initially during study conduct 

 
 
 
15 March 2024 
 
 
 
29 
and take corrective action for any delayed entries. Frequency of this tracking may be reduced to monthly 
for a subset of participants if initial review demonstrates consistent timely entry. 
 
Should independent monitoring become necessary, the PI will provide direct access to all trial related 
sites, source data/documents, and reports for the purpose of monitoring and auditing by the 
sponsor/funding agency, and inspection by local and regulatory authorities. 
 
10.1.9 DATA HANDLING AND RECORD KEEPING  
10.1.9.1 DATA COLLECTION AND MANAGEMENT RESPONSIBILITIES  
 
Data collection will be the responsibility of the clinical trial staff at the site under the supervision of the 
investigator. The investigator will be responsible for ensuring the accuracy, completeness, legibility, and 
timeliness of the data reported. 
 
All source documents will be completed in a neat, legible manner to ensure accurate interpretation of 
data.  Hardcopies of the study visit worksheets as applicable will be provided for use as source document 
worksheets for recording data for some data elements in the study.  Data recorded in the electronic case 
report form (eCRF) derived from source documents will be consistent with the data recorded on the 
source documents.  
 
Clinical data (including medications and other data) will be entered into REDCap,26 a 21 CFR Part 11-
compliant data capture system provided by Wake Forest University Health Sciences. The data system 
includes password protection and internal quality checks, such as automatic range checks, to identify data 
that appear inconsistent, incomplete, or inaccurate. Clinical data will be entered directly from the source 
documents, including the EHR as the source document for some data.  Some data will be entered directly 
by participants in the REDCap system as outlined earlier in the protocol. 
10.1.9.2 STUDY RECORDS RETENTION  
 
Study documents will be retained for a minimum of 3 years from the date of the Federal Financial Report 
(FFR) submission. These documents should be retained for a longer period, however, if required by local 
regulations. No records will be destroyed without the written consent of the sponsor/funding agency, if 
applicable. It is the responsibility of the sponsor/funding agency to inform the investigator when these 
documents no longer need to be retained. 
10.1.10 
PROTOCOL DEVIATIONS   
This protocol defines a protocol deviation as any noncompliance with the clinical trial protocol, 
International Council on Harmonisation Good Clinical Practice (ICH GCP), or Manual of Procedures (MOP) 
requirements. The noncompliance may be either on the part of the participant, the investigator, or the 
study site staff. As a result of deviations, corrective actions will be developed by the site and implemented 
promptly.  

 
 
 
15 March 2024 
 
 
 
30 
 
These practices are consistent with ICH GCP:  
• 
Section 4.5 Compliance with Protocol, subsections 4.5.1, 4.5.2, and 4.5.3  
• 
Section 5.1 Quality Assurance and Quality Control, subsection 5.1.1  
• 
Section 5.20 Noncompliance, subsections 5.20.1, and 5.20.2.  
 
It will be the responsibility of the investigator to use continuous vigilance to identify and report deviations 
within 7 working days of identification of the protocol deviation, or within 7 working days of the scheduled 
protocol-required activity. All deviations will be addressed in study source documents, reported to NCATS 
Program Official if required. Protocol deviations will be sent to the reviewing Institutional Review Board 
(IRB) per their policies. The investigator will be responsible for knowing and adhering to the reviewing IRB 
requirements. Further details about the handling of protocol deviations will be included in the MOP. 
10.1.11 
PUBLICATION AND DATA SHARING POLICY  
This study will be conducted in accordance with the following publication and data sharing policies and 
regulations: 
 
National Institutes of Health (NIH) Public Access Policy, which ensures that the public has access to the 
published results of NIH funded research. It requires scientists to submit final peer-reviewed journal 
manuscripts that arise from NIH funds to the digital archive PubMed Central upon acceptance for 
publication. 
 
This study will comply with the NIH Data Sharing Policy and Policy on the Dissemination of NIH-Funded 
Clinical Trial Information and the Clinical Trials Registration and Results Information Submission rule. As 
such, this trial will be registered at ClinicalTrials.gov, and results information from this trial will be 
submitted to ClinicalTrials.gov. In addition, every attempt will be made to publish results in peer-reviewed 
journals.  Data from this study may be requested from other researchers by contacting Dr. Heidi M. 
Munger Clary.  Considerations for ensuring confidentiality of these shared data are described in Section 
10.1.3. 
10.1.12 
CONFLICT OF INTEREST POLICY 
The independence of this study from any actual or perceived influence. Therefore, any actual conflict of 
interest of persons who have a role in the design, conduct, analysis, publication, or any aspect of this trial 
will be disclosed and managed. Furthermore, persons who have a perceived conflict of interest will be 
required to have such conflicts managed in a way that is appropriate to their participation in the design 
and conduct of this trial. The study leadership in conjunction with NCATS has established policies and 
procedures for all study group members to disclose all conflicts of interest and will establish a mechanism 
for the management of all reported dualities of interest. 
10.2 ADDITIONAL CONSIDERATIONS 
 
N/A 
 

 
 
 
15 March 2024 
 
 
 
31 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10.3 ABBREVIATIONS AND SPECIAL TERMS 
 
AE 
Adverse Event 
API 
Application Programming Interface 
CFR 
Code of Federal Regulations 
CLIA 
Clinical Laboratory Improvement Amendments 
CMP 
Clinical Monitoring Plan 
COC 
Certificate of Confidentiality 
CONSORT 
Consolidated Standards of Reporting Trials 
CRF 
Case Report Form 
DHHS 
Department of Health and Human Services 
EC 
Ethics Committee 
eCRF 
Electronic Case Report Forms 
EHR 
Electronic Health Record 
FDA 
Food and Drug Administration 
FDAAA 
Food and Drug Administration Amendments Act of 2007 
FFR 
Federal Financial Report 
GAD-7 
Generalized Anxiety Disorder-7 
GCP 
Good Clinical Practice 
HIPAA 
Health Insurance Portability and Accountability Act  
ICH 
International Council on Harmonisation  
ICMJE 
International Committee of Medical Journal Editors 
IRB 
Institutional Review Board 
ITT 
Intention-To-Treat 
MOP 
Manual of Procedures 

 
 
 
15 March 2024 
 
 
 
32 
NCT 
National Clinical Trial 
NDDI-E 
Neurological Disorder Depression Inventory-Epilepsy 
NIH  
National Institutes of Health 
OHRP 
Office for Human Research Protections 
PI 
Principal Investigator 
QA 
Quality Assurance 
QC 
Quality Control 
REDCap 
Research Electronic Data Capture 
SAE 
Serious Adverse Event 
SAP 
Statistical Analysis Plan 
SMART on 
FHIR 
Substitutable Medical Applications, Reusable Technologies (SMART) on Fast Healthcare 
Interoperability Resources (FHIR) 
SOA 
Schedule of Activities 
SOP 
Standard Operating Procedure 
UP 
Unanticipated Problem 
US 
United States 
 
 
 

 
 
 
15 March 2024 
 
 
 
33 
10.4 PROTOCOL AMENDMENT HISTORY 
The table below is intended to capture changes of IRB-approved versions of the protocol, including a 
description of the change and rationale.  
 
Version 
Date 
Description of Change  
Brief Rationale 
2.0 
April 28, 
2023 
Clarify pragmatic trial, standard 
care previsit questionnaires & 
clinical privacy policies to be 
followed, stratify by 
antidepressant status, consent 
information wording modified 
following stakeholder input 
Follow-up changes after 
funding received, stakeholder 
engagement and further 
communication with study 
team and privacy officer 
3.0 
Dec 5, 2023 
Corrected wording of brief 
consent to align with IRB 
recommended changes made 
after prior protocol submission. 
Ensure alignment between 
consent document and 
protocol wording. 
4.0 
March 15, 
2024 
-Update description of Aim 2 
procedures to align with 
necessary updates in technical 
approach identified in technical 
build of kit tools.  
-Add additional 
reproducibility/generalizability 
assessment enabled by 
Encompass harmonization. 
Ensure protocol captures 
accurate description of 
technical approach to aim 2a 
kit technology build and data 
collection approach. 
-Evaluate generalizability of 
Kit flowsheet REDCap plug-
in data collection on the 
same dataset in an additional 
instance of the Epic EHR, 
enabled by health system 
Epic instance transition and 
data transfer from prior 
instance. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
15 March 2024 
 
 
 
34 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
15 March 2024 
 
 
 
35 
11 REFERENCES  
1. 
Zareifopoulos N, Bellou A, Spiropoulou A, Spiropoulos K. Prevalence, Contribution to Disease Burden and 
Management of Comorbid Depression and Anxiety in Chronic Obstructive Pulmonary Disease: A Narrative 
Review. COPD. 2019;16(5-6):406-417. 
2. 
Matthews R, Ali Z. Co-morbid Mental Health Issues in Patients with Pemphigus Vulgaris and Pemphigus 
Foliaceus. Clin Exp Dermatol. 2021. 
3. 
Davis BE, Lakin L, Binns CC, Currie KM, Rensel MR. Patient and Provider Insights into the Impact of 
Multiple Sclerosis on Mental Health: A Narrative Review. Neurol Ther. 2021;10(1):99-119. 
4. 
Morris R, Jones J, Wilcox J, Cole S. Depression and anxiety screening after stroke: adherence to guidelines 
and future directions. Disabil Rehabil. 2012;34(9):733-739. 
5. 
Kwan P, Yu E, Leung H, Leon T, Mychaskiw MA. Association of subjective anxiety, depression, and sleep 
disturbance with quality-of-life ratings in adults with epilepsy. Epilepsia. 2009;50(5):1059-1066. 
6. 
Choi H, Hamberger MJ, Munger Clary H, et al. Seizure frequency and patient-centered outcome 
assessment in epilepsy. Epilepsia. 2014;55(8):1205-1212. 
7. 
Boylan LS, Flint LA, Labovitz DL, Jackson SC, Starner K, Devinsky O. Depression but not seizure frequency 
predicts quality of life in treatment-resistant epilepsy. Neurology. 2004;62(2):258-261. 
8. 
Palacios J, Khondoker M, Mann A, Tylee A, Hotopf M. Depression and anxiety symptom trajectories in 
coronary heart disease: Associations with measures of disability and impact on 3-year health care costs. J 
Psychosom Res. 2018;104:1-8. 
9. 
Scott AJ, Sharpe L, Hunt C, Gandy M. Anxiety and depressive disorders in people with epilepsy: A meta-
analysis. Epilepsia. 2017;58(6):973-982. 
10. 
Gilliam FG, Santos J, Vahle V, Carter J, Brown K, Hecimovic H. Depression in epilepsy: ignoring clinical 
expression of neuronal network dysfunction? Epilepsia. 2004;45 Suppl 2:28-33. 
11. 
Beyenburg S, Mitchell AJ, Schmidt D, Elger CE, Reuber M. Anxiety in patients with epilepsy: systematic 
review and suggestions for clinical management. Epilepsy & behavior : E&B. 2005;7(2):161-171. 
12. 
Munger Clary HM, Snively BM, Hamberger MJ. Anxiety is common and independently associated with 
clinical features of epilepsy. Epilepsy & behavior : E&B. 2018;85:64-71. 
13. 
Munger Clary HM, Wan M, Conner K, et al. Examining brief and ultra-brief anxiety and depression 
screening methods in a real-world epilepsy clinic sample. Epilepsy & behavior : E&B. 2021;118:107943. 
14. 
Patel AD, Baca C, Franklin G, et al. Quality improvement in neurology: Epilepsy Quality Measurement Set 
2017 update. Neurology. 2018;91(18):829-836. 
15. 
Bermeo-Ovalle A. Psychiatric Comorbidities in Epilepsy: We Learned to Recognize Them; It Is Time to Start 
Treating Them. Epilepsy Curr. 2016;16(4):270-272. 
16. 
Gandy M, Modi AC, Wagner JL, et al. Managing depression and anxiety in people with epilepsy: A survey 
of epilepsy health professionals by the ILAE Psychology Task Force. Epilepsia Open. 2021;6(1):127-139. 
17. 
Scott AJ, Sharpe L, Thayer Z, et al. How frequently is anxiety and depression identified and treated in 
hospital and community samples of adults with epilepsy? Epilepsy & behavior : E&B. 2021;115:107703. 

 
 
 
15 March 2024 
 
 
 
36 
18. 
Thompson AW, Kobau R, Park R, Grant D. Epilepsy care and mental health care for people with epilepsy: 
California Health Interview Survey, 2005. Preventing chronic disease. 2012;9:E60. 
19. 
Reid AY, Metcalfe A, Patten SB, Wiebe S, Macrodimitris S, Jette N. Epilepsy is associated with unmet 
health care needs compared to the general population despite higher health resource utilization--a 
Canadian population-based study. Epilepsia. 2012;53(2):291-300. 
20. 
Munger Clary HM, Croxton RD, Snively BM, et al. Neurologist prescribing versus psychiatry referral: 
Examining patient preferences for anxiety and depression management in a symptomatic epilepsy clinic 
sample. Epilepsy & behavior : E&B. 2021;114(Pt A):107543. 
21. 
Henry TL, Schmidt S, Lund MB, et al. Improving Depression Screening in Underserved Populations in a 
Large Urban Academic Primary Care Center: A Provider-Centered Analysis and Approach. Am J Med Qual. 
2020;35(4):315-322. 
22. 
Trivedi MH, Rush AJ, Wisniewski SR, et al. Evaluation of outcomes with citalopram for depression using 
measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 2006;163(1):28-40. 
23. 
Zupancic M, Yu S, Kandukuri R, Singh S, Tumyan A. Practice-based learning and systems-based practice: 
detection and treatment monitoring of generalized anxiety and depression in primary care. J Grad Med 
Educ. 2010;2(3):474-477. 
24. 
Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the 
primary care setting: a randomized controlled trial. Jama. 2002;288(22):2836-2845. 
25. 
Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane 
Database Syst Rev. 2012;10:CD006525. 
26. 
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--
a metadata-driven methodology and workflow process for providing translational research informatics 
support. Journal of biomedical informatics. 2009;42(2):377-381. 
27. 
Hussien H, Apetrii M, Covic A. Health-related quality of life in patients with chronic kidney disease. Expert 
Rev Pharmacoecon Outcomes Res. 2021;21(1):43-54. 
28. 
Lwin MN, Serhal L, Holroyd C, Edwards CJ. Rheumatoid Arthritis: The Impact of Mental Health on Disease: 
A Narrative Review. Rheumatol Ther. 2020;7(3):457-471. 
29. 
Zhao SS, Miller N, Harrison N, Duffield SJ, Dey M, Goodson NJ. Systematic review of mental health 
comorbidities in psoriatic arthritis. Clin Rheumatol. 2020;39(1):217-225. 
30. 
Siu AL, Force USPST, Bibbins-Domingo K, et al. Screening for Depression in Adults: US Preventive Services 
Task Force Recommendation Statement. Jama. 2016;315(4):380-387. 
31. 
Yano EM, Chaney EF, Campbell DG, et al. Yield of practice-based depression screening in VA primary care 
settings. J Gen Intern Med. 2012;27(3):331-338. 
32. 
Brown EL, Raue PJ, Halpert K. Evidence-Based Practice Guideline: Depression Detection in Older Adults 
With Dementia. J Gerontol Nurs. 2015;41(11):15-21. 
33. 
The American College of Obstetricians and Gynecologists Committee Opinion no. 630. Screening for 
perinatal depression. Obstet Gynecol. 2015;125(5):1268-1271. 
34. 
Sico JJ, Sarwal A, Benish SM, et al. Quality improvement in neurology: Neurology Outcomes Quality 
Measurement Set. Neurology. 2020;94(22):982-990. 

 
 
 
15 March 2024 
 
 
 
37 
35. 
Fazel S, Wolf A, Langstrom N, Newton CR, Lichtenstein P. Premature mortality in epilepsy and the role of 
psychiatric comorbidity: a total population study. Lancet. 2013;382(9905):1646-1654. 
36. 
Caller TA, Chen JJ, Harrington JJ, Bujarski KA, Jobst BC. Predictors for readmissions after video-EEG 
monitoring. Neurology. 2014;83(5):450-455. 
37. 
Au A, Leung P, Kwok A, Li P, Lui C, Chan J. Subjective memory and mood of Hong Kong Chinese adults with 
epilepsy. Epilepsy & behavior : E&B. 2006;9(1):68-72. 
38. 
Kanner AM, Barry JJ, Gilliam F, Hermann B, Meador KJ. Depressive and anxiety disorders in epilepsy: do 
they differ in their potential to worsen common antiepileptic drug-related adverse events? Epilepsia. 
2012;53(6):1104-1108. 
39. 
Petrovski S, Szoeke CE, Jones NC, et al. Neuropsychiatric symptomatology predicts seizure recurrence in 
newly treated patients. Neurology. 2010;75(11):1015-1021. 
40. 
Barry JJ, Ettinger AB, Friel P, et al. Consensus statement: the evaluation and treatment of people with 
epilepsy and affective disorders. Epilepsy & behavior : E&B. 2008;13 Suppl 1:S1-29. 
41. 
Kerr MP, Mensah S, Besag F, et al. International consensus clinical practice statements for the treatment 
of neuropsychiatric conditions associated with epilepsy. Epilepsia. 2011;52(11):2133-2138. 
42. 
Munger Clary H, Snively B, Duncan P, Topaloglu U, Brenes G. Patient reported outcomes of anxiety and 
depression in adults with epilepsy: 6-month outcomes during usual care in a pilot randomized trial of two 
remote outcome assessment methods. Annals of neurology. 2021;90:S109. 
43. 
Sharpe M, Walker J, Holm Hansen C, et al. Integrated collaborative care for comorbid major depression in 
patients with cancer (SMaRT Oncology-2): a multicentre randomised controlled effectiveness trial. Lancet. 
2014;384(9948):1099-1108. 
44. 
Pyne JM, Fortney JC, Curran GM, et al. Effectiveness of collaborative care for depression in human 
immunodeficiency virus clinics. Archives of internal medicine. 2011;171(1):23-31. 
45. 
Thota AB, Sipe TA, Byard GJ, et al. Collaborative care to improve the management of depressive disorders: 
a community guide systematic review and meta-analysis. Am J Prev Med. 2012;42(5):525-538. 
46. 
Katon WJ, Lin EH, Von Korff M, et al. Collaborative care for patients with depression and chronic illnesses. 
N Engl J Med. 2010;363(27):2611-2620. 
47. 
Panagioti M, Bower P, Kontopantelis E, et al. Association Between Chronic Physical Conditions and the 
Effectiveness of Collaborative Care for Depression: An Individual Participant Data Meta-analysis. JAMA 
psychiatry. 2016;73(9):978-989. 
48. 
Huffman JC, Niazi SK, Rundell JR, Sharpe M, Katon WJ. Essential articles on collaborative care models for 
the treatment of psychiatric disorders in medical settings: a publication by the academy of psychosomatic 
medicine research and evidence-based practice committee. Psychosomatics. 2014;55(2):109-122. 
49. 
Ciechanowski P, Chaytor N, Miller J, et al. PEARLS depression treatment for individuals with epilepsy: a 
randomized controlled trial. Epilepsy & behavior : E&B. 2010;19(3):225-231. 
50. 
Chaytor N, Ciechanowski P, Miller JW, et al. Long-term outcomes from the PEARLS randomized trial for 
the treatment of depression in patients with epilepsy. Epilepsy & behavior : E&B. 2011;20(3):545-549. 
51. 
Miller DP. Tablet-based screening identifies 5x as many at-risk primary care patients. American Medical 
Informatics Association 2020 Clinical Informatics Conference; May 2020, 2020; Virtual. . 

 
 
 
15 March 2024 
 
 
 
38 
52. 
Moura L, Magliocco B, Ney JP, Cheng EM, Esper GJ, Hoch DB. Implementation of quality measures and 
patient-reported outcomes in an epilepsy clinic. Neurology. 2019;93(22):e2032-e2041. 
53. 
Patt D, Wilfong L, Hudson KE, et al. Implementation of Electronic Patient-Reported Outcomes for 
Symptom Monitoring in a Large Multisite Community Oncology Practice: Dancing the Texas Two-Step 
Through a Pandemic. JCO Clin Cancer Inform. 2021;5:615-621. 
54. 
Jeffrey J, Do MT, Hajal N, et al. Using web-based technology to improve depression screening in primary 
care settings. BMJ Open Qual. 2021;10(1). 
55. 
Bunnell BE, Davidson TM, Winkelmann JR, et al. Implementation and Utility of an Automated Text 
Messaging System to Facilitate Symptom Self-Monitoring and Identify Risk for Post-Traumatic Stress 
Disorder and Depression in Trauma Center Patients. Telemed J E Health. 2019;25(12):1198-1206. 
56. 
Evans WD, Wallace Bihm J, Szekely D, et al. Initial outcomes from a 4-week follow-up study of the 
Text4baby program in the military women's population: randomized controlled trial. J Med Internet Res. 
2014;16(5):e131. 
57. 
Food and Drug Administration USDoHaHS. Guidance for Industry: Electronic source data in clinical 
investigations. In:2013. 
58. 
Garza MY, Rutherford MW, Adagarla B, et al. Evaluating Site-Level Implementations of the HL7 FHIR 
Standard to Support eSource Data Exchange in Clinical Research. Stud Health Technol Inform. 
2021;281:397-401. 
59. 
Garza MY, Rutherford M, Myneni S, et al. Evaluating the Coverage of the HL7 ((R)) FHIR ((R)) Standard to 
Support eSource Data Exchange Implementations for use in Multi-Site Clinical Research Studies. AMIA 
Annu Symp Proc. 2020;2020:472-481. 
60. 
Metke-Jimenez A, Hansen D. FHIRCap: Transforming REDCap forms into FHIR resources. AMIA Jt Summits 
Transl Sci Proc. 2019;2019:54-63. 
61. 
Ahmad FA, Payne PRO, Lackey I, et al. Using REDCap and Apple ResearchKit to integrate patient 
questionnaires and clinical decision support into the electronic health record to improve sexually 
transmitted infection testing in the emergency department. J Am Med Inform Assoc. 2020;27(2):265-273. 
62. 
Hawley S, Yu J, Bogetic N, et al. Digitization of Measurement-Based Care Pathways in Mental Health 
Through REDCap and Electronic Health Record Integration: Development and Usability Study. J Med 
Internet Res. 2021;23(5):e25656. 
63. 
Mahabee-Gittens EM, Dexheimer JW, Khoury JC, Miller JA, Gordon JS. Development and Testing of a 
Computerized Decision Support System to Facilitate Brief Tobacco Cessation Treatment in the Pediatric 
Emergency Department: Proposal and Protocol. JMIR Res Protoc. 2016;5(2):e64. 
64. 
Munger Clary HM, Croxton RD, Allan J, et al. Who is willing to participate in research? A screening model 
for an anxiety and depression trial in the epilepsy clinic. Epilepsy & behavior : E&B. 2020;104(Pt 
A):106907. 
65. 
Munger Clary H, Snively B, Topaloglu U, et al. Pragmatic patient reported outcomes via EHR patient portal 
vs. telephone: A pilot randomized controlled trial among adults with epilepsy and anxiety or depression 
symptoms. American Medical Informatics Association Annual Symposium; 2021; San Diego, CA. 
66. 
Corporation ES. Interconnect: INT401 Programming for App Orchard-Basics. In: Verona, WI2021. Accessed 
Oct 9, 2021. 

 
 
 
15 March 2024 
 
 
 
39 
67. 
Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and 
designing behaviour change interventions. Implement Sci. 2011;6:42. 
68. 
Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the 
GAD-7. Archives of internal medicine. 2006;166(10):1092-1097. 
69. 
Micoulaud-Franchi JA, Lagarde S, Barkate G, et al. Rapid detection of generalized anxiety disorder and 
major depression in epilepsy: Validation of the GAD-7 as a complementary tool to the NDDI-E in a French 
sample. Epilepsy & behavior : E&B. 2016;57(Pt A):211-216. 
70. 
Gilliam FG, Barry JJ, Hermann BP, Meador KJ, Vahle V, Kanner AM. Rapid detection of major depression in 
epilepsy: a multicentre study. The Lancet Neurology. 2006;5(5):399-405. 
71. 
Gill SJ, Lukmanji S, Fiest KM, Patten SB, Wiebe S, Jette N. Depression screening tools in persons with 
epilepsy: A systematic review of validated tools. Epilepsia. 2017;58(5):695-705. 
72. 
Mula M, McGonigal A, Micoulaud-Franchi JA, May TW, Labudda K, Brandt C. Validation of rapid suicidality 
screening in epilepsy using the NDDIE. Epilepsia. 2016;57(6):949-955. 
73. 
Giambarberi L, Munger Clary H. Suicide and Epilepsy. Current neurology and neuroscience reports. 2022(In 
Press).