Participatory Design of Electronic Health Record Tools for Problem Solving Therapy
Phase: NA
Conditions: Depression
Sponsor: University of Washington

 
 
Study Title:  
Participatory design of electronic health 
record tools for problem solving therapy 
Document type:  
Study Protocol and Statistical Analysis Plan 
NCT#:  
NCT03516513 
Document date:  
15-May-2018 
 

Page 396 
Specific Aims 
 
 
SPECIFIC AIMS. Although evidence-based psychosocial interventions (EBPIs) are a preferred treatment 
option by vulnerable populations, they are rarely available in community primary care settings and when 
available, are often delivered with poor fidelity.1 High quality delivery of evidence-based psychosocial 
interventions (EBPIs) in primary care medicine is a function of many variables, including clinician training 
(UWAC Study 1) and usability of the intervention. Several studies find that for EBPIs to be delivered with 
sustained quality, on-going supervision and guidance is critical (this study’s focus).2-5 While the availability of 
clinicians trained in EBPIs is scarce, the availability to supervisors trained in EBPIs is even more limited. 
Given the ubiquity of electronic health records, automated decision support tools and feedback systems6 have 
been found to be effective in supporting sustained quality EBPIs7, but in practice have had mixed success on 
outcomes8,9 such that they may actually hinder clinical care10,11 and are often ignored by clinicians.12 In a 
report by the Agency for Healthcare Research and Quality, a significant barrier to the use of decision support 
tools is that these tools have not been developed with input from the clinician or in consideration of their work 
environment. Using the Center’s Discover, Design, Build, Test (DDBT) framework, we will work with clinicians 
from 13 BHIP sites to create a clinical decision tool that addresses the common decisional dilemmas clinicians 
face when implementing EBPIs. We hypothesize that creating tools to support EBPIs will result in improved 
clinician competency and sustained skill (target) to EBPIs, compared to clinicians without these supports, 
resulting in better patient outcomes (mediation, Fig. 1). The specific aims of this study are: 
Aim 1: Discover Phase (6 months). Using PAR informed user-centered design methods we will interview 
clinicians in primary care about challenges they face in the delivery of two EBPIs, Behavioral Activation and 
Problem Solving Treatment, observe them delivering these EBPIs, and receiving feedback on cases from 
experts in these EBPIs. This process will help us to identify the common decisional dilemma’s clinician’s face 
in delivering EBPIs, their preferences for expert guidance strategies, and how decision support tools could be 
embedded into clinic workflow to reduce obstacles and enhance the delivery of EBPIs. Contribution to Center: 
Data from this phase will be used to inform the Typology of EBPI Targets. 
Aim 2: Design/Build Phase (6 months). Based on information obtained in the discover phase, we will engage 
in a rapid cycle iterative prototype development and testing of decision support tools to support PST-PC will be 
carried out using user-centered design (UCD). The build of these tools will include the development of 
prototypes for user testing and refinement with input from care managers across the 13 BHIP sites. 
Contribution to the Center. Data from this phase will be used to inform the Matrix of EBPI Modifications. 
Aim 3: Test Phase (18 months) In the second to third year of the proposed project we will test the decision- 
support tools in a small pilot trial with six providers and thirty patients randomized to the use of the decision 
support tools. H1: Clinicians with access to decision tools will report better acceptability, usability, and less 
burden when using PST-PC than clinicians without the tools (Acceptability of Intervention Measure, 
Intervention Appropriateness Measure, Feasibility Of Intervention Measure, System Usability, Modified Self-
Efficacy Scale, and User Burden Scales). H2: Clinicians randomized to decision support tools will more 
competently deliver EBPI elements (fewer number of sessions rated below standard by experts and more time 
to first session rated below standard after certification) than clinicians randomized to unsupported EBPI. H3: 
Patients treated by clinicians with access to decision tools will have better patient-reported outcomes than 
patients treated by clinicians without access to these tools as assessed with functional disability (Sheehan 
Disability Scale) and change in depression symptoms over time (PHQ-9). We will conduct preliminary analysis 
to determine if change in clinical outcomes is mediated by clinician 
adherence to protocol. 
Exploratory Aim: Use the data collected during Aims 1-3 to 
demonstrate proof-of-concept work on using machine learning 
and natural language processing to automate and potentially 
personalize decision support tools. 
 
This study addresses IOM Recommendation 6-3: Conduct 
research to design and evaluate strategies that can influence 
the quality of psychosocial interventions, and NIMH SP 4.3 
Develop innovative service delivery models to improve 
dramatically the outcomes of mental health services received 
in diverse communities and populations 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 397 
Research Strategy 
 
 
  
SIGNIFICANCE 
 
1. Evidence-Based Depression Care is Lacking Among Vulnerable Populations. Depressive disorders are 
among the leading causes of disability, for which many effective treatments exist. Patients from low income 
and/or racial/ethnic minority populations are less likely to receive care for this prevalent and debilitating 
disorder.13,14 Our research group has shown that, in contrast to patients with private insurance, patients with 
Medicaid insurance have poor quality of depression care, which highlights a wide disparity in care delivery 
related to income and insurance source.15,16 Low-income populations with Medicaid insurance also have a 
higher rate of mood disorders.17 Up to half of patients with depression receive no evaluation or treatment for 
mental health issues and fewer than one-quarter who are evaluated and found to be depressed receive any 
mental health care.18 The increase in delivery of evidence-based depression care to vulnerable populations is 
a public health priority as described by the Healthy People 2020 objective for the mental health topic (MDMH 9 
and 9.2 – increasing the proportion of adults with major depressive disorder who receive treatment). Delivery of 
effective depression treatments to vulnerable populations is a major opportunity to reduce the burden of 
depression among the US population.19 
2. Collaborative Care has demonstrated efficacy, but the sustained delivery of EBPIs is low. 
Collaborative Care (CC) is an effective integrated care model with a strong evidence base for vulnerable 
populations.20,21 CC includes the use of EBPIs such as Problem Solving Treatment for primary care (PST-
PC)22. Because of its evidence base, the CC model is now the subject of substantial dissemination and 
implementation support through state, regional and national efforts, including the implementation of CC-
specific billing codes for Medicare insurance recipients. Central to the CC model is the “care manager,” who 
provides many of the core features of CC, including patient assessment, education, medication management, 
and psychosocial interventions (BA and PST-PC).23-25 Although many organizations elect to train care 
managers in PST-PC, the use of antidepressant medications is the predominant mode of treatment in settings 
using this model of care, despite patient preferences for EBPIs.25,26 Part of the challenge in the limited 
availability of EBPIs in CC is the lack of on-going decision support to care managers in the use of EBPIs. An 
integral part of the CC model is on-going decision support by psychiatrists or other prescribing professionals, 
who are typically not trained in EBPIs. Thus, care managers who are trained by experts in EBPIs, loose 
access to critical support in treatments that patients prefer. 
3. Clinical Decision Support Systems: Application to EBPIs. Although access to expert opinion when 
delivering EBPIs results in sustained delivery of high quality care3,27, the availability of this expertise is limited 
and costly. To mitigate this problem, recent efforts have turned out the use of Clinical Decision Support 
Systems (CDSS), data analytic tools embedded in electronic health records that compile patient information, 
synthesize and visualizes the information to support clinicians in making treatment decisions. Preliminary 
evidence suggests that these tools can be effective in supporting the use of EBPIs6,7 in clinical practices, 
however, clinician uptake of these tools is low, owing to perceived burden and poor clinical utility.9,11,12. 
According to AHRQ’s 2010 report on Clinical Decision Supports28, poor uptake is due to developers lack of 
familiarity with clinician work flows and limited clinician involvement in their creation. Many CDSSs are tied to 
treatment algorithms and are largely alert sed systems, meaning the decision support tool is activated with a 
patient is not responding to care. Few incorporate complex decision rules, and in our experience, do not 
provide the type of support care managers seek when implementing EBPIs. As was discussed in UWAC Study 
2, clinicians face many challenges in the implementation of EBPIs. Challenges relevant to sustained 
competency are memorability of the EBPI elements and ability to apply them with minimal guidance, ease of 
recovery from misapplications of EBPI content, amount of cognitive load the EBPI places on the clinician, in 
the form of task structure and number of steps required to implement it in a given session and across sessions, 
and flexibility of the EBPI to be implemented in different contexts.29 For CDSS tools to be usable and effective 
for care managers delivering EBPIs in busy primary care clinics, they must address the decisional dilemma’s 
that care managers feel they need the most support. 
4.Participant Action Research informed User-centered design (UCD) as a solution to creating a usable 
CDSS for EBPIs. Working from the AHRQ recommendations, we will employ our Discover, Design/Build and 
Teat (DDBT) Framework to create a CDSS to support sustained competency in the delivery of EBPIs (see 
Methods Core for more detailed explanation of the framework). Working with care managers from 13 primary 
care clinics that uses CC as their integrated care model, we will determine from care managers when they feel 
the need the most support in providing EBPIs, how that support should look, and how support tools would be 
integrated into the Care Management Tracking System (CMTS) a shared electronic health record that supports 
CC. Our intention is to develop a CDSS that is user friendly and effective, and is more than an alert system, 
but provides guidance on common decisional dilemma’s care managers face. 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 398 
Research Strategy 
 
 
The purpose of this study is to test whether an EBPI CDSS created with UCD-based clinician input 
throughout the process will result in a usable decision support tool, whether this tool will be (1) used by care 
managers, (2) improves the sustained EBPI competency, and (3) results in better patient outcomes through 
enhanced clinician ability to adhere to, and competently deliver, treatment (target mechanism of action; Fig. 1). 
5.Impact. Decision support and access to expert opinion on the delivery of EBPIs is limited and significantly 
impacts the quality of care3. This problem is recognized by many large health care systems who want to 
support the use of EBPIs, for instance, the Veteran’s Hospital Administration has invested in EBPIs, and have 
developed cost–efficient on-going support systems called communities of practice30, these communities are 
hard to create and sustain31, and do not guarantee against drift in clinician competencies. CDSS embedded in 
electronic health records appears the be the most efficient and sustainable method, but they are limited in the 
type of guidance they can provide and in their usability. The results from this pilot has the potential to make an 
impact on implementation and intervention science. By creating a decision support system that is informed by 
clinician input, we have the potential to impact the quality of EBPI care. This study is also part of the larger 
Center mission to begin creating a Typology of EBPI Targets for future intervention modification and potentially 
novel psychosocial interventions. It will also serve to inform the Center mission of identifying best methodology 
for EBPI modification (Matrix of EBPI Modifications). 
  
INNOVATION 
 
The main innovation from this project will be the implementation of UCD methods to create a clinical decision 
support system for sustained EBPI quality. The development of CDSS that clinicians will find useful (rather 
than burdensome), would be an innovation in the field; however, given the use of PAR and UCD in the 
development of this system, we may discover that the electronic health record is not the best way to provide 
the support clinicians want or need. Although we have planned this project to create a CDSS for an existing 
electronic health record, we may find that clinicians will uncover a unique process for clinical decision support. 
  
APPROACH 
 
Team Expertise. The proposed research project brings together a novel team of investigators with 
complementary skills including primary care mental health integration and PAR (Bennett), and EBPIs for 
Collaborative Care (Raue). Dr. Munson (UCD expert) will provide our design/build incubator expertise from the 
Center Methods Core. Dr. Bennett’s decades-long work in the implementation of mental health services in 
real-life primary care settings serving vulnerable populations informs the view that building EHR tools focusing 
on the support of EBPI delivery will result in greater uptake and resulting improved patient outcomes. 32-34 Dr. 
Raue is the lead trainer of EBPIs for the UW Aims Center and brings >10 years of experience working with 
clinicians in primary care on this EBPI.5  
P roblem-Solving Therapy for Primary Care (PST-PC). We will focus on developing support tools for PST-
PC because it is a common and evidence based depression intervention in primary care. Currently, no 
decision support tools for PST-PC post certification have been created, and thus, we have the unique 
opportunity to build on the existing decision support literature to tools informed by clinician input. Commonly, 
PST-PC support and access to experts ends once the clinician has been certified. Although certified clinicians 
may seek support as needed from the AIMS Center, we have yet to have anyone do so post-certification. 
C onceptual Frameworks. We drew upon several conceptual frameworks to structure the planned research 
project. First, The “Dynamic Systems Framework” for implementation and sustainability of interventions into 
real life contexts is a key model for the current study and underlies our proposed mechanism of action 
(Figure1).38 This implementation science (IS) framework emphasizes the need for interventions, including 
EBPIs, to adapt to particular clinical contexts in order to be effective and sustained. Central to this model is the 
view that the better matched to the needs of the setting, the better the implementation, and the greater the 
effectiveness of the intervention will be. 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 399 
Research Strategy 
 
 
 
 
S tudy Setting. The study will be carried out within the Behavioral Health Integration Program (BHIP) of the 
primary care network of the University of Washington Health System. This network provides care to 
approximately 250,000 unique patients annually across 13 sites with a majority from vulnerable populations, 
including 60% Medicare or Medicaid insured, and a high proportion of patients from racial/ethnic minority 
populations. This is a high-need and high-cost population, with 38,000 visits annually associated with a 
diagnosis of depression. These multi-disciplinary sites deliver general primary care and utilize an evidence- 
based model of the team-based collaborative care system for the BHIP program. 22,39-43 BHIP has been in 
place since 2011 with over three thousand patients participating in treatment. 
S tudy Overview. To accomplish the aims of the study, this project will be broken down into 3 phases that are 
tied to each aim of the study: The Discover Phase, (Aim 1), The Design and Build Phase (Aim 2), and the Test 
Phase (Aim 3; see Fig.1). For phases 1 and 2, we will identify 15 representative clinicians working in BHIP 
clinics partnering. For phase 3, we will work with 6 care managers and 30 patient participants, >18 years old 
with a PHQ9 of 10-20. The study will be conducted over a 2.5-year period during Y02 through Y04. Steps 1 
and 2 will be conducted in the first 12 months of the study and Step 3 in the last 18 months (See timeline). 
 
P hase 1 / Aim 1: Discover (6 months). The purpose of this phase is to uncover common decisional 
dilemmas and challenges in the sustained use of PST-PC. We will have the advantage of Study 2’s Discover 
phase data that will have already identified modification targets of key components of PST-PC, including 
identification of user needs and user testing of PST-PC. We will replicate the user needs assessment and user 
testing process as is described in Study 2, with 15 representative clinicians to confirm that the information 
collected in rural FQHCs generalizes to BHIP clinics. We will need to conduct a new contextual evaluation and 
create an As-Is work-flow diagram focused on how clinicians interact with CMTS. This data will be useful in 
informing how CMTS can be used to support quality implementation of PST-PC elements. 
 
Usability testing. Usability testing will be carried out using a “discounted usability engineering” approach.44 
Observation of PST-PC from UWAC Study 2 will be used for PST-PC usability, but we will conduct usability 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 400 
Research Strategy 
 
 
testing with the current CMTS care management system. This will be carried out using a range of typical case 
scenarios. The design team will monitor clinician interaction with CMTS while using a Think-Aloud protocol to 
describe their experiences. Patient use testing will involve reviews of current patient care materials and 
workflows. These sessions typically will last one hour each. These usability testing sessions will be recorded 
for later reference. 
 
Data synthesis and confirmation of findings. The Methods Core will conduct this analysis, through the 
creation of a matrix of decisional dilemmas and expert advice. The cells of this matrix will be populated with 
common decisional dilemma’s clinicians face in implementing PST-PC, and accompanying expert advice. This 
matrix will be shared with the CPB, EAB, (see Admin Core) and the EBPI experts in the Methods Core 
(Aisenberg, Areán and Kaysen). For the CMTS based tool, an as-Is workflow diagram of challenges and 
opportunities for use of CMTS will be created to allow for a pictorial representation of CMTS use in the context 
of the care managers’ day. As an example, we will need to determine whether care mangers use the CMTS 
before each patient, after each patient, or at different times of day (morning or at the end of day). This will 
determine if a CDSS embedded into the electronic health records would provide timely information. The 
generated matrix and diagrams will be reviewed by the 15 clinicians for confirmation of accuracy. Finally, data 
collected on the implementation challenges of PST-PC through these sources will be given to the Methods 
Core directors for use in the development of the Typology of EBPI Targets. 
P hase 2 / Aim 2: Design and Build (6 months). The next step in the process will be to create initial 
prototypes of decision support tools, based on the findings in the Discover phase. The Methods Core designer, 
working in consultation with Drs. Munson, will create 3-5 initial prototypes. The process, described in the 
Methods Core, involves the same clinicians who participated in the Discover phase and has them work with 
the designer to create initial prototypes. If the clinicians recommend the creation of CDSS for the electronic 
health record, prototypes will be embedded in a mockup of the CMTS, which will then be uploaded into the 
remote prototyping portal for care managers to interact with and give further feedback for additional 
modification. As they interact with the prototype, they will engage in a think-aloud process and will assess each 
prototype based on the System Usability Scale (SUS; see Methods Core for description). This rapid iteration 
will end when we create a prototype with a SUS score of 80 or greater. Information from the think-aloud 
practice will be used to inform the prototype modifications.44,45 Periodically there will be meetings with the 
CMTS developer group to ensure that aspects of the prototypes under development are technically feasible 
within the CMTS system and alternative approaches will be elicited to ensure that the prototypes are headed in 
a direction that can realistically be built into the EHR. Should the Discover phase reveal that a CDSS 
embedded in electronic health records is not the optimal method for decision support, prototypes of potential 
solutions will follow a similar process as described above. 
Once a set of sufficiently advanced prototypes have been developed, we will move to the Build phase of the 
project. If the solution is a CDSS, at this point the CMTS developer team will be brought into the design 
process and the tools and desired functionalities will be described until it is clear to all parties. Feedback on the 
capacity of the CMTS system to accommodate specific designs will be reviewed. When an adequate 
agreement on the nature of the designs is achieved, the CMTS development team will build the proposed tools 
in a development environment. Whatever the solution is, a final prototype will undergo initial feasibility 
evaluation in pilot field deployments with Dr. Raue. He will test the system with one or two clinicians in the 
BHIP program using the prototype with test patient scenarios. Once we have completed the design of PST-PC 
support system so that it (1) meets needs of its clinicians (i.e., is useful), and (2) is easy to use and 
understand (i.e., is usable), we will move to the Test phase of this study. Data from this process will be used to 
inform the Matrix of EBPI Modifications. 
P hase 3 / Aim 3: Test Phase (18 months). We will conduct a pilot study comparing clinicians randomized to 
decision support to clinicians randomized to no support. Six care managers who will each see 30 patients will 
be identified and complete informed consent. Clinicians at all 13 BHIP sites will be offered the opportunity to 
participate in the trial. 
 
Patient eligibility. Patient participants will be identified through clinic electronic health records. Participants 
must be 18 years old or older and suffering from depression (score between 10 and 20 on the PHQ-9). 
Methods Core research assistants will contact potentially eligible patients to conduct a preliminary screening for 
eligibility, provide a brief description of the study, and obtain informed consent. 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 401 
Research Strategy 
 
 
Clinician recruitment and training. Clinicians will be incentivized to participate in the trial by receiving free 
training in the PST-PC EBPI, culminating in a certification that can be used for career advancement for these 
clinical social workers. The training in this EBPI is under consideration to be delivered by the AIMS Center to 
all clinicians at the BHIP sites within the next several years. The results of the trial will be used to guide the 
remaining trainings. Six care managers in the BHIP program will be randomized to usual training or training 
with the use of the smart tools within the CMTS system. We will follow the PST-PC training protocol developed 
by the AIMS Center, which includes training in basic brief treatment skills, therapeutic elements of the 
intervention and therapy process. This is followed by simulated case training. Clinicians must pass an 
introductory session, middle and last session before deploying the intervention with patients. In PST-PC this is 
determined by obtaining a score of 3 (“satisfactory”) or greater on the PST Adherence Scales, which utilizes a 
Likert-type rating scale (0 = very poor, 5 = very good). Dr. Raue will provide clinician training in PST-PC to all 
clinicians. Clinicians randomized to the decision support condition will also receive a training manual and in-
person training in the use of these tools prior to the initiation of PST-PC training. Once a care manager in either 
arm of the study is certified to provide PST-PC, they will proceed to the four-month fidelity phase of the trial. 
Five active patients identified from the existing site registry of each care manager will receive treatment during 
the trial (30 total). 
Clinician Data Collection. Three types of data will be collected from clinicians: clinician assessment of 
appropriateness and acceptability, intervention usability, and clinician ability to learn the intervention and 
maintain competency. To measure decision support usability, clinicians will complete the Acceptability of 
Intervention Measure, Intervention Appropriateness Measure, Feasibility Of Intervention Measure, System 
Usability, Modified Self-Efficacy Scale, and User Burden Scales), which are core measures for this Center. 
These scales will be collected through the UWAC data portal after clinicians have completed their intervention 
with 5 patients. To measure sustained competency, clinicians will collect audio recordings of patient 
sessions for review by experts. Dr. Raue will conduct these reviews using the PST Therapist Adherence 
Scales. Sustained competency will be defined by (1) time to first session rated below average after certification 
in PST-PC and (2) number of sessions rated as <3 (“satisfactory”). Clinicians will not be given feedback on 
session performance. 
 
Patient data collection. We will use the UWAC data portal49 designed specifically for research on mobile 
depression interventions to confirm eligibility, complete the consent process, and conduct baseline and 
outcomes assessment. Patient data will be collect before treatment initiation and after treatment ends. 
Patients will complete a demographic survey to determine gender, age, ethnicity, income categories, and 
educational level. Patients will complete a series of brief clinical measures to determine the presence of a 
depressive disorder and if there are any important comorbidities. The Sheehan Disability Scale(SDS) 50,51 is a 
brief analog too assess functioning in work, social, and health domains, using visual-spatial, numeric and 
verbal anchors. The scale has been validated in medical and psychiatric populations with a variety of 
psychiatric diagnoses.50,51. The 9-item Patient Health Questionnaire52 (PHQ-9) consists of 9 DSM depression 
symptoms and one disability item. It has been found to have excellent sensitivity to change over time 
(sensitivity = .88, sensitivity = .80).53 
Data Analysis. Aim 3 focuses on pilot feasibility data to inform a larger (future) clinical trial comparing decision 
support of PST-PC to no decision support. Thus, analyses do not focus on hypothesis-driven inferential 
statistics but on descriptive statistics, graphical summaries, and basic effect sizes. H1: Usability. 
Differences in clinician reported usability between the two clinician groups will be plotted using a dotplot and 
tested using a t-test using the SUS and UBS as dependent variables. H 2: Fidelity/Competence.  
We will compare groups on the proportion of clinicians who reach certification (i.e. score of 3 or greater on 
treatment competence on at least two sessions). Between treatment differences will be tested with a 2-sample 
proportion test. (Future analyses with larger sample size will use a generalized linear mixed model approach to 
account for the nested data.)54,55 H3: 
Patient Reported Outcomes. Histograms and kernel density estimates will be plotted by treatment condition 
to explore differences on depression outcomes (PHQ-9 total score) and overall daily functioning (SDS). 
Descriptive statistics (M and SD) of these outcomes will also be examined by clinician group, using Cohen’s d 
as an effect size summary. A t-test will be used to examine treatment differences in these patient outcomes. 
Of ultimate interest is whether the modified treatment (i.e., decision-supported PST-PC vs. unsupported PST-
PC) effects patient outcomes through enhanced usability and clinician fidelity. We will use graphical 
summaries of the data to explore the relationships that are consistent with mediation (e.g., correlation of 
usability with patient outcomes [b pathway], and treatment differences on patient outcomes with fidelity 

Contact PD/PI: Arean, Patricia A. 
Research-Projects-003 (645) 
Page 402 
Research Strategy 
 
 
partialed out [c’ pathway’]). The primary focus here is whether the data and relationships appear consistent 
with mediation, as opposed to a formal test, which will be dramatically under-powered. 
The Exploratory Aim will use data collected from Aims 1-3 to inform hypotheses about how modern 
computational tools for written text and spoken language might further extend tools to support PST-PC. 
Conducting such analyses are beyond the scope of the current project, given the anticipated data quantities, 
but rather would be the focus of future research. Potential targets for machine learning and NLP could include: 
a) understanding mechanisms of decision-supported PST-PC via predicting change in PHQ9 from content of 
written notes, and b) using NLP to evaluate the quality of PST-PC from CMTS-based notes. 
Power and Sample Size. Sample sizes for Aims 1 and 2 were based on estimates from the UCD literature on 
necessary number of participants to capture critical design information, where the recommendation is 5-10 end 
users. Aim 3 is focused on gathering information (feasibility, recruitment and retention rates, response and 
attrition rates, etc.) for a future R01 application. As such, the sample size was set primarily for practical 
reasons; rather, it used estimated effect sizes and was not driven by power to test hypotheses.