JMIR FORMATIVE RESEARCH

Dimeff et al

Original Paper

A Suicide Prevention Digital Technology for Individuals
Experiencing an Acute Suicide Crisis in Emergency Departments:
Naturalistic Observational Study of Real-World Acceptability,
Feasibility, and Safety
Linda A Dimeff1,2, PhD; Kelly Koerner2, PhD; Kandi Heard2, MA; Allison K Ruork2, PhD; Angela Kelley-Brimer2,
MS; Suzanne T Witterholt3, MD; Mary Beth Lardizabal3, DO; Joseph R Clubb3, MSW, LICSW; Julie McComish4,
MBA; Arpan Waghray4, MD; Roger Dowdy4, LICSW, MHA; Sara Asad-Pursley4, MD; Maria Ilac2, BS; Hannah
Lawrence2, BA; Frank Zhou2, BS; Blair Beadnell5, PhD
1Evidence-Based Practice Institute, Beaverton, OR, United States

2

Evidence-Based Practice Institute, Seattle, WA, United States

3

Mental Health and Addiction Clinical Service Line, Allina Health, Minneapolis, MN, United States

4

Providence's Well Being Trust, Providence, Renton, WA, United States

5

Evaluation Specialists, Carlsbad, CA, United States

Corresponding Author:
Linda A Dimeff, PhD
Evidence-Based Practice Institute9450 SW Gemini Dr
PMB 68735
Beaverton, OR, 97008-7105
United States
Phone: 1 253 765 0455
Email: linda.dimeff@jasprhealth.com

Abstract
Background: Emergency departments (EDs) are the front line in providing suicide care. Expert consensus recommends thedelivery of several suicide prevention evidence-based interventions for individuals with acute suicidal ideation in the ED. ED
personnel demands and staff shortages compromise delivery and contribute to long wait times and unnecessary hospitalization.
Digital technologies can play an important role in helping EDs deliver suicide care without placing further demands on the careteam if their use is safe to patients in a routine care context.
Objective: This study evaluates the safety and effectiveness of an evidence-based digital technology (Jaspr Health) designed
for persons with acute suicidal ideation seeking psychiatric crisis ED services when used as part of routine ED-based suicidecare. This study deployed Jaspr Health for real-world use in 2 large health care systems in the United States and aimed to evaluate(1) how and whether Jaspr Health could be safely and effectively used outside the context of a researcher-facilitated clinical trial,and (2) that Jaspr’s use would be associated with improved patient agitation and distress.
Methods: Under the auspices of a nonsignificant risk device study, ED patients with acute suicidal ideation (N=962) from 2
health care systems representing 10 EDs received access to Jaspr Health as part of their routine suicide care. Primary outcome
measures included how many eligible patients were assigned Jaspr Health, which modules were assigned and completed, andfinally, the number of adverse events reported by patients or by medical staff. Secondary outcome measures were patient agitation,
distress, and satisfaction.
Results: The most frequent modules assigned were Comfort and Skills (98% of users; n=942) and lethal means assessment(90% of patient users; n=870). Patient task completion rates for all modules ranged from 51% to 79%. No adverse events werereported, suggesting that digital technologies can be safely used for people seeking ED-based psychiatric services. Statisticallysignificant (P<.001) reductions in agitation and distress were reported after using the app. Average patient satisfaction ratingsby site were 7.81 (SD 2.22) and 7.10 (SD 2.65), with 88.8% (n=325) and 84% (n=90) of patients recommending the app to others.

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e52293 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Dimeff et al

Conclusions: Digital technologies such as Jaspr Health may be safely and effectively integrated into existing workflows to help
deliver evidence-based suicide care in EDs. These findings hold promise for the use of digital technologies in delivering
evidence-based care to other vulnerable populations in complex environments.
(JMIR Form Res 2024;8:e52293) doi: 10.2196/52293
KEYWORDS
suicide; emergency department; ED; digital technology; suicide prevention best practices; individual; particular; suicide prevention;evidence-based intervention; Emergency department; hospital; vulnerable population; Jaspr Health; psychiatric; psychiatrist;
care; safety

Introduction
Globally, suicide remains a significant public health problem
accounting for approximately 700,000 deaths annually [1] andis the fourth leading cause of death for people aged 15-29 years
worldwide [2]. In the United States, suicide is the ninth leading
cause of death for individuals aged 10-64 years, and the second
leading cause of death for individuals aged 10-14 and 25-34
years [3]. In 2021, a total of 47,646 suicides were recorded in
the United States—a 4% increase from the previous year [4].
In addition to the lives lost, far more people in the United States
seriously think about (12.2 million), make a plan for (3.2
million), or attempt (1.2 million) suicide annually [5]. The
impact of suicide and suicidal behaviors are far greater, asfamilies, coworkers, and communities also suffer from the lossof their loved one to suicide or from seeking capable help [6-9].
In the United States, emergency departments (EDs) are the frontline for suicide prevention, providing psychiatric crisis care forthose with acute suicidal ideation [10-12]. Nearly 650,000
people are evaluated for suicide attempts in US EDs annually[13-15]; approximately 1.4 million ED visits annually are forsuicidal ideation or deliberate nonsuicidal self-injury [16-18].
Recognizing the unique challenges in treating suicidality in the
ED, The Joint Commission (TJC) recommends targeting threedomains: (1) address patients’ needs (eg, rapid nonsedating
treatment of agitation, mitigating stressors, initiating active
treatment), (2) support ED staff (eg, in reducing patient’s
distress; assessing patient’s progress over time to determine
readiness for discharge), and (3) improve the environment (eg,avoid psychiatric hospitalization) [19].
Unfortunately, ED-based behavioral health emergencies
continue to rise in number, with 6%-10% of all ED patients
presenting with a psychiatric emergency [20,21]. This rise in
the need for psychiatric care produces an increased burden on
staff [22-26]. Furthermore, behavioral health emergencies
require approximately 42% more time than nonpsychiatric visits
and result in twice the rate of inpatient admission [27], thus,
contributing to the problem of boarding as patients can remainin the ED for up to 4 hours after a decision is made to admit
them [19,22]. The burden of treating patients with acute suicidal
ideation in the ED cannot be overstated [19,23]—longer ED
stays affect patients and providers, and cost the system an
estimated US $2264 per patient [19,25,28]. Such challenges
make it difficult for many EDs to comply with TJC
recommendations [29] and there are few tools to help ensurethat EDs meet these targets, without placing increased demands
on under-resourced settings (eg, additional staffing, training,https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

oversight is often near impossible in more rural EDs). Thus,
novel approaches are needed to address the significant publichealth impact of suicide.
Digital technologies can efficiently and effectively deliver
evidence-based interventions, and are highly scalable, [30-32]
critical in the case of ED-based suicide care. In addition, digitalinterventions also appear to facilitate more complete patient
disclosure [33]. Moreover, digital interventions can deliver
helpful tools to patients, and simultaneously ease the problemof patient boarding by facilitating the collection of necessary
information for discharge (eg, safety plans), while patients are
waiting for providers [34]. Streamlining of data collection anddelivery of helpful tools without the need of additional personnel
may be particularly helpful in systems with limited behavioral
health resources (eg, EDs in more rural areas). What is needed
is a study on the deployment of a digital intervention to support
suicide care in EDs, specifically examining how such atechnology would be used by staff and its impact on patient
safety and stabilization.

Background on the Intervention
To help address the needs of providers and improve the qualityof care for individuals in acute suicide crisis, we developed atablet-based crisis app (Jaspr Health; “Jaspr”) for people
experiencing suicidality and seeking suicide crisis care in EDs.
Jaspr was developed in consultation with suicide experts, witha goal of translating standards of clinical care for suicidecommon across intervention models (eg, risk assessment, safety
planning, lethal means assessment, and management) [35]. Jaspr
helps medical care teams deliver these expert-recommended
evidence-based suicide care [36-39]. Specifically, Jaspr digitizes
a comprehensive suicide risk assessment, assessment of lethalmeans, and building of a crisis stability plan for review by thepatient’s ED provider, all of which becomes part of the patient’s
health record via integration with electronic health record
software (eg, Epic). Furthermore, it strongly aligns with all 3
domains recommended by TJC: patients can view videos ofpeople with lived experience sharing their wisdom and teaching
behavioral skills intended to reduce patient agitation anddistress; Jaspr-at-Home, a companion app, bundles Jaspr forhome use during the high-risk postdischarge period. In addition,
it bundles these evidence-based suicide care practices in 1
system for ease of delivery. Results from a small pilot
randomized controlled trial (RCT) comparing Jaspr (n=14) tocare-as-usual (CAU; n=17) in ED adults with acute suicidal
ideation, demonstrated its superiority to CAU. Jaspr participants
were significantly more likely to receive evidence-based
procedures in the ED, reported significant decreases in agitation
JMIR Form Res 2024 | vol. 8 | e52293 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
and distress over time, and reported significant increases in theircapacity to cope with suicidal distress. Jaspr participants also
reported higher overall satisfaction with their ED experience(approached significance: P=.06; large effect: Cohen d=.8),with 100% of users recommending its use [37]. (The ED-based
study was discontinued due to the COVID-19 pandemic, whichbegan impacting EDs 6 weeks after the RCT’s start). While
RCT findings were encouraging, large-scale data without aresearch team driving participation was necessary to understandhow Jaspr would be implemented in real-world settings.

This Study
This study deployed Jaspr for real-world use in 2 large healthcare systems in the United States, and aimed to evaluate (1)how and whether Jaspr could be safely and effectively usedoutside of the context of a researcher-facilitated clinical trial,
and assuming that with delivery of evidence-based care comes
improved patient stabilization and satisfaction (2) that Jaspr’s
use would be associated with improved patient outcomes,
consistent with the pilot RCT [37]. To evaluate Aim 1,
exploratory data were collected on how medical staff deployed
Jaspr, including (1a) frequency of module assignment and (1b)completion rates for individual Jaspr modules, as well as (1c)presence of adverse events (AEs) related to study participation,
including use of Jaspr. For Aim 2, hypotheses were consistent
with the prior RCT, specifically, (2a) that Jaspr use would
produce significant reductions in self-reported agitation anddistress and (2b) high care satisfaction ratings.

Methods
Setting
We implemented Jaspr at 10 sites serving patients with acutesuicidal ideation throughout 2 geographically diverse healthcare systems, Allina Health (“Allina”) and Providence. Allina
includes 13 hospitals located throughout Minnesota, with its
largest campuses located near the high-density cities of
Minneapolis and St. Paul. As part of the initial Allina
implementation, sites were selected by each system’s behavioral
health leadership. Criteria for ED selection included: highvolume of behavioral health emergencies in the ED, high needfor behavioral health resources, strong ED leadership with priorsuccess launching other innovative programs, and a request to
participate. With 51 hospitals in its system, Providence serves
patients across 7 states, including Alaska, California, Montana,
New Mexico, Oregon, Texas, and Washington. Implementation
at Providence was based on testing Jaspr in a wide variety ofEDs, that varied by setting (eg, rural vs urban), workflow (eg,with mental health clinicians or rapid response teams vs delivery
by nursing or other staff), and resources. Data from thesesettings were collected between January 17, 2022, and December
21, 2022.
Each system assembled an implementation team to work
collaboratively on their respective rollouts. Each team met
weekly with Jaspr’s implementation team for several months
in preparation for its use with patients. Topics addressed
included: selection of EDs, sequencing of ED rollout, analyzing
existing workflows for optimal insertion of Jaspr, selection ofsite leaders, eligibility criteria for Jaspr use, information
https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

Dimeff et al
technology’s review of Jaspr’s HIPAA (Health Insurance
Portability and Accountability Act) compliance standards andother security measures, and review of institutional review board(IRB) approval and nonsignificant risk device study status. In
total, 6 EDs from Allina (Mercy Hospital, Coon Rapids,
Minnesota [MN]; Unity Hospital, Fridley, MN; Abbott
Northwestern Hospital, Minneapolis, MN; United Hospital, St.
Paul, MN; St. Francis Regional Medical Center, Shakopee, MN;
Cambridge Medical Center; Cambridge, MN) and 4 from
Providence (St. Mary’s Hospital, Walla Walla, WA; St. Patrick’s
Hospital, Missoula, MT; Swedish Edmonds, Edmonds, WA;
Portland Medical Center, Portland, OR) implemented Jaspr in
their EDs.

Ethical Considerations
All procedures were approved by Association for the
Accreditation of Human Research Protection Programs,
Inc–accredited Sterling IRB and each system’s IRB (Sterling
IRB #9187, WCG IRB #1841871). External monitoring wasprovided by an independent Data Safety Monitoring Board
comprising suicide experts. An IRB-approved consent document
was displayed at the start of participation in Jaspr; participants
could either affirm consent by pressing “I understand and agree
to take part in this research” or could simply not press forward
and return the tablet to staff. Use of Jaspr was not incentivized
and there were no penalties or other changes to care for patients
not participating. Study data were collected solely within the
Jaspr environment, which was accessed via dedicated tablet andstored within the patient’s medical record. Deidentified data
were stored in a HIPAA-compliant cloud-based server for
analyses.

Participants
In total, 706 Allina patients and 256 Providence patients(N=962) were assigned activities in Jaspr. Eligible participants
were English-speaking individuals seeking psychiatric crisis
services for suicidality in the ED, as well as individuals admitted
to a psychiatric inpatient unit for suicidality (Allina). Ageeligibility differed between the 2 systems: Allina participants
were 18 years or older; Providence participants were 13 years
or older with parent or legal guardian permission collected whererequired by state law for those under 18 years of age. Individuals
who were acutely psychotic, severely agitated, or significantly
impaired by alcohol or drugs were excluded from participating
because they would be unable to provide informed consent,
meaningfully engage with Jaspr, and because of safety concerns
involving access to a tablet that could be weaponized.

Procedures
Eligibility was determined by nurses or behavioral healthspecialists. Once deemed eligible, a medical provider introduced
the patient to Jaspr and invited them to use the app. For thoseinterested, the provider then activated the app and assigned
specific Jaspr modules based on the patient’s needs. When first
accessing Jaspr, patients were oriented to the purpose of the app
and their rights: Jaspr use was voluntary and was intended tosupplement their care, not replace treatment provided by their
medical care team. Consenting patients then engaged in using
their assigned Jaspr activities. Patients used the app for as long
JMIR Form Res 2024 | vol. 8 | e52293 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
or little as they wished, while waiting to receive direct care fromtheir care teams. Medical care teams were expected to review
the patient’s answers to the comprehensive suicide assessment,
lethal means assessment, and crisis stability plan as part of theirown clinical assessment; they could edit the responses basedon discussion with their patient. Patients wishing to access theapp’s content at home could register for Jaspr-at-Home forcontinued use post discharge at no cost. Care teams were
instructed to help patients with the Jaspr-at-Home app
installation process when possible, and when not (eg, units that
did not allow cell phones) provide instructions for setup in thepatient’s discharge paperwork.

Measures
Primary Outcomes (Aim 1): Assignment, Completion,and Safety
As our primary aim was to evaluate how and whether Jaspr was
deployed in real-world settings, outcome measures included
how many eligible patients were assigned Jaspr, which modules
were assigned and completed, and finally, number of AEs
reported within the app by patients or by medical staff. To
ensure the ease of reporting AEs, patients could directly reportan AE within the app itself or alert their medical team, who
could also record AEs from within their own Jaspr portal.
Patients and providers could also directly contact Jaspr
researchers via phone or email. AEs could be reported at any
point during app usage or post discharge through Jaspr-at-Home.
The researchers were electronically alerted immediately upon
report of an AE.

Secondary Outcomes (Aim 2): Patient Care andSatisfaction
Patients completed brief self-report measures within Jaspr toassess agitation, distress, and care satisfaction. In general, thecurrent study replicated measures used in the RCT [37] where
measures were selected for their brevity and simplicity for
individuals with acute suicidal ideation. Two items from the
Safety and Imminent Distress Questionnaire, a 4-item face-valid
self-report survey based on Boudreaux’s Keeping Myself Safe
Subject Usability Survey [40] were used to assess agitation anddistress. Participants rated their feelings in the present using a
10-point scale. The following items were included: intensity ofemotional distress (1=no distress; 10=highest distress ever felt)
and the extent to which they felt calm or agitated (1=very calm;
10=very frustrated or agitated). One item from the 7-item
Emergency Room Patient Satisfaction Survey, developed inconsultation with the patient experience division of a large
health care organization, was selected to assess overallsatisfaction with their ED experience (1=worst experience;
5=best experience). One item involved providing rating of Jaspr(1=poor; 10=excellent). Finally, patients were asked if theywould recommend Jaspr to others in their situation (yes/no).

Intervention
Jaspr is intended to help ED health care professionals deliver
evidence-based suicide care for individuals with acute suicidal
ideation [37]. Patients are welcomed into the app via a video
by suicide experts and an individual with lived experience.
Patients are walked through several evidence-based practices
https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

Dimeff et al
in a conversational tone, including a risk assessment flow, safety
and stability plan creation, lethal means assessment, and distress
tolerance skills practices (these elements are based on the
Collaborative Assessment and Management of Suicidality(CAMS) Suicide Status Interview (SSI, Sections A and B)[41,42], CAMS crisis stability plan and lethal means assessment(SSI, Section C), and behavioral skills training for managing
emotional distress [43,44] respectively). In addition, there iscontent related to suicide prevention psychoeducation and an
extensive library of videos by people with lived experiencesharing their own recovery stories and generating hope.
Psychoeducation videos span a range of topics including what
to expect in the ED, when is inpatient hospitalization indicated,and common reasons why people feel suicidal. “Shared Stories”
by people with lived experience include coping with shame,
strategies for staying well, and messages of hope (“My Wishfor You”). For a more thorough description of Jaspr, see Dimeff
et al [37] and supplementary file.
Jaspr tools were bundled for provider ease in assigning them topatients based on need. Bundles included: Comprehensive
Suicide Assessment, Safety Planning (crisis stabilization andlethal means assessment), Comfort and Skills (behavioral skills
training), and Shared Stories. The election of bundles to assign
was at the discretion of the care team and varied based on thepatient's needs, the care the patient received before Jaspr, andthe estimated time before the discharge or transfer to an inpatient
floor. If patients were not assigned a bundle, they could notaccess it. However, all bundles included lethal means assessment(paired with the first bundle assigned). Additionally, if a user
was assigned the Safety Planning bundle, they could accessvideo content from Comfort and Skills. Due to TJC’s
requirement that all patients with suicidal ideation receive thecrisis stability plan before discharge, system leaders encouraged
the Safety Planning bundle at a minimum.

Statistical Analyses
Analyses focused on assignment and completion rates, pre- topost-intervention change, and satisfaction. All analyses wereconducted separately for Allina and Providence patients to
identify patterns across systems, and out of recognition thateach system has its own workflows, policies, and procedures.
Finally, paired sample t tests were conducted in SPSS (version
27; IBM Corp) [45] to compare pre- and posttest scores foragitation and distress. There was no a priori sample size
requirement for analysis; the sample was based only on the total
number of participants who provided data during the datacollection window.

Results
Aim 1a: Assignment
In total, 706 Allina patients and 256 Providence patients(N=962) were assigned at least one Jaspr bundle. Across both
systems, Comfort and Skills was by far the most assigned
feature, with 98% of all users being assigned its content (n=942).
Jaspr’s lethal means assessment was the second most commonly
assigned feature, assigned to 90% of patient users (n=870).
Overall, 86% were assigned to use Jaspr to build a crisis stability
plan (n=830). The Comprehensive Suicide Assessment was the
JMIR Form Res 2024 | vol. 8 | e52293 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Dimeff et al

least assigned bundle, assigned to fewer than half of Jaspr users(n=470).

Aim 1b: Completion
Assigned bundles were completed at moderately high rates.
Across both health care systems, the Comprehensive Suicide
Assessment had the highest rate of completion (74%; n=348),

followed by lethal means assessment (68%; n=591), and thenbuilding a crisis stability plan (59%; n=489). Since there are
numerous hours of Comfort and Skills available within Jaspr,
there is no a priori determinant of completion. In total, 42.8%
of Allina patients (n=302) and 29% of Providence patients(n=75) set up Jaspr-at-Home (Table 1).

Table 1. Assignment and completion rates of Jaspr’s modules per hospital system.
Allina (n=706)aProvidence (n=256)

Assigned

Completed

Completed (%)

Assigned

Completed

Completed (%)

CSAa

327

259

79

143

89

62

CSPb

625

384

61

205

105

51

Lethal Means

643

451

70

227

140

62

C&Sc

689

—d

—

253

—

—

CSA: Comprehensive Suicide Assessment.

b

CSP: Crisis Safety Plan.

c

C&S: Comfort and Skills.

d

Not applicable.

Aim 2a: Treatment OutcomesAim 1c: AEs
Of the 962 ED patient users of Jaspr with acute suicidal ideation
to date, there have been no serious AEs reported. Moreover,
only one report of a potential AE was made, which after review
by the Data Safety Monitoring Board was classified as asoftware “bug” requiring fixing, not an AE. Specifically, aprovider noted that a patient was able to exit the Jaspr app andaccess the internet. No harm to the patient was reported and the
bug was quickly addressed.

Statistically significant decreases in agitation and distress wereobserved across patients in both health care systems after using
Jaspr (Table 2). Pre-post within-condition effect sizes at Allina
were medium for agitation (d=0.46) and distress (d=0.56). At
Providence, effect sizes were also medium for agitation (d=0.51)
and distress (d=0.51).

Table 2. Paired sample t tests and means for pre- and postvisit distress and agitation levels among participants by hospital system.
N

Pre, mean (SD)Post, mean (SD)t test (df)Cohen dSig. (2-tailed)

Allina366

5.46 (2.60)

4.35 (2.63)

10.69 (365)

0.56

<.001

Providence

112

5.58 (2.45)

4.59 (2.53)

5.42 (111)

0.51

<.001

Allina366

4.63 (2.75)

3.65 (2.63)

8.75 (365)

0.46

<.001

Providence

109

5.09 (2.84)

4.14 (2.52)

5.32 (108)

0.51

<.001

Distress

Agitation

Aim 2b: Satisfaction
High levels of satisfaction were reported by patients at both
health care systems. Specifically, of those who provided
satisfaction ratings, 88.8% of Allina patients (n=325) and 84%
of Providence patients (n=90) recommended Jaspr using a binary
true or false response. The average Jaspr satisfaction rating on
the 10-point satisfaction survey was 7.81 (SD 2.22) for Allina
patients and 7.10 (SD 2.65) for Providence patients. The average
ED rating, scored on a 5-point scale, was 3.96 (SD 1.08) for
Allina and 3.79 (SD 1.13) for Providence.

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

Discussion
Relevance and Findings
EDs remain the frontline in suicide prevention. For this reason,consensus-derived standards define ED care for persons withacute suicidal ideation seeking psychiatric crisis services, someof which are now regulatory requirements. Yet, few EDs areadequately equipped with the resources to deliver theseimportant life-saving interventions. Jaspr was designed to help
ED health care providers in delivering evidence-based suicidecare at the point of need. Findings from a previous pilot RCT
supported Jaspr’s feasibility and efficacy: in comparison to
CAU,
Jaspr
significantly
increased
deliveryof
JMIR Form Res 2024 | vol. 8 | e52293 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
expert-recommended suicide care for patients with acute suicidal
ideation in the ED, reduced agitation and distress, and increased
capacity to cope with a suicide crisis. This study sought toextend these findings and determine its safety when used inreal-world conditions, where the app’s use was integrated into
EDs’ natural workflows and without the tightly controlled
research procedures. Results suggest that Jaspr was regularly
assigned to patients, who completed the various modules atmoderately high rates. It was safe to use, associated withdecreases in agitation and distress, and reported high satisfaction
rates.
Across 10 EDs and 1 inpatient unit within 2 large health caresystems, Jaspr was used by ED health care teams to aid theirsuicide care to 962 individuals with acute suicidal ideation
seeking psychiatric crisis services. No AEs occurred, supporting
the safety of the Jaspr crisis app in particular, and digital aids
in general, for individuals with acute suicidal ideation, even ina highly complex, fast-paced ED. Consistent with our previous
research, patients also experienced significant reductions inagitation and distress over the course of their use, a behavioral
target encouraged by TJC. While the clinical significance of
these specific changes is not clear, decreases in agitation anddistress may improve ED outcomes for patients with suicidal
ideation by decreasing the need for pharmacologicalinterventions or restraint, and help facilitate other ED
interventions (eg, staff assessment, treatment, and dischargeplanning) [11,46,47].
Of note is also how Jaspr was used by medical providers andpatients. In both health care systems, providers were most likely
to assign content that helped patients distract from their suicide
crisis and learn new behavioral skills to better cope withnegative emotions. This may account for the significant
reductions in agitation and distress. The comprehensive suicide
assessment was the least assigned, though it had the highest
completion rate by patients when assigned. The low assignment
may be because both health care systems routinely use a
different instrument to assess suicidality (the Columbia Suicide
Severity Rating Scale [48]) from the assessment instrument
embedded in Jaspr (CAMS SSI). While overall usage wascomparatively low, 348 patients—over a third of the total
sample—fully completed an evidence-based suicide risk
assessment. Other life-saving components were assigned at ahigh rate: lethal means assessment was assigned in 91% of allcases and safety planning in 86% of all cases. Even though only
68% fully completed the lethal means assessment and 59% built
a crisis stability plan using Jaspr, a sizeable number of patients
received and completed these important interventions to fidelity(n=591 and n=489, respectively). These results are particularly
promising as crisis stability planning and lethal means
assessment are practices recommended by TJC [49]. Future
research to clarify under what conditions these modules werediscontinued would be helpful in improving the intervention
and ensuring delivery of recommended care. In addition, nearly
all patients who provided ratings from both systems positively
endorsed Jaspr (89% at Allina; 84% at Providence) andrecommended it for others in the midst of a suicide crisis in theED.

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

Dimeff et al
These data suggest that when used in conjunction with
provider-led care, tools like Jaspr allow for more standardizeddelivery of evidence-based suicide care, particularly deliveryof safety planning and lethal means assessment. Suchstandardization is beneficial for the patient, who is stabilized
and can return home more readily. However, the system andpayers also benefit, as it reduces costs, reduces the problems
associated with boarding in the ED [19,22,25], saves providers
time by gathering and documenting necessary information(standardized documentation also reduces the potential for
lawsuits) all without requiring allocation of additional personnel.
Future research would benefit from a more granular examinationof the benefits of tools like Jaspr, including the aspects of theintervention that are most helpful, the relation to patient
outcomes, in both the short (eg, agitation, distress) and longterms (eg, lost productivity, subsequent hospitalization), as well
as system costs, and discharge timeline. However, barriers to
implementation may need to be, at least partially, addressed forsuch research to occur.
Unfortunately, while these results indicate interventions like
Jaspr are promising, there are barriers to more widespread
implementation of these types of tools, particularly with regard
to payment, as well as regulation and its associated costs. Thereis not currently a Current Procedural Terminology (CPT) code,
which allows for adequate standardizing costs and billing for
the use of such tools to payers. A CPT code is needed for more
complex delivery platforms to help underwrite the costs
associated with a software platform that is HIPAA compliant
and integrates into an electronic health record. The lack of CPT
code is a significant barrier to widespread adoption, as it is not
feasible for each health care system to negotiate its own rate
for each individual technology. Moreover, while theseinterventions are potentially powerful, for problems like suicide,
they will likely require at a minimum, an application to the Foodand Drug Administration to determine whether oversight isrequired. If it is, this is an extremely time-consuming, expensive
process, the cost of which might need to be incorporated into
the cost associated with a CPT code and would ensure that thesystem is compensated in a way where they can afford the actual
cost of a tool like Jaspr. Given that many tools like Jaspr aredeveloped by start-up companies or academic laboratories on
shoe-string budgets, models for an effective pathway toward
addressing these barriers is needed. Preliminary, real-world datawith large samples on how these tools are used by providers
and their associated patient outcomes, are an important step in
helping provide a rationale for solving these implementation
barriers.

Limitations
Despite the strengths of this study, several limitations arenotable. First, the data were gathered in a real-world context
with no experimental controls. Without a control condition,
blinded assessment, and other experimental controls, it is
impossible to know whether decreases in agitation and distresswere due to the delivery of evidence-based care via the app
itself, the passing of time, or other interventions received in the
ED. This finding is, however, consistent with outcomes fromour previously published RCT [37]. In addition, a follow-up
assessment or information about subsequent hospitalizations
JMIR Form Res 2024 | vol. 8 | e52293 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
would provide additional data on benefits to patients due toJaspr use. Second, it is impossible to know whether the AE data
accurately reflect actual AEs that may have occurred throughuse of Jaspr or Jaspr-at-Home. While medical personnel andpatients could easily report an AE via the app or to staff, it ispossible that providers were unaware that an AE occurred (eg,they were not with the patient when it happened; too busy
responding to medical emergencies to record an AE) or patients
may have failed to report an AE (eg, could not easily
discriminate further distress caused by app usage; lacked
motivation to record an AE). Third, in addition, there was noassessment of whether app use reduced personal contact timewith ED staff (ie, substitution effects) or increased wait times
for patients using Jaspr (eg, triaging care to those who had
declined). Fourth, since Jaspr records a “completed” event in abinary fashion, it is not possible to identify the number of casesthat were near completion. It is possible, for example, that usage
and engagement are far greater than captured in the conservative“completion” data. Fifth, there may be specific barriers toengagement and ongoing use that were not assessed in thecontext of this study. For instance, age could be a factor; it ispossible that older adults and others less conversant with digitaltechnologies may struggle to use Jaspr during a period ofheightened distress (ie, in the ED), and be less likely to engage
in ongoing use post discharge. Sixth, no data on functionalchanges in patient behavior were captured in this data set. Futurestudies should seek to capture this data. Seventh, there was noway to reliably determine why patients discontinued Jaspr use(eg, transfer/discharge vs dislike of the app). Eighth, it is also
possible that there were ED conditions that impacted
implementation that were not captured, but relevant for morewidespread adoption (eg, increased ED chaos leading todecreased Jaspr assignment). Ninth, data on the number of users
who were eligible but not offered Jaspr, as well as users who

Dimeff et al
were offered but declined to participate was not collected, would
provide helpful context to understanding the implementationof Jaspr. Tenth, there is an assumption that stabilization (ie,reductions in distress and agitation) follows delivery ofevidence-based practices via the Jaspr app; however, further
investigation is needed to determine the mechanisms by whichthese changes occur. Eleventh, finally the specific data points
are limited to a modest number of variables because ofprioritizing reduced assessment burden in the context ofreal-world use, and because of the method of data collection(ie, internal Jaspr data analytics) we do not have data on sample
that would provide important information on generalizability
and potential sources of bias (eg, patient demographics for
subgroup analyses).

Conclusions
This study of an evidence-based app for individuals with acutesuicidal ideation seeking suicide care in the ED supports its
safety when integrated into natural workflows, without clinical
trial procedural controls. Comprising 10 EDs and 1 inpatientsetting across 2 large US-based health care systems, 962 patientswith acute suicidal ideation were assigned the Jaspr crisis app
as part of their suicide care. At least 942 patients used Jaspr;
489 patients used Jaspr to thoroughly and safely complete anevidence-based crisis stability plan, and another 591 thoroughly
and safely completed an evidence-based lethal means
assessment. Given the significant shortage of ED medical
providers available to deliver robust suicide care, digitaltechnologies like Jaspr may facilitate provision of thorough andlife-saving evidence-based suicide care at the point of need. To
the extent that Jaspr is designed for, tested with, and safe foruse with a highly vulnerable population, this research may also
pave the way for similar behavioral health digital interventions
for others in need of evidence-based care.

Acknowledgments
This work was supported by the National Institute of Mental Health (R44MH108222). The sponsor had no role in the review orapproval of this manuscript. The authors wish to express their gratitude to Adam Haim, PhD, a National Institute of Mental Healthprogram officer, for his continued guidance of this work. This work would also not have been possible but for the work andguidance of our Allina Health and Providence colleagues who served as tireless leadership champions and on-the-ground supporters.
Thanks to our Allina colleagues for leading the way and making this implementation effort possible at scale: Kathleen M Tuenge,
Shannon T Woulfe, PhD, Jennifer A Abraham, Victoria B Reimers, Coralyn R Malles, and Joseph R Clubb, MSW LICSW. We
are also very grateful to our Providence implementation team who made this work possible in their settings: Huynh Chhor,
LICSW; Jennifer Mckay, MSW, LICSW; Jeremy Hamill, MD; Jeremy Williams, RN; Lance Somerfeld, MD; Emily Jacobson,
BSN, RN; Kristin Powers, MSW; Elizabeth O’Rourke, LPC; Jessica Monego, RN; Mike Ruiz; Vince Torres, MD; Alysa Kyle,
RN; Kim Beierschmitt, RN; Lee Turner; Maggie Weller, MS; Monique Gill, PhD, MPH; Megan Holtorf, MPH, CCRP; Kathleen
Yancy, RN, BS; and Melissa Bowe, MSN, RN, CEN. Special thanks to Brian Palmer, MD, for his contribution to this project.
No generative AI was used in the writing of this manuscript.

Data Availability
The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.

Conflicts of Interest
LAD, PhD, and KK, PhD, are Founders of the Evidence-Based Practice Institute, Inc, a for-profit company that owns Jaspr Health,and are its principal shareholders. To manage investigators’ potential conflicts of interest, statistical consultant BB, PhD,
independently oversaw all data management, data analyses, and written interpretation of all results and verified the accuracy andvalidity of all study data before any data have been publicly presented.
https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e52293 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Dimeff et alMultimedia Appendix 1
Jaspr Workflow and Screenshots.
[PDF File (Adobe PDF File), 4884 KB-Multimedia Appendix 1]

References
1.

2.
3.
4.
5.

6.
7.

8.
9.
10.

11.
12.

13.

14.
15.
16.

17.

18.

19.
20.
21.

Martinez-Ales G, Hernandez-Calle D, Khauli N, Keyes K. Why are suicide rates increasing in the United States? Towards
a multilevel reimagination of suicide prevention. Curr Top Behav Neurosci. 2020;46:1-23. [FREE Full text] [doi:10.1007/7854_2020_158] [Medline: 32860592]
Suicide. World Health Organization. 2023. URL: https://www.who.int/news-room/fact-sheets/detail/suicide [accessed2024-08-09]
CDC. CDC WONDER: Underlying cause of death, 1999-2019. US Department of Health Human Services. 2020. URL:
https://wonder.cdc.gov/Deaths-by-Underlying-Cause.html [accessed 2024-08-09]
Curtin SC, Garnett MF, Ahmad FB. Provisional Numbers and Rates of Suicide by Month and Demographic Characteristics.
US. National Center for Health Statistics, CDC; 2021.
Key substance use and mental health indicators in the United States: results from the 2020 national survey on drug use andhealth. In: Center for Behavioral Health Statistics and Quality. Rockville, MD. Substance Abuse and Mental Health Services
Administration; 2021.
Cerel J, Jordan JR, Duberstein PR. The impact of suicide on the family. Crisis. 2008;29(1):38-44. [doi:10.1027/0227-5910.29.1.38] [Medline: 18389644]
Causer H, Spiers J, Efstathiou N, Aston S, Chew-Graham CA, Gopfert A, et al. The impact of colleague suicide and thecurrent state of postvention guidance for affected co-workers: a critical integrative review. Int J Environ Res Public Health.
2022;19(18):11565. [FREE Full text] [doi: 10.3390/ijerph191811565] [Medline: 36141837]
Jordan JR. Is suicide bereavement different? A reassessment of the literature. Suicide Life Threat Behav. 2001;31(1):91-102.
[doi: 10.1521/suli.31.1.91.21310] [Medline: 11326773]
Chapman AL, Dixon-Gordon KL. Emotional antecedents and consequences of deliberate self-harm and suicide attempts.
Suicide Life Threat Behav. 2007;37(5):543-552. [doi: 10.1521/suli.2007.37.5.543] [Medline: 17967121]
Wasserman D, Iosue M, Wuestefeld A, Carli V. Adaptation of evidence-based suicide prevention strategies during andafter the COVID-19 pandemic. World Psychiatry. 2020;19(3):294-306. [FREE Full text] [doi: 10.1002/wps.20801] [Medline:
32931107]
Guzmán EM, Tezanos KM, Chang BP, Cha CB. Examining the impact of emergency care settings on suicidal patients: acall to action. Gen Hosp Psychiatry. 2020;63:9-13. [doi: 10.1016/j.genhosppsych.2018.07.004] [Medline: 30077397]
True G, Pollock M, Bowden CF, Cullen SW, Ross AM, Doupnik SK, et al. Strategies to care for patients being treated in
the emergency department after self-harm: perspectives of frontline staff. J Emerg Nurs. 2021;47(3):426-436.e5. [FREEFull text] [doi: 10.1016/j.jen.2020.12.016] [Medline: 33610311]
Canner JK, Giuliano K, Selvarajah S, Hammond ER, Schneider EB. Emergency department visits for attempted suicideand self harm in the USA: 2006-2013. Epidemiol Psychiatr Sci. 2018;27(1):94-102. [FREE Full text] [doi:10.1017/S2045796016000871] [Medline: 27852333]
Owens PL, Mutter R, Stocks C. Mental health and substance abuse-related emergency department visits among adults,
2007. Agency for Healthcare Research and Quality (US). 2010. [Medline: 21413214]
Spicer RS, Miller TR. Suicide acts in 8 states: incidence and case fatality rates by demographics and method. Am J Public
Health. 2000;90(12):1885-1891. [doi: 10.2105/ajph.90.12.1885] [Medline: 11111261]
Owens P, Fingar K, Heslin K, Mutter R, Booth C. Emergency department visits related to suicidal ideation, 2006-2013:
statistical brief #220. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. US. Agency for Healthcare
Research and Quality; 2017.
Rui P, Kang K. National hospital ambulatory medical care survey emergency department summary tables, table 15. Centers
for Disease Control and Prevention. 2015. URL: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2015_ed_web_tables.
pdf [accessed 2024-08-09]
Schmutte T, Olfson M, Xie M, Marcus SC. National study of emergency department disposition for high suicide risk
geriatric patients. Gen Hosp Psychiatry. 2019;58:67-70. [FREE Full text] [doi: 10.1016/j.genhosppsych.2019.03.005][Medline: 30933688]
The Joint Commission. ED boarding of psychiatric patients - a continuing problem. Quick Saf. 2015. URL: https://www.
jointcommission.org/-/media/tjc/newsletters/quick-safety-19-ed-boarding-update-7-27-21.pdf [accessed 2024-08-09]
McCaig LF, Burt CW. National hospital medical care survey: 2000 emergency department summary. Adv Data.
2002;326(326):1-33. [Medline: 12666256]
Pitts SR, Niska RW, Xu J, Burt CW. National hospital ambulatory medical care survey: 2006 emergency department
summary. Natl Health Stat Report. 2008;(7):1-38. [FREE Full text] [Medline: 18958996]

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e52293 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
22.

23.

24.
25.
26.
27.
28.

29.
30.
31.

32.

33.
34.

35.

36.

37.

38.
39.
40.

41.
42.

43.
44.
45.
46.

Nordstrom K, Berlin JS, Nash SS, Shah SB, Schmelzer NA, Worley LLM. Boarding of mentally ill patients in emergency
departments: American psychiatric association resource document. WestJEM. 2019;20(5):690-695. [doi:10.5811/westjem.2019.6.42422] [Medline: 31539324]
Abid Z, Meltzer AC, Lazar D, Pines JM. Psychiatric boarding in U.S. EDs: a multifactorial problem that requires
multidisicplinary solutions. Center for Health Care Quality. 2014. URL: https://core.ac.uk/download/pdf/230843231.pdf[accessed 2024-08-09]
Chang G, Weiss A, Kosowsky JM, Orav EJ, Smallwood JA, Rauch SL. Characteristics of adult psychiatric patients withstays of 24 hours or more in the emergency department. PS. 2012;63(3):283-286. [doi: 10.1176/appi.ps.201000563]
Nicks BA, Manthey DM. The impact of psychiatric patient boarding in emergency departments. Emerg Med Int.
2012;2012:360308. [FREE Full text] [doi: 10.1155/2012/360308] [Medline: 22888437]
Simpson SA, Joesch JM, West II, Pasic J. Who's boarding in the psychiatric emergency service? West J Emerg Med.
2014;15(6):669-674. [FREE Full text] [doi: 10.5811/westjem.2014.5.20894] [Medline: 25247041]
ACEP psychiatric and substance abuse survey. American College of Emergency Physicians. 2008. URL: https://www.
emergencyphysicians.org/newsroom?o=25 [accessed 2019-07-18]
Stamy C, Shane DM, Kannedy L, Van Heukelom P, Mohr NM, Tate J, et al. Economic evaluation of the emergency
department after implementation of an emergency psychiatric assessment, treatment, and healing unit. Acad Emerg Med.
2021;28(1):82-91. [FREE Full text] [doi: 10.1111/acem.14118] [Medline: 32869891]
Hogan MF, Grumet JG. Suicide prevention: an emerging priority for health care. Health Aff (Millwood).
2016;35(6):1084-1090. [doi: 10.1377/hlthaff.2015.1672] [Medline: 27269026]
Car LT, Dhinagaran DA, Kyaw BM, Kowatsch T, Joty S, Theng Y, et al. Conversational agents in health care: scoping
review and conceptual analysis. J Med Internet Res. 2020;22(8):e17158. [doi: 10.2196/17158] [Medline: 32763886]
Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is
effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10):e13196. [FREE Full text] [doi:10.1371/journal.pone.0013196] [Medline: 20967242]
Kiluk BD, Ray LA, Walthers J, Bernstein M, Tonigan JS, Magill M. Technology-delivered cognitive-behavioral interventions
for alcohol use: a meta-analysis. Alcohol Clin Exp Res. 2019;43(11):2285-2295. [FREE Full text] [doi: 10.1111/acer.14189][Medline: 31566787]
Lucas GM, Gratch J, King A, Morency LP. It’s only a computer: virtual humans increase willingness to disclose. Computersin Human Behavior. 2014;37:94-100. [doi: 10.1016/j.chb.2014.04.043]
Larkin C, Djamasbi S, Boudreaux ED, Varzgani F, Garner R, Siddique M, et al. ReachCare mobile apps for patients
experiencing suicidality in the emergency department: development and usability testing using mixed methods. JMIR Form
Res. 2023;7(1):e41422. [FREE Full text] [doi: 10.2196/41422] [Medline: 36705961]
Dimeff LA, Jobes DA, Tyndal T, Zhang I, Stefan S, Kako N, et al. Using the delphi method for determining key performanceelements for delivery of optimal suicide-specific interventions in emergency departments. Arch Suicide Res.
2023;27(2):246-260. [doi: 10.1080/13811118.2021.1984347] [Medline: 34632952]
Dimeff LA, Jobes DA, Chalker SA, Piehl BM, Duvivier LL, Lok BC, et al. A novel engagement of suicidality in theemergency department: virtual collaborative assessment and management of suicidality. Gen Hosp Psychiatry.
2020;63:119-126. [doi: 10.1016/j.genhosppsych.2018.05.005] [Medline: 29934033]
Dimeff LA, Jobes DA, Koerner K, Kako N, Jerome T, Kelley-Brimer A, et al. Using a tablet-based app to deliver
evidence-based practices for suicidal patients in the emergency department: pilot randomized controlled trial. JMIR Ment
Health. 2021;8(3):e23022. [FREE Full text] [doi: 10.2196/23022] [Medline: 33646129]
Dimeff LA, Koerner K. Fulfilling the promise of behavioral health technologies to improve public health impact and reduce
public health disparities: a commentary. Clin Psychol Sci Pract. 2019;26(1):e12276. [doi: 10.1111/cpsp.12276]
Suicide Prevention Resource Center. Caring for adult patients with suicide risk: a consensus guide for emergency departments.
Educ Dev Cent Inc. 2015. URL: https://sprc.org/sites/default/files/EDGuide_full.pdf [accessed 2024-08-09]
Boudreaux ED, Brown GK, Stanley B, Sadasivam RS, Camargo CA, Miller IW. Computer administered safety planningfor individuals at risk for suicide: development and usability testing. J Med Internet Res. 2017;19(5):e149. [FREE Fulltext] [doi: 10.2196/jmir.6816] [Medline: 28506957]
Jobes DA. Managing Suicidal Risk: A Collaborative Approach. New York, NY. Guilford Press; 2016.
Swift JK, Trusty WT, Penix EA. The effectiveness of the collaborative assessment and management of suicidality (CAMS)
compared to alternative treatment conditions: a meta-analysis. Suicide Life Threat Behav. 2021;51(5):882-896. [doi:10.1111/sltb.12765] [Medline: 33998028]
Linehan MM. DBT Skills Training Handouts and Worksheets, Second Edition. New York, NY. Guildford Press; 2014.
Linehan MM. Skills Training Manual for Treating Borderline Personality Disorder. New York, NY. Guildford Press; 1993.
IBM SPSS Statistics for Windows, Version 27.0. Armonk, NY. IBM Corp; 2020.
Kraft CM, Morea P, Teresi B, Platts-Mills TF, Blazer NL, Brice JH, et al. Characteristics, clinical care, and disposition
barriers for mental health patients boarding in the emergency department. Am J Emerg Med. 2021;46:550-555. [doi:10.1016/j.ajem.2020.11.021] [Medline: 33279330]

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

Dimeff et al

JMIR Form Res 2024 | vol. 8 | e52293 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
47.
48.

49.

Dimeff et al

Holloman G, Zeller S. Overview of project BETA: best practices in evaluation and treatment of agitation. West J Emerg
Med. 2012;13(1):1-2. [FREE Full text] [doi: 10.5811/westjem.2011.9.6865] [Medline: 22461914]
Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-suicide severity rating scale:initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry.
2011;168(12):1266-1277. [FREE Full text] [doi: 10.1176/appi.ajp.2011.10111704] [Medline: 22193671]
National patient safety goals. The Joint Commission. URL: https://www.jointcommission.org/standards/
national-patient-safety-goals/ [accessed 2024-09-08]

Abbreviations
AE: adverse event
CAMS: Collaborative Assessment and Management of Suicidality
CAU: care-as-usual
CPT: Current Procedural Terminology
ED: emergency department
HIPAA: Health Insurance Portability and Accountability Act
IRB: institutional review board
RCT: randomized controlled trial
SSI: Suicide Status Interview
TJC: The Joint Commission

Edited by A Mavragani; submitted 29.08.23; peer-reviewed by C Laranjeira; comments to author 28.12.23; revised version received
19.06.24; accepted 06.07.24; published 16.09.24
Please cite as:
Dimeff LA, Koerner K, Heard K, Ruork AK, Kelley-Brimer A, Witterholt ST, Lardizabal MB, Clubb JR, McComish J, Waghray A,
Dowdy R, Asad-Pursley S, Ilac M, Lawrence H, Zhou F, Beadnell B
A Suicide Prevention Digital Technology for Individuals Experiencing an Acute Suicide Crisis in Emergency Departments: Naturalistic
Observational Study of Real-World Acceptability, Feasibility, and Safety
JMIR Form Res 2024;8:e52293
URL: https://formative.jmir.org/2024/1/e52293
doi: 10.2196/52293
PMID:©Linda A Dimeff, Kelly Koerner, Kandi Heard, Allison K Ruork, Angela Kelley-Brimer, Suzanne T Witterholt, Mary BethLardizabal, Joseph R Clubb, Julie McComish, Arpan Waghray, Roger Dowdy, Sara Asad-Pursley, Maria Ilac, Hannah Lawrence,Frank Zhou, Blair Beadnell. Originally published in JMIR Formative Research (https://formative.jmir.org), 16.09.2024. This is
an open-access article distributed under the terms of the Creative Commons Attribution License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information,a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

https://formative.jmir.org/2024/1/e52293

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e52293 | p. 10(page number not for citation purposes)