JMIR FORMATIVE RESEARCH

Shinno et al

Original Paper

Feasibility of Guided Internet-Based Cognitive Behavioral Therapy
for Panic Disorder and Social Anxiety Disorder in Japan: Pilot
Single-Arm Trial
Seina Shinno1*, MMSc; Kazuki Matsumoto2,3*, PhD; Sayo Hamatani3,4, PhD; Yosuke Inaba5, PhD; Yoshihito Ozawa5,
BSc; Yohei Kawasaki5, PhD; Tomoki Ikai6, MD, PhD; Chihiro Sutoh3, MD, PhD; Hiroyuki Hayashi1,7, MD, PhD;
Eiji Shimizu3,8, MD, PhD
1

Department of Integrated Medical Sciences, Graduate School of Medicine, University of Fukui, Fukui, Japan

2

Division of Clinical Psychology, Kagoshima University Medical and Dental Hospital, Kagoshima, Japan

3

Research Center for Child Mental Development, Chiba University, Chiba, Japan

4

Research Center for Child Mental Development, University of Fukui, Fukui, Japan

5

Biostatistics Section, Clinical Research Center, Chiba University Hospital, Chiba, Japan

6

Faculty of Medical Sciences, Division of Medicine Primary Health Care, University of Fukui, Fukui, Japan

7

Department of Emergency and General Medicine, Fukui University Hospital, Fukui, Japan

8

Department of Cognitive Behavioral Physiology, Graduate School of Medicine, Chiba University, Chiba, Japan*these authors contributed equally

Corresponding Author:
Kazuki Matsumoto, PhD
Division of Clinical Psychology
Kagoshima University Medical and Dental Hospital
1-35-8 Sakuragaoka
Kagoshima, 8908520
Japan
Phone: 81 099 275 5707
Email: k2782199@kadai.jp

Abstract
Background: Cognitive behavioral therapy (CBT) is effective in treating anxiety disorders. Accessibility to CBT has been
limited in Japan due to the shortage of therapists. While an open-source e-learning system can be used to create a simple
internet-based cognitive behavioral therapy (ICBT) program, the safety and outpatient acceptance of this treatment approach
have not been explored in Japan.
Objective: The aim of this study was to investigate whether outpatients with anxiety disorders could accept and successfullycomplete the ICBT program with guidance by CBT therapists when implementing therapeutic modules and CBT tasks. Due tobeing in the initial phase of a novel treatment in Japan, this study was intended for verification with a small sample size.
Methods: In total, 6 adults, including 4 male participants and 2 female participants, were enrolled in a single-arm trial. Theintervention involved guided ICBT comprising 12 sessions, including CBT text, comprehension confirmation tests, and explanatory
videos about cognitive behavioral models, accessible through a website. The therapist guided the participants in accessing the
ICBT program and answering their questions using a chat tool. The primary outcome was anxiety severity assessed using theState-Trait Anxiety Inventory-Trait. Secondary outcomes included the Panic Disorder Severity Scale, Liebowitz Social Anxiety
Scale (LSAS), Beck Anxiety Inventory (BAI), Patient Health Questionnaire–9, Generalized Anxiety Disorder–7, and Working
Alliance Inventory–Short Form (WAI-SF). Statistical analyses were performed using paired 2-tailed t tests to assess the changes
in clinical symptoms. The total WAI-SF score at the final session was used to evaluate the therapeutic alliance. For statisticalanalyses, mean changes for total State-Trait Anxiety Inventory-Trait, BAI, Panic Disorder Severity Scale, LSAS, Patient Health
Questionnaire–9, and Generalized Anxiety Disorder–7 scores were analyzed using the paired 2-tailed t test. The 2-sided significancelevel for hypothesis testing was set at 5%, and 2-sided 95% CIs were calculated.
Results: Most participants diligently engaged with the ICBT program. No adverse events were reported. The mean total scoresfor the primary outcome decreased by 11.0 (SD 9.6) points (95% CI –22.2 to 0.20; Hedges g=0.95), but it was not statistically
https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e53659 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shinno et al

significant. The mean total scores for the secondary outcomes that assess clinical symptoms decreased, with a significant reduction
observed in the BAI of 15.7 (SD 12.1) points (95% CI –28.4 to –3.0; P=.03; Hedges g=1.24). The mean total scores for PDSS
and LSAS decreased significantly, by 12.0 (SD 4.24) points (95% CI –50.1 to 26.1; P=.16; Hedges g=1.79) and 32.4 (SD 11.1)
points (95% CI –59.7 to –4.3; P=.04; Hedges g=1.38), respectively. Of the participants, 67% (n=4) showed treatment response,and 50% (n=3) achieved remission after the intervention. The therapeutic alliance, measured using the WAI-SF, was moderate.
Conclusions: Guided ICBT may be feasible for the treatment of outpatients with panic disorder and social anxiety disorder inJapan.
Trial Registration:
University Hospital Medical Information Network Clinical Trials Registry UMIN0000038118;
https://center6.umin.ac.jp/cgi-open-bin/ctr/ctr_view.cgi?recptno=R000043439(JMIR Form Res 2024;8:e53659) doi: 10.2196/53659
KEYWORDS
cognitive behavioral therapy; internet intervention; panic disorder; social anxiety; feasibility trail; adult; adults; anxiety disorder;internet-based; e-learning; Japan; statistical analyses; therapist; therapists; intervention; severity; symptoms; therapeutic alliance;
mobile phone

Introduction
Anxiety disorders such as social anxiety disorder (SAD) andpanic disorder (PD) involve prolonged and significant anxiety,leading to substantial impairment in daily functioning. In Japan,
the 12-month prevalence of anxiety disorders is 5.3%, making
them the most commonly diagnosed mental disorders [1].
Cognitive behavioral therapy (CBT) has been proven to beeffective in treating anxiety disorders, particularly SAD and PD[2,3]. As per the clinical guidelines [4,5], CBT is the primary
treatment option for patients with SAD or PD. However, CBT
has rarely been integrated into the Japanese psychiatric clinicalpractice. The limited availability of CBT therapists and highimplementation costs have been major obstacles significantly
impacting the accessibility of CBT [6,7]. According to a 2018
report, the implementation rate of CBT in Japanese psychiatricclinics was 6.2% [7]. When CBT is applied, it is rarely basedon an evidence-based cognitive behavioral model. In most CBTpractices, therapists independently select and combine CBTtechniques [7].
Since the latter half of the 1990s, self-help CBT programs havebeen provided in rural areas via the internet. CBT delivered
through the internet is known as internet-based cognitive
behavioral therapy (ICBT) [8]. In the ICBT approach, therapists
guide patients via phone or email to enhance their treatment
experience. Previous studies of guided ICBT have demonstrated
significant therapeutic effects on anxiety disorders. According
to 2 systematic reviews with meta-analyses, guided ICBT islikely as effective as face-to-face CBT [9,10]. Most studies ofguided ICBT in these reviews were conducted in Europe,
particularly, Northern Europe [10-13]. Due to the varying
internet infrastructure and ICT literacy across different countries,
evidence from clinical trials in Eastern countries is crucial.
Recently, several studies on ICBT with treatment programs
adapted to specific cultures have been conducted in China,
Korea, and Pakistan, yielding results supporting the effectivenessof ICBT [14-16]. Although some culturally adapted ICBT
programs for mental disorders have begun to emerge in Japan[17-20], there is a lack of evidence of clinical trials thatspecifically target anxiety disorders [21]. A previous single-arm
https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

study in Japan demonstrated the feasibility of CBT via
videoconference as another remote format for 20 adults with
SAD or PD [22]. In this study, remote therapists provided
individual CBT in real time. The results indicated that Japaneseindividuals with anxiety disorders find remote interventions
preferable and readily acceptable. However, in the guided ICBT
format, the absence of face-to-face interaction with the therapist
is fundamental, and it is anticipated that the treatment experiencemay qualitatively differ, potentially leading to less favorable
patient acceptance and treatment responses.
To carefully assess the feasibility of guided ICBT, a novel
treatment in Japan, we conducted a rigorously controlled clinical
trial with a small sample size, monitoring for the risk ofdeterioration and unexpected adverse events. The participants’
guided ICBT experiences were also summarized in a brief caseseries included as Multimedia Appendix 1.

Methods
Study Design
This prospective single-arm open trial was conducted between
September 2019 and March 2020 at 2 university hospitals anda psychiatric clinic in Japan. Patients with PD or SAD wererecruited between September and November 2019 throughposters and leaflets placed at medical institutions in the 2
prefectures and on the university’s home page. The participants
continued to receive treatment from general medical
practitioners during the study period, and their authorization
was obtained before enrollment. Personal IDs and passwords
to the treatment program and chat tools were provided toparticipants.

Ethical Considerations
This study project and the present clinical trial protocol adhered
to the Ethical Guidelines for Medical and Biological Research
Involving Human Subjects in Japan [23]. This study wasapproved by the institutional review board of Chiba University
Hospital (G2019004). Written informed consent was obtained
from all participants after they were thoroughly informed of thestudy protocol. No compensation was provided for participatingin the research, although the guided ICBT was provided for
JMIR Form Res 2024 | vol. 8 | e53659 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
free. This pilot single-arm trial was registered at the University
Hospital Medical Information Network Clinical Trials Registry(UMIN0000038118).

Recruitment
Figure 1 illustrates the flow of participants throughout the study.
Initially, 8 patients were recruited in the study. The screening
through psychological assessments was conducted by 3 clinicalpsychologists (SS, KM, and SH), and it was ultimately
confirmed by 2 physicians (HH and ES) whether the participants
met the diagnostic criteria of the Diagnostic and Statistical
Manual of Mental Disorder, Fifth Edition [24]. After thescreening, 2 patients were excluded either because they did not

Shinno et al
meet the inclusion criteria or because of pending cancellations
as part of the participant management process. In total, 6 patients
underwent a baseline assessment and were subsequently enrolled
in the study. Following enrollment, 1 patient disagreed with theexposure in session 6 during the intervention. Notably, there
were no dropouts in this trial, and no adverse events werereported during treatment. Of the 6 patients, 5 (83%)
successfully completed the treatment modules and participatedin the telephonic posttreatment assessment conducted by anindependent assessor. One patient with PD who could not be
assessed over the phone received all symptom rating scales via
mail except for the Panic Disorder Severity Scale (PDSS).

Figure 1. Participant flow.

Inclusion Criteria
Inclusion criteria were (1) age between 18 and 65 years, (2) aprimary diagnosis of PD or SAD according to the
Mini-International Neuropsychiatric Interview [25], (3)moderate symptoms of PD or SAD: PDSS total score >9 or
Liebowitz Social Anxiety Scale (LSAS) total score >70 [26-29],
and (4) the ability to send emails and access the e-learning
system.

Exclusion Criteria
Exclusion criteria were (1) psychosis and bipolar disorder, (2)current high risk of suicide, (3) substance abuse or dependence
diagnosed within the past 12 months (antisocial personality
disorder), and (4) IQ<80 measured by the Japanese Adult Rating
Test [30].
https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

Primary Outcome Measure
The primary outcome was a change in anxiety symptoms
assessed using the Japanese version of the State-Trait Anxiety
Inventory-Trait (STAI-T) [31]. The STAI-T is a self-reported
scale comprising 40 items that quantitatively measures anxiety
and can be evaluated using 2 scales [32]. The STAI-T comprises
20 items rated on a 4-point scale, and the total score ranges from
20 to 80, divided into 5 levels (I=very low, II=low, III=normal,
IV=high, and V=very high). The validity and reliability of theJapanese version of the STAI-T have been demonstrated [33].

Secondary Outcome Measures
In addition to the primary outcome, several secondary outcomeswith established reliability and validity were selected based on
the research objectives. The Beck Anxiety Inventory (BAI) was
JMIR Form Res 2024 | vol. 8 | e53659 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
used to quantify comprehensive anxiety symptoms [34]. The
BAI is a 21-question multiple-choice self-report inventory used
to assess anxiety severity [34,35]. Other secondary outcome
measures included the PDSS for participants with a primary PD
diagnosis [26,27] and the LSAS for those with a primary SAD
diagnosis [28,29].
The PDSS is a 7-item clinical interview rating scale that
evaluates the core PD characteristics and demonstrates good
psychometric properties [36]. As the PDSS is meant to beadministered by professionals knowledgeable of the clinicalmanifestations of PD, an independent assessor (SH) was used
to conduct and interpret this scale. The treatment response andremission rates were calculated using the LSAS and PDSS. For
SAD, treatment response was defined as a 31% or greater
reduction in the total LSAS score, and remission was defined
as a final LSAS score of ≤36 [29]. For PD, treatment response
was defined as a 40% or greater reduction in the total PDSS
score, and remission was defined as a final PDSS score of ≤5[37].
The trial also assessed the psychological bond between therapists
and participants using the Working Alliance Inventory-ShortForm (WAI-SF) [38], depressive symptoms using the Patient
Health Questionnaire–9 (PHQ-9) [39,40], and generalized
anxiety symptoms using the Generalized Anxiety Disorder–7(GAD-7) [39,41]. The WAI-SF assesses the strength of thetherapeutic alliance between the therapist and patient through12 items rated on a scale of 1 (never) to 7 (always). The total
score ranges from 12 to 84, reflecting the overall strength of
the therapeutic bond between the therapist and patient.
Both the PHQ-9 and GAD-7 were scored on a 4-point scale(0=none, 1=a few days, 2=more than half, and 3=almost daily).
The PHQ-9 scores range from 0 to 27, with the cutoff value forclinically significant depressive symptoms set at 10.
Symptomatology was categorized as follows: 0-4=none,
5-9=mild, 10-14=moderate, 15-19=moderate to severe, and20-27=severe depressive state. The Japanese version of the
PHQ-9 has demonstrated adequate validity [40]. GAD-7 scores
range from 0 to 21, with a cutoff value for clinically significant
generalized anxiety set at 10. Symptomatology was defined asfollows: 0-4=none, 5-9=mild, 10-14=moderate, and15-21=severe general anxiety. The GAD-7 has demonstrated
good reliability and validity [42].

Intervention
We developed ICBT programs based on the Clark and Wells[43] model for SAD and Seki and Shimizu’s model for PD [44].
Each ICBT program consisted of 12 modules. Therapeutic
guidance was provided by a clinical psychologist (KM) withextensive experience in face-to-face CBT for patients with PD
and SAD. The therapist, a clinical psychologist with a PhD, hadcompleted the CBT training course [45]. After each session,
the therapist conducted peer supervision with the third author.
The intervention involved guided ICBT comprising 12 sessions,
including CBT text, comprehension confirmation tests, andexplanatory videos about cognitive behavioral models,accessible through a website. The therapist guided theparticipants in accessing the ICBT program and answering theirquestions using a chat tool.
https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

Shinno et al
The ICBT program for PD included the following modules:
guidance for ICBT program (session 0), psychological educationand case conceptualization (week 1), review of safe behaviors(week 2), modification of self-image (week 3), attention shifttraining (week 4), behavioral experiments with internal sensory
exposure (week 5), staged exposure (week 6 and 7), intervention
on memory of the first panic attack (week 8), examination ofothers’ interpretations of panic attacks (week 9), stop dwellingon the panic attacks (week 10), reconstruction of remaining
beliefs (schema work; week 11), and relapse prevention (week12).
The ICBT program for SAD included the following modules:
guidance for ICBT program (session 0), psychological educationand case conceptualization (week 1), examination of safety
behavior (week 2), video feedback (week 3), attention shifttraining (week 4), behavioral experiment 1 (week 5), behavioral
experiment 2 (week 6), creation of anxiety hierarchy form andgraded exposure (week 7), validating negatively rated
expectations (week 8), stop dwelling on things (week 9),
rewriting the meaning of memories linked to self-image (week10), reconstruction of remaining beliefs (schema work; week11), and relapse prevention (week 12).
The participants in this study were generally instructed not toinitiate, discontinue, or modify their pharmacotherapy after theintervention. The participants were required to inform theresearch team if there were any changes to their medication
regimen.

Hardware and Software
The participants used personal PCs, tablet PCs, or smartphones.
The software used included LearningBox (Tatsuno Information
System Co) and MediLine (ShareMedical Co), a medical chatservice for delivering ICBT. LearningBox is an e-learning
system enabling administrators to create and manage educational
materials, handle member groups, and record and assess grades.
Although e-learning systems store and manage user results, they
do not include personal or sensitive information.
MediLine is a medical chat service (medical social networking
service) designed to replace email and phone calls. It
incorporates robust encryption to prevent military-level
information leaks and operates in a double-encrypted state,
following the Japanese government guidelines. End-to-end
encryption is performed in real time on the server duringcommunication, and temporary memory is used in the terminal.
The lectures for each CBT session were recorded as video
footage, uploaded to YouTube [46,47], and shared with theparticipants.

Data Setting
SH was responsible for collecting outcomes at baseline (week1) and postintervention (week 12). The collected data wereregistered on a server and managed by the Data Management
Office of Chiba University Hospital.

Statistical Analysis
Statistical analyses were conducted according to theintention-to-treat principle by a team of statistical analysts (YI,
YO, and YK). For screening assessments at baseline, summary
JMIR Form Res 2024 | vol. 8 | e53659 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shinno et al

statistics were generated, including proportions for categorical
data and means and SDs for continuous variables. All outcomesthat could be expressed as continuous variables were analyzed
using paired 2-tailed t tests before and after the intervention.
Specifically, total STAI-T, BAI, PDSS, LSAS, PHQ-9, andGAD-7 scores were analyzed. The 2-sided significance levelfor hypothesis testing was set at 5%, and 2-sided 95% CIs were
calculated. The total WAI-SF score, which measures the strengthof the therapeutic alliance, was submitted as a raw score after
the completion of ICBT. One male participant with PD did not
submit PDSS data after the intervention. Imputation of missing
values was not performed.

Results
Demographic Data and Clinical Characteristics
In total, 6 participants (4 male participants and 2 female
participants) with a mean age of 41 (SD 8.2; range 26-51) years
were enrolled in the clinical trial. Table 1 shows thedemographic data and the participants’ clinical characteristics.
All participants continued their pharmacotherapy during thetrial with specific medications such as paroxetine hydrochloride
hydrate (n=3), venlafaxine hydrochloride (n=1), sertralinehydrochloride (n=1), and a combination of sertralinehydrochloride and ezilam (n=1). The participants’ estimated
IQs tended to be higher than the mean (110, SD 4.2).

Table 1. Demographic characteristics.

aParticipantParticipant 2

Participant 3

Participant 4

Participant 5

Participant 6

Sex

Male

Male

FemaleMale

FemaleMale

Age (years)

37

44

26

51

40

48

Education (years)

16

12

9

16

14

14

Employment status

Full time

Full time

Unemployed

Full time

Unemployed

Unemployed

Age at onset (years)

16

20

12

51

40

38

SAD

b

PD

PD

PD

None

MDDc

APd

AP

AP

104

108

110

112

116

Primary diagnosis

SAD

a

SAD

Comorbidity

None

Estimated IQ

110

SAD: social anxiety disorder.

b

PD: panic disorder.

c

MDD: major depressive disorder.

d

AP: agoraphobia.

Adverse Events
No mental or physical adverse events were reported after theintervention.

Evaluation Outcomes
Table 2 provides a detailed overview of the outcomes. In the
primary outcome, the mean total score on the STAI-T decreasedby 11.0 (SD 9.6) points, although this did not reach statisticalhttps://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

significance (95% CI –22.2 to 0.20; P=.05; Hedges g=0.95).
However, several secondary outcomes demonstrated significant
improvements. The mean total BAI score significantly decreased
by 15.7 (SD 12.1) points (95% CI –28.4 to –3.0; P=.03; Hedges
g=–1.24). The mean total PDSS score also decreased
significantly, by 12.0 (SD 4.2) points (95% CI –50.1 to 26.1;
P=.16; Hedges g=1.79). Furthermore, the mean total LSAS
score significantly decreased by 32.0 (SD 11.1) points (95% CI–59.7 to –4.3; P=.04; Hedges g=1.38).

JMIR Form Res 2024 | vol. 8 | e53659 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shinno et al

Table 2. Outcomes from pre- to postintervention (N=6).

a

Scale

Preintervention, mean (SD)

Postintervention, mean (SD)P value

STAI-Ta

56.8 (12.5)

45.8 (10.5)

.05

BAIb

34.2 (13.9)

18.5 (11.1).03

PDSSc (n=3)

15.3 (6.03)

3.0 (4.24).16

LSASd (n=3)

88.3 (17.0)

56.2 (28.2).40

PHQ-9e12.5 (5.61)

8.7 (5.61).16

GAD-7f9.17 (6.85)

6.0 (4.5).27

STAI-T: State-Trait Anxiety Inventory-Trait.

b

BAI: Beck Anxiety Inventory.

c

PDSS: Panic Disorder Severity Scale.

d

LSAS: Liebowitz Social Anxiety Scale.

e

PHQ-9: Patient Health Questionnaire–9.

f

GAD-7: Generalized Anxiety Disorder–7.

Regarding depressive symptoms, the mean total PHQ-9 score
decreased from 12.5 (SD 5.61) to 8.7 (SD 5.61) points, but the
reduction was not statistically significant (95% CI –9.3 to 1.6;
P=.13; Hedges g=0.68). Similarly, for generalized anxiety
symptoms, the mean total GAD-7 score decreased from 9.17
(SD 6.85) to 6.0 (SD 4.47) points, but the reduction was notstatistically significant (95% CI –9.8 to 3.5; P=.27; Hedges
g=0.55).
At the postintervention assessment (week 12), the treatment
response rates were 67% (n=2) for PD and 67% (n=2) for SAD.
The remission rates were 67% (n=2) for PD and 33% (n=1) for
SAD.

Acceptance of Guided ICBTOf the 4 male participants, 3 completed all treatment modules.
These 3 participants completed the treatment without asking
questions to the therapist and were able to understand the content
and execute therapeutic tasks independently. A female
participant with SAD avoided even minimal communication
with therapists through chat tools due to severe social anxiety
symptoms. While she accepted the self-help format of CBT,the chat with the therapist itself became burdensome rather thanbeneficial. A female participant with PD and fear of panic attack
reported being encouraged by the therapist’s empathy andencouragement. A male participant with PD self-discontinued
the treatment module and subsequently lost contact with thetherapist. He frequently struggled to control his emotions,making it challenging to address his concerns effectively. Fora brief case series of guided ICBT, please refer to Multimedia
Appendix 1.

Discussion
Principal Results
This study explored the feasibility of guided ICBT in outpatientswith PD and SAD in Japan. In total, 5 of the 6 participants
successfully completed all modules, and no adverse events werereported throughout the study period. While anxiety symptoms
decreased in most patients after the intervention, with ahttps://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

substantial effect size, the primary outcome (STAI-T) did notshow a significant difference. However, significant
improvements were observed in the secondary outcome, as
measured by the BAI. Our results suggest that guided ICBT has
a positive impact on anxiety symptoms, as evidenced by thenotable changes in BAI scores. Most participants diligently
engaged with the ICBT programs. Meanwhile, patients who
considered themselves to have enough understood the materials
understood the materials hardly contacted the therapists.

Limitations
When interpreting our study, it is important to consider thefollowing limitations. First, the small sample size prevented us
from determining whether the findings were due to a type II (β)
error. Second, this was a single-arm trial without a control group.
More robust and statistically rigorous results could be obtained
through a clinical randomized controlled trial in which thesample size is determined based on the effect sizes observed inour study. A randomized controlled trial in Japan would allow
for better investigation of the effectiveness of ICBT in Japanesepatients with anxiety disorders.
Additionally, we did not assess the extent to which theparticipants implemented ICBT program strategies in their daily
lives. While the therapist encourages participants to practicewhat they learned through chat tools, phone calls, and in-person
interactions, patients might face challenges in implementing
these changes or coping with symptoms [48]. In this study,
patients with PD had difficulty applying learned techniques indaily life. Future studies should evaluate the impact of practiceon symptom improvement to gain a more comprehensive
understanding of the therapeutic processes.

Comparison With Prior Work
Our results showed that participants engaged in video lectures
explaining the treatment modules, potentially enhancing theirunderstanding and motivation for treatment. Despite the small
sample size, guided ICBT with these lecture videos showedsignificant effectiveness as reflected in a treatment response
rate of 67% (n=4) and a remission rate of 50% (n=3). These
JMIR Form Res 2024 | vol. 8 | e53659 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
results are consistent with a representative meta-analysis
examining treatment response predictors in 1162 patients
undergoing ICBT [9]. The reduction in the total score on the
primary symptom rating scales before and after the trial, alongwith the effect size, underscored the efficacy of guided ICBT
that incorporated lecture videos. Notably, the within-groupeffect sizes for the primary symptoms in this study (Hedges
g=1.38 for LSAS and 1.79 for PDSS) were as substantial as
those observed in previous representative studies [49-51].
In this study, the dropout rate was 17% (1 of 6 participants),
which is comparable to face-to-face CBT (26%) [52]. In
contrast, another study of guided ICBT reported an average
dropout rate of 32% for depression [53]. Notably, one dropout
from our trial was a male patient with a significantly lower
therapeutic alliance score (12 points on the WAI-SF total score).
Consistent with previous studies on dropout predictors, patientswith low therapeutic alliance scores tended to drop out [54].
When the therapeutic alliance is weak, therapists and patientsshould engage in discussions regarding treatment goals andtasks. However, rebuilding therapeutic alliances in guided ICBT
contexts may be more challenging than in face-to-face contexts.
Participants with anxiety disorders might willingly participate
in CBT programs but may not always practice cognitive
behavioral skills [8]. For instance, one participant with PD(participant 5) experienced severe panic symptoms related to
employment status. Conversely, some patients demonstrate
independent therapeutic progress without extensive therapist
interactions. Overall, our findings align with the existing
research, indicating that severe anxiety, coexisting depression,and low socioeconomic status are associated with poor outcomes[55]. Please refer to Multimedia Appendix 1 for details regarding
each participant’s treatment course.
In the case of SAD, all 3 participants successfully applied theacquired skills to their daily lives. However, none of theparticipants with SAD used a communication tool with their
therapist. This suggests that using a chat tool may beburdensome for individuals with social anxiety, potentiallyreducing the accessibility of ICBT, as these patients often dislike

Shinno et al
high-intensity communication. However, the therapists’guidance may have encouraged them to engage in self-help
activities [56]. For patients with social anxiety who find it
challenging to ask questions or express their thoughts, therapists
could consider expressing their ability to provide answers asneeded instead of engaging in routine email interactions.
It is worth noting that even patients with social anxiety who
avoid chatting may still be willing to receive treatment, as
indicated by a guided ICBT trial conducted in China, in which
patients with SAD completed more modules of the treatment
program than the nonclinical group [15]. However, if thesymptoms are severe, patients may struggle to practice cognitive
behavioral skills independently in their daily lives. Participants
who showed symptom improvement had relatively high social
functioning, continued employment, and direct community
involvement (participants 1, 2, 4, and 6).
Conversely, patients with severe disorders (participant 3) andthose resistant to exposure-mediated interventions (participant5) may require a significant amount of therapeutic attention.
For instance, individual face-to-face CBT might be a moresuitable option for someone like participant 3, who has SAD.
Similarly, for participant 5, a personalized face-to-face CBTapproach in which the therapist helps analyze the gains andlosses of exposure and safety behaviors and demonstrates
internal sensory exposure techniques could be beneficial.
Clinicians and therapists who are going to conduct CBT should
consider disclosing treatment modules to patients and assess
the level of support required before providing ICBT.

Conclusions
Guided ICBT may be a feasibility treatment approach forpatients with PD and SAD in Japan. If the patients have highmotivation and the ability to understand and practice treatment
modules, supports might not be strictly necessary during thetreatment from the CBT therapist. The empathetic words fromthe therapist and encouragement for program implementation
may assist some patients in overcoming fear and engaging in
the assigned tasks.

Acknowledgments
This study was funded by the Japan Society for the Promotion of Science KAKENHI Grant-in-Aid for Scientific Research (grant
18K03130).

Data Availability
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions
SS played a significant role in study design, recruitment, and paper writing. KM contributed to all aspects of the research process.
KM, SH, and ES were involved in the development and implementation of the cognitive behavioral therapy course. TI, HH, andES contributed to the study design, participant recruitment, and the overall administration of the clinical trial. YI, YO, and YK
were responsible for conducting the statistical analyses. All authors thoroughly reviewed the paper before submission and granted
their approval for publication.

Conflicts of Interest
None declared.
https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e53659 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shinno et alMultimedia Appendix 1
Case series of the 6 participants.
[DOCX File , 23 KB-Multimedia Appendix 1]

References
1.

2.

3.

4.
5.
6.

7.

8.
9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

Demyttenaere K, Bruffaerts R, Posada-Villa J, Gasquet I, Kovess V, Lepine JP, et al. Prevalence, severity, and unmet needfor treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA.
2004;291(21):2581-2590. [FREE Full text] [doi: 10.1001/jama.291.21.2581] [Medline: 15173149]
Carpenter JK, Andrews LA, Witcraft SM, Powers MB, Smits JAJ, Hofmann SG. Cognitive behavioral therapy for anxiety
and related disorders: a meta-analysis of randomized placebo-controlled trials. Depress Anxiety. 2018;35(6):502-514.
[FREE Full text] [doi: 10.1002/da.22728] [Medline: 29451967]
Hofmann SG, Smits JAJ. Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized
placebo-controlled trials. J Clin Psychiatry. 2008;69(4):621-632. [FREE Full text] [doi: 10.4088/jcp.v69n0415] [Medline:
18363421]
Generalized anxiety disorder and panic disorder in adults: management (CG113). National Institute for Health and ClinicalExcellence. 2011. URL: http://tinyurl.com/5cbf4c3z [accessed 2024-02-08]
Social anxiety disorder: recognition, assessment and treatment (CG159). National Institute for Health and Clinical Excellence.
2013. URL: https://www.nice.org.uk/guidance/cg159 [accessed 2024-02-08]
Kawakami N, Takeshima T, Ono Y, Uda H, Hata Y, Nakane Y, et al. Twelve-month prevalence, severity, and treatmentof common mental disorders in communities in Japan: preliminary finding from the World Mental Health Japan Survey
2002-2003. Psychiatry Clin Neurosci. 2005;59(4):441-452. [FREE Full text] [doi: 10.1111/j.1440-1819.2005.01397.x][Medline: 16048450]
Takahashi F, Takegawa S, Okumura Y, Suzuki S. Actual condition survey on the implementation of cognitive behavioral
therapy at psychiatric clinics in Japan. Fumito Takahashi Lab. 2018. URL: http://tinyurl.com/2mbuvbtm [accessed2024-02-08]
Andersson G. The Internet and CBT: A Clinical Guide. Boca Raton, FL. Taylor & Francis Group; 2014.
Andersson G, Carlbring P, Rozental A. Response and remission rates in internet-based cognitive behavior therapy: anindividual patient data meta-analysis. Front Psychiatry. 2019;10:749. [FREE Full text] [doi: 10.3389/fpsyt.2019.00749][Medline: 31708813]
Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior
therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther.
2018;47(1):1-18. [FREE Full text] [doi: 10.1080/16506073.2017.1401115] [Medline: 29215315]
Ciuca AM, Berger T, Crişan LG, Miclea M. Internet-based treatment for panic disorder: a three-arm randomized controlled
trial comparing guided (via real-time video sessions) with unguided self-help treatment and a waitlist control. PAXPD
study results. J Anxiety Disord. 2018;56:43-55. [doi: 10.1016/j.janxdis.2018.03.009] [Medline: 29625834]
Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided Acceptance and Commitment
Therapy for social anxiety disorder and/or panic disorder provided via the internet and a smartphone application: a randomized
controlled trial. J Anxiety Disord. 2016;44:27-35. [doi: 10.1016/j.janxdis.2016.09.012] [Medline: 27721123]
Jolstedt M, Wahlund T, Lenhard F, Ljótsson B, Mataix-Cols D, Nord M, et al. Efficacy and cost-effectiveness oftherapist-guided internet cognitive behavioural therapy for paediatric anxiety disorders: a single-centre, single-blind,randomised controlled trial. Lancet Child Adolesc Health. 2018;2(11):792-801. [doi: 10.1016/S2352-4642(18)30275-X][Medline: 30241993]
Amin R, Iqbal A, Naeem F, Irfan M. Effectiveness of a culturally adapted cognitive behavioural therapy-based guided
self-help (CACBT-GSH) intervention to reduce social anxiety and enhance self-esteem in adolescents: a randomized
controlled trial from Pakistan. Behav Cogn Psychother. 2020;48(5):503-514. [doi: 10.1017/S1352465820000284] [Medline:
32450939]
Chen H, Rodriguez MA, Qian M, Kishimoto T, Lin M, Berger T. Predictors of treatment outcomes and adherence ininternet-based cognitive behavioral therapy for social anxiety in China. Behav Cogn Psychother. 2020;48(3):291-303. [doi:10.1017/S1352465819000730] [Medline: 31928568]
Seol SH, Kwon JS, Kim YY, Kim SN, Shin MS. Internet-based cognitive behavioral therapy for obsessive-compulsive
disorder in Korea. Psychiatry Investig. 2016;13(4):373-382. [FREE Full text] [doi: 10.4306/pi.2016.13.4.373] [Medline:
27482237]
Hamatani S, Matsumoto K, Ishibashi T, Shibukawa R, Honda Y, Kosaka H, et al. Development of a culturally adaptableinternet-based cognitive behavioral therapy for Japanese women with bulimia nervosa. Front Psychiatry. 2022;13:942936.
[FREE Full text] [doi: 10.3389/fpsyt.2022.942936] [Medline: 36081468]
Hamatani S, Matsumoto K, Takahashi J, Shiko Y, Ozawa Y, Niitsu T, et al. Feasibility of guided internet-based cognitive
behavioral therapy for patients with anorexia nervosa. Internet Interv. 2022;27:100504. [FREE Full text] [doi:10.1016/j.invent.2022.100504] [Medline: 35257002]

https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e53659 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
19.

20.

21.

22.

23.
24.
25.

26.
27.
28.
29.
30.

31.
32.
33.
34.
35.

36.

37.

38.
39.
40.

41.

Matsumoto K, Hamatani S, Makino T, Takahashi J, Suzuki F, Ida T, et al. Guided internet-based cognitive behavioral
therapy for obsessive-compulsive disorder: a multicenter randomized controlled trial in Japan. Internet Interv. 2022;28:100515.
[FREE Full text] [doi: 10.1016/j.invent.2022.100515] [Medline: 35242595]
Hamatani S, Matsumoto K, Andersson G, Tomioka Y, Numata S, Kamashita R, et al. Guided internet-based cognitive
behavioral therapy for women with bulimia nervosa: protocol for a multicenter randomized controlled trial. JMIR Res
Protoc. 2023;12:e49828. [FREE Full text] [doi: 10.2196/49828] [Medline: 37725414]
Yoshinaga N, Thew GR, Hayashi Y, Matsuoka J, Tanoue H, Takanashi R, et al. Preliminary evaluation of translated andculturally adapted internet-delivered cognitive therapy for social anxiety disorder: multicenter, single-arm trial in Japan.
JMIR Form Res. 2023;7:e45136. [FREE Full text] [doi: 10.2196/45136] [Medline: 37145850]
Matsumoto K, Sutoh C, Asano K, Seki Y, Urao Y, Yokoo M, et al. Internet-based cognitive behavioral therapy with real-timetherapist support via videoconference for patients with obsessive-compulsive disorder, panic disorder, and social anxiety
disorder: pilot single-arm trial. J Med Internet Res. 2018;20(12):e12091. [FREE Full text] [doi: 10.2196/12091] [Medline:
30559094]
Ethical guidelines for medical and biological research involving human subjects (only Japanese text available). Ministryof Education, Culture, Sports, Science and Technology. 2023. URL: http://tinyurl.com/yck98zkm [accessed 2024-01-27]
American Psychiatric Association; DSM-5 Task Force. Diagnostic and Statistical Manual of Mental Disorders: DSM-5,
5th Edition. Washington, DC. American Psychiatric Publishing Inc; 2013.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International NeuropsychiatricInterview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and
ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22-33;quiz 34. [Medline: 9881538]
Houck PR, Spiegel DA, Shear MK, Rucci P. Reliability of the self-report version of the Panic Disorder Severity Scale.
Depress Anxiety. 2002;15(4):183-185. [doi: 10.1002/da.10049] [Medline: 12112724]
Katagami M. The self-report version of the Panic Disorder Severity Scale: reliability and validity of the Japanese version.
Jpn J Psychosom Med. 2007;47(5):331-338. [FREE Full text] [doi: 10.15064/jjpm.47.5_331]
Asakura S, Inoue S, Sasaki F, Sasaki Y, Kitagawa N, Inoue T, et al. Consideration of validity and reliability for Liebowitz
Social Anxiety Scale (LSAS) Japanese version. J Clin Psychiatry. 2002;44(10):1077-1084. [doi: 10.1037/t75069-000]
Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141-173. [doi: 10.1159/000414022] [Medline:
2885745]
Matsuoka K, Uno M, Kasai K, Koyama K, Kim Y. Estimation of premorbid IQ in individuals with Alzheimer's diseaseusing Japanese ideographic script (Kanji) compound words: Japanese version of National Adult Reading Test. Psychiatry
Clin Neurosci. 2006;60(3):332-339. [FREE Full text] [doi: 10.1111/j.1440-1819.2006.01510.x] [Medline: 16732750]
Hida N, Fukuhara T, Iwawaki M, Soga Y, Spielberger CD. New Edition STAI Manual: State-Trait Anxiety Inventory-Form
JYZ. Tokyo. Practical Education Publishing; 2000.
Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the State-Trait Anxiety Inventory. Palo Alto,
CA. Consulting Psychologists Press; 1983.
Himeno C, Nakazawa H, Hamada S, Ikeda H, Kitajiri M, Yamashita T. An investigation of State-Trait Anxiety Inventory(STAI) in dizzy patients. Equilibrium Res. 2005;64(4):225-232. [FREE Full text] [doi: 10.3757/jser.64.225]
Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult
Clin Psychol. 1988;56(6):893-897. [doi: 10.1037//0022-006x.56.6.893] [Medline: 3204199]
Leyfer OT, Ruberg JL, Woodruff-Borden J. Examination of the utility of the Beck Anxiety Inventory and its factors as ascreener for anxiety disorders. J Anxiety Disord. 2006;20(4):444-458. [doi: 10.1016/j.janxdis.2005.05.004] [Medline:
16005177]
Shear MK, Brown TA, Barlow DH, Money R, Sholomskas DE, Woods SW, et al. Multicenter collaborative Panic Disorder
Severity Scale. Am J Psychiatry. 1997;154(11):1571-1575. [FREE Full text] [doi: 10.1176/ajp.154.11.1571] [Medline:
9356566]
Furukawa TA, Shear MK, Barlow DH, Gorman JM, Woods SW, Money R, et al. Evidence-based guidelines for interpretationof the Panic Disorder Severity Scale. Depress Anxiety. 2009;26(10):922-929. [FREE Full text] [doi: 10.1002/da.20532][Medline: 19006198]
Tracey TJ, Kokotovic AM. Factor structure of the Working Alliance Inventory. Psychol Assess. 1989;1(3):207-210. [doi:10.1037/1040-3590.1.3.207]
Muramatsu K. Patient Health Questionnaire (PHQ-9, PHQ-15) Japanese version and Generalized Anxiety Disorder-7
Japanese version up to date. J Clin Psychol. 2014;7:35-39. [FREE Full text]
Muramatsu K, Miyaoka H, Kamijima K, Muramatsu Y, Yoshida M, Otsubo T, et al. The Patient Health Questionnaire,
Japanese version: validity according to the Mini-International Neuropsychiatric Interview-Plus. Psychol Rep. 2007;101(3Pt 1):952-960. [doi: 10.2466/pr0.101.3.952-960] [Medline: 18232454]
Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary carestudy. Primary care evaluation of mental disorders. Patient Health Questionnaire. JAMA. 1999;282(18):1737-1744. [FREEFull text] [doi: 10.1001/jama.282.18.1737] [Medline: 10568646]

https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

Shinno et al

JMIR Form Res 2024 | vol. 8 | e53659 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
42.
43.
44.

45.

46.
47.
48.

49.

50.

51.

52.
53.
54.

55.

56.

Shinno et al

Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7.
Arch Intern Med. 2006;166(10):1092-1097. [FREE Full text] [doi: 10.1001/archinte.166.10.1092] [Medline: 16717171]
Clark DM, Welles A. A cognitive model of social phobia. In: Heimberg RG, Libowitz MR, Hope DA, Schneier FR, editors.
Social Phobia: Diagnosis, Assessment, and Treatment. New York. The Guilford Press; 1995;69-96.
Seki Y, Nagata S, Shibuya T, Yoshinaga N, Yokoo M, Ibuki H, et al. A feasibility study of the clinical effectiveness andcost-effectiveness of individual cognitive behavioral therapy for panic disorder in a Japanese clinical setting: an uncontrolled
pilot study. BMC Res Notes. 2016;9(1):458. [FREE Full text] [doi: 10.1186/s13104-016-2262-5] [Medline: 27717407]
Kobori O, Nakazato M, Yoshinaga N, Shiraishi T, Takaoka K, Nakagawa A, et al. Transporting Cognitive Behavioral
Therapy (CBT) and the Improving Access to Psychological Therapies (IAPT) project to Japan: preliminary observations
and service evaluation in Chiba. J Ment Health Train Educ Pract. 2014;9:155-166. [doi: 10.1108/jmhtep-10-2013-0033]
Matsumoto K. Cognitive behavioral therapy for panic disorder. YouTube. 2019. URL: http://tinyurl.com/b688f9c8 [accessed2024-02-08]
Matsumoto K. Cognitive behavioral therapy for social anxiety disorder. YouTube. 2019. URL: https://www.youtube.com/
watch?v=dCYsUfhWopc [accessed 2024-02-08]
Bendelin N, Hesser H, Dahl J, Carlbring P, Nelson KZ, Andersson G. Experiences of guided internet-based
cognitive-behavioural treatment for depression: a qualitative study. BMC Psychiatry. 2011;11:107. [FREE Full text] [doi:10.1186/1471-244X-11-107] [Medline: 21718523]
Allen AR, Newby JM, Mackenzie A, Smith J, Boulton M, Loughnan SA, et al. Internet cognitive-behavioural treatment
for panic disorder: randomised controlled trial and evidence of effectiveness in primary care. BJPsych Open.
2016;2(2):154-162. [FREE Full text] [doi: 10.1192/bjpo.bp.115.001826] [Medline: 27703768]
Hedman E, El Alaoui S, Lindefors N, Andersson E, Rück C, Ghaderi A, et al. Clinical effectiveness and cost-effectivenessof internet- vs. group-based cognitive behavior therapy for social anxiety disorder: 4-year follow-up of a randomized trial.
Behav Res Ther. 2014;59:20-29. [doi: 10.1016/j.brat.2014.05.010] [Medline: 24949908]
Nordgreen T, Gjestad R, Andersson G, Carlbring P, Havik OE. The effectiveness of guided internet-based cognitive
behavioral therapy for social anxiety disorder in a routine care setting. Internet Interv. 2018;13:24-29. [FREE Full text][doi: 10.1016/j.invent.2018.05.003] [Medline: 30206515]
Fernandez E, Salem D, Swift JK, Ramtahal N. Meta-analysis of dropout from cognitive behavioral therapy: magnitude,
timing, and moderators. J Consult Clin Psychol. 2015;83(6):1108-1122. [doi: 10.1037/ccp0000044] [Medline: 26302248]
Schmidt ID, Forand NR, Strunk DR. Predictors of dropout in internet-based cognitive behavioral therapy for depression.
Cognit Ther Res. 2019;43(3):620-630. [FREE Full text] [doi: 10.1007/s10608-018-9979-5] [Medline: 32879540]
Watson HJ, Levine MD, Zerwas SC, Hamer RM, Crosby RD, Sprecher CS, et al. Predictors of dropout in face-to-face andinternet-based cognitive-behavioral therapy for bulimia nervosa in a randomized controlled trial. Int J Eat Disord.
2017;50(5):569-577. [FREE Full text] [doi: 10.1002/eat.22644] [Medline: 27862108]
Kelly JM, Jakubovski E, Bloch MH. Prognostic subgroups for remission and response in the Coordinated Anxiety Learning
and Management (CALM) trial. J Clin Psychiatry. 2015;76(3):267-278. [FREE Full text] [doi: 10.4088/JCP.13m08922][Medline: 25562579]
Gershkovich M, Herbert JD, Forman EM, Schumacher LM, Fischer LE. Internet-delivered acceptance-based
cognitive-behavioral intervention for social anxiety disorder with and without therapist support: a randomized trial. Behav
Modif. 2017;41(5):583-608. [doi: 10.1177/0145445517694457] [Medline: 28776431]

Abbreviations
BAI: Beck Anxiety Inventory
CBT: cognitive behavioral therapy
GAD-7: Generalized Anxiety Disorder–7
ICBT: internet-based cognitive behavioral therapy
LSAS: Liebowitz Social Anxiety Scale
PD: panic disorder
PDSS: Panic Disorder Severity Scale
PHQ-9: Patient Health Questionnaire–9
SAD: social anxiety disorder
STAI-T: State-Trait Anxiety Inventory-Trait
WAI-SF: Working Alliance Inventory-Short Form

https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e53659 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shinno et al

Edited by A Mavragani; submitted 18.10.23; peer-reviewed by L Yang; comments to author 25.01.24; revised version received 25.01.24;
accepted 28.01.24; published 29.02.24
Please cite as:
Shinno S, Matsumoto K, Hamatani S, Inaba Y, Ozawa Y, Kawasaki Y, Ikai T, Sutoh C, Hayashi H, Shimizu E
Feasibility of Guided Internet-Based Cognitive Behavioral Therapy for Panic Disorder and Social Anxiety Disorder in Japan: Pilot
Single-Arm Trial
JMIR Form Res 2024;8:e53659
URL: https://formative.jmir.org/2024/1/e53659
doi: 10.2196/53659
PMID: 38421717©Seina Shinno, Kazuki Matsumoto, Sayo Hamatani, Yosuke Inaba, Yoshihito Ozawa, Yohei Kawasaki, Tomoki Ikai, Chihiro
Sutoh, Hiroyuki Hayashi, Eiji Shimizu. Originally published in JMIR Formative Research (https://formative.jmir.org), 29.02.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information,a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

https://formative.jmir.org/2024/1/e53659

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e53659 | p. 11(page number not for citation purposes)