JMIR FORMATIVE RESEARCH

Kang & Hong

Original Paper

Digital Interventions for Reducing Loneliness and Depression in
Korean College Students: Mixed Methods Evaluation
Boyoung Kang, MBA, ME; Munpyo Hong, PhD
Sungkyunkwan University, Seoul, Republic of Korea

Corresponding Author:
Boyoung Kang, MBA, ME
Sungkyunkwan University
25-2, Sungkyunkwan-ro, Jongno-gu
Seoul, 03063
Republic of Korea
Phone: 82 2 740 1770
Email: bykang2015@gmail.com

Abstract
Background: The COVID-19 pandemic has exacerbated the prevalence of loneliness and depression among college students.
Digital interventions, such as Woebot (Woebot Health, Inc) and Happify (Twill Inc), have shown promise in alleviating these
symptoms.
Objective: This study aims to investigate the effectiveness and acceptability of Woebot and Happify in reducing loneliness anddepression among college students after the COVID-19 pandemic.
Methods: A mixed methods approach was used over 4 months. A total of 63 participants aged 18 to 27 years from Sungkyunkwan
University in Seoul, South Korea, were initially recruited, with an inclusion criterion of University of California, Los Angeles(UCLA) Loneliness Scale score ≥34. The final sample consisted of 27 participants due to attrition. Participants were randomly
assigned to Woebot (15/27, 55%); Happify (9/27, 33%); or a control group using Bondee (Metadream), a metaverse social network
messenger app (3/27, 11%). Quantitative measures (UCLA Loneliness Scale and Patient Health Questionnaire-9) and qualitative
assessments (user feedback and focused interviews) were used.
Results: Although mean decreases in loneliness and depression were observed in the control and intervention groups after theintervention, the differences between the control and intervention groups were not statistically significant (UCLA Loneliness:
P=.67; Patient Health Questionnaire-9: P=.35). Qualitative data indicated user satisfaction, with suggestions for improved app
effectiveness and personalization.
Conclusions: Despite limitations, this study highlights the potential of well-designed digital interventions in alleviating college
students’ loneliness and depression. The findings contribute to the growing body of research on accessible digital mental healthtools and underscore the importance of comprehensive support systems. Further research with larger and more diverse samples
is needed to better understand the effectiveness and optimization of such interventions.
Trial Registration: Clinical Research Information Service KCT0009449; https://bit.ly/4d2e4Bu(JMIR Form Res 2024;8:e58791) doi: 10.2196/58791
KEYWORDS
loneliness; depression; digital interventions; college students; mental health; mixed methods evaluation; Woebot; Happify

Introduction
Background
The rising rates of depression and anxiety among college
students worldwide, further compounded by the effects of the
COVID-19 pandemic, emphasize the critical need for effective
interventions to address these mental health challenges. In South
Korea, the proportion of people in their 20s and 30s at the risk
https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderXof depression increased 6-folds, to 22.8%, in 2021 compared
to 2018 [1]. Similarly, in the United States, anxiety anddepression symptoms among college students rose by 75% from2019 to 2021 [2]. However, this trend is not solely attributable
to the pandemic; college students’ mental health has been
consistently declining over the past decade [3].
Depression, a potentially fatal illness, is a leading cause ofsuicide, claiming approximately 800,000 lives worldwide each
JMIR Form Res 2024 | vol. 8 | e58791 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
year [4]. South Korea’s suicide rate surpasses the Organization
for Economic Cooperation and Development average, with anotable increase in suicide rates among patients with depression[5,6]. The seriousness of depression is underscored by its highmortality rate; complex etiology involving genetic,
environmental, and psychological factors; and the persistent
stigma surrounding mental health [4-6].
Social isolation and loneliness are significant risk factors for
mortality [7] and are particularly prevalent among college
students [8-10]. The college years are a period of heightened
emotional vulnerability, with students being particularly
susceptible to depression and loneliness [11,12]. Loneliness canlead to social withdrawal and sleep disturbances and exacerbate
depressive symptoms [8,13]. Hawkley and Cacioppo [13]demonstrated that loneliness can indeed lead to depression,
particularly in young adults. Their research suggests that evenwhen controlling for factors such as stress and social support,
loneliness remains a predictor of depression in college students[13]. The correlation between loneliness and depression ispartially attributed to changes in neurotransmitters that regulate
mood, such as serotonin and dopamine [14]. In addition,
loneliness can trigger negative coping behaviors such as alcoholand drug use, further exacerbating feelings of depression [8].
Cacioppo and Hawkley [15] and Matthews et al [9] emphasized
the importance of addressing loneliness and depression in young
adults for their overall well-being and development. Untreateddepression in this age group can lead to academic difficulties,
interpersonal relationship issues, and increased risk of self-harmor suicide. Timely intervention is essential to prevent
progression to more severe or chronic depression. Various
intervention methods suggest that loneliness and depression inyoung adults can be effectively alleviated. Key to theseinterventions is social support through meaningful connections
formed through family, friends, school, or community activities[10]. The college years are crucial in career development stages
and characterized by increased independence, intimacy, and the
dynamic exploration of life goals [11,16]. However, college
students often face challenges such as stress while adapting to
new environments, anxiety about academic and career paths,
and difficulties in relationships, all of which can negatively
impact their psychological and emotional well-being [11,12].
Despite these challenges, the college years also provide
unprecedented opportunities for intellectual growth anddevelopment [17,18].
Digital mental health interventions have shown promise inovercoming the limitations of traditional mental health services[19]. Grounded in theories such as self-determination theory
and cognitive behavioral therapy (CBT), digital interventions
have demonstrated effectiveness in reducing symptoms ofdepression and anxiety [19,20]. Self-determination theory
emphasizes the importance of autonomy, competence, andrelatedness in engaging individuals and initiating sustainedchanges [21], while CBT posits that depression arises fromnegative thoughts and beliefs, which can be modified throughcognitive restructuring and behavioral activation [22]. The
anonymity and accessibility of digital interventions make them
particularly appealing to digital-native college students [23].
https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

Kang & HongThis Study
This study explores the feasibility and acceptability of 2 digital
therapy apps, Woebot (Woebot Health, Inc) and Happify (Twill
Inc), among college students experiencing loneliness. Inspiredby the research of Lim et al [24] on +Connect, this study has alarger sample size and an extended intervention period for acomprehensive evaluation. In addition, the research by Boucher
et al [25] and Fitzpatrick et al [26] provided a foundation for
selecting these specific digital interventions. The primary aim
was to determine whether these digital interventions canmeaningfully reduce feelings of loneliness and depression
among college students in South Korea. The study also
investigated the role of participants’ beliefs and behaviors in
the effectiveness of the interventions. The hypotheses were as
follows:
•

•

•

•

Hypothesis 1: Participants using Woebot and Happify willshow a greater reduction in loneliness and depression scorescompared to the control group using the Bondee(Metadream) app.
Hypothesis 2: Participants who expect this intervention
experiment to reduce loneliness and depression will
experience greater reductions in these symptoms (perceived
benefits of action).
Hypothesis 3: Participants who believe that even digitalapplications can alleviate loneliness and depression will
experience greater reductions in these symptoms (perceived
barriers to action).
Hypothesis 4: Participants with low help-seeking behavior
will experience greater reductions in loneliness anddepression after using the digital interventions (help-seeking
behavior).

In addition, qualitative feedback will provide insights into theuser experience, perceived effectiveness, and areas for
improvement of these digital interventions.

Methods
Experimental Design
This study aimed to explore the feasibility and acceptability ofdigital interventions in reducing loneliness and depression
among university students. The study was conducted fromMarch to June 2023, over a 4-month period, with participantsfrom Sungkyunkwan University’s Colleges of Natural Sciencesand Humanities and Social Sciences in Seoul, South Korea. The
study used a mixed methods approach, incorporating bothquantitative and qualitative data collection and analyses. The
clinical trial was registered with the Clinical ResearchInformation Service, a World Health Organization International
Clinical Trials Registry Platform–linked registry, under theregistration KCT0009449.

Intervention
Given the importance of selecting effective digital interventions
for addressing loneliness and depression among college students,
Woebot and Happify were chosen for their demonstrated
efficacy in prior research. Participants were provided with an
instruction manual (Multimedia Appendix 1) to guide themthrough the use of the assigned intervention app. Woebot, a
JMIR Form Res 2024 | vol. 8 | e58791 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
chatbot based on CBT principles, has shown effectiveness in
reducing symptoms of depression and anxiety among young
adults, as evidenced by studies such as the randomized
controlled trial conducted by Fitzpatrick et al [26]. In addition,
Happify has demonstrated promise in alleviating loneliness,particularly during challenging periods such as the COVID-19
pandemic, as reported by Boucher et al [25].
Moreover, both Woebot and Happify are readily downloadable
in South Korea, providing convenient and accessible platforms
for individuals seeking to improve their mental well-being.
These chatbots were also selected based on the performance
results of other mental health chatbots that have been verified
for effectiveness and satisfaction in clinical experimental studies.
Building on the promising findings of a recent study by Li etal [27], regarding their top-tier performance in terms ofengagement (Woebot: mean 4.10, SD 0.14; Happify: mean 4.60,

Kang & HongSD 0.00), functionality (Woebot: mean 4.65, SD 0.21; Happify:
mean 4.25, SD 0.00), and overall quality (Woebot: mean 4.31,
SD 0.09; Happify: mean 4.15, SD 0.02) on a 5-point scale,
Woebot and Happify were selected as the interventions for thisstudy. The features and instructions of use of Woebot andHappify are shown in Table 1.
Given the lack of Korean-language versions of Woebot andHappify, all interactions with the apps were conducted in
English. This decision was made to ensure uniformity acrossall intervention groups, despite the potential challenge it posed
to participants with lower English proficiency. No special
cultural adaptations for Korean users were implemented,
although compensation was provided in the form of a choice
between equivalent-value coffee coupons and cash, with allparticipants opting for the cash incentive.

Table 1. Digital interventions: Woebot and Happify.

aWoebotHappify

Type of therapy

CBTa

CBT and mindfulness

Delivery method

Chatbot

Chatbot, games, and activities

Cost

Free

Free and premium (paid)

Availability

Use in academic researchCommercial use

Other features

Weekly mood description

Mood tracking, social community, and rewards

Benefits

Can help improve mood, reduce stress, and improve cogni- Can help improve mood, reduce stress, and improve cognitive function
tive function

Previous representative research

Fitzpatrick et al [26]

Boucher et al [25]Instructions

Step 1: download the app from the app store or Google
Play. Step 2: create an account by entering your email address, password, date of birth, and nickname. Step 3: open
the app and use it for at least 15 minutes every day. You
can choose from a variety of topics, including Focusing on
Positive, Relationship, Mindfulness & Meditation, andManaging Emotions. You must also use the gratitude
journal. Step 4: capture and save the journal 3 times a week.
This is necessary for record keeping and monthly participant rewards.

Step 1: download the app from the app store or Google
Play. Step 2: create an account by entering your sex, age,
race, occupation, family relationships, and current mental
state. Step 3: open the app and use it for at least 15 minutes
every day. You can choose from a variety of activities in
the Instant Play section, including Serenity Scene, Guided
Meditation, and Negative Knockout. Thank mode is required for each activity. You can also use the Mindfulness& Meditation track (free). Step 4: capture and save the activity tasks 3 times a week. This is necessary for record
keeping and monthly participant rewards.

CBT: cognitive behavioral therapy.

Participant Recruitment and Selection
The target sample size was based on previous studies on digitalmental health interventions. Initially, 63 undergraduate andgraduate students aged 18 to 27 years were recruited throughthe university portal (see Multimedia Appendix 2 for the
recruitment notice and survey questionnaires). Participants withvery low loneliness scores (<34) were excluded. While therecruitment notice mentioned that participants should be
confident in using English, as the Woebot and Happify apps do
not have Korean versions, no applicants were excluded basedon their English proficiency. To ensure fairness, all participants,
including those in the control group using Bondee, were required
to interact with the apps in English. Due to high attrition rates,
the final sample consisted of 27 participants (Woebot: n=15,55%; Happify: n=9, 33%; Bondee: n=3, 11%). To enhance
https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

retention, participants received a US $30 incentive forcompleting the study and an additional US $30 for completingthe postintervention assessment [28].

Randomization
Participants were assigned to 1 of the 3 groups (Woebot,
Happify, or Bondee) using a stratified randomization method.
The participants’ names were blinded, and they grouped suchthat there were similar distributions of age, sex, and lonelinessscores across the groups. The decision to have fewer participantsin the control group was made to ensure sufficient power in theintervention groups, given the limited overall sample size andanticipated high attrition rates in the control group. This decision
was made based on the intention of the research to ensure thestudy focused on the effectiveness of the interventions.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Assessment Tools and Data Collection
Quantitative Data
Quantitative data were collected using the University ofCalifornia, Los Angeles (UCLA) Loneliness Scale (Korean
version) and the Patient Health Questionnaire-9 (PHQ-9;
Korean-translated version) before and after the intervention (seeMultimedia Appendix 3 for the questionnaires). The primary
outcome was reduction in loneliness and depression scores. The
secondary outcomes were user satisfaction and perceived
benefits of the digital interventions.

Cronbach α
In addition to examining the quantitative outcomes, Cronbach
α coefficients were calculated to assess the reliability of theKang & Hongmeasurement scales used in the study (Table 2). The consistently
high Cronbach α coefficients (range 0.848-0.928) for the UCLA
Loneliness Scale across all time points indicate strong internalconsistency. This high internal consistency enhances the internal
validity of the test and underscores the reliability of theloneliness scores obtained throughout the study. The robustnessof the measurement instrument strengthens the confidence in
the interpretation of loneliness levels among participants,
thereby bolstering the internal validity of our experiment.
The PHQ-9 coefficients were generally lower due to the shorter
scale length (9 items). However, all PHQ-9 coefficients, except
for the baseline coefficient (0.680), were above the acceptable
threshold of 0.6 for short scales [29].

Table 2. Internal consistency reliability of measurement scales: Cronbach α coefficients.
Cronbach α

Measurement tool and time pointa

UCLA Loneliness Scale
Baseline

0.848

1-month follow-up

0.893

2-month follow-up

0.928

After the intervention

0.904

b

PHQ-9

a

Baseline

0.680c1-month follow-up

0.702

2-month follow-up

0.746

After the intervention

0.769

UCLA: University of California, Los Angeles.

b

PHQ-9: Patient Health Questionnaire-9.

c

The PHQ-9 coefficients were generally lower due to the shorter scale length (9 items). However, all PHQ-9 coefficients except for the baseline (0.680)
were above the acceptable threshold of 0.6 for short scales [29].

Qualitative Data
Qualitative data were gathered through open-ended survey
questions and 5 focus group interviews conducted via Zoom
(Zoom Video Communications, Inc), each lasting approximately
90 minutes with 5 to 6 participants per group. Participants were
given a preliminary orientation via KakaoTalk (Kakao Corp),
a popular Korean social networking service, which included an
introduction to the concept of focus group interviews, themethodology, and the anticipated questions. The interviews
explored participants’ experiences and coping mechanisms forloneliness and app features. Participants were provided with anorientation document introducing the concept and methodology
of focus group interviews as well as containing a list of
anticipated questions. However, due to time constraints, theactual interviews primarily focused on a subset of thesehttps://formative.jmir.org/2024/1/e58791

XSL• FO
RenderXquestions, that is, those related to coping with loneliness anduser experiences with the apps (Multimedia Appendix 4).

Interview Questions
The following questions were asked during the focus groupinterviews.
••••

What features of the app did you find most helpful?
What challenges did you encounter while using the app?
How did the app impact your feelings of loneliness anddepression?
What improvements would you suggest for the app?

Data Analysis
Textbox 1 provides the list of statistics considered and tests
conducted during qualitative and quantitative analyses.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Textbox 1. Summary of data analysis methods.
Quantitative analysis•

Descriptive statistics: mean, median, IQR, and SD [30]•Group comparisons: 2-tailed t tests for normally distributed data and Mann-Whitney U tests for nonnormally distributed data [31]•Statistical software: data analysis was conducted using Python (numpy, scipy, and matplotlib) after extracting results from Google Surveys in
CSV format ChatGPT was used to refine the data statistical analysis code.

•

Significance level: α=.05•

CI: 95%•The data analysis code used in this study is available in Multimedia Appendix 5.

Qualitative analysis•

Coding approach: an inductive approach was used to identify themes from the data [32]•Software: Zoom was used for real-time recording, Naver Clova was used for transcription, and Python-based text mining was used to extract key
themes and related content•

Process: themes were identified and refined through an iterative review of transcripts. The primary author conducted the coding, with periodic
reviews by another researcher to ensure consistency AI tools, including Claude AI, were used to check for duplicates and assist in the refinementof the coding process.

•Validity and reliability: data validity and reliability were ensured through triangulation, member checking, and maintaining a detailed audit trail[29]•

Confidentiality: data were deidentified to maintain participant confidentiality•

Homogeneity: focus groups consisted of participants using the same app, primarily third-year university students•

Data saturation: achieved through repetitive themes identified across the focus groups

Ethical Considerations
This study was approved by the institutional review board (IRB)of Sungkyunkwan University, Seoul, South Korea (IRB file:
SKKU 2023-02-043; approval date: February 27, 2023; see
Multimedia Appendix 6 for the IRB approval letter). All
participants provided informed consent before their involvementin the study (see Multimedia Appendix 7 for the informed
consent form). Data were deidentified to ensure confidentiality.
Participants were compensated for their time and participationin the study. Compensation varied based on participation, witha maximum amount of up to KRW 200,000 (US $154).
Participants received KRW 10,000 (US $7.70) per survey, KRW
20,000 (US $15.40) per interview, up to KRW 30,000 (US$23.10) for experimental participation, and an additional reward
at the end of the study (amount to be determined). Compensation

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

was calculated based on participation frequency and compliance,and payments were made at the end of each month after theinterviews were completed. The details of this compensation
were clearly outlined in the informed consent form provided to
all participants.

Consolidated Standards of Reporting Trials Reporting
This study followed the CONSORT (Consolidated Standardsof Reporting Trials) reporting guidelines for randomized
controlled trials. The CONSORT flow diagram, illustrating theflow of participants through the study, is presented in Figure 1.
The CONSORT-EHEALTH (Consolidated Standards ofReporting Trials of Electronic and Mobile Health Applications
and Online Telehealth) checklist (version 1.6) [33] was also
completed and is available as Multimedia Appendix 8.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Figure 1. CONSORT (Consolidated Standards of Reporting Trials) flow diagram of participants.

Results
Demographic and Clinical Variables of Participantsat Baseline
Table 3 presents the demographic and clinical variables ofparticipants at baseline. The table includes measures of central
tendency and dispersion, such as mean, SD, median, and IQR
for both the UCLA Loneliness Scale and PHQ-9.
Among the initial 63 applicants, the average PHQ-9 score was
9.23 (SD 6.15), with 25 (40%) individuals classified as being

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

at risk for depression (PHQ-9≥10). At the baseline survey, 23(43%) out of 53 participants were categorized as being at riskfor depression, which is considered a very high proportion
according to the Ministry of Health and Welfare’s criteria [1].
These findings highlight the urgent need for interventions toaddress the mental health crisis among college students.
In addition, participants with a UCLA Loneliness Scale scoreof ≥65, indicating severe loneliness, and those with a PHQ-9
score of ≥20, indicating severe depression, were advised to seek
further counseling.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Table 3. Demographic and clinical variables of participants at baseline (n=53).
Variable

Woebot (n=22)

Happify (n=21)

Control (Bondee) (n=10)

Values, mean (SD)53.18 (9.14)

53.67 (11.05)

53.50 (7.09)

Values, median (IQR)

53.5 (47-59)

53.0 (45-62)

50.5 (48-59)

Values, mean (SD)10.55 (5.89)

9.14 (5.84)

12.50 (7.47)

Values, median (IQR)

9.5 (7-12)

8.0 (4-11)

12.5 (8-15)

Values, mean (SD)23.5 (1.78)

23.0 (2.14)

22.9 (1.85)

Values, median (IQR)

22 (21-25)

23 (22-24)

23 (22-24)

Female14 (64)

11 (52)5 (50)

Male

7 (32)

9 (43)

5 (50)

Prefer not to disclose

1 (5)

1 (5)

0 (0)

Humanities and social science

11 (50)10 (48)4 (40)Natural science and engineering

11 (50)11 (52)6 (60)

UCLAa Loneliness

Depression (PHQ-9b)

Age (y)Sex, n (%)

Majors, n (%)

a

UCLA: University of California, Los Angeles.

b

PHQ-9: Patient health Questionnaire-9.

Participant Flow
Figure 1 displays the CONSORT flow diagram, illustrating theflow of participants through the study. Of the initial 63
respondents, 10 (16%) were excluded due to loneliness scores<34. The remaining 53 (84%) participants were randomly
assigned to the intervention groups (Woebot: n=22, 42%;
Happify: n=21, 40%) and the control group (Bondee: n=10,19%). Due to attrition, the final sample consisted of 27
participants (Woebot: n=15, 55%; Happify: n=9, 33%; Bondee:
n=3, 11%).

intervention. However, these effects did not reach statisticalsignificance, likely due to the small sample size. The control
group (Bondee) saw a significant decrease in user numbers(baseline: n=10; after the intervention: n=3), complicatingcomparisons. Consequently, hypothesis 1 was not verified.
To address the unbalanced sample sizes (Bondee: 3/27, 11%;
Woebot: 15/27, 55%; Happify: 9/27, 33%), Welch ANOVA
was used, as it is robust to violations of equal variances.
However, the overall small sample size limited the power todetect significant effects, and the quantitative results should beinterpreted with caution.

Quantitative FindingsIntervention Effects and Trends
The quantitative data analysis results, presented in Figure 2 andTable 4, show decreases in loneliness and depression after the

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Figure 2. (A) University of California, Los Angeles (UCLA) Loneliness Scale and (B) Patient Health Questionnaire-9 (PHQ-9) trend charts.

Table 4. Descriptive statistics and group comparisons for University of California, Los Angeles (UCLA) Loneliness Scale and Patient Health
Questionnaire-9 (PHQ-9) scoresa.
Variable and time point

Woebot (n=15),
mean (SD)Happify (n=9),mean (SD)Control (Bondee;n=3), mean (SD)Group comparison
test

P value

Baseline

51.13 (8.73)

51.44 (8.76)

53 (6.08)

t test.93

1 month

48.40 (10.27)

49.78 (10.51)

50 (10.39)

Mann-Whitney U.68

2 months

49.53 (10.26)

46.56 (10.72)

52.33 (8.96)

t test.51

Postintervention

48.47 (10.26)

46.78 (10.92)

51.33 (4.16)

t test.67

Baseline

8.87 (4.88)

8.33 (3.81)

11 (5.29)

t test.78

1 month

7 (5.13)

6.78 (5.33)

8 (4.36)

Mann-Whitney U.83

2 months

7.73 (4.38)

5 (3.91)

8.33 (3.51)

t test.14

Postintervention

7.07 (4.67)

4.78 (3.73)

6.33 (3.21)

Mann-Whitney U.35

UCLA Loneliness

PHQ-9 Depressiona

P values indicate the significance level of group comparisons. α level was set at.05. t test (2-tailed) was used for normally distributed data with equalvariances. Mann-Whitney U test was used when data did not meet normal distribution criteria. Detailed data analysis procedures are provided in thePatient Health Questionnaire-9 (PHQ-9) and University of California, Los Angeles (UCLA) Loneliness Trend Analysis code in Multimedia Appendix
5.

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Description of Results
UCLA Loneliness Scale
In control group using Bondee, the mean UCLA Loneliness
Scale score remained relatively stable, from 53 (SD 6.08) atbaseline to 51.33 (SD 4.16) after the intervention. The groupcomparison test (t test) did not show a statistically significant
difference among the 3 groups at baseline (P=.93) or after theintervention (P=.67). In the group using Woebot, the mean
UCLA Loneliness Scale score decreased from 51.13 (SD 8.73)
at baseline to 48.47 (SD 10.26) after the intervention. In thegroup using Happify, the mean UCLA Loneliness Scale score
decreased from 51.44 (SD 8.76) at baseline to 46.78 (SD 10.92)
after the intervention.
PHQ-9 Depression Scale
In the control group using Bondee, the mean PHQ-9 depression
score decreased from 11 (SD 5.29) at baseline to 6.33 (SD 3.21)
after the intervention. The group comparison test (t test) did notshow a statistically significant difference among the 3 groups
at baseline (P=.78), and the Mann-Whitney U test did not show
a significant difference between the control and intervention
groups after the intervention (P=.35). In the group using
Woebot, the mean PHQ-9 depression score decreased from 8.87
(SD 4.88) at baseline to 7.07 (SD 4.67) after the intervention.
In the group using Happify, the mean PHQ-9 depression score
decreased from 8.33 (SD 3.81) at baseline to 4.78 (SD 3.73)
after the intervention.
Overall, although the intervention groups (using Woebot andHappify) showed decreases in both UCLA Loneliness Scaleand PHQ-9 depression scores from baseline to after theintervention, the group comparison tests did not reveal
statistically significant differences among the groups either atbaseline or after the intervention for both measures.

Hypothesis Testing
As hypothesized in the Introduction section, we investigatedthe effectiveness of the digital interventions and the role ofparticipants’ beliefs and behaviors in their outcomes.
Hypothesis 1: Intervention Effect
The quantitative results showed decreases in both UCLA
Loneliness Scale and PHQ-9 depression scores from baseline
to after the intervention in the intervention groups (using Woebot
and Happify). However, the group comparison tests (t test fornormally distributed data and Mann-Whitney U test for
nonnormally distributed data) did not reveal statisticallysignificant differences between the control and intervention
groups after the intervention (UCLA Loneliness Scale: P=.67;
PHQ-9 depression: P=.35). Therefore, hypothesis 1 was notsupported.

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

Kang & Hong
Hypothesis 2: Perceived Benefits of Action
The positive group, which expected this intervention experiment
to reduce loneliness and depression, showed higher reduction
rates in scores than the negative group. However, Mann-Whitney
U tests did not reveal statistically significant differences between
the groups (UCLA Loneliness Scale: P=.93; PHQ-9: P=.48).
Therefore, hypothesis 2 was not supported.
Hypothesis 3: Perceived Barriers to Action
For participants believing in the effectiveness of digitalapplications in general, the difference in UCLA Loneliness
Scale scores (P=.60) and PHQ-9 scores (P=.47) between thepositive and negative groups was not statistically significant
according to Mann-Whitney U tests. Therefore, hypothesis 3
was not supported.
Hypothesis 4: Help-Seeking Behavior
The results for self-help behavior attitudes showednonsignificant differences between the groups (UCLA
Loneliness Scale: P=.34; PHQ-9: P=.62) based on
Mann-Whitney U tests. Therefore, hypothesis 4 was notsupported.
It is important to note that the small and unbalanced sample
sizes across the study groups may have limited the statisticalpower and generalizability of the hypothesis tests. The notably
smaller control group compared to the intervention groups, aswell as the overall limited sample size, could have influenced
the nonsignificant results. These limitations should be taken
into account when interpreting the findings, and future studies
with larger and more balanced samples are needed to further
investigate these hypotheses and to obtain more conclusive
results.
Figure 3 visually presents the comparisons between the positive
and negative groups for each of the hypotheses (hypotheses 2,
3, and 4). For hypotheses 2 (perceived benefits of action), 3(perceived barriers to action), and 4 (help-seeking behavior),
the differences between the positive and negative groups were
not statistically significant for either UCLA Loneliness Scaleor PHQ-9 depression scores, as indicated by the P values >.05.
In summary, although the intervention groups showed reductions
in loneliness and depression scores, the quantitative analyses
did not provide statistically significant evidence to support the
hypothesized intervention effect (hypothesis 1) or the influenceof participants’ beliefs and behaviors on the outcomes(hypotheses 2, 3, and 4). The nonsignificant results should beinterpreted with caution due to the limitations imposed by thesmall sample size.
From the quantitative analysis, we added the self-reported
mental health literacy levels and app satisfaction ratings. The
results are described in the subsequent sections.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Figure 3. Health behavior belief versus loneliness and depression. (A) Hypothesis 2: perceived benefits of action (the positive group is the group ofpeople who answered, “I expected my loneliness and depression would decrease after the experiment”). (B) Hypothesis 3: perceived barriers to action(the positive group is the group of people who answered, “even digital application can help people relieve loneliness”). (C) Hypothesis 4: help-seeking
behavior (the positive group is the group of people who answered, “it is difficult to control loneliness and depression on one’s own will and efforts”).
PHQ-9: Patient Health Questionnaire-9; UCLA: University of California, Los Angeles.

Self-Reported Cohort Mental Health Literacy
Participants rated their mental health literacy on a 5-point scale(1=poor, 5=abundant). The mean score was 2.46, indicating
relatively low self-perceived mental health literacy (Figure 4).
Figure 4. Self-reported cohort mental health literacy.

App Satisfaction Rating
User satisfaction was moderate (Table 5). Most users (16/24,
67%) were somewhat satisfied, with few (2/24, 8%) reporting
https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

high satisfaction. Low retention rates suggest that evensomewhat satisfied users may not have found the apps
compelling enough for continued use.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Table 5. Satisfaction ratingsa.
QuestionaVery satisfied

Somewhat satisfied

Not satisfied

WoebotHappify

WoebotHappify

WoebotHappify

Overall satisfaction

1

0

12

4

2

6

Easy to use

0

0

11

4

4

8

Novelty

0

0

5

2

10

7

Effectiveness

2

0

11

5

2

4

Intention to maintain

0

0

6

3

9

6

Not satisfied=1-3, somewhat satisfied=4-7, and very satisfied=8-10.

Interpretation of Nonsignificant Results
Despite nonsignificant quantitative results, qualitative insights
highlight practical benefits and potential improvements for theinterventions.

Qualitative Findings
The qualitative findings from the open-ended survey questions
and focus group interviews provided valuable insights into
participants’ experiences with the digital interventions and theircoping mechanisms for loneliness.

Favored App Features and Areas for Improvement
Participants highlighted favored features and areas for
improvement in the digital interventions. Woebot users
appreciated the app’s responsiveness and tone, while Happify
users valued features supporting meditation and gratitude
practice (Table 6). Suggestions for improvement included
enhancing app effectiveness, personalization, and user interface.
Woebot and Happify users reported several limitations and areas
for improvement during the first month of the intervention(Table 7). One of the most significant drawbacks was that both
applications supported only English, which could be a major
barrier for non–English speaking users. This lack of language
diversity highlights the need for mental health apps that caterto a wider range of users.
Another common issue was the repetitive and limited nature of
the responses provided by the apps. Despite Woebot’s diverse
content, users noted that certain parts felt repetitive. Happify
users also reported a lack of content variety. This suggests thateven with advanced conversational agents, the interactions canstill feel scripted and constrained by rule-based systems.
Woebot users encountered problems with the user input system,
which sometimes led to infinite loops or errors in theconversation flow. This could be frustrating for users seeking
smooth and intuitive interactions. Happify users also found theinterface less intuitive than expected.
In terms of conversational abilities, both apps had limitations.
Woebot’s responses could feel fixed and repetitive, while

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

Happify’s questions were sometimes perceived as repetitive.
These issues underscore the challenges in creating truly dynamic
and engaging conversational experiences.
The attrition rates were notably different between the 2intervention groups. The Woebot group saw a decrease from22 to 15 participants, whereas the Happify group saw a decrease
from 21 to 9 participants. This significant attrition in the Happify
group suggests that user satisfaction may have played a role inretention rates. Participants’ feedback indicated that issues suchas nonintuitive design and pressure for premium subscription
might have contributed to the higher dropout rate in the Happify
group compared to the Woebot group.
Overall, users expressed a need for more relevant and applicable
content in both apps. Woebot users desired content that was
better tailored to their specific needs, while Happify users
reported dissatisfaction with the overall experience and felt
pressured to subscribe to premium services.
Table 8 summarizes the participants’ suggestions for enhancing
the apps, which include improving the chatbot’s conversational
abilities, providing more personalized content, offering a wider
variety of interactive features, and enhancing the user interface.
Participants also emphasized the importance of integrating theseapps with professional mental health support systems.
These findings highlight the need for designing mental healthapps that are linguistically inclusive, offer diverse and engaging
content and intuitive user experiences, and enable effective
interactions. Addressing these areas for improvement could
greatly enhance the effectiveness and user satisfaction of digitalmental health interventions. Moreover, this feedback
underscores the essential need to integrate digital platforms withboth web-based and offline medical systems, ensuring thatstudents have seamless access to professional support. As
Lambert and Barley [34] highlighted, therapeutic relationshipis crucial for psychotherapy’s effectiveness, and our study’s
participants similarly emphasized the importance of connectingwith mental health professionals for effective digitalapplications.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 11(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Table 6. Participants’ favored features.
Category

Woebot (n=17)

Happify (n=9)

User experience

Conversational feel, daily journal, and warm messages

Well-chosen name and versatility

Contents

Gratitude journal and psychological knowledge

Thank activity and guided meditation

Features

Fast response, varied content, and cute images

Thank activity, guided meditation, and word game

Table 7. Areas for improvement.
Category

WoebotHappify

Language support

English only

English only

Content variety

Variety but some repetition

Content seems a bit lacking

User experience

User input system or infinite loop problem

Nonintuitive and occurrence of errors

Conversational features

Fixed responses

Repetitive questions

Overall feedback

Desire for more applicable contents

Not satisfactory and pressure to subscribe

Table 8. App enhancement suggestions (n=31).

aSuggested enhancement

Mentions, n (%)

Inclusion of a system for connecting with mental health professionals

12 (39)Reward system for activities performed

7 (23)

High-quality feedback for gratitude journals

5 (16)

Home button and feature descriptions on pages

4 (13)

Alarm system for activities to be performed

4 (13)

Personalized opt-in function based on individual needs

4 (13)

Various video content for users

3 (10)

Podcast on loneliness and depression

2 (6)UCCa and peer-to-peer networking

2 (6)UCC: user-created content.

Coping Mechanisms for Loneliness
During the focus group interviews, participants shared a diverse
array of coping mechanisms for loneliness (Textbox 2). These
strategies encompassed goal-setting, distraction, self-care, andsocial connection.

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

Participants emphasized redirecting focus toward tangible
achievements and using diverse forms of distraction to manage
loneliness. Self-care activities and interpersonal engagement
were highlighted as crucial aspects of coping with loneliness.
These insights suggest that a multifaceted approach addressing
individual coping strategies and broader social connections isessential.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 12(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Textbox 2. Coping strategies for loneliness.
Goal-setting•

Participants emphasized redirecting their focus toward tangible achievements, such as studying for English proficiency certificates or scholarships.
“I’m motivated to achieve things. I like to find a way out and do various things. After graduating from high school, I started reading books
seriously, even though I avoided classic novels like Frankenstein because they were difficult.”

Distraction•

Overcoming emotional entanglement, participants turned to diverse forms of distraction. “I used to get caught up in my emotions, but lately, I’ve
been trying to get out of them. For example, I watch interesting TV shows. I listen to new songs, both upbeat and ballads. I find something to do
to distract myself. I watch TV or YouTube. I find something I can immerse myself in. I watch games or dramas. I watched ‘Iltascansdal.’ I watch
games or dramas.”

Self-care•Self-care strategies include going for walks, reading specific books, and consciously managing negative thoughts. “Life is a journey, so it’s
important to be strong on your own. I’ve been working out at the gym recently.” Participants emphasized regular exercise routines, waking up
on time, and engaging in activities such as writing during challenging times.

Social connection•

Interpersonal engagement emerged as a key coping mechanism, with participants highlighting the significance of talking to people, especiallyclose friends, to alleviate loneliness. “I’ve tried various things, but talking to people is the only way to forget those feelings. I prefer to talk to
close friends rather than new people. I miss being around people. I cope by talking to other people. I talk to other people and express my feelings.”

Gratitude Journaling and Positive Experiences
Participants’ responses to the gratitude journaling feature of theapps revealed instances of positive emotional transformation.
Engaging in gratitude journaling enhanced self-compassion,
fostered acceptance, and strengthened social bonds. Theinteractive nature of the chatbots and the sense of care conveyed
through warm greeting messages were particularly appreciatedby participants dealing with loneliness or isolation.
The word cloud in Multimedia Appendix 9 visually represents
participants’ positive experiences with the app, based on their
responses to questions about specific moments they foundhelpful. The word size represents the frequency of mentions,with “gratitude,” “daily,” “conversation,” “meditation,”“response,” and “writing” being the most common words. This
indicates that participants frequently benefited from the app indaily gratitude practices, meditation, conversations, and writing.

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

The word cloud clearly highlights these common themes andexperiences.
Table 9 presents the specific instances when participants foundthe apps most beneficial, categorized by the positive changes
they experienced, such as emotion enhancement, gratitude
journaling, daily life improvement, and physical healthimprovement. Participants reported increased happiness, reduced
negative emotions, better sleep quality, enhanced concentration,and even physical health benefits such as reduced blood pressureand fatigue through app use.
The examples in Table 9 suggest that digital mental healthinterventions have the potential to positively impact various
aspects of college students’ well-being, even if these effects
were not fully captured in the quantitative analysis. The
qualitative data indicates that consistent use of these apps could
lead to meaningful improvements in users’ mental health andoverall well-being.

JMIR Form Res 2024 | vol. 8 | e58791 | p. 13(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Table 9. Specific examples of best used cases.
Category and subcategory

App

Positive change

Specific example

Gratitude journal
Positive emotion enhance- Woebot and Hap- Increased happiness and sense of accom- •
ment
pify
plishment, reduced negative emotions,and the development of a positive mindset•“When I wrote down what I was grateful for every day in Woebot, I was able to be grateful foreven the small things.”“Through Happify’s Thank You, there were times
during the week when I thought about the things
I was grateful for and felt happy.”•“When I used the journal to write down three
things I was grateful for each day in Woebot, I
was able to reflect on the day.”“Writing a gratitude journal in Happify made me
think about the things I was grateful for.”Self-reflection

Woebot and Hap- Reflecting on the day, understanding
pify
oneself, and exploring positive values•

Daily life improvement

Woebot and Hap- Reduced stress, increased mental stabili- •pifyty, and improved sleep quality•“When I felt really frustrated, writing a gratitude
journal in Woebot helped me to feel neutral.”“When I tried to submit a screenshot of my gratitude journal in Happify, I was surprised to see
the old gratitude I had forgotten.”

Meditation and deep breathing
Concentration improvement

Happify

Improved concentration, reduced distrac- •tions, and efficient time management forgoal achievement“On the way home on the subway, or in bed before going to sleep, I wrote thank activity, and I
was able to look back on the day and feel relievedthat I was able to get through the day safely, andthat there were things to be grateful for, even if
they were small.”

Physical health improvement

WoebotReduced blood pressure and heart rate, •increased immunity, relaxed muscles andrelieved pain, improved sleep quality,
and reduced fatigue

Brief meditation before sleep for emotional regulation: One participant mentioned, “Before going
to bed, I used the brief meditation, which helpedme organize my emotions and feel more at ease.”

Emotion regulation

WoebotReduced anxiety, stress, and depression; •improved emotion recognition and regulation ability; and maintenance of peace •of mind“There was a moment when I was able to regulate
my emotions.”“One day, when someone I was in touch with said
they were going to confess, I suddenly thoughtof the app and wrote that I was grateful.”

BondeeEmotional stability and healing effect•“The experience of just floating sky photos when
I’m tired.”Miscellaneous
Positive emotions

Discussion

compared Woebot and Happify, offering insights into theireffectiveness and user preferences.

Principal Findings

The quantitative data showed a moderate positive correlation
between loneliness and depression (r=0.37; P<.001),
highlighting their complexity. This suggests that while related,loneliness and depression are distinct, and other factors may
also influence college students’ mental health outcomes (seethe PHQ-9 and UCLA Loneliness Trend Analysis code in
Multimedia Appendix 5).

This study adds to the research on digital mental healthinterventions by evaluating the effectiveness of Woebot andHappify in reducing loneliness and depression among college
students. Compared to the study by Fitzpatrick et al [26], ourstudy had a longer intervention period and a postintervention
follow-up for a more comprehensive evaluation. While the study
by Lim et al [24] had a similar intervention period, our study
had a larger sample size and more extensive analyses. Unlike
Boucher et al [25], who focused on qualitative insights during
a shorter period, our mixed methods approach that spanned alonger duration provided a better understanding of theinterventions’ efficacy and user experiences. Our study uniquely

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

The moderate correlation between loneliness and depression
highlights their intricate, bidirectional relationship. Lonelinesscan lead to depressive symptoms due to a lack of social
connection, while depression can increase loneliness throughsocial withdrawal. This suggests that interventions targeting
either issue may positively affect the other. However, lonelinessand depression are distinct constructs that require different
JMIR Form Res 2024 | vol. 8 | e58791 | p. 14(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hongapproaches: social skills and connections for loneliness and
CBT and emotion regulation for depression.

Additional Considerations for the South Korean
Context

In addition, the moderate correlation suggests that other factors,
such as stress, coping mechanisms, and resilience, also play arole in mental health outcomes. This underscores the need forcomprehensive, multifaceted interventions that address thediverse challenges faced by college students. Even within thesmall control group using Bondee, the slight decrease inloneliness and depression scores suggests that any form ofstructured interaction can have beneficial effects, emphasizing
the importance of social engagement and connection.

In South Korea, research on digital interventions for reducing
loneliness and depression among college students is extremely
rare. There are very few studies, and none involve structured
intervention experiments. This highlights the importance ofdeveloping and validating Korean-language digital mental healthinterventions. Given the severe mental health crisis among
Korean college students, immediate intervention is crucial.

Several factors might explain why the interventions did notyield significant effects. First, the lack of a Korean version of
the apps may have reduced engagement, especially for thoseless proficient in English.
In addition, the demanding academic schedule and competitiveenvironment, coupled with daily stressors, may have exacerbateddepression and loneliness, reducing intervention effectiveness.
This underscores the need for holistic digital mental healthinterventions considering these factors.

South Korean college students highly valued conversational
elements, positive reinforcement, educational features, andreward mechanisms in digital interventions. This study serves
as a critical pilot for the future development of Korean-language
support features as well as reward-based CBT chatbots
integrating positive psychology and empathetic support. The
outcomes of this research provide a foundational understandingthat will guide the creation of more effective and culturally
relevant digital mental health tools for South Korean college
students.

Implications
The findings of this study have several important implications
for future research and practice in the field of digital mental
health interventions for college students (Textbox 3).

Textbox 3. Implications of this study.
Implications
1.

Personalized, user-centered design: future interventions should be tailored to the unique needs and preferences of college students to enhance
engagement and effectiveness. For example, conducting in-depth user interviews to develop customized opt-in features and various personas cansignificantly improve the personalization of digital mental health tools. As mentioned inTable 8, features such as alarm systems for activities and personalized opt-in functions based on individual needs were frequently suggested by participants.

2.

Integration with professional and peer support: digital interventions should seamlessly connect with both web-based and offline mental healthprofessionals. In addition, peer support groups, for example, the University of Michigan’s Wolverine Support Network [35] and listener systems
such as 7 Cups [36], can be effective alternatives for providing emotional support and fostering a sense of community.

3.

Mental health literacy education: incorporating mental health literacy education into college curricula and developing holistic support systems
is crucial for addressing the persistent challenges faced by students. Institutions should prioritize comprehensive approaches to mental healthpromotion.

4.

Longitudinal research: further research with larger, more diverse samples and extended follow-up periods is needed to validate the efficacy andlong-term impact of digital interventions. This will help in understanding their sustained benefits and potential limitations.

5.

Development of large language model–based chatbots: enhancing chatbots with Korean-language support, positive psychology, empathy, andreward mechanisms can potentially better meet the mental health needs of South Korean college students, improving engagement and effectiveness.
Future research should focus on developing and evaluating such chatbots tailored to the specific needs and preferences of South Korean college
students.

Limitations
This study has several limitations that should be acknowledged.
First, the small and unbalanced sample sizes across the study
groups may have limited the statistical power andgeneralizability of the results. The notably smaller control groupcompared to the intervention groups could have influenced thenonsignificant findings. In addition, due to the limited sample
size and the need to ensure sufficient power in the intervention
groups, an uneven distribution of depression scores wasobserved at baseline (P=.78). Although the mean depression
scores were within a similar range across the groups, thishttps://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

limitation should be considered when interpreting the results,
as the groups may not be perfectly comparable in terms of initialdepression severity. Future studies with larger and morebalanced samples are needed to confirm the efficacy of digitalinterventions for college students.
Second, the reliance on self-report measures may haveintroduced response bias. Incorporating objective measures anddiagnostic assessments could enhance the validity of futurestudies. Third, the use of English-language interventions may
have influenced engagement and satisfaction among participants
with lower English proficiency. Future research should explore
JMIR Form Res 2024 | vol. 8 | e58791 | p. 15(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
the development and evaluation of interventions in participants’native languages. Finally, the study’s duration may have been
insufficient to capture long-term effects. Longer follow-up
periods are needed to assess the sustainability of treatment
effects and identify potential barriers to long-term engagementwith digital mental health tools.

Conclusions
This mixed methods study provides valuable insights into theefficacy and user experiences of digital mental healthinterventions for college students. While the quantitative
findings did not show significant effects, the qualitative data
revealed the potential of well-designed digital interventions topositively impact various aspects of college students’ mental
health and well-being. Participants’ experiences with gratitude
journaling, meditation, and other app features suggest that

Kang & Hongconsistent use of these tools could lead to meaningfulimprovements in emotional, cognitive, and physical health.
However, the study also underscores the importance ofconsidering contextual factors and individual preferences when
designing and implementing digital mental health interventions.
Future research should focus on developing personalized,
user-centered interventions that address the unique needs ofcollege students and explore ways to integrate these tools withprofessional support systems. Moreover, a comprehensive
approach to promoting mental health among college students
is needed, which includes mental health education and theprovision of holistic support systems. Further large-scale,longitudinal studies are necessary to confirm the efficacy andsustained impact of digital interventions on college students’mental health outcomes.

Acknowledgments
This research was conducted with funding from the K-medi Global Talent Development Project of the Ministry of Health andWelfare of the Republic of Korea (assignment: HI22C2185).
This research was also supported by the Sung Kyun Kwan University and the BK21 FOUR (Graduate School Innovation) fundedby the Ministry of Education (MOE, South Korea) and National Research Foundation of Korea (NRF).
The authors acknowledge the use of artificial intelligence (AI) language models in the preparation of this manuscript. Claude AI[37] was used for proofreading, editing (removing redundant expressions), and formatting references. ChatGPT [38] assisted inrefining the data statistical analysis code. In addition, AI tools were used to check for duplicates during the thematic analysis
process. These AI tools were used to enhance the manuscript’s readability and provide technical support, while the researchcontent, analysis, and results remain entirely the work of the authors.

Authors' Contributions
BK designed the study, collected and analyzed the data, and drafted the manuscript. MH provided feedback and critically reviewed
the manuscript.

Conflicts of Interest
None declared.

Multimedia Appendix 1
Participants instruction manual (in Korean).
[PDF File (Adobe PDF File), 346 KB-Multimedia Appendix 1]

Multimedia Appendix 2
Recruitment notice and survey questionnaires (in Korean).
[PDF File (Adobe PDF File), 43 KB-Multimedia Appendix 2]

Multimedia Appendix 3
Depression and University of California, Los Angeles Loneliness scale questionnaires (self-reported, in Korean).
[PDF File (Adobe PDF File), 338 KB-Multimedia Appendix 3]

Multimedia Appendix 4
Focus group interview orientation (in Korean).
[PDF File (Adobe PDF File), 54 KB-Multimedia Appendix 4]

Multimedia Appendix 5
Data analysis code.
[PDF File (Adobe PDF File), 53 KB-Multimedia Appendix 5]
https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 16(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Multimedia Appendix 6
Institutional review board approval letter.
[PDF File (Adobe PDF File), 172 KB-Multimedia Appendix 6]

Multimedia Appendix 7
Informed consent form (in Korean).
[PDF File (Adobe PDF File), 485 KB-Multimedia Appendix 7]

Multimedia Appendix 8
CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online
Telehealth) checklist (in English).
[PDF File (Adobe PDF File), 1025 KB-Multimedia Appendix 8]

Multimedia Appendix 9
Word cloud showcasing students’ positive experiment experiences.
[PNG File , 213 KB-Multimedia Appendix 9]

References
1.

2.
3.

4.
5.
6.

7.

8.
9.

10.
11.
12.

13.
14.

15.
16.

National survey on the mental health of Koreans during COVID-19 [Article in Korean]. Ministry of Health and Welfare
Korea. Dec 15, 2021. URL: https://www.mohw.go.kr/board.es?mid=a10503010100&bid=0027&tag=&act=view&list_no=372545&cg_code= [accessed 2024-08-22]
Mental health - household pulse survey - COVID-19. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/nchs/covid19/pulse/mental-health.htm [accessed 2024-08-11]
Lipson SK, Lattie EG, Eisenberg D. Increased rates of mental health service utilization by U.S. college students: 10-year
population-level trends (2007-2017). Psychiatr Serv. Jan 01, 2019;70(1):60-63. [FREE Full text] [doi:10.1176/appi.ps.201800332] [Medline: 30394183]
Depression. World Health Organization. URL: https://www.who.int/health-topics/depression#tab=tab_1 [accessed 2024-08-22]
Causes of death statistics. Statistics Korea. URL: https://kostat.go.kr/menu.es?mid=a20205010000 [accessed 2024-08-22]
Che SE, Geun GY, Lee JY, Lee H, Yun K, Sim B, et al. Trends in patient suicide rate after psychiatric discharge in Korea
from 2010 to 2018: a nationwide population-based study. J Affect Disord. Feb 15, 2023;323:860-865. [doi:10.1016/j.jad.2022.12.009] [Medline: 36526113]
Holt-Lunstad J, Smith TB, Baker M, Harris T, Stephenson D. Loneliness and social isolation as risk factors for mortality:
a meta-analytic review. Perspect Psychol Sci. Mar 2015;10(2):227-237. [doi: 10.1177/1745691614568352] [Medline:
25910392]
Eres R, Lim MH, Lanham S, Jillard C, Bates G. Loneliness and emotion regulation: implications of having social anxiety
disorder. Aust J Psychol. Aug 05, 2020;73(1):46-56. [doi: 10.1111/ajpy.12296]
Matthews T, Danese A, Wertz J, Odgers CL, Ambler A, Moffitt TE, et al. Social isolation, loneliness and depression inyoung adulthood: a behavioural genetic analysis. Soc Psychiatry Psychiatr Epidemiol. Mar 2016;51(3):339-348. [FREEFull text] [doi: 10.1007/s00127-016-1178-7] [Medline: 26843197]
Cuijpers P, Stringaris A, Wolpert M. Treatment outcomes for depression: challenges and opportunities. Lancet Psychiatry.
Nov 2020;7(11):925-927. [doi: 10.1016/S2215-0366(20)30036-5] [Medline: 32078823]
Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. May
2000;55(5):469-480. [Medline: 10842426]
Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, et al. Mental disorders among college students inthe World Health Organization World Mental Health Surveys. Psychol Med. Oct 2016;46(14):2955-2970. [FREE Full text][doi: 10.1017/S0033291716001665] [Medline: 27484622]
Hawkley LC, Cacioppo JT. Loneliness matters: a theoretical and empirical review of consequences and mechanisms. Ann
Behav Med. Oct 2010;40(2):218-227. [FREE Full text] [doi: 10.1007/s12160-010-9210-8] [Medline: 20652462]
Spithoven AW, Cacioppo S, Goossens L, Cacioppo JT. Genetic contributions to loneliness and their relevance to theevolutionary theory of loneliness. Perspect Psychol Sci. May 07, 2019;14(3):376-396. [doi: 10.1177/1745691618812684][Medline: 30844327]
Cacioppo JT, Hawkley LC. Perceived social isolation and cognition. Trends Cogn Sci. Oct 2009;13(10):447-454. [FREEFull text] [doi: 10.1016/j.tics.2009.06.005] [Medline: 19726219]
Adolescent health. World Health Organization. URL: https://www.who.int/health-topics/adolescent-health#tab=tab_1[accessed 2024-08-11]

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 17(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
17.
18.
19.

20.

21.
22.
23.

24.

25.

26.

27.
28.
29.
30.
31.
32.
33.

34.
35.
36.
37.
38.

Kang & Hong

Magolda MB. Making Their Own Way: Narratives for Transforming Higher Education to Promote Self-Development.
Milton Park, UK. Taylor & Francis; 2001.
Chickering AW, Reisser L. Education and Identity, Second Edition. San Francisco, CA. Jossey-Bass; 1993.
Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions fordepression, anxiety, and enhancement of psychological well-being among college students: systematic review. J MedInternet Res. Jul 22, 2019;21(7):e12869. [FREE Full text] [doi: 10.2196/12869] [Medline: 31333198]
Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescentsand young people: systematic overview. JMIR Ment Health. Apr 29, 2021;8(4):e25847. [FREE Full text] [doi: 10.2196/25847][Medline: 33913817]
Ryan R, Deci EL. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. New
York, NY. Guilford Press; 2017.
Beck AT, Dozois DJ. Cognitive therapy: current status and future directions. Annu Rev Med. 2011;62:397-409. [doi:10.1146/annurev-med-052209-100032] [Medline: 20690827]
Karyotaki E, Miguel C, Panagiotopoulou OM, Harrer M, Seward N, Sijbrandij M, et al. Digital interventions for common
mental disorders in low- and middle-income countries: a systematic review and meta-analysis. Glob Ment Health (Camb).
Sep 15, 2023;10:e68. [FREE Full text] [doi: 10.1017/gmh.2023.50] [Medline: 38024798]
Lim MH, Rodebaugh TL, Eres R, Long KM, Penn DL, Gleeson JF. A pilot digital intervention targeting loneliness in youth
mental health. Front Psychiatry. Aug 23, 2019;10:604. [FREE Full text] [doi: 10.3389/fpsyt.2019.00604] [Medline:
31507469]
Boucher EM, McNaughton EC, Harake N, Stafford JL, Parks AC. The impact of a digital intervention (Happify) on lonelinessduring COVID-19: qualitative focus group. JMIR Ment Health. Feb 08, 2021;8(2):e26617. [FREE Full text] [doi:10.2196/26617] [Medline: 33498011]
Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression
and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health. Jun
06, 2017;4(2):e19. [FREE Full text] [doi: 10.2196/mental.7785] [Medline: 28588005]
Li LS, Wong LL, Yap KY. Quality evaluation of stress, anxiety and depression apps for COVID-19. J Affect Disord Rep.
Dec 2021;6:100255. [FREE Full text] [doi: 10.1016/j.jadr.2021.100255] [Medline: 34704086]
Cotter RB, Burke JD, Loeber R, Navratil JL. Innovative retention methods in longitudinal research: a case study of thedevelopmental trends study. J Child Fam Stud. Dec 2002;11(4):485-498. [doi: 10.1023/a:1020939626243]
Vaske JJ, Beaman J, Sponarski CC. Rethinking internal consistency in Cronbach's Alpha. Leis Sci. 2016;39(2):163-173.
[doi: 10.1080/01490400.2015.1127189]
Delacre M, Leys C, Mora YL, Lakens D. Taking parametric assumptions seriously: arguments for the use of Welch’s F-test
instead of the classical F-test in one-way ANOVA. Int Rev Soc Psychol. 2019;32(1):13. [doi: 10.5334/irsp.198]
Nachar N. The Mann-Whitney U: a test for assessing whether two independent samples come from the same distribution.
Quant Methods Psychol. Mar 01, 2008;4(1):13-20. [FREE Full text] [doi: 10.20982/tqmp.04.1.p013]
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [doi:10.1191/1478088706qp063oa]
Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports ofweb-based and mobile health interventions. J Med Internet Res. Dec 31, 2011;13(4):e126. [FREE Full text] [doi:10.2196/jmir.1923] [Medline: 22209829]
Lambert MJ, Barley DE. Research summary on the therapeutic relationship and psychotherapy outcome. Psychother Theory
Res Pract Train. 2001;38(4):357-361. [doi: 10.1037/0033-3204.38.4.357]
Wolverine Support Network. URL: https://www.umichwsn.org/ [accessed 2024-08-11]
7 Cups. URL: https://www.7cups.com/ [accessed 2024-08-17]
Claude login. Claude. URL: https://claude.ai/login?returnTo=%2F%3F [accessed 2024-08-28]
ChatGPT homepage. ChatGPT. URL: https://chatgpt.com/ [accessed 2024-08-28]

Abbreviations
CBT: cognitive behavioral therapy
CONSORT: Consolidated Standards of Reporting Trials
CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications
and Online Telehealth
IRB: institutional review board
PHQ-9: Patient Health Questionnaire-9
UCLA: University of California, Los Angeles

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 18(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Kang & Hong

Edited by A Mavragani; submitted 25.03.24; peer-reviewed by HF Sit, M Almutairi, H Omar; comments to author 06.06.24; revised
version received 19.06.24; accepted 08.07.24; published 12.09.24
Please cite as:Kang B, Hong M
Digital Interventions for Reducing Loneliness and Depression in Korean College Students: Mixed Methods Evaluation
JMIR Form Res 2024;8:e58791
URL: https://formative.jmir.org/2024/1/e58791
doi: 10.2196/58791
PMID:©Boyoung Kang, Munpyo Hong. Originally published in JMIR Formative Research (https://formative.jmir.org), 12.09.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information,a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

https://formative.jmir.org/2024/1/e58791

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e58791 | p. 19(page number not for citation purposes)