Brief Internet Interventions to Support Mental Health
Phase: NA
Conditions: Loneliness, Mental Health Issue, Distress, Emotional, Depression, Anxiety, Stress
Sponsor: University of California, Irvine

  
  
  
  
  
  
 
 
 
  
  
  
  
  
Brief Internet Interventions to Support Mental Health Study Protocol  
(Includes analysis plans and information sheets) 
April 6, 2023 
 
 
 
 
 
 
 
 
 
 
 
 

Aim 1: Evaluating a Single-session Intervention for Loneliness 
Procedure 
Sample recruitment  
Researchers will use social media and the University of California, Irvine Social 
Ecology subject pool to recruit a diverse online sample of English-speaking participants aged 
16+ with access to the internet and a computer, tablet, or smartphone. Researchers will 
advertise the study on social media using both paid and unpaid posts on several platforms, 
including Instagram, Reddit, Tumblr, Twitter, Facebook, LinkedIn, 4chan, and others. 
Researchers may also use the online subject recruitment platform Researchmatch.  
Recruitment materials will explain that participants must be at least 16 years old, speak 
English, and have access to internet and a computer, smartphone, or tablet to be allowed to 
participate in the study. After a brief introduction to the study, researchers will first ask 
participants to complete the ULS-3 loneliness measure. To ensure a sufficient number of 
participants struggling with loneliness for analysis, if one’s score on the measure meets the 
criterion for “struggling with loneliness”, they will be invited to participate in the study and 
be offered a reimbursement of a $10 Amazon gift card that researchers will email them once 
they complete the final follow-up 8 weeks later. Individuals will be considered to be 
struggling with loneliness if their score on the ULS-3 is 6 or higher and they answer 
affirmatively that their loneliness is causing them distress (Steptoe et al., 2013). 
Researchers will only allow participants to complete the study once. If one’s score on 
the loneliness measure does not meet the criterion, researchers will still give them the 
opportunity to complete the study without compensation. Researchers will also offer them 
access to the loneliness programs if they prefer to not participate in the study. Researchers 
will offer those who participate through the UC Irvine subject pool 1 hour of participation 
credit for completing the first part of the study. If UC Irvine subject pool participants meet 
criteria for struggling with distress, they will also be offered a $10 gift card for completing all 
parts of the study. 
 
Study process  

After participants read the study information and agree to participate, they complete 
baseline self-report measures of mental health outcomes and demographics. Next, they 
complete the intervention (see materials for intervention information). After they complete the 
intervention, participants complete a few brief measures of hope, agency, meaning in life, and 
engagement with the intervention platform. 4 and 8 weeks later, they receive links via email to 
complete follow-up measures of mental health outcomes and engagement. After the 8-week 
follow-up, participants are offered access to all three conditions in PDF and browser forms. 
The study will be conducted using the secure online survey and "experience management" 
platform Qualtrics. Researchers will include anti-fraud / anti-bot measures in the survey.  
Materials 
Researchers created a 25-minute loneliness SSI and a 3-week loneliness intervention 
by adapting Käll et al.’s 9-week therapist-supported iCBT for loneliness (Käll, Shafran, et al., 
2020). Käll et al.’s iCBT focuses on person-level psychological factors contributing to the 
maintenance of loneliness; it aims to help challenge maladaptive thoughts and encourage 
social behaviors (more information on Käll et al.'s iCBT available at 
www.iterapi.se/sites/solus/) (Käll, Shafran, et al., 2020). Following Käll et al.’s iCBT, the 
loneliness intervention includes content relating to understanding loneliness, setting goals 
and values, testing thoughts and behaviors, making plans, gaining social confidence, and 
overcoming barriers. Broadly, the intervention uses behavioral activation, cognitive 
restructuring, and behavioral experimentation strategies. The loneliness interventions also 
pull content from Jessica Schleider et al.'s single-session mental health interventions 
(http://www.schleiderlab.org) and take inspiration from their interventions’ aesthetics and 
their strategy of using Qualtrics to develop and provide the intervention. Both the 30-minute 
SSI and 3-week intervention are text-based and self-guided. 
The 3-week intervention includes three 10-20-minute self-guided sessions 
(participants complete the first session when they start the study, the next session a week 
later after they receive an email with it, and the final portion the next week when they 
receive another email). 
In the control condition, participants complete a supportive therapy SSI, called 
“The Sharing Feelings Project”, of roughly equivalent length and style to the loneliness 
SSI (J. L. Schleider & Weisz, 2019). The supportive therapy SSI, which researchers 

borrowed from the Lab from Scalable Mental health, “encourages participants to express 
emotions to close others”, but “does not teach specific skills”. (J. L. Schleider et al., 2022; 
J. Schleider & Weisz, 2018). Previous work showed this control to be slightly less 
efficacious in improving mental health outcomes in adolescents than a growth mindset 
SSI and a behavioral activation SSI (J. L. Schleider et al., 2022; J. Schleider & Weisz, 
2018). Researchers modified the supportive therapy SSI to make it relevant to both teens 
and adults, as it was originally designed for teens only (link to SSI: https://osf.tvio/u4axs/, 
license: http://creativecommons.org/licenses/by-nc-sa/4.0/) (J. L. Schleider & Weisz, 
2019).  
Hypotheses 
Researchers’ primary hypothesis is that participants randomized to the loneliness SSI will 
have greater reductions in self-reported loneliness between baseline and 8-week follow-up than 
those who are randomized to the active control SSI. 
As secondary hypotheses, researchers predict greater improvements in distress (DASS-9), 
social anxiety, general self-efficacy, hopelessness, meaning and purpose in life, and adaptive 
actions and thoughts in the loneliness SSI condition relative to the control SSI. Researchers also 
hypothesize that participants who are randomized to the 3-week version of the loneliness 
intervention will report greater reductions in loneliness than those who are randomized to either 
the control condition or the 30-minute SSI. 
Exploratory Hypotheses 
1. Having an insight experience during the intervention is associated with greater improvement 
in mental health outcomes between baseline and week 8.  
2. Greater self-efficacy at baseline is associated with greater reduction in loneliness between 
baseline and week 8. 
3. Greater engagement with the SSI platform predicts greater improvement in loneliness 
between baseline and week 8. 

 
Analysis plan 
Main analysis 
The main DV is the change in loneliness (UCLA Loneliness Scale - 20 item Version 3) 
from baseline to 8-week follow-up. In the main analysis, researchers will test between-group 
differences across experimental conditions in change in loneliness from baseline to 8-week 
follow-up. Researchers will use a mixed-effects model predicting loneliness with condition, 
measurement time point, and the 2-way interaction between condition and time point as 
predictors and a participant identifier as a random intercept. Using the “lme4” package in R, 
these analyses will take the following form: 
lmer(loneliness ~ time*condition + (1|study_id)) 
Secondary analysis 
Researchers will re-run the main analysis separately for each secondary DV, 
although the “time” variable will compare across different timepoints depending on the DV 
(see table 1). 
Inclusion criteria 
Researchers will limit the main analyses to only those participants who 1) meet criteria 
for struggling with loneliness, 2) are randomized to a condition in analyses (ie, those who 
consented to participate and completed the baseline measures), 3) did not indicate that they 
did not participate seriously at the end of the study, 4) completed the study only once, and 5) 
spent at least 3 minutes completing the study introduction and baseline measures (ie, were not 
clearly bots or speeding past most of the content). 
Update February 20, 2023: 
Due to many suspected bots and scammers taking the survey (particularly from 
12/22/2022 to 1/22/2023), researchers identified an array of suspicious factors suggesting that 
the same person may have taken the survey multiple times or that some form of automated bot 
was used to complete the survey. Based on these criteria, researchers decided to add an 

additional column to the dataset with a suspicion score for each response and another column 
indicating whether researchers consider the response as fraudulent, real, or likely fraudulent. 
 The factors that researchers identified as suspicious are as follow: 
- Using windows NT 6 
- Using android 9 or below 
- Multiple IP Addresses with same first 8 digits, particularly when taken within a close time 
period 
- Reporting that one found the study on facebook, Instagram, or Twitter after Dec 10, 2022 or 
Reddit after Jan 13, 2023 
- Email address contains 6 numbers in a row and is from gmail 
- Responses to the question about what they liked and what they would change about the 
intervention are specific and identical 
- Reports one found the study on Researchmatch, but one is not on the Researchmatch email list 
- Many participants completing each study session within a short amount of time, often using 
similar devices 
- duration < 8 minutes if finished whole survey  
-  more than 5 responses with the exact same longitude and latitude, many have  >20 or 30 
- Researchers emailed suspected bots/scammers and asked them to indicate where they found the 
study and what they learned from participating in the study. In suspicious responses, one's email 
address did not match one's stated name, emails used broken English, and some email responses 
were highly similar to others. Many who took the survey within a few minutes of one another 
also replied to the emails within a few minutes of one another. 
  
Missing data handling 
Although the researchers take efforts to avoid missing data (e.g., a financial incentive 
for completing the study and reminder emails), it is not necessary to impute missing data 
before performing a longitudinal mixed-model analysis (Twisk et al., 2013). Participants with 
missing data at some assessments will be included in the analysis and results will be modeled 
on the basis of the available data. For all analyses, researchers will use the standard p < 0.05 
criterion for determining statistical significance. 

To test for faulty randomization, researchers will use chi-squared tests with Yates's 
continuity correction and t-tests with Welch's approximation of degrees of freedom and 
assuming unequal variance. If any effects are found, researchers will adjust for them using the 
confounding variable as a covariate in model testing and will note them in the interpretation 
of results. Researchers will also use these tests to check for differential dropout. 
  
Additional analyses 
1.  Researchers will duplicate the main analysis with the subset of participants who did 
not meet criteria for struggling with loneliness and thereby participated in the study 
without being compensated. 
  
2.  Researchers will duplicate the main analysis with the subset of participants who completed 
the week 8 follow-up. 
  
3.  Researchers will duplicate the main analysis but will include the following covariates 
in the mixed-effects model: gender (multiple choice), age (continuous), ethnicity 
(categorical), sexual orientation (categorical), education level (categorical), socioeconomic 
status (continuous), if participants completed the study on a computer, tablet, or phone, 
and whether participants answered at least one written-response question. 
 
 
4.  Researchers will use a one-way ANOVA to test between-group differences in loneliness at 
8-week follow-up. If the omnibus ANOVA produces a significant F-score, researchers will 
conduct two planned contrasts to compare the experimental condition to each other condition, 
and will compute Cohen’s ds of these comparisons. Researchers will run the same ANOVA 
on the 4-week follow-up data. Researchers will also conduct these ANOVAs on secondary 
outcomes and at post-intervention for outcomes collected then. 
  
5.  Researchers will calculate Cohen’s d effect sizes reflecting within-group intervention 
effects on the loneliness for each condition from baseline to 4-week and 8-week follow-
ups. They will also do so for each secondary DV   

 
6. Researchers will run four logistic regression models with dropout as the outcome 
baseline measures as predictors to determine if they predict which participants drop out 
either before completing the intervention, before the post-intervention questions, or before 
the 4- and 8-week follow-ups. 
  
Additional analyses testing exploratory hypotheses 
1. To compare change in mental health outcomes across presence of an insight experience, 
researchers will conduct a mixed-effects model with loneliness as the outcome, 
timepoint, insight experience, and the interaction between the two as predictors, and a 
participant identifier nested within experimental condition as a random intercept. 
2. To test if baseline self-efficacy is associated with change in loneliness between baseline 
and week 8, researchers will run a mixed-effects model with loneliness as the outcome, 
timepoint, baseline self-efficacy (as measured by the GSE-6), and the interaction between 
the two as predictors, and a participant identifier nested within experimental condition as 
a random intercept. 
3. To test if engagement with the intervention predicts change in loneliness, researchers will 
run a mixed-effects model with loneliness as the outcome, timepoint, engagement 
(measured as the number of characters entered across all text entry prompts within 
interventions), and the interaction between the two as predictors, and participant 
identifier nested within experimental condition as a random intercept.  
 
 
 
 
 

Tables describing measures 
Table 1. Full list of primary, secondary, and screening variables collected during the 
study, including when during the trial they were assessed. 
 
Measure (Primary, Secondary, Other) Baseline 
Post- 
Intervention 
4-week 
follow-up 
8-week 
follow-up 
UCLA Loneliness Scale (primary) 
X  
X 
X 
Depression Anxiety and Stress Scale – 
Short Form (secondary) 
X  
X 
X 
3-item Mini Social Phobia Inventory 
(secondary) 
X  
X 
X 
Short form of the General Self-Efficacy 
Scale (secondary) 
X X 
X 
X 
Beck Hopelessness Scale – 4-item 
(secondary) 
X X 
X 
X 
Frequency of actions and thoughts scale 
(secondary) 
X  
X 
X 
Program Feedback Scale (secondary) 
 X 
 
 
Credibility and Expectancy Questionnaire 
(secondary) 
 X 
 
 
Insight experience (secondary) 
 X 
 
 
Star rating (secondary) 
 X 
 
 
MacArthur Scale of Subjective Social 
Status – Youth Version (secondary) 
X  
 
 
UCLA loneliness scale – 3 item 
(screening) 
X  
 
 
Patient Health Questionnaire – 4-item  
(screening)   
X  
 
 
 

Table 2. Measures organized by dimension 
 
Dimension 
Application to 
project 
Study Measurement(s) 
Effectiveness Improvements in 
symptoms and 
functioning 
Primary: ULS-20 (loneliness) 
Secondary: DASS-9 (depression, anxiety, and stress), 
Mini-SPIN (social anxiety), GSE-6 (general self- 
efficacy), Beck Hopelessness Scale, FATS (frequency 
of adaptive behaviors and thoughts related to 
therapeutic change), 
Engagement 
Satisfaction with 
intervention and 
aspects of 
experience 
Intervention appeal: PFS (satisfaction with SSI), CEQ 
(intervention credibility and expectations of impact), 
insight experience, frequency of using one’s list of 
popular online content links, insight experience, star 
rating (program quality) 
Screening 
Inclusion in main 
analyses and 
reimbursement 
ULS-3 (loneliness), PHQ-4 (depression/anxiety) 
 
 
 
 
 
 
 
 
 
 
 
 

University of California, Irvine  
Study Information Sheet 
 
Brief Internet Interventions to Support Mental Health    
 
Lead Researcher  
Benjamin Kaveladze  
Psychological Science 
Phone: (949) 342-6253  
Email: bkavelad@uci.edu 
 
Faculty Sponsor  
Dr. Stephen Schueller  
Psychological Science 
Phone:(949) 824-3850  
Email: s.schueller@uci.edu 
 
• 
Please read the information below and ask questions about anything that you do 
not understand. A researcher listed above will be available to answer your 
questions. 
 
• 
You are being asked to participate in a research study. Participation in this study is 
voluntary. You may choose to skip a question or a study procedure. You may refuse 
to participate or discontinue your involvement at any time without penalty or loss of 
benefits. You are free to withdraw from this study at any time. If you decide to 
withdraw from this study, you should notify the research team immediately. 
 
• 
You are eligible to participate in this study if you are at least 16 years of age and fluent 
in English, and if you have access to the internet, a computer or smartphone, and an 
email address for the next 8 weeks. You may only complete the study once. 
 

• 
You are being asked to participate in a research study to examine if different self-
directed online programs can help people to overcome loneliness. The programs 
include written information based on evidence from research on well-being, as well as 
brief exercises. The programs can be completed on a computer, tablet, or smartphone. 
 
• 
All research data collected will be stored securely and confidentially in the 
encrypted servers of the cloud-based survey platform Qualtrics. When the data is 
downloaded for analysis, it will be stored in an encrypted format on a locked 
computer. 
 
• 
The research procedures involve answering some anonymous questions about yourself 
and your feelings, then completing an online program, and then giving your feedback 
on the program and answering more questions about yourself. 4 weeks and 8 weeks 
after you complete the initial survey, we will e-mail you with a few more questions to 
see what impact the program may have had. In total, this should take about 1 hour and 
10 minutes broken up into 3 sessions or 2 hours and 10 minutes broken up into 5 
sessions, depending on the intervention to which you are randomly assigned 
[highlighted portion only present in study 1]. At the end of the study, we will give you 
access to all the interventions we have available. 
 
• 
Possible risks/discomforts associated with the study are as follow: feeling distress at 
some of the questions we ask about your life, finding the content of the program 
upsetting, and a potential breach of confidentiality if our data are compromised 
(although we take extensive measures to avoid such a breach).  
 
• 
There are no direct benefits from participation in the study. However, this study may 
teach us how to make more helpful online interventions for loneliness and other mental 
health-related struggles. If a program is found to be useful, we will do our best to make 
it available to as many people as we can.  
 
• 
If you do not wish to participate in this study but still want to try the program, please 

email the research team and we will be happy to send it to you.  
 
• 
You will receive a $10 gift card to Amazon.com after you complete the 8-week follow-
up study, provided you completed the first two parts of the study and have not 
completed the study before. [only appears for those who meet criteria to be paid] 
 
• 
There is no cost to you for your participation in this study.  
 
• 
The research team, authorized UCI personnel, and regulatory entities may have access 
to your study records to protect your safety and welfare. While the research team will 
make every effort to keep your personal information confidential, it is possible that an 
unauthorized person might see it. We cannot guarantee total privacy.  
 
• 
We will use the information you provide us to conduct our study. Once the study is 
done, we plan to share a completely anonymous version of this information publicly 
online so that other researchers can also use it for their studies. The information we 
share will be de-identified, meaning it will not contain any private identifiable 
information that could reveal it came from you. We will not ask you for additional 
permission to share this de- identified information.  
 
• 
[If participating through the UCI Subject Pool] You will receive extra course credit for 
an eligible course through the UCI Social Ecology human subjects’ pool. You will only 
receive course credit for the first part of the study. You will receive a ½ unit of course 
credit for each ½ hour of participation in this study. Total amount of credit you may 
earn for this study is 1 credit. The course instructor offering extra course credit for 
participation in research must provide alternatives to earn extra course credit. The 
alternative assignment must require equal or less time and effort for the amount of 
extra credit that can be earned through participation in research. 
 
• 
In accordance with UC Office of the President policy, information will be retained for 
10 years after the end of the calendar year in which the research is completed.  

 
• 
If, during the course of this study, significant new information becomes available that 
may relate to your willingness to continue to participate, this information will be 
provided to you by the research team listed at the top of the form.  
 
• 
If you have any comments, concerns, or questions regarding the conduct of this 
research please contact the researchers listed at the top of this form.   
 
• 
It is important that you promptly tell the researchers if you believe that you have been 
injured because of taking part in this study. You can call the lead researcher at the 
number listed at the top of this form. 
 
• 
Stephen Schueller has financial interests in Headspace, Inc., a company with interests 
related to this study. Stephen Schueller is a consultant for the company and receives 
stock options, which is in addition to their salary from the University of California, 
Irvine. The nature of this financial interest and the design of the study have been 
reviewed by the UCI Conflict of Interest Oversight Committee (COIOC). The COIOC 
has determined that the researcher’s financial interests are appropriately managed as to 
avoid compromising the quality or reliability of the study and furthermore, the UCI 
Institutional Review Board has determined that appropriate safeguards are in place to 
avoid adversely affecting your safety and welfare. 
 
• 
Please contact the UCI Institutional Review Board by phone, (949) 824-6662, by e-
mail at IRB@research.uci.edu or at 160 Aldrich Hall, Irvine, CA 92697-7600 if you 
are unable to reach the researchers listed at the top of the form and have general 
questions; have concerns or complaints about the research; have questions about your 
rights as a research subject; or have general comments or suggestions. What is an 
IRB? An Institutional Review Board (IRB) is a committee made up of scientists and 
non-scientists. The IRB’s role is to protect the rights and welfare of human subjects 
involved in research. The IRB also assures that the research complies with applicable 
regulations, laws, and institutional policies. 

Study 2: Trial of a popular online content-based single-session intervention for 
psychological distress 
 
Procedure 
Sample recruitment 
Researchers will use social media and the University of California, Irvine Social Ecology subject 
pool to recruit a diverse online sample of English-speaking participants aged 16+ with access to 
the internet and a computer, tablet, or smartphone. To advertise the study on social media, 
researchers will use both paid and unpaid posts on several social media platforms, potentially 
including Instagram, Reddit, Tumblr, Twitter, Facebook, LinkedIn, 4chan, and others. 
Researchers may also use the online subject recruitment platform Researchmatch. Finally, 
researchers may collaborate with Healthy Gamer, a web-based mental wellness company, to 
share information about the study with Healthy Gamer’s online community Discord and 
Subreddit, reaching 20,000-25,000 people. 
To ensure a sufficient number of participants for analysis who are experiencing distress, 
if one's score on the PHQ-4 meets the study criterion for “struggling with distress”, one will be 
offered reimbursement of a $10 Amazon gift card that researchers will email them once they 
complete the final follow-up 8 weeks later, provided they completed all 4 evaluations (pre-test, 
post-test, 4-week, and 8-week). Participants will be considered to be struggling with distress if 
their score on either the depression or anxiety subscale is at least 3 (range 0-6), as these scores 
are suggestive of a depressive or anxiety disorder (Kroenke et al., 2009). 
Participants who do not meet the criterion for "struggling with distress" will still have the 
opportunity to complete the study without compensation. They will also be allowed to access 
the ABC Project and Blu Surfer programs directly if they prefer to not participate in the study. 
Participants are only allowed to complete the study once. 
Those who participate through the UC Irvine subject pool will receive 1 hour of 
participation credit for completing the first part of the study. If UC Irvine subject pool 
participants meet criteria for struggling with distress, they will also be offered a $10 gift card for 
completing all parts of the study. 
 
Study process 

Participants will first complete the PHQ-4 measure, which will determine if they are offered a 
$10 gift card for their participation in the study in the study information portion. After 
participants read the study information and agree to participate, they complete baseline self-
report measures of mental health outcomes and demographics. Next, they complete the 
intervention (see Materials section below for intervention information). After completing the 
intervention, participants are offered access to a browser and PDF version of the intervention 
they completed to keep and/or share with others. They are also referred to other online resources 
for people struggling with mental health. Next, they complete a few brief measures of hope, 
agency, meaning in life, and engagement with the intervention platform. 4 and 8 weeks later, 
they receive links via email to complete follow-up measures of mental health outcomes and 
engagement. At the end of the 8-week follow-up, participants are given access to all three 
conditions in PDF and browser forms. The entire study is conducted using the secure online 
experience management platform Qualtrics and several anti-fraud / bot measures are included in 
the survey. All contact between researchers and participants is asynchronous. 
 
Materials 
The experimental condition will be the Blu Surfer Program, a 25-minute SSI that the research 
team developed centered on popular online content relevant to mental health. In the SSI, each 
user is first asked to select the kinds of support they would like to view. Next, the user is asked to 
explore a library of annotated popular online content (filtered by the kinds of support they chose) 
and to select the content they find personally valuable. Researchers crowdsourced this library by 
asking people from a range of online communities and an undergraduate psychology course 
lecture to contribute one piece of popular online content that they believed could help someone 
overcome a difficult time or mental health challenge in their life. Finally, the SSI provides the 
user a list of the content they selected to keep and draw from, build on, or share with friends. 
The active comparison condition will be the ABC (Action Brings Change) Project 
(TEAM Lab Version). The original ABC Project is a 30-minute SSI for teens based on 
behavioral activation that was found to be efficacious for reducing depression, anxiety, 
hopelessness, and self-hate and increasing perceived control and agency in youth (link to SSI: 
https://osf.io/ch2tg/, license: http://creativecommons.org/licenses/by-nc-sa/4.0/). The TEAM Lab 
version slightly modifies the intervention to make it more relevant to both teens and adults.  

The third condition is online help-seeking as usual, which aims to emulate how one 
might find support using the internet on one’s own. In this condition, participants are asked to 
browse the internet for 25 minutes to find popular online content for overcoming personal 
problems and create an annotated list of links to the content they find personally useful. This 
condition is similar to the “self-study” condition in another study that found self-study and a 
video intervention equally increased mental health knowledge (Yamaguchi et al., 2019). The 
survey platform provides text entry boxes to help each participants create their guide and then 
provides a text version of their guide for them to keep. 
 
Hypotheses 
Researchers’ primary hypothesis is that participants randomized to the popular online 
content SSI (The Blu Surfer Program) will have greater reductions in distress between baseline 
and 8-week follow-up than those randomized to the researcher-created SSI (the ABC Project). 
As secondary hypotheses, researchers predict greater improvements in loneliness, general 
self-efficacy, hopelessness, meaning and purpose in life, and adaptive actions and thoughts in 
the popular online content SSI condition relative to the researcher-created SSI. Researchers also 
predict that participants randomized to the popular online content SSI will report greater 
reductions in distress than those randomized to the online help-seeking as usual condition. 
 
Exploratory hypotheses:  
1. Demographics and baseline mental health characteristics predict one’s willingness to 
choose a certain kind of content. 
2. The kinds of content one chooses predict change in distress between baseline and 8-week 
follow-up.  
3. Participants add more pieces of content to their guide when provided an annotated library 
to choose from VS when they have to seek out their own content without a library. 
4. Participants are more satisfied with their mental health resource guides after completing it 
when they choose content from an annotated library than when they choose it themselves. 
5. Participants are more likely to view their mental health resource guide after 8 weeks 
when they choose content from a curated library than when they select it all on their own. 
6. Younger participants will experience a greater reduction in distress after the popular 

online content interventions than older participants at 8 weeks. 
7. The annotated library condition (blu surfer) is more likely to lead to an insight experience 
than online help-seeking as usual. 
8. Having an insight experience during an intervention is associated with greater 
improvement in distress at 8 weeks. 
9. Higher baseline general self-efficacy is associated with greater improvement in distress at 
8 weeks. 
10. Engagement with the SSI predicts greater improvement in distress at 8 weeks. 
 
Analysis Plan  
The main DV is the change between baseline and 8-week follow-up in distress, measured 
by the total score of the Depression and Stress Scale - 9 item version. 
Main analysis  
The main analysis compares change in distress from baseline to 8-week follow-up across 
experimental conditions. Researchers will use a mixed-effects model predicting distress with 
condition, measurement time point (1 = baseline, 2 = 4-week follow-up, 3= 8-week follow-up), 
and the 2-way interaction between condition and measurement time point as IVs and a 
participant identifier as a random intercept. Using the “lme4” package in R, this analysis will 
take the following form: 
 
lmer(distress ~ time*condition + (1|study_id)) 
 
Secondary analysis  
Researchers will re-run the main analysis separately for each secondary outcome, although the 
timepoints measured by the “time” variable will differ across outcomes (see table 3).  
 
Inclusion criteria 
Main analyses will be limited to only those participants who 1) met criteria for struggling with 
distress, 2) were randomized to a condition (ie, those who consented to participate and completed 
the baseline measures), 3) did not indicate that they did not participate seriously at the end of the 
study, 4) completed the study only once, and 5) were not clearly bots, responding mischievously, 

or speeding past most of the content. Researchers will examine whether a response is fraudulent 
on a case-by-case basis, taking the suspicious factors mentioned in study 1’s inclusion criteria 
into account. 
Missing data handling 
Although researchers will take efforts to avoid missing data (e.g., a financial incentive for 
completing the study and reminder emails), it is not necessary to impute missing data before 
performing a longitudinal mixed-model analysis (Twisk et al., 2013). Participants with missing 
data at some assessments will be included in the analysis and results will be modeled on the basis 
of the available data. For all analyses, the standard p < 0.05 criterion will be used to determine 
statistical significance. 
To test for faulty randomization, researchers will use chi-squared tests on demographic 
and outcome variables with Yates's continuity correction and t-tests with Welch's approximation 
of degrees of freedom and assuming unequal variance. If any faulty randomization effects are 
found, researchers will adjust for them using the confounding variable as a covariate in model 
testing and will consider them in interpretation of results. These tests will also be used to check 
for differential dropout. 
 
Additional analyses 
1. Duplicate the main analysis with the subset of participants who did not meet criteria for 
struggling with distress and thereby participated in the study without being compensated. 
2. Duplicate the main analysis comparing the change in distress from baseline to week 4 
follow-up rather than to week 8. 
3. Duplicate the main analysis with the subset of participants who completed the week 8 
follow-up. 
4. Duplicate the main analysis but with the following covariates included in the mixed-
effects model: gender (multiple choice), age (continuous), ethnicity (categorical), sexual 
orientation (categorical), education level (categorical), socioeconomic status 
(continuous), and if participants completed the study on a computer, tablet, or phone. 
5. Use a one-way ANOVA to test between-group differences in distress at 8-week follow-
up. If the omnibus ANOVA produces a significant F-score, researchers will conduct two 
planned contrasts to compare the experimental condition to each other condition, and will 

compute Cohen’s ds of these comparisons. Researchers will run the same ANOVA on the 
4-week follow-up data and at post-intervention for outcomes collected then. Researchers 
will also conduct these ANOVAs on secondary outcomes. 
6. Calculate Cohen’s d effect sizes reflecting within-group intervention effects on distress 
for each condition from baseline to 4-week and 8-week follow-ups. 
7. Run four logistic regression models with dropout as the outcome baseline measures as 
predictors to determine if they predict which participants drop out either before 
completing the intervention, before the post-intervention questions, or before the 4- and 
8-week follow-ups. 
8. Duplicate the main analysis including data from suspected bots/scammers. 
 
Additional analyses testing exploratory hypotheses 
1. To test if demographic and baseline mental health characteristics predict one’s 
willingness to choose a certain kind of content, researchers will conduct a separate 
multiple logistic regression for each kind of content one can choose in the content 
selection portion of the blu surfer program. Researchers define a “kind” of content as a 
topic (eg, emotional support, motivation, personal experience) or format (eg, video, 
image, article, activity). As demographic predictors, researchers will include age, gender, 
ethnicity, country, education level, employment status, socio-economic status, and having 
received professional mental health support in the past. As baseline mental health 
predictors, researchers will include distress, self-efficacy, hopelessness, meaning and 
purpose in life, adaptive actions and thought, and loneliness. 
2. To test if the kinds of content one chooses predict change in distress over 8 weeks, 
researchers will conduct an MLM regression with each kind (defined as in the above 
analysis) as a binary predictor, study timepoint as another predictor, and the interaction 
between selected kind and timepoint as predictors, as well as a participant identifier as a 
random intercept.  
3. To test how many pieces of content one adds when they pull content from an annotated 
library vs when they find it independently online, researchers will conduct a t-test of 
numbers of content added to one’s guide across the “blu surfer” and “online help seeking 
as usual” conditions. 

4. To test if participants are more satisfied in the blu surfer condition than the online help 
seeking as usual condition, researchers will conduct a t-test comparing mean satisfaction 
across those conditions. 
5. To test if participants are more likely to view their mental health resource guide when 
they choose content from a curated library than when they select it all on their own, 
researchers will conduct a t-test comparing the mean number of times participants viewed 
their guide across the blu surfer and online help-seeking as usual conditions at 8 weeks 
after creating the guide. 
6. To evaluate how age moderates change in distress over 8 weeks among participants 
completing popular online content interventions, researchers will conduct a mixed effects 
model with distress as the outcome, age, timepoint, and their interaction as predictors, 
and a participant identifier as a random intercept. 
7. To compare the frequency of insight experiences across the blu-surfer and online help-
seeking as usual conditions, researchers will conduct a Chi-squared test. 
8. To test how having an insight experience moderates the intervention’s impact on change 
in distress over 8 weeks, researchers will run a mixed effects model with distress as the 
outcome, insight experience, measurement time point, and their interaction as predictors, 
and a participant identifier nested within experimental condition as a random intercept. 
9. To test how baseline self-efficacy moderates the impact of the intervention on change in 
distress over 8 weeks, researchers will run a mixed effects model with distress as the 
outcome, baseline self-efficacy, measurement time point, and their interaction as 
predictors, and a participant identifier nested within experimental condition as a random 
intercept. 
10. Researchers will test how engagement predicts improvement in distress at 8 weeks in 
two ways. First, researchers will run a mixed effects model with distress as the outcome, 
number of resources viewed from the list at week 8, measurement time point, and their 
interaction as predictors, and a participant identifier nested within experimental 
condition as a random intercept. Second, researchers will run a mixed effects model 
with distress as the outcome, number of pieces of content added to one’s list, 
measurement time point, and their interaction as predictors, and a participant identifier 

nested within experimental condition as a random intercept. These analyses will be 
limited to participants in the two popular online content conditions. 
Tables describing measures 
 
Table 3. Full list of primary, secondary, and screening variables collected during the study, 
including when during the trial they were assessed. 
Measure (Primary, Secondary, Other) Baseline 
Post- 
Intervention 
4-week 
follow-up 
8-week 
follow-up 
Depression Anxiety and Stress Scale – 9 
item total score (primary) 
X 
 
X 
X 
3-item UCLA Loneliness Scale 
(secondary)  
X  
X 
X 
Short form of the General Self-Efficacy 
Scale (secondary) 
X 
X 
X 
X 
Beck Hopelessness Scale – 4-item 
(secondary) 
X 
X 
X 
X 
Frequency of actions and thoughts scale 
(secondary) 
X 
 
X 
X 
PROMIS Meaning and Purpose in Life – 
4-item (secondary  
X  
X 
X 
Program Feedback Scale (secondary) 
 
X 
 
 
Credibility and Expectancy Questionnaire 
(secondary) 
 
X 
 
 
Insight experience (secondary) 
 
X 
 
 
Star rating (secondary)  
X 
 
 
Number of times one viewed a resource 
from one’s list of popular online content 
(secondary) 
 
 
X 
X 
Patient Health Questionnaire – 4-item 
(screening) 
X 
 
 
 
 

 
Table 4. Measures organized by dimension 
Dimension  
Application to project 
Study Measurement(s) 
Effectiveness Improvements in 
symptoms and 
functioning 
Primary: DASS-9 (depression, anxiety, and stress; 
used in this study as a proxy for psychological 
distress) 
Secondary: ULS-3, GSE-6 (general self-efficacy), 
Beck Hopelessness Scale, Meaning and Purpose in 
Life measure, FATS (frequency of adaptive behaviors 
and thoughts related to therapeutic change) 
Engagement 
Satisfaction with 
intervention and 
aspects of experience 
PFS (satisfaction with SSI), CEQ (intervention 
credibility and expectations of impact), insight 
experience, star rating (program quality), number of 
times viewed resource 
Screening 
Inclusion in main 
analyses and 
reimbursement 
PHQ-4 (depression/anxiety, used as proxy for 
psychological distress)  
 
 
 
 
 
 
 
 
 
 
 
 

Study 3: Trial of a popular online content-based single-session intervention for loneliness 
 
Procedure 
 
Researchers will use social media and the University of California, Irvine Social Ecology 
subject pool to recruit a diverse online sample of English-speaking participants aged 16+ with 
access to the internet and a computer, tablet, or smartphone. To advertise the study on social 
media, researchers will use both paid and unpaid posts on several social media platforms, 
potentially including Instagram, Reddit, Tumblr, Twitter, Facebook, LinkedIn, 4chan, and others. 
Researchers may also use the online subject recruitment platform Researchmatch. Finally, 
researchers may collaborate with Healthy Gamer, a web-based mental wellness company, to 
share information about the study with Healthy Gamer’s online community Discord and 
Subreddit, reaching 20,000-25,000 people. 
To ensure a sufficient number of participants for analysis who are experiencing 
loneliness, if one's score on the ULS-3 meets the study criterion for “struggling with loneliness”, 
one will be offered reimbursement of a $10 Amazon gift card that researchers will email them 
once they complete the final follow-up 8 weeks later, provided they completed all 4 evaluations 
(pre-test, post-test, 4-week, and 8-week). Researchers will consider an individual to be struggling 
with loneliness if their score on the ULS-3 is 6 or higher and they answer affirmatively that their 
loneliness is causing them distress (Steptoe et al., 2013). 
Participants who do not meet the criteria for "struggling with loneliness" will still have 
the opportunity to complete the study without compensation. They will also be allowed to access 
the two non-control conditions (see Materials for more information) directly if they prefer to not 
participate in the study. Participants are only allowed to complete the study once. 
Those who participate through the UC Irvine subject pool will receive 1 hour of 
participation credit for completing the first part of the study. If UC Irvine subject pool 
participants meet criteria for struggling with distress, they will also be offered a $10 gift card for 
completing all parts of the study.  
Study process 
After participants complete the loneliness screen, read the study information, and agree to 
participate, they complete baseline self-report measures of mental health outcomes and 
demographics. Next, they complete an intervention (see materials for intervention information). 

After completing the intervention, participants are offered access to a browser and PDF version 
of the intervention they completed to keep and/or share with others. They are also referred to 
other online resources for people struggling with mental health. Next, they complete a few brief 
measures of hope, agency, meaning in life, and engagement with the intervention platform. 4 and 
8 weeks later, they will receive links via email to complete follow-up measures of mental health 
outcomes and engagement. At the end of the 8-week follow-up, participants are given access to 
all three conditions in PDF and browser forms. The entire study will be conducted using the 
secure online experience management platform Qualtrics and several anti-fraud / bot measures 
will be included in the survey. 
Materials 
The Overcoming Loneliness program is a 25-minute SSI for loneliness that the 
researchers created by adapting Käll et al.’s 9-week therapist-supported iCBT for 
loneliness (Käll, Shafran, et al., 2020). Käll et al.’s iCBT focuses on person-level 
psychological factors contributing to the maintenance of loneliness; it aims to help 
challenge maladaptive thoughts and encourage social behaviors (more information on Käll 
et al.'s iCBT available at www.iterapi.se/sites/solus/) (Käll, Shafran, et al., 2020). It is 
text-based and self-guided. Following Käll et al.’s iCBT, the loneliness intervention 
includes content relating to understanding loneliness, setting goals and values, testing 
thoughts and behaviors, making plans, gaining social confidence, and overcoming 
barriers. Broadly, the intervention uses behavioral activation, cognitive restructuring, and 
behavioral experimentation strategies. The loneliness interventions also pull some content 
from Jessica Schleider et al.'s single-session mental health interventions 
(http://www.schleiderlab.org) and take inspiration from their interventions’ aesthetics and 
their strategy of using Qualtrics to develop and provide the intervention. 
The Lonely Blu Surfer program is a 25-minute SSI that the research team developed 
centered on popular online content relevant to mental health. In the SSI, each user is first 
asked to select the kinds of support they would like to view. Next, the user is asked to 
explore a library of annotated popular online content (filtered by the kinds of support they 
chose) and to select the content they find personally valuable. The user is asked to explore 
a library of annotated popular online content and to select the content they find personally 
valuable.  Researchers crowdsourced this library by asking people from online 

communities and an undergraduate psychology course lecture to contribute one piece of 
popular online content that they believed could help someone struggling with loneliness. 
Researchers also pulled some content relevant to loneliness from the original Blu Surfer 
program’s library, which included content for psychological distress and other mental 
health challenges. Finally, the SSI provides the user a list of the content they selected to 
keep and draw from, build on, or share with friends. 
In the control condition, participants complete a supportive therapy SSI, called “The 
Sharing Feelings Project”, of roughly equivalent length and style to the loneliness SSI (J. L. 
Schleider & Weisz, 2019). Previous work showed this control to be slightly less efficacious in 
improving mental health outcomes in adolescents than a growth mindset SSI and a behavioral 
activation SSI (J. L. Schleider et al., 2022; J. Schleider & Weisz, 2018). The supportive 
therapy SSI “encourages participants to express emotions to close others”, but “does not teach 
specific skills”. (J. L. Schleider et al., 2022; J. Schleider & Weisz, 2018). Researchers 
borrowed the supportive therapy SSI from the Lab from Scalable Mental health and slightly 
modified it to make it more applicable to both teens and adults, as it was originally designed 
for teens only (link to SSI: https://osf.tvio/u4axs/, license: 
https://creativecommons.org/licenses/by-nc-sa/4.0/) (J. L. Schleider & Weisz, 2019). 
Hypotheses 
Researchers’ primary hypothesis is that participants randomized to the popular online 
content loneliness SSI (the Lonely Blu Surfer program) will have greater reductions in loneliness 
between baseline and 8-week follow-up than those who are randomized to the active control 
condition (the Sharing Feelings program). 
As secondary hypotheses, researchers predict greater improvements in distress, general 
self-efficacy, hopelessness, meaning and purpose in life, and adaptive actions and thoughts in 
the popular online content loneliness SSI condition relative to the researcher created SSI 
condition. Researchers also predict that participants who are randomized to the popular online 
content SSI (the Lonely Blu Surfer program) will report greater reductions in loneliness than 
those randomized to the researcher-created loneliness SSI (the Overcoming Loneliness program). 
 
Exploratory hypotheses 

1. Demographics and baseline mental health characteristics predict one’s willingness to 
choose a certain kind of content. 
2. The kinds of content one chooses predict change in loneliness between baseline and 8-
week follow-up.  
3. Younger participants will experience a greater reduction in loneliness after the popular 
online content interventions than older participants at 8 weeks. 
4. The popular online content SSI condition (the Lonely Blu Surfer program) is more likely 
to lead to an insight experience than the researcher-created SSI (Overcoming Loneliness 
program) condition. 
5. Having an insight experience during an intervention is associated with greater 
improvement in loneliness at 8 weeks. 
6. Higher baseline general self-efficacy is associated with greater improvement in loneliness 
at 8 weeks. 
7. Engagement with the researcher-created SSI predicts greater improvement in loneliness 
at 8 weeks. 
8. Engagement with the popular online content SSI predicts greater improvement in 
loneliness at 8 weeks. 
 
Analysis Plan 
Main analysis 
The main DV is change in loneliness (UCLA Loneliness Scale - 20 item Version 3) from 
baseline to 8-week follow-up. The main analysis compares change in loneliness from baseline to 
8-week follow-up across experimental conditions. Researchers will use a mixed-effects model 
predicting distress with condition, measurement time point (1 = baseline, 2 = 4-week follow-up, 
3= 8-week follow-up), and the 2-way interaction between condition and measurement time point 
as IVs and a participant identifier as a random intercept. Using the “lme4” package in R, these 
analyses will take the following form: 
 
lmer(loneliness ~ time*condition + (1|study_id)) 
 
Secondary analysis 

Researchers will re-run the main analysis separately for each secondary outcome, although the 
timepoints measured by the “time” variable will differ across outcomes (see table 5).  
 
Inclusion criteria 
Main analyses will be limited to only those participants who 1) met criteria for struggling with 
loneliness, 2) were randomized to a condition (ie, those who consented to participate and 
completed the baseline measures), 3) did not indicate that they did not participate seriously at the 
end of the study, 4) completed the study only once, and 5) were not clearly bots, responding 
mischievously, or speeding past most of the content. Researchers will examine whether a 
response is fraudulent on a case-by-case basis, taking the suspicious factors mentioned in study 
1’s inclusion criteria into account. 
 
Missing data handling  
Although researchers will take efforts to avoid missing data (e.g., a financial incentive for 
completing the study and reminder emails), it is not necessary to impute missing data before 
performing a longitudinal mixed-model analysis (Twisk et al., 2013). Participants with missing 
data at some assessments will be included in the analysis and results will be modeled on the basis 
of the available data. For all analyses, the standard p < 0.05 criterion will be used to determine 
statistical significance. 
To test for faulty randomization, researchers will use chi-squared tests on demographic 
and outcome variables with Yates's continuity correction and t-tests with Welch's approximation 
of degrees of freedom and assuming unequal variance. If any faulty randomization effects are 
found, researchers will adjust for them using the confounding variable as a covariate in model 
testing and will consider them in interpretation of results. These tests will also be used to check 
for differential dropout. 
 
Additional analyses 
1. Duplicate the main analysis with the subset of participants who did not meet criteria for 
struggling with loneliness and thereby participated in the study without being 
compensated. 

2. Duplicate the main analysis comparing the change in loneliness from baseline to week 4 
follow-up rather than to week 8. 
3. Duplicate the main analysis with the subset of participants who completed the week 8 
follow-up. 
4. Duplicate the main analysis but with the following covariates included in the mixed-
effects model: gender (multiple choice), age (continuous), ethnicity (categorical), sexual 
orientation (categorical), education level (categorical), socioeconomic status 
(continuous), and if participants completed the study on a computer, tablet, or phone. 
5. Use a one-way ANOVA to test between-group differences in loneliness at 8-week 
follow-up. If the omnibus ANOVA produces a significant F-score, researchers will 
conduct two planned contrasts to compare the experimental condition to each other 
condition, and will compute Cohen’s ds of these comparisons. Researchers will run the 
same ANOVA on the 4-week follow-up data and at post-intervention for outcomes 
collected then. Researchers will also conduct these ANOVAs on secondary outcomes. 
6. Calculate Cohen’s d effect sizes reflecting within-group intervention effects on loneliness 
for each condition from baseline to 4-week and 8-week follow-ups. 
7. Researchers will run four logistic regression models with dropout as the outcome baseline 
measures as predictors to determine if they predict which participants drop out either 
before completing the intervention, before the post-intervention questions, or before the 
4- and 8-week follow-ups. 
8. Duplicate the main analysis including data from suspected bots/scammers. 
 
Additional analyses testing exploratory hypotheses 
1. To test if demographic and baseline mental health characteristics predict one’s 
willingness to choose a certain kind of content, researchers will conduct a separate 
multiple logistic regression for each kind of content one can choose in the content 
selection portion of the lonely blu surfer program. Researchers define a “kind” of content 
as a topic (eg, emotional support, motivation, personal experience) or format (eg, video, 
image, article, activity). As demographic predictors, researchers will include age, gender, 
ethnicity, country, education level, employment status, socio-economic status, and having 
received professional mental health support. As baseline mental health predictors, 

researchers will include loneliness, distress, self-efficacy, hopelessness, meaning and 
purpose in life, and adaptive actions and thought. 
2. To test if the kinds of content one chooses predict change in loneliness, researchers will 
conduct an MLM regression with each kind (defined as in the above analysis) as a binary 
predictor, study timepoint as another predictor, and the interaction between each kind and 
timepoint as predictors, as well as a participant identifier as a random intercept. This 
analysis will be limited to participants in the popular online content condition. 
3. To evaluate how age moderates change in loneliness among participants in popular online 
content interventions across 8 weeks, researchers will conduct a mixed effects model 
with loneliness as the outcome, age, timepoint, and their interaction as predictors, and a 
participant identifier as a random intercept. 
4. To compare the frequency of insight experiences across the popular online content 
(lonely blu surfer) and researcher-created content (overcoming loneliness) conditions, 
researchers will conduct a Chi-squared test. 
5. To test how having an insight experience moderates the impact of the intervention on 
change in loneliness across 8 weeks, researchers will run a mixed effects model with 
loneliness as the outcome, insight experience, measurement time point, and their 
interaction as predictors, and a participant identifier nested within experimental condition 
as a random intercept. 
6. To test how baseline self-efficacy moderates the impact of the intervention on change in 
loneliness across 8 weeks, researchers will run a mixed effects model with loneliness as 
the outcome, baseline self-efficacy, measurement time point, and their interaction as 
predictors, and a participant identifier nested within experimental condition as a random 
intercept. 
 
 
 
 
 
 
 

Tables describing measures 
 
Table 5. Full list of primary, secondary, and screening variables collected during the study, 
including when during the trial they were assessed. 
 
Measure (Primary, Secondary, Other) Baseline 
Post- 
Intervention 
4-week 
follow-up 
8-week 
follow-up 
UCLA Loneliness Scale (primary) 
X  
X 
X 
Depression Anxiety and Stress Scale – 
Short Form (secondary) 
X  
X 
X 
3-item Mini Social Phobia Inventory 
(secondary) 
X  
X 
X 
Short form of the General Self-Efficacy 
Scale (secondary) 
X X 
X 
X 
Beck Hopelessness Scale – 4-item 
(secondary) 
X X 
X 
X 
Frequency of actions and thoughts scale 
(secondary) 
X  
X 
X 
PROMIS Meaning and Purpose in Life – 
4-item (secondary)   
X  
X 
X 
Program Feedback Scale (secondary) 
 X 
 
 
Credibility and Expectancy Questionnaire 
(secondary) 
 X 
 
 
Insight experience (secondary) 
 X 
 
 
Star rating (secondary) 
 X 
 
 
MacArthur Scale of Subjective Social 
Status – Youth Version (secondary) 
X  
 
 
UCLA loneliness scale – 3 item 
(screening) 
X  
 
 

Patient Health Questionnaire – 4-item  
(screening)   
X  
 
 
 
Table 2. Measures organized by dimension 
 
Dimension 
Application to 
project 
Study Measurement(s) 
Effectiveness Improvements in 
symptoms and 
functioning 
Primary: ULS-20 (loneliness) 
Secondary: DASS-9 (depression, anxiety, and stress), 
Mini-SPIN (social anxiety), GSE-6 (general self- 
efficacy), Beck Hopelessness Scale, FATS (frequency 
of adaptive behaviors and thoughts related to 
therapeutic change), Meaning and Purpose in Life-4  
Engagement 
Satisfaction with 
intervention and 
aspects of 
experience 
Intervention appeal: PFS (satisfaction with SSI), CEQ 
(intervention credibility and expectations of impact), 
insight experience, frequency of using one’s list of 
popular online content links, insight experience, star 
rating (program quality) 
Screening 
Inclusion in main 
analyses and 
reimbursement 
ULS-3 (loneliness), PHQ-4 (depression/anxiety)