JMIR MHEALTH AND UHEALTH

Chaves et al

Original Paper

The Effectiveness of a Cell Phone eHealth App in Changing
Knowledge, Stigmatizing Attitudes, and Intention to Seek Help
Associated With Obsessive-Compulsive Disorder: Pilot
Questionnaire Study
Antonio Chaves1, PhD; Sandra Arnáez2, PhD; Gemma García-Soriano2, PhD
1

Departamento de Orientación Educativa, IES Cid Campeador, Conselleria d'Educació, Cultura i Esport, Valencia, Spain

2

Departamento de Personalidad, Evaluación y Tratamientos Psicológicos, Universitat de València, Valencia, Spain

Corresponding Author:
Gemma García-Soriano, PhD
Departamento de Personalidad, Evaluación y Tratamientos Psicológicos
Universitat de València
Avda Blasco Ibáñez, 21
Valencia, 46010
Spain
Phone: 34 963983389
Email: gemma.garcia@uv.es

Abstract
Background: Obsessive-compulsive disorder (OCD) is a disabling disorder associated with high interference in people’s lives.
However, patients with OCD either do not seek help or delay seeking help. Research suggests that this could be explained bypoor mental health literacy about the disorder and the associated stigma.
Objective: This study aims to evaluate the feasibility, acceptability, and preliminary effectiveness of a mental health mobile
app, esTOCma, developed to improve knowledge about OCD and its treatment, increase help-seeking intention, and reduce
stigmatizing attitudes and social distance associated with OCD.
Methods: We used preintervention, postintervention, and 3-month follow-up assessments in this single-arm pilot intervention.
Overall, 90 participants were recruited from the community using the snowball sampling method. We used esTOCma to defeat
the “stigma monster” over the course of 10 missions. The participants completed the sociodemographic information andObsessive-Compulsive Inventory–Revised at preassessment and an acceptability questionnaire at postassessment. All othermeasures were completed at the preassessment, postassessment, and 3-month follow-up (ie, the Spanish Mental Illness Stigma
Attribution Questionnaire–27, the General Help-Seeking Questionnaire, the Social Distance Scale, and the Mental Health Literacy
Questionnaire).
Results: Of the 90 participants from the community that were assessed for eligibility, 86% (n=78) were allocated to intervention.
Of these 78 participants, 79% (n=62) completed the game and answered the postintervention assessment (completer group).
Overall, 69% (43/62) of the participants also completed the 3-month follow-up assessment. The participants completing the study
were older (P=.003) and had a higher baseline knowledge of OCD (P=.05). The participants took an average of 13.64 (SD 10.50)
days to complete the intervention, including the pre- and postassessments. The participants spent an average of 4.56 (SD 3.33)
days completing the 10 missions included in the app. Each mission took a mean of between 2 (SD 3.01) and 9.35 (SD 3.06)
minutes. The app was rated as useful or very useful by the vast majority of participants 90% (56/62). Moreover, 90% (56/62) of
the participants reported that they had learned or learned a lot, and 98% (61/62) of the participants reported that they would
recommend the app to a friend. Repeated measures ANOVA (43/62, 69%) showed that after the intervention participants showed
an increased knowledge of mental health and intention to seek help as well as fewer stigmatizing attitudes and less social distance.
Conclusions: Preliminary data show that esTOCma is a feasible and acceptable app, and after completing its 10 missions, there
is an increase in the understanding of OCD and help-seeking intention along with a decrease in the social stigma and social
distance associated with OCD that lasts for at least 3 months. The results support the potential of technology-based interventions
to increase the intention to seek help and reduce the stigma associated with OCD. A larger, community-controlled study is also
recommended.
https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 1
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Chaves et al(JMIR Mhealth Uhealth 2024;12:e48027) doi: 10.2196/48027
KEYWORDS
obsessive-compulsive disorder; OCD; mental health literacy; stigma; app; help-seeking intention; seek help; mobile phone

Introduction
BackgroundObsessive-compulsive disorder (OCD) is a clinicallyheterogeneous condition characterized by obsessions,compulsions or both that cause clinically significant levels of
distress or functional impairment [1,2]. At present, effective
treatments exist for OCD [3,4]; however, many people delay
seeking treatment [5,6], and this may contribute to its chronic
course [7,8]. Research suggests that this delay in seeking
treatment by patients with OCD may be explained by social
stigma and poor mental health literacy (MHL) about the disorder[5,9,10].
To date, many interventions have proved to be effective in
reducing stigma associated with mental disorders, producing
knowledge, and achieving attitudinal improvements [11-13].
Data show that contact and education strategies produce
small-to-medium short-term reductions in stigmatizing attitudes,
and there is limited evidence on long-term effectiveness [14,15].
A few of these interventions have benefited from the advantagesof new technologies (eg, video games and electronic contact
with patients), showing medium effects on reducing social
stigma and suggesting that new technologies are a useful tool
to decrease stigmatizing attitudes toward mental disorders, atleast in the young population (the mean age of participants
ranged between 15.7 and 24 years) [16]. However, none of thesestudies have examined the benefits of using app-based
interventions. Furthermore, most of these interventions havefocused on reducing the stigma of mental disorders such as
schizophrenia [17-19], depression [20-22] or bipolar disorder[23,24].
In the case of OCD, programs are scarce, and most proposals
have studied the impact of the educational mechanism [25-28],
showing a reduction in stigmatizing attitudes and an
improvement in participants’ MHL. Furthermore, a proposal
has reported a significant reduction in social stigma and social
distance using an indirect contact strategy through a video of apatient with OCD and a family member talking about theirexperience with the problem [29]. However, none of theseinterventions are based on innovative technologies, and only 1of them includes >1 intervention strategy (ie, contact andeducation) [29]. In this context, as a response to the limited
interventions focused on OCD and the need to bridge the gap
between the onset of symptoms and seeking help, a gamified
mental health app named esTOCma has been developed [30](refer to the Methods section). In this way, the intervention willtake advantage of mobile mental health interventions as low-costtools that are available 24 hours a day for a large number ofpeople [31,32], in addition to including gaming benefits suchas providing immediate feedback, motivating users to achievegoals, and being easy to use [33].

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

Objective and Hypothesis
The aim of this study is to explore the feasibility, acceptability,and preliminary effectiveness of the beta version of the eHealthmobile app esTOCma. Regarding the effectiveness, and basedon the reviewed literature on interventions to reduce stigma andincrease knowledge of OCD [25-29] and other mental disorders[11,14,16], we hypothesize that the esTOCma intervention will(1) improve knowledge associated with OCD and its treatment,(2) decrease stigmatizing attitudes and social distance, and (3)increase intention to seek help. Moreover, we hypothesize thatchanges will be maintained at the 3-month follow-up assessment.

Methods
Study Design
This study was a single-arm pilot intervention with 3 measuresat the pretest, posttest, and 3-month follow-up. Data werecollected from people residing in Spain.

Participants and Procedure
Data were collected from a convenience sample. Participants
were recruited from the general community and university
setting by snowball sampling after providing relevant
information via face-to-face classes and inviting the participants
to share information about the study with their acquaintances.
The inclusion criteria for this study were as follows: (1) being
aged >18 years, (2) residing in Spain, (3) owning a smartphone,and (4) self-reporting not having an OCD diagnosis. Interested
participants were invited to participate in a study consisting of
downloading an app (esTOCma) in an Android Package Kit
file format, playing with it, and completing a set ofquestionnaires before and after using the app. The participants
performed all the tasks individually at home and at their own
pace at the time they deemed most convenient. Furthermore,they chose the rate at which they completed the game, although
the app recommends completing 1 mission a day.
The participants signing the informed consent form were given
an identification number automatically generated by the app.
The data from the questionnaires and the game were matchedwith the personal player ID. The participants were randomizedby the app using a sampling without replacement method to 1of 6 vignettes describing a person with obsessive-compulsive(OC) symptoms from 1 out of 6 types of content (ie, aggressionor harm; sexual; religious, blasphemous, or immoral;
contamination or washing; doubts or checking; or superstition,
symmetry, or order). The vignettes consisted of descriptions ofpatients presenting with OC symptoms, with similar severity.
Furthermore, interference and impairment in quality of life were
described. The descriptions were based on real clinical cases[34]. All of them meet the diagnosis criteria following the
Diagnostic and Statistical Manual of Mental Disorders, Fifth
Edition, and the labels of obsessions or compulsions were
avoided. All the described patients were referred to as A; sex
JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 2
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
was not specified, and they were middle aged. Furthermore, alldescriptions included between 166 and 175 words.
After reading the assigned vignette, the participants were asked
by the app to complete the preintervention measures. Most ofthem (ie, Spanish Mental Illness Stigma Attribution
Questionnaire–27 [AQ-27-E)], General Help-Seeking
Questionnaire [GHSQ], Social Distance Scale [SDS], and MHLquestionnaire) were answered in reference to the assigned
vignette. Only after completing the preassessment were theparticipants able to begin playing. The participants whocompleted the 10 missions and finished the game were asked
through the app to complete the postintervention assessment
and the 3-month follow-up assessment. To encourage theparticipants to complete the game, they were given entries to a
prize draw for a voucher to spend on the web.

Chaves et al

Intervention Program
The esTOCma beta version is a serious game whose content
and videos were developed by Doctor of Philosophy–level
clinical psychologists who are experts on OCD, together witha usability expert. A professional designer developed the graphic
elements, and a computer engineer developed the app. After
testing multiple prototypes, this version was developed(Multimedia Appendix 1).
During the game, participants are asked to fight against the
OCD “stigma monster” with their knowledge by accomplishing
10 missions and freeing the 10 characters who are affected by
the esTOCma monster, a creature that feeds on false beliefs andmisinformation in society [35] (Figure 1). The participants are
guided through the game by a woman who describes herself as
an OCD expert.

Figure 1. The missions, set along a mountain road.

The game is organized into three different intervention
mechanisms: (1) psychoeducation (general information about
OCD, OCD heterogeneity, OCD dimensionality, OCD cognitivemodel, evidence-based treatments, and options for seeking help
in OCD), which includes 5 missions, and 1 of them—mission
https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

3—includes a video explaining the OCD cognitive model (1min); (2) indirect contact (including 6 videos of approximately
2 to 3 minutes of 3 patients diagnosed with OCD who talk about
their own experience with OCD: symptom description,interference, how long it took them to seek help, their experience
JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 3
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
in disclosing that they had a disorder, and experience withpsychotherapy), which is organized into 2 missions; and (3)cognitive restructuring to replace dysfunctional beliefs related
to rejection toward people with OCD, variables involved in
OCD development, and treatment options and effectiveness,
which includes 3 missions.
The missions are organized as follows: (1) the expert describes
the objective of the mission and introduces the character to be
freed, (2) the expert presents activities associated with themission (between 6 and 8 activities, mostly consisting of readinga text and answering questions about it), and (3) the freed
character appears. During the game, users receive basic
internet-based rewards (ie, a key to free the character), together
with a message of reinforcement and a video of the character
actually being freed from esTOCma. At the end of the game,
there is new visual reinforcement through a video in which all
the characters are freed from the monster. Moreover, users
receive reinforcement through a diploma that certifies them as
OCD experts. A further description of the game can be foundin the study by Chaves et al [35].

Measures
The participants completed the sociodemographic information
and Obsessive-Compulsive Inventory–Revised at preassessment
and an acceptability questionnaire at postassessment. All othermeasures were completed at the preassessment, postassessment,
and 3-month follow-up assessment.

Sociodemographic Variables
The sociodemographic variables include gender, age,educational level, level of information and communications
technology (ICT) knowledge (from 1 little or none to 5 expert
level), and the question of whether they have an OCD diagnosis.

Acceptability Questionnaire
It consisted of 3 questions developed ad hoc to assess theacceptability of the esTOCma app. The survey included 2
questions related to usefulness (ie, “Did you find the app
useful?” [from very useful to not useful at all] and “Did you
learn from the app?” [from I learned a lot to I learned nothing])
and 1 question related to satisfaction with the app (ie, “Would
you recommend this app to a friend?” [from a lot to not at all]).
The questions were multiple-choice questions with 4alternatives.

AQ-27-E Measures
This measures the social stigma associated with a vignette
describing a person showing OC symptoms through 27 items
rated on a Likert-type scale ranging from 1 to 9 [36,37]. It
includes 9 subscales with 3 items: responsibility, pity, anger,
dangerousness, fear, no help, coercion, segregation, andavoidance. Higher scores indicate higher social stigma. In thisstudy, the AQ-27-E scales showed acceptable to excellent
internal consistency across subscales (from 0.70 [anger, 3-monthfollow-up] to 91 [fear, 3-month follow-up]), except for
responsibility (ranging from 0.40 [postassessment] to 0.63
[preassessment]) and pity (ranging from 0.36 [preassessment]
to 0.52 [3-month follow-up]).

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

Chaves et al

GHSQ Measures
This measures the intention to seek help from 10 different
sources with regard to the specific content described in avignette describing a person showing OC symptoms [38,39].
Participants rated 10 items regarding their help-seeking
intentions if they were experiencing from symptoms similar to
those described in the vignette on a 7-point Likert-type scaleranging from 1 (extremely unlikely) to 7 (extremely likely).
Scores were calculated by summing up the items and dividing
by 10. Higher scores indicate a higher intention to seek help.
In this, the GHSQ showed acceptable internal consistency atthe different assessment points (Cronbach α ranging from 0.68
[preassessment] to 0.75 [postassessment]).

SDS Measures
This assesses an individual’s willingness to interact with anindividual with a mental disorder described in a vignette across
7 different situations on a 4-point Likert scale from 0 (definitely
willing) to 3 (definitely unwilling) [40]. The total score has been
calculated by adding the scores and dividing by the 7 items,with higher scores indicating a greater preference for social
distance. In this, the SDS showed between good and excellent
internal consistency at all assessment points; Cronbach α ranged
from 0.86 at postassessment to 0.93 at the 3-month follow-up.

MHL QuestionnaireThis is an instrument developed for this study based on previous
studies [25,41] and assesses 1 of the components defined by
Kutcher et al [42] as MHL but only referring to OCD: theunderstanding of OCD and its treatment. Part 1 has 4
multiple-choice questions including between 2 and 7 alternative
answers, only 1 of which is correct. The questions refer to theassigned vignette (person A) and evaluate the following
dimensions: (1) problem recognition (ie, “What happens to A
is cause for concern?”; response alternatives: yes and no), (2)OCD identification (ie, “What do you think might be happening
to A?” This question includes 7 response alternatives: family
problems, adjustment problems, anxiety disorder, generalized
anxiety disorder, schizophrenia, OCD, and depression), (3)perception of causality, and (4) effective treatment option. Part
2 has four multiple-choice questions with 3 alternative answers,
only 1 of which is correct, and refers to participants’ generalknowledge of OCD: (1) identification of OCD as a mental
disorder (ie, OCD is [a] a learning disorder, [b] a mental
disorder, or [c] a set of manias); (2) definition of obsession; (3)definition of a compulsion; and (4) role played by compulsions
and other control strategies in the maintenance of obsessions.
A total score has been calculated as the sum of the correct
answers, thus ranging from 0 to 8.

Obsessive-Compulsive Inventory–Revised Measures
This is an 18-item self-report questionnaire assessing distresscaused by OC symptoms and rated on a 5-point Likert scaleranging from 0 (not at all) to 4 (extremely) [43,44]. A total score
was calculated. The total score of the Obsessive-Compulsive
Inventory-Revised showed excellent internal consistency at allassessment points in this study (Cronbach α ranging from 0.89
[postassessment] to 0.90 [preassessment]).

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 4
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Statistical Analysis
Descriptive statistics (eg, means, SD, and frequencies orpercentages) were used to analyze demographic data, study
variables, and app use patterns. Chi-square and 1-tailed t tests
were used to test whether the groups (completers vsnoncompleters) had preexisting differences. The change in study
variables over time was determined by a repeated measures
ANOVA. Partial eta–squared was used to report the effect sizeof the intervention on the dependent measures. A mixed model
was used to determine whether the pattern of use of the app
affected its effectiveness. The within-participants factor wastime, and the between-participants factor was whether the user
followed the 1 mission per day recommendation. The statisticalsignificance level was set at P=.05. SPSS Statistics (version 26;
IBM Corp) was used for statistical analysis.

Ethical Considerations
All procedures described in the study have been approved by
the Human Research Ethics Committee of the University of

Chaves et al
Valencia, Spain (1276901). All study participants provided
informed consent before study enrollment.

Results
Feasibility
Recruitment
A total of 90 participants were enrolled in the study; they
downloaded the app and met the inclusion criteria. Of these 90
participants, 86% (78/90) were allocated to intervention and
13% (12/90) were excluded as they did not provide informed
consent or did not complete the preassessment intervention(Figure 2). Of these 78 participants, 20% (16/78) did notcomplete the game or the postassessment (noncompleter group)and 79% (62/78) completed the game and answered thepostintervention assessment (completer group). In the completergroup, 69% (43/62) of the participants completed the 3-monthfollow-up assessment, and the remaining 30% (19/43) of theparticipants were lost at the 3-month follow-up.

Figure 2. Participants’ flowchart.

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 5
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Chaves et al

Participants’ Characteristics
The completer group had a mean age of 36.74 years, ranging
from 18 to 71 years; were mostly women; and had university

studies and moderate ICT knowledge. Differences insociodemographic characteristics and preintervention assessment(baseline) between the completer group and the noncompletergroup were calculated (Table 1).

Table 1. Differences between the completers and noncompleters on demographic and study variables (N=90).
Variable or measure

Completers (n=62)

Noncompleters (n=16)

Chi-square (df)t test (df)

P valueAge (y), mean (SD)36.74 (14.41)

27.19 (9.26)

10.4 (2)

N/Aa

.003

Gender (women), n (%)

38 (61)7 (44)

2.0 (2)N/A.35

2.2 (2)N/A.31

2.8 (2)

N/A.58

Education level, n (%)
Primary

3 (5)

0 (0)

Secondary

11 (18)1 (6)University

48 (77)15 (94)

Knowledge of ICTb, n (%)

a

Little or none

1 (2)

0 (0)

Low

5 (8)

0 (0)

Moderate

35 (57)

10 (63)

Advanced

18 (29)4 (25)

Expert level

3 (5)2 (13)MHLc (totald), mean (SD)6.79 (1.50)

5.93 (1.61)

N/A

1.992 (76, 36.104).05

AQ-27-Ee (totalf), mean (SD)84.77 (24.81)

77.25 (25.52)

N/A

1.075 (76, 36.104).28

SDSg (totalh), mean (SD)1.00 (0.64)

0.78 (0.70)

N/A

1.207 (76, 36.104).23

GSHQi (totalj), mean (SD)

4.20 (0.92)

4.27 (0.83)

N/A

0.308 (76, 36.104).38

OCI-Rk (totall), mean (SD)18.17 (13.23)

20.06 (10.70)

N/A

0.526 (76, 36.104).60

N/A: not applicable.

b

ICT: information and communications technology.

c

MHL: Mental Health Literacy Questionnaire.

d

Total score ranging from 0 to 8.

e

AQ-27-E: Spanish Mental Illness Stigma Attribution Questionnaire–27.

f

Total scoring from 27 to 243.

g

SDS: Social Distance Scale.

hTotal score ranging from 0 to 3.

i

GHSQ: General Help-Seeking Questionnaire.

j

Total scoring from 1 to 7.

k

OCI-R: Obsessive-Compulsive Inventory–Revised.

l

Total scoring from 0 to 72.

Statistically significant differences were only observed in age,with the completer group being older, and MHL scores, whichwere higher in the completer group. In addition, in the completergroup, we explored the differences between the participants
who completed the 3-month assessment (43/62, 69%) and those
who did not (19/62, 31%). The only discernible difference
observed was in age (t60=2.167; P=.01), with participants whocompleted the follow-up assessment being older.
Of the 43 participants included in the effectiveness analyses,
most were women (n=28, 65%) with a mean age of 39.30 (SD14.58; range 21-71) years, with university-level education (n=35,81%), and with ICT knowledge between moderate (n=23, 54%)https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

and advanced (n=15, 35%; only 3 (7%) participants describedhaving between little or none and low ICT knowledge level).

App Use Pattern
An analysis of the app use pattern was conducted with thecompleter group. First, we explored the number of days thatelapsed from the preassessment to the postassessment. The
participants took a mean of 13.64 (SD 10.50; range 1-44) days
to complete the app (which includes having done thepreassessment, the 10 missions and the postassessment), witha mode of 1, that is, the most frequent pattern was conducting
the pre- and postassessments and missions of the app in 1 day.
Second, we analyzed the number of days the person spent
JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 6
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
performing the missions. The participants spent between 1 and
10 days performing the missions, with a mean of 4.56 (SD 3.33)
days and a mode of 1. In total, 30% (19/62) of the participants
completed the 10 missions in 1 day, whereas another 21%(13/62) of the participants spent between 9 and 10 days
completing the app, that is, approximately 1 session per day.

Chaves et al
Finally, we explored the minutes spent performing each mission,
first excluding the participants who stayed on 1 mission for >20
minutes, as we assumed that they left the app open without
using it. The data on the participants completing the missionsin <20 minutes is presented in Table 2.

Table 2. Time (in min) spent in completing each of the 10 missions for participants who complete the mission in <20 minutes.

aParticipants, n (%)aTime (min), mean (SD; range)

Mode (min)Mission 1

58 (94)3.93 (1.02; 1-6)

4

Mission 2

54 (87)4.44 (1.90; 2-10)

4Mission 3

35 (56)3.80 (1.37; 1-8)

3Mission 4

35 (56)2.06 (1.64; 0-10)2Mission 5

35 (56)5 (3.11; 2-20)

4Mission 6

56 (96)

8.71 (1.78; 7-20)

8Mission 7

29 (47)9.35 (3.06; 7-20)

8Mission 8

33 (53)3.85 (1.48; 2-8)

3Mission 9

58 (94)2 (3.01; 0-19)

0Mission 10

56 (90)2.09 (2.26; 0-9)0n (%) of participants completing the mission in ≤20 minutes.

The data are displayed in minutes.
Between the participants who completed missions in ≤20
minutes, and considering the mode, missions included in module
1 (ie, missions 1 to 5) took between 2 and 4 minutes, in module
2 (ie, missions 6 to 7) took 8 minutes, and in module 3 (ie,missions 8 to 10) took between <1 minute and 3 minutes.

Acceptability
After completing the app until the end of the game, mostparticipants (56/62, 90%) perceived the app as useful or very
useful. Moreover, 90% (56/62) of the participants indicated thatthey had learned or learned a lot, and 98% (61/62) of theparticipants indicated that they would recommend the app to afriend.

Preliminary Effectiveness: Differences Between Preand Postintervention Assessments and 3-Month
Follow-Up

follow-up on knowledge of OCD and its treatments (MHL
questionnaire), stigmatizing attitudes (AQ-27-E), social distance(SDS), and intention to seek help (GHSQ) associated with OCD(Table 3). The results showed statistically significant differences(P≤.05) in all variables with medium-to-large effect sizes, except
for the second part of the MHL questionnaire and the pity andcoercion subscales (AQ-27-E). In general, the results show thatusing esTOCma until the end of the game results in an increasein MHL and intention to seek treatment (GHSQ) and a decreasein stigmatizing attitudes (AQ-27-E) and social distance desire(SDS). Post hoc pairwise comparisons showed statisticallysignificant differences between preintervention and the other 2
assessment points (postintervention and 3-month follow-up).
No significant differences were observed between
postintervention and the 3-month follow-up in the variables
assessed, except for the MHL total score, in which follow-up
scores did not differ from pre- to postintervention.

Repeated measures ANOVA was conducted to examine
differences between pre- and postintervention and the 3-monthhttps://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 7
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Chaves et al

Table 3. Means (SDs) and repeated measures ANOVA on pre-, post-, and 3-month follow-up intervention scores (n=43).
ηp2bPretreatment,
mean (SD)

Posttreatment, mean 3 month follow-up,(SD)mean (SD)F test (df)aPart 1d3.32 (0.80)e3.72 (0.54)e3.62 (0.61)e6.523 (1.566, 65.769) .005

0.134Part 2f

3.58 (0.69)

3.72 (0.50)

3.77 (0.57)

1.896 (1.513, 63.526) .16

0.043

Total score

6.90 (1.34)e7.44 (0.93)e

7.39 (0.90)e5.754 (1.325, 55.662) .01

0.120

Responsibility

9.04 (4.07)e

6.97 (3.70)e7.30 (4.35)e8.103 (2, 84)

.001

0.162

Pity

17.30 (4.15)

18.62 (4.36)

18.20 (4.68)

1.984 (2, 84)

.14

0.045

Anger

8.23 (4.44)e

5.81 (3.42)e5.76 (3.19)e

10.554 (1.614,
67.813)

<.001

0.201

Dangerousness

7.11 (4.31)e4.72 (2.65)e5.37 (4.01)e

7.386 (2, 84)

<.001

0.150

Fear

6.39 (4.26)e

5.00 (3.72)e

4.23 (2.42)e6.489 (2, 84)

.002

0.134

No help

8.09 (4.68)e

5.69 (3.32)e

5.83 (4.05)e

9.320 (2, 84)

<.001

0.182

Coercion

12.00 (5.30)

11.18 (5.50)

10.93 (6.47)

0.983 (1.490, 62.601) .35

0.023

Segregation

5.16 (3.92)e

4.00 (1.96)e

4.04 (2.22)e

4.762 (1.555, 65.297) .01

0.102

Avoidance

11.88 (5.66)e

8.58 (6.35)e

7.76 (5.48)e

16.938 (2, 84)

<.001

0.287

SDSh

0.99 (0.58)e

0.64 (0.65)e0.59 (0.63)e

10.597 (2, 84)

<.001

0.201

GSHQi

4.33 (0.84)e

4.84 (0.86)e

4.58 (1.03)e

6.818 (2, 84)

.002

0.140

Variable or measure

P value

MHLc

AQ-27-Ega

Dfs were Greenhouse-Geisser corrected where appropriate.

bηp2:partial eta squared for within-subject contrasts (ANOVA).

c

MHL: Mental Health Literacy Questionnaire, total score ranging from 0 to 8.

dPart 1 scoring from 0 to 4.

eSignificant differences among groups (P≤.05).

f

Part 2 scoring from 0 to 4.

g

AQ-27-E: Spanish Mental Illness Stigma Attribution Questionnaire–27; subscales ranging from 3 to 27.

h

SDS: Social Distance Scale, ranging from 0 to 3.

i

GHSQ: General Help-Seeking Questionnaire, scoring from 1 to 7.

Finally, we examined whether the pattern of use of the app
influenced the effectiveness of the intervention with a 2 (group:
individuals who follow the recommendation of 1 mission per
day [ie, 9/10, % days] [12/N, %; percentile 75] vs individuals
who did it in another way [31/N, %])×3 (time: pre-, post-, andfollow-up assessments) repeated measures mixed ANOVA. The
results show that there was no significant group×time interaction(F26,16=0.838; P=.67). Univariate follow-up analyses also
indicated no significant group×time effect for any of themeasured variables (P>.05).

Discussion
Principal Findings
This study is the first to investigate the feasibility, acceptability,and effectiveness of a mobile health app designed to enhanceseveral variables related to OCD, which were identified by
Kutcher et al [42] as components of MHL: understanding OCD
and its treatments, decreasing stigma associated with OCD, andhttps://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderXenhancing help-seeking effectiveness. Our findings show that
esTOCma was feasible and acceptable and that after using it
until game completion, there was a positive change in thevariables of interest, which lasted for at least 3 months.
Of the participants allocated to the intervention, approximately
80% (78/90) completed the app, and of the who completed theintervention, approximately 70% (62/78) completed thefollow-up assessment. This adherence rate is similar to or higher
than that reported in other internet-based studies with self-help
interventions, with dropouts being one of the main challenges
of interventions with mental health apps [45-47].
The participants who completed the study were older and had
higher knowledge of OCD than those who began the study but
did not complete it. However, there were no differences inparticipants’ knowledge of new technologies, which suggeststhat the app is easy to use and does not require a significant
amount of knowledge to become involved in its use, although>50% (35/62) of the participants described themselves as having
JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 8
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
moderate ICT knowledge. Regarding the pattern of use, although
it was recommended that participants complete 1 mission per
day and participants were reminded of this recommendation
after finishing each mission, the most common pattern of use
was to complete the app, including pre- and postevaluations, in
1 day. In fact, only 21% (13/62) of the participants completing
the intervention followed the recommendation to perform 1
mission per day. It seems that it is more comfortable forparticipants to perform more missions per day; otherwise, they
forget to complete the mission the following day. In fact, theparticipants completed their participation in the study within arange of 1 to 44 days.
Regarding the time invested in each mission, although missions
were quick to complete and always took <10 minutes, theparticipants often left missions midway and continued later thesame day or days later. Furthermore, certain missions wereabandoned in the middle more frequently, suggesting that they
could be reformulated to make them more “attractive.” This
was the case for mission 7, which was interrupted by >50%
(31/62) of the participants, as well as for missions 3, 4, 5, and
8. The pattern of use and interruptions does not seem to be
associated with the intervention mechanism on which themission is based but perhaps with the content or the durationof the mission. If we analyze missions 1, 2, 6, 9, and 10, thosethat were carried out without interruption, we see that missions
1 and 2 are the initial ones and deal with content describing
obsessions and compulsions; mission 6 includes the first videos
that also describe symptomatology and interference, without
additional theoretical content to read; and missions 9 and 10(cognitive restructuring) are characterized by being very brief
and with less theoretical content than missions 1 to 5
(psychoeducation). It seems that the description ofsymptomatology, as well as more dynamic and shorter missions,
result in more attractive missions or at least in missions that
capture participants’ attention to a greater extent.
In general, the participants seemed satisfied with the app, as
approximately 100% (62/62) would recommend it to a friend
and >90% (56/62) consider it useful and that they have learned
about OCD.
Regarding the effectiveness of the app, the results show anincrease in OCD knowledge. Our data match those of theprevious interventions that have found increases in generalknowledge of OCD after offering written information about thedisorder [25,26]. The data are also consistent with othertechnology interventions that have increased MHL levels on
different mental health problems [48-50]. However, thedifferences were not statistically significant in those questions
that asked about OCD in general (eg, the definition of anobsession), that is, not referring to the description of a person
showing OC symptoms. This could be due to a ceiling effect,as the scores were already high in the preassessment evaluation.
In fact, they were higher among those participants who playedwith the app until the game was over in comparison with those
who did not finish the game. In this sense, our first hypothesis
was only partially supported.
Regarding the hypothesized decrease in stigmatizing attitudes,
the results support our hypothesis, as they suggest that after
https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

Chaves et alcompleting the 10 missions of the app, there was a decrease insome stereotypes or public attitudes, such as the perception of
OC symptoms as dangerous; emotional reactions of anger orfear toward people showing OC symptoms; discriminating
behaviors such as the intention of not helping, segregating, or
avoiding people showing OC symptoms; or the desire to
maintain social distance. Although a video-based intervention
decreased social distance desire [29] and 2 interventions
centered on reading the Diagnostic and Statistical Manual of
Mental Disorders, Fifth Edition diagnostic criteria for OCD
decreased negative attitudes about violent and sexual thoughts[28,51], other interventions consisting of reading information
on OCD (psychoeducation) showed small changes instigmatizing attitudes [25]. The results suggest that theesTOCma intervention, which includes psychoeducation but
also incorporates components such as contact, seems to change
stigmatizing attitudes to a greater extent, with medium-to-large
effect sizes.
Our data are also consistent with previous research also using
new technologies to reduce stigma associated with other mental
disorders that have reported a decrease in dangerousness, anger,
fear, segregation, and avoidance [15,52,53]; a decrease in stigma
as a general measure [53-55]; an increase in the help factor [52];or a decrease in social distance [52,56]. This is a remarkable
result, as traditional antistigma interventions not using
innovative technologies often report small-to-medium effectsizes [14,52]. Moreover, the results are based on a community
sample with an average age higher than those used in studies
reporting interventions using innovative technologies [16],suggesting that these types of interventions could also be usefulfor older people.
In addition to the changes in stigmatizing attitudes after using
esTOCma, there were no significant changes in 2 of the social
stigma dimensions measured by the AQ-27-E: pity and coercion.
Previous studies have also reported a lack of changes in pity
[57]. In fact, it has been suggested that pity should beconsidered, at least in some contexts, not to be a factor of stigma
but rather a reflection of compassion and the capacity toempathize with people with mental health problems [58], andresearch has shown associations between pity and the tendency
to help [37]. Regarding coercion, there is no significant change
after the intervention in the belief that people with OCD should
receive treatment, even if they refuse it (coercion). This isconsistent with previous interventions in the OCD field [25].
Finally, the results also support our third hypothesis, as there
was a large effect size improvement in the intention to seek help
when experiencing symptoms similar to those represented in
the different vignettes. Thus, increasing knowledge of OCD
and its treatment could have acted as a help-seeking facilitator[59,60]. To the best of our knowledge, there are no interventions
to improve help seeking associated with OCD, and thus, ourresults are of great relevance, as research shows that early help
seeking is associated with a better treatment response, earlier
remission of symptoms, and recovery from the disorder [9,61].
The data are consistent with other studies that, through mental
health apps [62] and other technology-based interventions, have
improved the intention to seek help for other mental healthproblems [63].
JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 9
(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Chaves et al

Moreover, our study showed a maintained effect in the study
variables at the 3-month follow-up assessment. This is especiallyrelevant as most of the interventions do not include a follow-up
assessment [16,50,63,64], and only 1 intervention on OCD has
included a follow-up assessment [29].

3 to 5, 7, and 8 should be revised, as although they were quick
to complete (between 3 and 8 min), the participants often leftthem midway and continued later. They could perhaps be
shortened or made more dynamic.

Limitations and Recommendations

The results suggest the potential for the use of this intervention
app and provide the basis for developing a larger randomized
controlled study to validate the use of esTOCma (version 1.0),
as has been proposed [35]. Importantly, we will also explore
differences in OCD types of content through a controlled study,
as research shows that there are differences in OCD recognitionand stigma considering the different OCD types of content[41,64,65]. Furthermore, it will be of interest to explore theeffectiveness of each intervention mechanism (ie,psychoeducation, indirect contact, and cognitive restructuring)
to understand whether there are differences between them. This
analysis would allow us to develop a new app that would
eliminate those mechanisms that are less effective or improvethem. Future studies should also examine the usefulness of thisapp in OCD cohorts, especially in the first stages of diagnosis,
and their families. Through esTOCma, people with OCD could
receive (or reinforce) cognitive-behavioral psychoeducation
and be assisted in their search for empirical-based treatments.
Furthermore, esTOCma could eliminate self-stigma, improving
their quality of life. In terms of family involvement, esTOCma
could assist them in comprehending and managing OCD
symptoms as well as support their relatives with OCD in seeking
effective treatment.

This study has limitations. As a pilot study, the study sample
was small and did not include a control group. There was alsoa considerable dropout rate that could be associated with thefact that participants forgot to complete the app (to do all 10
missions) as well as the large number of assessment
questionnaires included. Furthermore, considering that OCD isa heterogeneous disorder and that there are differences in stigma
and OCD recognition between different types of content[41,64,65], we decided to randomize participants to 6 vignettes
that represent OCD heterogeneity. However, this decision could
be considered as a limitation of the design of this study that
could affect the effectiveness of the data.
Despite these limitations, our results provide valuable
information for a broader implementation of the app. First, as
future lines of work, we propose making an effort to motivate
participants, given that higher adherence was associated withinitial knowledge of OCD or sensitivity toward mental healthproblems. Second, a potential strategy to attract a younger
audience, as adherence was also associated with older age, could
involve transforming the app into a serious game, aligning
esTOCma with the immersive experience of video games.
Actively engaging participants through gamification elements
could further contribute to increasing adherence. Looking ahead,
evolving esTOCma into a serious game with an adventure gametheme holds promise for future developments and for addressing
the stigma associated with OCD. Third, pop-ups and emails toremind participants to “play” could be incorporated, as thoseparticipants who completed the participation took up to 44 days
to finish an app recommended to be conducted in 10 days.
Fourth, given that the number of missions per day seems not to
influence the effectiveness of the app, the recommendation toplay 1 mission per day could be omitted, as this is not thepreferred pattern of use of participants. Finally, given that somemissions were completed with interruptions by approximately
half of the participants, it would be recommended to inform
users of how long each mission will last. Furthermore, missions

Future Research

Conclusions
Our findings show that esTOCma is a feasible and acceptable
app and that after completing its 10 missions, there is an increasein the understanding of OCD and help-seeking intention as well
as a decrease in the social stigma and social distance associatedwith OCD that lasts for at least 3 months. These changes might
result in less delay in seeking help and a better treatment
response to the problem and prognosis. Moreover, providing
mental health knowledge to the community population as to thenature and universality of intrusive thoughts may protect andprevent the general population from developing OCD and,furthermore, reduce the economic and personal costs associatedwith OCD.

Acknowledgments
Funding for this study (RTI2018-098349-B-I00 and PID2021-124409OB-I00) was provided by the Ministry of Science andInnovation—State Research Agency of Spain and by the European Union European Regional Development Fund: “A way ofmaking Europe.” None of the funders had any role in the design, preparation, review, or approval of the manuscript or in thedecision to submit the manuscript for publication.

Authors' Contributions
The work was conceptualized by AC and GG-S. The methodology and research, and the writing of the original draft were carriedout by AC, GG-S and SA. Formal analyses were developed by SA. The final manuscript was reviewed and edited by GG-S.

Conflicts of Interest
None declared.

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 10(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH

Chaves et alMultimedia Appendix 1
Screenshots of the esTOCma app with examples of the Psychoeducation mechanism.
[PDF File (Adobe PDF File), 960 KB-Multimedia Appendix 1]

References
1.
2.

3.
4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.
16.

17.
18.

19.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 5th Edition. Washington, DC.
American Psychiatric Association; 2013.
McKay D, Abramowitz JS, Calamari JE, Kyrios M, Radomsky A, Sookman D, et al. A critical evaluation ofobsessive-compulsive disorder subtypes: symptoms versus mechanisms. Clin Psychol Rev. Jul 2004;24(3):283-313. [doi:10.1016/j.cpr.2004.04.003] [Medline: 15245833]
Abramowitz JS, Blakey SM, Reuman L, Buchholz JL. New directions in the cognitive-behavioral treatment of OCD: theory,
research, and practice. Behav Ther. May 2018;49(3):311-322. [doi: 10.1016/j.beth.2017.09.002] [Medline: 29704962]
Olatunji BO, Davis ML, Powers MB, Smits JA. Cognitive-behavioral therapy for obsessive-compulsive disorder: ameta-analysis of treatment outcome and moderators. J Psychiatr Res. Jan 2013;47(1):33-41. [doi:10.1016/j.jpsychires.2012.08.020] [Medline: 22999486]
Belloch A, Del Valle G, Morillo C, Carrió C, Cabedo E. To seek advice or not to seek advice about the problem: thehelp-seeking dilemma for obsessive-compulsive disorder. Soc Psychiatry Psychiatr Epidemiol. Apr 31, 2009;44(4):257-264.
[doi: 10.1007/s00127-008-0423-0] [Medline: 18668187]
Pinto A, Mancebo MC, Eisen JL, Pagano ME, Rasmussen SA. The brown longitudinal obsessive compulsive study: clinicalfeatures and symptoms of the sample at intake. J Clin Psychiatry. May 15, 2006;67(5):703-711. [FREE Full text] [doi:10.4088/jcp.v67n0503] [Medline: 16841619]
Fullana MA, Mataix-Cols D, Caspi A, Harrington H, Grisham J, Moffitt TE, et al. Obsessions and compulsions in thecommunity: prevalence, interference, help-seeking, developmental stability, and co-occurring psychiatric conditions. Am
J Psychiatry. Mar 2009;166(3):329-336. [FREE Full text] [doi: 10.1176/appi.ajp.2008.08071006] [Medline: 19188283]
Subramaniam M, Abdin E, Vaingankar JA, Chong SA. Obsessive--compulsive disorder: prevalence, correlates, help-seeking
and quality of life in a multiracial Asian population. Soc Psychiatry Psychiatr Epidemiol. Dec 17, 2012;47(12):2035-2043.
[doi: 10.1007/s00127-012-0507-8] [Medline: 22526825]
García-Soriano G, Rufer M, Delsignore A, Weidt S. Factors associated with non-treatment or delayed treatment seeking
in OCD sufferers: a review of the literature. Psychiatry Res. Dec 15, 2014;220(1-2):1-10. [doi:10.1016/j.psychres.2014.07.009] [Medline: 25108591]
Williams MT, Domanico J, Marques L, Leblanc NJ, Turkheimer E. Barriers to treatment among African Americans withobsessive-compulsive disorder. J Anxiety Disord. May 2012;26(4):555-563. [FREE Full text] [doi:10.1016/j.janxdis.2012.02.009] [Medline: 22410094]
Griffiths KM, Carron-Arthur B, Parsons A, Reid R. Effectiveness of programs for reducing the stigma associated withmental disorders. A meta-analysis of randomized controlled trials. World Psychiatry. Jun 02, 2014;13(2):161-175. [FREEFull text] [doi: 10.1002/wps.20129] [Medline: 24890069]
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Cochrane Bias Methods Group, et al. Cochrane
Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. Oct 18,
2011;343(oct18 2):d5928. [FREE Full text] [doi: 10.1136/bmj.d5928] [Medline: 22008217]
Clement S, Lassman F, Barley E, Evans-Lacko S, Williams P, Yamaguchi S, et al. Mass media interventions for reducing
mental health-related stigma. Cochrane Database Syst Rev. Jul 23, 2013;2013(7):CD009453. [FREE Full text] [doi:10.1002/14651858.CD009453.pub2] [Medline: 23881731]
Morgan AJ, Reavley NJ, Ross A, Too LS, Jorm AF. Interventions to reduce stigma towards people with severe mental
illness: systematic review and meta-analysis. J Psychiatr Res. Aug 16, 2018;103(4):120-133. [FREE Full text] [doi:10.1016/j.jpsychires.2018.05.017] [Medline: 29843003]
Maunder RD, White FA. Intergroup contact and mental health stigma: a comparative effectiveness meta-analysis. Clin
Psychol Rev. Aug 2019;72:101749. [doi: 10.1016/j.cpr.2019.101749] [Medline: 31254936]
Rodríguez-Rivas ME, Cangas AJ, Cariola LA, Varela JJ, Valdebenito S. Innovative technology-based interventions to
reduce stigma toward people with mental illness: systematic review and meta-analysis. JMIR Serious Games. May 30,
2022;10(2):e35099. [FREE Full text] [doi: 10.2196/35099] [Medline: 35635744]
Jorm AF, Kitchener BA, Fischer JA, Cvetkovski S. Mental health first aid training by e-learning: a randomized controlled
trial. Aust N Z J Psychiatry. Dec 01, 2010;44(12):1072-1081. [doi: 10.3109/00048674.2010.516426] [Medline: 21070103]
Corrigan PW, River LP, Lundin RK, Penn DL, Uphoff-Wasowski K, Campion J, et al. Three strategies for changing
attributions about severe mental illness. Schizophr Bull. Jan 01, 2001;27(2):187-195. [doi:10.1093/oxfordjournals.schbul.a006865] [Medline: 11354586]
Penn DL, Chamberlin C, Mueser KT. The effects of a documentary film about schizophrenia on psychiatric stigma. Schizophr
Bull. Jan 01, 2003;29(2):383-391. [doi: 10.1093/oxfordjournals.schbul.a007012] [Medline: 14552511]

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 11(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
20.

21.

22.

23.

24.
25.

26.

27.
28.
29.

30.
31.

32.

33.
34.
35.

36.
37.

38.
39.

40.
41.

42.

Kiropoulos LA, Griffiths KM, Blashki G. Effects of a multilingual information website intervention on the levels ofdepression literacy and depression-related stigma in Greek-born and Italian-born immigrants living in Australia: a randomized
controlled trial. J Med Internet Res. Apr 19, 2011;13(2):e34. [FREE Full text] [doi: 10.2196/jmir.1527] [Medline: 21504872]
Gulliver A, Griffiths KM, Christensen H, Mackinnon A, Calear AL, Parsons A, et al. Internet-based interventions to promotemental health help-seeking in elite athletes: an exploratory randomized controlled trial. J Med Internet Res. Jun 29,
2012;14(3):e69. [FREE Full text] [doi: 10.2196/jmir.1864] [Medline: 22743352]
Farrer L, Christensen H, Griffiths KM, Mackinnon A. Web-based cognitive behavior therapy for depression with andwithout telephone tracking in a national helpline: secondary outcomes from a randomized controlled trial. J Med Internet
Res. Jun 27, 2012;14(3):e68. [FREE Full text] [doi: 10.2196/jmir.1859] [Medline: 22738715]
Patten SB, Remillard A, Phillips L, Modgill G, Szeto AC, Kassam A, et al. Effectiveness of contact-based education for
reducing mental illness-related stigma in pharmacy students. BMC Med Educ. Dec 05, 2012;12(1):120. [FREE Full text][doi: 10.1186/1472-6920-12-120] [Medline: 23216787]
Rusch LC, Kanter JW, Angelone AF, Ridley RC. The impact of in our own voice on stigma. Am J Psychiatr Rehabil. Oct13, 2008;11(4):373-389. [doi: 10.1080/15487760802397660]
Chaves A, Arnáez S, Roncero M, García-Soriano G. Teachers' knowledge and stigmatizing attitudes associated withobsessive-compulsive disorder: effectiveness of a brief educational intervention. Front Psychiatry. Jun 2, 2021;12:677567.
[FREE Full text] [doi: 10.3389/fpsyt.2021.677567] [Medline: 34149482]
Jassi AD, Kolvenbach S, Heyman I, Macleod T, Rose J, Diamond H. Increasing knowledge about obsessive compulsive
disorder and support for parents and schools: evaluation of initiatives. Health Educ J. Jul 28, 2016;75(5):600-609. [doi:10.1177/0017896915608513]
White SW, Sukhodolsky DG, Rains AL, Foster D, McGuire JF, Scahill L. J Dev Phys Disabil. Aug 3, 2010;23(1):5-14.
[doi: 10.1007/s10882-010-9209-x]
Snethen C, Warman DM. Effects of psychoeducation on attitudes towards individuals with pedophilic sexual intrusive
thoughts. J Obsessive Compuls Relat Disord. Oct 2018;19:92-98. [doi: 10.1016/j.jocrd.2018.10.001]
Gürbüz AA, Yorulmaz O, DURNA G. Reducing the social stigma associated with obsessive compulsive disorder: a controlled
trial of an intervention program in a Turkish community sample. J Evid Based Psychother. Sep 1, 2020;20(2):101-120.
[doi: 10.24193/jebp.2020.2.14]
Home page. EsTOCma. URL: http://estocma.com/ [accessed 2024-03-05]
Miralles I, Granell C, Díaz-Sanahuja L, Van Woensel W, Bretón-López J, Mira A, et al. Smartphone apps for the treatmentof mental disorders: systematic review. JMIR Mhealth Uhealth. Apr 02, 2020;8(4):e14897. [FREE Full text] [doi:10.2196/14897] [Medline: 32238332]
Lecomte T, Potvin S, Corbière M, Guay S, Samson C, Cloutier B, et al. Mobile apps for mental health issues: meta-review
of meta-analyses. JMIR Mhealth Uhealth. May 29, 2020;8(5):e17458. [FREE Full text] [doi: 10.2196/17458] [Medline:
32348289]
Fitzgerald M, Ratcliffe G. Serious games, gamification, and serious mental illness: a scoping review. Psychiatr Serv. Feb
01, 2020;71(2):170-183. [doi: 10.1176/appi.ps.201800567] [Medline: 31640521]
Belloch A, Cabedo E, Carrió C. TOC: Obsesiones y Compulsiones: Tratamiento Cognitivo del Trastorno Obsesivo
Compulsivo. Madrid, Spain. Alianza Editorial; 2011.
Chaves A, Arnáez S, Castilla D, Roncero M, García-Soriano G. Enhancing mental health literacy in obsessive-compulsive
disorder and reducing stigma via smartphone: a randomized controlled trial protocol. Internet Interv. Sep 2022;29:100560.
[FREE Full text] [doi: 10.1016/j.invent.2022.100560] [Medline: 35874968]
Corrigan P, Markowitz FE, Watson A, Rowan D, Kubiak MA. An attribution model of public discrimination towards
persons with mental illness. J Health Soc Behav. Jun 2003;44(2):162. [doi: 10.2307/1519806]
Muñoz M, Guillén AI, Pérez-Santos E, Corrigan PW. A structural equation modeling study of the Spanish Mental Illness
Stigma Attribution Questionnaire (AQ-27-E). Am J Orthopsychiatry. May 2015;85(3):243-249. [doi: 10.1037/ort0000059][Medline: 25985111]
Wilson CJ, Deane FP, Ciarrochi JV, Rickwood D. Measuring help seeking intentions: properties of the general help seeking
questionnaire. Can J Couns Psychother. 2005;39:15-28. [doi: 10.1037/t42876-000]
Pacheco del Castillo LA. Dominican college students’ experiences of distress, help-seeking and stigma. Western Michigan
University. 2017. URL: https://scholarworks.wmich.edu/cgi/viewcontent.cgi?article=4111&context=dissertations [accessed2024-03-05]
Link BG, Cullen FT, Frank J, Wozniak JF. The social rejection of former mental patients: understanding why labels matter.
Am J Sociol. May 1987;92(6):1461-1500. [doi: 10.1086/228672]
García-Soriano G, Roncero M. What do Spanish adolescents think about obsessive-compulsive disorder? Mental healthliteracy and stigma associated with symmetry/order and aggression-related symptoms. Psychiatry Res. Apr 2017;250:193-199.
[doi: 10.1016/j.psychres.2017.01.080] [Medline: 28161615]
Kutcher S, Bagnell A, Wei Y. Mental health literacy in secondary schools: a Canadian approach. Child Adolesc Psychiatr
Clin N Am. Apr 2015;24(2):233-244. [doi: 10.1016/j.chc.2014.11.007] [Medline: 25773321]

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

Chaves et al

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 12(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
43.
44.

45.
46.

47.

48.

49.

50.

51.
52.

53.

54.

55.

56.

57.

58.
59.

60.

61.
62.

63.

Foa EB, Huppert JD, Leiberg S, Langner R, Kichic R, Hajcak G, et al. The obsessive-complusive inventory: development
and validation of a short version. Psychol Assess. 2002;14(4):485-495. [doi: 10.1037//1040-3590.14.4.485]
Fullana MA, Tortella-Feliu M, Caseras X, Andión O, Torrubia R, Mataix-Cols D. Psychometric properties of the Spanish
version of the Obsessive-Compulsive Inventory--revised in a non-clinical sample. J Anxiety Disord. Jan 2005;19(8):893-903.
[doi: 10.1016/j.janxdis.2004.10.004] [Medline: 16243637]
Eysenbach G. The law of attrition. J Med Internet Res. Mar 31, 2005;7(1):e11. [FREE Full text] [doi: 10.2196/jmir.7.1.e11][Medline: 15829473]
Kelders SM, Van Gemert-Pijnen JE, Werkman A, Nijland N, Seydel ER. Effectiveness of a web-based intervention aimed
at healthy dietary and physical activity behavior: a randomized controlled trial about users and usage. J Med Internet Res.
Apr 14, 2011;13(2):e32. [FREE Full text] [doi: 10.2196/jmir.1624] [Medline: 21493191]
Ludden GD, van Rompay TJ, Kelders SM, van Gemert-Pijnen JE. How to increase reach and adherence of web-based
interventions: a design research viewpoint. J Med Internet Res. Jul 10, 2015;17(7):e172. [FREE Full text] [doi:10.2196/jmir.4201] [Medline: 26163456]
Brijnath B, Protheroe J, Mahtani KR, Antoniades J. Do web-based mental health literacy interventions improve the mental
health literacy of adult consumers? Results from a systematic review. J Med Internet Res. Jun 20, 2016;18(6):e165. [FREEFull text] [doi: 10.2196/jmir.5463] [Medline: 27323907]
Li TM, Chau M, Wong PW, Lai ES, Yip PS. Evaluation of a web-based social network electronic game in enhancing mental
health literacy for young people. J Med Internet Res. May 15, 2013;15(5):e80. [FREE Full text] [doi: 10.2196/jmir.2316][Medline: 23676714]
Tay JL, Tay YF, Klainin-Yobas P. Effectiveness of information and communication technologies interventions to increasemental health literacy: a systematic review. Early Interv Psychiatry. Dec 13, 2018;12(6):1024-1037. [doi: 10.1111/eip.12695][Medline: 29897166]
Warman DM, L. Phalen P, Martin JM. Impact of a brief education about mental illness on stigma of OCD and violent
thoughts. J Obsessive Compuls Relat Disord. Apr 2015;5:16-23. [doi: 10.1016/j.jocrd.2015.01.003]
Mullor D, Sayans-Jiménez P, Cangas AJ, Navarro N. Effect of a serious game (Stigma-Stop) on reducing stigma among
psychology students: a controlled study. Cyberpsychol Behav Soc Netw. Mar 2019;22(3):205-211. [doi:10.1089/cyber.2018.0172] [Medline: 30855993]
Rodríguez-Rivas ME, Cangas AJ, Fuentes-Olavarría D. Controlled study of the impact of a virtual program to reduce stigma
among university students toward people with mental disorders. Front Psychiatry. Feb 9, 2021;12:632252. [FREE Full text][doi: 10.3389/fpsyt.2021.632252] [Medline: 33633613]
Cangas AJ, Navarro N, Parra JM, Ojeda JJ, Cangas D, Piedra JA, et al. Stigma-stop: a serious game against the stigma
toward mental health in educational settings. Front Psychol. Aug 21, 2017;8:1385. [FREE Full text] [doi:10.3389/fpsyg.2017.01385] [Medline: 28878702]
Yuen AS, Mak WW. The effects of immersive virtual reality in reducing public stigma of mental illness in the university
population of Hong Kong: randomized controlled trial. J Med Internet Res. Jul 14, 2021;23(7):e23683. [FREE Full text][doi: 10.2196/23683] [Medline: 34259636]
Ferchaud A, Seibert J, Sellers N, Escobar Salazar N. Reducing mental health stigma through identification with video gameavatars with mental illness. Front Psychol. Sep 9, 2020;11:2240. [FREE Full text] [doi: 10.3389/fpsyg.2020.02240] [Medline:
33013585]
Maunder RD, White FA, Verrelli S. Modern avenues for intergroup contact: using E-contact and intergroup emotions to
reduce stereotyping and social distancing against people with schizophrenia. Group Process Intergr Relat. Sep 19,
2018;22(7):947-963. [doi: 10.1177/1368430218794873]
Saavedra J, Arias-Sánchez S, Corrigan P, López M. Assessing the factorial structure of the mental illness public stigma in
Spain. Disabil Rehabil. Sep 06, 2021;43(18):2656-2662. [doi: 10.1080/09638288.2019.1710769] [Medline: 31906732]
Aguirre Velasco A, Cruz IS, Billings J, Jimenez M, Rowe S. What are the barriers, facilitators and interventions targetinghelp-seeking behaviours for common mental health problems in adolescents? A systematic review. BMC Psychiatry. Jun
11, 2020;20(1):293. [FREE Full text] [doi: 10.1186/s12888-020-02659-0] [Medline: 32527236]
Picco L, Abdin E, Pang S, Vaingankar JA, Jeyagurunathan A, Chong SA, et al. Association between recognition andhelp-seeking preferences and stigma towards people with mental illness. Epidemiol Psychiatr Sci. Dec 08, 2016;27(1):84-93.
[doi: 10.1017/s2045796016000998]
Eisen JL, Pinto A, Mancebo MC, Dyck IR, Orlando ME, Rasmussen SA. A 2-year prospective follow-up study of thecourse of obsessive-compulsive disorder. J Clin Psychiatry. Aug 15, 2010;71(08):1033-1039. [doi: 10.4088/jcp.08m04806blu]Wiljer D, Shi J, Lo B, Sanches M, Hollenberg E, Johnson A, et al. Effects of a mobile and web app (Thought Spot) on
mental health help-seeking among college and university students: randomized controlled trial. J Med Internet Res. Oct30, 2020;22(10):e20790. [FREE Full text] [doi: 10.2196/20790] [Medline: 33124984]
Johnson JA, Sanghvi P, Mehrotra S. Technology-based interventions to improve help-seeking for mental health concerns:
a systematic review. Indian J Psychol Med. Jul 21, 2022;44(4):332-340. [FREE Full text] [doi: 10.1177/02537176211034578][Medline: 35949632]

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

Chaves et al

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 13(page number not for citation purposes)

JMIR MHEALTH AND UHEALTH
64.
65.

Chaves et al

Ponzini GT, Steinman SA. A systematic review of public stigma attributes and obsessive–compulsive disorder symptom
subtypes. Stig Health. Feb 2022;7(1):14-26. [doi: 10.1037/sah0000310]
Ponzini GT, Signorelli M, Claydon EA, Lilly C, Steinman SA. Stereotypes and OCD-symptom presentations: amixed-methods evaluation using male-character vignettes. Clinical Psychological Science. Sep 24, 2023. [doi:10.1177/21677026231192893]

Abbreviations
AQ-27-E: Spanish Mental Illness Stigma Attribution Questionnaire–27
GHSQ: General Help-Seeking Questionnaire
ICT: information and communications technology
MHL: mental health literacy
OC: obsessive-compulsive
OCD: obsessive-compulsive disorder
SDS: Social Distance Scale

Edited by L Buis; submitted 09.04.23; peer-reviewed by B Kalema, R McCarty, B Pascual, AM Ros; comments to author 25.08.23;
revised version received 17.12.23; accepted 01.02.24; published 29.03.24
Please cite as:
Chaves A, Arnáez S, García-Soriano G
The Effectiveness of a Cell Phone eHealth App in Changing Knowledge, Stigmatizing Attitudes, and Intention to Seek Help AssociatedWith Obsessive-Compulsive Disorder: Pilot Questionnaire StudyJMIR Mhealth Uhealth 2024;12:e48027
URL: https://mhealth.jmir.org/2024/1/e48027
doi: 10.2196/48027
PMID: 38551629©Antonio Chaves, Sandra Arnáez, Gemma García-Soriano. Originally published in JMIR mHealth and uHealth(https://mhealth.jmir.org), 29.03.2024. This is an open-access article distributed under the terms of the Creative Commons
Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete
bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license
information must be included.

https://mhealth.jmir.org/2024/1/e48027

XSL• FO
RenderX

JMIR Mhealth Uhealth 2024 | vol. 12 | e48027 | p. 14(page number not for citation purposes)