Evaluation of the Effectiveness of the FOCUS ADHD Mobile Health Platform in the Monitoring of Adults With Attention-Deficit/ Hyperactivity Disorder (ADHD)
Phase: NA
Conditions: ADHD
Sponsor: Hospital de Clinicas de Porto Alegre

Evaluation of the effectiveness of the FOCUS
ADHD App in monitoring adults with
attention-deficit/hyperactivity disorder
Luiz Roberto Carvalho1
, Letícia M. Haas2, Gregory Zeni3, Marcelo M. Victor3,
Stefania P. Techele3, Júlia Marrone Castanho3, Isabel Meneghetti Coimbra3,
Anthony de Freitas de Sousa3, Nathalia Ceretta3, Alia Garrudo4,
Eugenio Horacio Grevet3 and Luis Augusto Rohde3,5,6
1Institute of Psychiatry, Faculty of Medicine of the São Paulo, São Paulo, Brazil; 2Laboratorio Interdisciplinar de
Neurociencias Clinicas (LiNC), Department of Psychiatry, Federal University of São Paulo, São Paulo, Brazil;
3Attention Deficit/Hyperactivity Program and Developmental Psychiatry Program, Hospital de Clinicas de Porto
Alegre, Federal University of Rio Grande do Sul, Porto Alegre, Brazil; 4Institute of Mathematics and Statistics,
University of São Paulo, São Paulo, Brazil; 5Medical Council, UniEduK, São Paulo, Brazil and 6Center for Research and
Innovation in Mental Health, National Institute of Developmental Psychiatry, São Paulo, Brazil
Abstract
Background. The current investigation assessed a) the performance of the FOCUS ADHD
mobile health application (App) in increasing pharmacological treatment adherence and
improving patients’ knowledge of attention-deficit/hyperactivity disorder (ADHD) and b) the
impact of implementing a financial incentive for using the App (i.e., a discount on medication).
Methods. In a randomized, blind, parallel-group clinical trial, 73 adults diagnosed with ADHD
were allocated into three groups for 3 months: a) Pharmacological treatment as usual (TAU); b)
TAU and the App (App Group); and c) TAU and the App + a commercial discount on the
purchase of medication prescribed for ADHD treatment (App + Discount Group).
Results. There was no significant difference in mean treatment adherence between groups,
assessed as a medication possession ratio (MPR). However, the App + Discount Group exhibited
greater medication intake registrations compared with the App Group during the initial phase of
the trial. The financial discount also produced a 100% App adoption rate. App use did not
increase ADHD knowledge, though knowledge scores were high at baseline. The usability and
quality of the App were rated favorably.
Conclusions. The FOCUS ADHD App achieved a high adoption rate and positive evaluations by
users.UseoftheAppdidnotincreaseadherencetotreatmentasmeasuredbyMPR,but,forAppusers,
the addition of a financial incentive to use the App produced an increase in treatment adherence in
terms of medication intake registrations. The present results offer encouraging data for combining
incentives with mobile digital health solutions to positively impact treatment adherence in ADHD.
Introduction
Attention-deficit/hyperactivity disorder (ADHD) is a chronic condition characterized by
inattention, hyperactivity, and impulsivity, affecting approximately 5.3% of children and ado-
lescents globally [1]. ADHD symptoms in adults are associated with significant functional
impairments, such as lower quality of life, poor academic and work performance, lower income,
impairments in social life, and difficulty maintaining relationships [2]. The disorder is also
associated with an increased risk of substance use, crime, incarceration, and premature death due
to accidents or even suicide [2].
The ADHD diagnosis is based on clinical evaluation. As a chronic condition, patients with
ADHD should undergo periodic reassessments and receive long-term monitoring [3]. Effective
treatment is multi-faceted and includes psychoeducation, psychological treatment, and the use of
medication [4]. Psychostimulants are the first-line pharmacological agents for ADHD treatment
with high evidence of efficacy, tolerability, and safety [5]. The main treatment objective is to
maximize functionality [6] by reducing symptoms, which, in turn, will reduce negative outcomes
and improve quality of life [7, 8].
Despite data showing the efficacy and tolerability of available treatment interventions in
clinical trials, less than 20% of adults with ADHD are diagnosed [9], only 11% of those who meet
the diagnostic criteria have access to treatment [10], and only 12% of patients who start treatment
persist after 12 months [11]. Both lack of treatment and low adherence to treatment are associated
with significant negative outcomes to the patients and their ecosystem [12]. These factors result in
expenses that threaten the viability of the entire Health System [13, 14].
European Psychiatry
www.cambridge.org/epa
Research Article
Cite this article: Carvalho LR, Haas LM,
Zeni G, Victor MM, Techele SP, Marrone
Castanho J, Meneghetti Coimbra I, de Freitas
de Sousa A, Ceretta N, Garrudo A, Grevet EH,
Rohde LA (2023). Evaluation of the
effectiveness of the FOCUS ADHD App in
monitoring adults with attention-deficit/
hyperactivity disorder. European Psychiatry,
66(1), e53, 1–9
https://doi.org/10.1192/j.eurpsy.2023.2422
Received: 01 February 2023
Revised: 15 May 2023
Accepted: 16 May 2023
Keywords:
adherence; attention-deficit/hyperactivity
disorder; digital; discount; mobile app
Corresponding author:
Luiz Roberto. Carvalho;
Email:beto@soulbeegood.com.br
© The Author(s), 2023. Published by Cambridge
University Press on behalf of the European
Psychiatric Association. This is an Open Access
article, distributed under the terms of the
Creative Commons Attribution licence (http://
creativecommons.org/licenses/by/4.0), which
permits unrestricted re-use, distribution and
reproduction, provided the original article is
properly cited.
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

Treatment adherence
Treatment adherence is a complex phenomenon that depends on a
set of interrelated domains including patient symptomatology,
treatment plan, socioeconomic aspects, and the healthcare system
itself [15, 16]. Though treatment adherence in ADHD remains a
central challenge in patient follow-up [17], the variables affecting
adherence might be preventable or amenable by relatively simple
interventions [17–22].
Numerous strategies have been evaluated to improve treat-
ment adherence in mental health. The optimization of health
services design, involving specialized and/or multidisciplinary
management [18] can improve patient engagement and treatment
outcomes [19–21]. The implementation of patient education pro-
grams, nursing support lines, and behavioral therapy are examples
of effective interventions [22]. Psychoeducational interventions
aim to increase patients’ knowledge and acceptance of their health
condition [23]. Providing information about ADHD, its treat-
ment, and symptom management strategies improves treatment
satisfaction and treatment adherence, and enhances self-esteem
and quality of life [19, 24]. Among the studies conducted with
adults, positive results were obtained regarding inattention symp-
toms [25, 26], self-esteem [27, 28], anxiety and depressive symp-
toms [27, 29], and relationships with others [28, 29]. Recently, a
randomized controlled trial evaluating a smartphone-assisted
psychoeducation program showed a higher homework compli-
ance rate (93.9%) compared with a brochure-assisted psychoedu-
cation group (66.2%) [30].
Other interventions include contingency management strat-
egies (i.e., offering patients financial incentives to increase
adherence). In substance use disorders, the use of contingency
strategies has a positive impact on medication adherence, treatment
attendance, and treatment goal completion, including completing
homework, signing treatment plans, and reducing dysfunctional
behavior [31, 32]. This strategy has also been applied to other
mental health conditions [31], with positive and cost-effective
results on medication adherence [32].
Mobile digital health solutions for treatment adherence
Another promising strategy for increasing treatment adherence in
mental health disorders [33], including ADHD [22], involves the
use of technology-based interventions. Mobile digital health
(mHealth) solutions, delivered through smartphones and other
mobile devices, may help in patient education, monitoring, and
treatment adherence by improving patient communication with
their healthcare professionals and services [34]. In addition, the use
of mHealth solutions could facilitate the tracking of treatment
initiation, adherence, and efficacy [22]. Features such as reminders
for medication intake, medical appointments, or prescription
renewal are simple but potentially effective solutions for treatment
adherence support.
To date, few studies have been conducted on ADHD evaluating
the use of text messages (short message service [SMS]) to send
medication reminders as an intervention to improve pharmaco-
logical treatment adherence. Two such studies evaluated the impact
of SMS messages on promoting timely prescription refills in adult
patients undergoing stimulant medication treatment in an experi-
mental [35] and primary care [36] setting. Both studies showed that
significantly more patients in the SMS intervention groups (68%
and 81%, respectively) refilled their prescriptions in a timely man-
ner compared to the control groups (34% and 36%, respectively).
The primary care study also showed that the technological inter-
vention led to 96% of subjects engaging with the intervention,
successfully completing the 37 days of the SMS program. Finally,
additional strategies using digital medication adherence solutions
based on cognitive and behavioral approaches are currently being
evaluated, such as gamified interfaces and/or contingency manage-
ment strategies [37]. Interestingly, early evidence suggests that
combining multiple strategies including gamification, dosage
reminders, incentives, education, and social community interven-
tions appears to be a promising direction for managing medication
adherence [38].
Present study: the FOCUS ADHD app
The FOCUS ADHD App was developed to help monitor patients
with ADHD in the ADHD outpatient program (PRODAH) of the
Federal University of Rio Grande do Sul UFRGS) at the Hospital
de Clínicas de Porto Alegre (HCPA), Brazil. Through a Task
Manager contained within the App, FOCUS combines multiple
features and functionalities to support all aspects of the treatment
trajectory for ADHD patients including treatment information
registration (therapeutic plan including medication, dosage, fre-
quency, and other clinical information), patient management
support (daily pill reminders and adherence tracking, weekly
ADHD symptoms evaluation, and adverse events registering),
an option to add a user-customized reward system, and a robust
library of psychoeducational contents. Finally, the App allows the
integration of a treatment support network that can be shared
with family members and health professionals, at the user’s
discretion.
Although promising, many mHealth solutions lack robust sci-
entific clinical evidence of their efficacy [39, 40]. Thus, the main
aims of the current investigation were to a) assess the performance
of the FOCUS ADHD mobile health App in increasing treatment
adherence and improving patients’ knowledge of ADHD and b)
determine the impact of implementing a financial incentive as a
contingency strategy (i.e., providing a discount on the purchase of
psychostimulant medication).
The main hypotheses considered for this study were a) the use of
the FOCUS ADHD App during pharmacological treatment (App
Group) would increase treatment adherence compared with a
treatment as usual (TAU) only Control Group; b) providing a
significant discount on stimulant medication (25%), combined
with the use of the App (App + Discount Group) would increase
medication treatment adherence compared with the TAU-only
Control Group and the App Group; and c) users of the App (App
Group and App + Discount Group) would obtain higher scores on
an ADHD knowledge scale compared with the TAU-only Control
Group.
Methods
Participants and recruitment
A three-arm randomized clinical trial was conducted to monitor
adult participants with ADHD for 3 months. The participants were
recruited through a public call led by the PRODAH-Adult division
between June 2021 and October 2022.
The participants were evaluated according to the following
inclusion criteria: a) fulfill diagnostic criteria for ADHD accord-
ing to the DSM-5, b) age range between 18 and 45 years of age, c)
own a smartphone, d) have a clinical ASRS score at diagnosis of
2
Carvalho et al.
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

≥24, e) have at least a high school diploma, and f) accept the use of
pharmacological treatment for ADHD. The exclusion criteria for
the study were a) participants with any unstable or chronic
clinical disease without adequate treatment, as hypertension,
heart, kidney, or liver diseases; b) the presence of any significant
neurological disease (e.g., delirium, dementia, epilepsy, ALS,
TAND, multiple sclerosis, and stroke); and c) the presence of
unstable psychiatric comorbidities requiring immediate treat-
ment such as depression with risk of suicide, substance abuse/
dependence, anxiety, bipolar disorder, and current or past history
of psychosis.
Prior to the trial, participants were evaluated by a clinically
experienced psychiatrist using the Structured Clinical Interview
for DSM-IV and the Mini International Neuropsychiatric Inter-
view. Sociodemographic and medical history information was
also collected. Participants answered the ASRS-18 questionnaire
version validated in Brazil [41] for the evaluation of ADHD
symptoms.
Before starting the trial, participants were given a Baseline
clinical evaluation by a psychiatrist with extensive experience
with ADHD who confirmed the diagnosis based on a clinical
interview and the application of the Portuguese version of the
ADHD module of the K-SADS-E adapted for adults [42]. Parti-
cipants needed to meet all DSM-5 criteria for ADHD in adults
and have a minimum clinical ASRS score of 24 at baseline (see
the study flowchart in Figure 1, and the protocol workflow in
Supplementary Figure 1).
Sample description/characteristics
The eligible participants were then randomized into three groups
by an outside member of the research team who did not participate
in their follow-up. Participants randomized to the first group
followed pharmacological treatment as usual (TAU-only Control
Group). TAU consisted of clinical pharmacotherapy administered
by a psychiatrist experienced in the treatment of adult ADHD. The
second group, in addition to TAU, used the FOCUS ADHD App
(App Group). Participants in the third group followed TAU and
used the FOCUS ADHD App (like the App Group), but also
received a discount on their medication purchase, conditioned on
participating in at least 80% of the planned monthly medication
intakes as established by the treatment plan registered in the App
(App + Discount Group).
Clinical follow-up assessments were conducted by the same
psychiatrist who conducted the Baseline clinical evaluation after
4 weeks (second clinical evaluation) and 8 weeks (third clinical
evaluation). The clinician performed treatment adjustments,
refilled the prescriptions, and evaluated ADHD symptoms
using the ASRS scale. A final follow-up was also performed at
12 weeks using electronic self-assessment questionnaires sent to
participants.
Treatment adherence was monitored through weekly contacts
by research assistants with all participants. The instrument used to
evaluate adherence to pharmacological treatment was the medica-
tion possession rate (MPR) – defined as the average percentage of
time (days covered by the medication dosage) a patient has access to
treatment according to their treatment plan between two prescrip-
tions. A cutoff of 80% (or a mean MPR of <0.8) was used to indicate
low treatment adherence [22].
The knowledge assessment was performed based on an
adapted questionnaire [43] to identify participants’ awareness
and understanding of ADHD. This consisted of an 11-item Likert
Scale from 1 (totally disagree) to 5 (totally agree), where a total
score equal to or higher than 7 points was used as a cutoff for a
“good” understanding of ADHD consistent with previous inves-
tigations [43]. The participants’ knowledge of ADHD was
assessed at the beginning of the study at the Baseline clinical
evaluation and at the end of the follow-up period (12 weeks).
During the protocol duration, the App users received psychoedu-
cational multimedia ADHD contents twice a week that could be
accessed directly in the App or at the FOCUS ADHD website. The
publications included short videos and texts covering different
facets of adult ADHD-related information such as what the dis-
order is, its potential causes, clinical presentations and comorbid-
ities, and strategies to deal with ADHD. Users were sent push
notifications and messages to inform them about content updates.
Participants assigned to the two FOCUS ADHD App groups
were given instructions concerning App download and registration
by the study coordinator. They were also informed about the App
functionalities such as the pill reminders, weekly symptoms assess-
ment, the option to invite a support network, and the twice-a-week
psychoeducation contents that would be published during the
protocol duration. The App + Discount Group was also informed
about the incentive eligibility criteria, namely, to register 80% of
their medication intakes on the App according to the treatment
plan registered on the tool by the user. Once a week, the App
database was analyzed to check participants’ adherence to deter-
mine eligibility for the discount at the end of the first and second
months. If a participant was eligible, instructions were provided
explaining how to access a 25% discount from a partner Pharmacy
Chain.
Participant ratings of App quality were evaluated through the
User Version of the Mobile Application Rating Scale (uMARS)
[44] that was adapted to Portuguese. The uMARS is a 20-item
Likert Scale from 1 (low) to 5 (high) that includes four objective
quality subscales – Engagement, Functionality, Esthetics, and
Information Quality – and one subjective Quality subscale
[44]. The total score, ranging from 1 to 5, is calculated as the
mean of the four objective subscales. A score above 3 or higher
indicates satisfactory app quality. The participants rated the App
using the uMARS scale, after interacting with the App for 2 weeks.
The questionnaire was sent again after 12 weeks of usage (i.e., at
study completion).
Statistical analysis
The final analyzed sample comprised all participants with available
data in at least one of the primary outcomes, namely, treatment
adherence, knowledge of ADHD, or quality of the App.
All analyses were performed using JAMOVI software
(Version 2.3) [45]. Mean, standard deviation, and absolute and
relative frequencies were used to describe the sample. Normality
tests and distribution techniques were used to explore quantita-
tive distribution variables. Categorical data were analyzed using
X2 and Fisher exact tests. The t-Student or U-Mann–Whitney
test was applied to compare the effect of the discount incentive
on the App + Discount Group compared to the App Group. An
ANOVA or Kruskal Wallis test was used to compare the three
groups, and post hoc analyses with Holm’s correction were
performed for multiple pairwise comparisons. To study the
between-subjects factor (Group) and within-subject factor
(Time), we performed a repeated measure two-way ANOVA
European Psychiatry
3
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

including the interaction term. Finally, we used Spearman’s
correlation analysis to evaluate the association between quanti-
tative variables. The threshold for statistically significant effects
was set at 5% (p ≤0.05).
Results
Seventy-three participants (Mean age = 35 ± 7 years) were ran-
domized, of which 68 were included in the analysis (see Figure 1). A
statistically significant difference was found for age between groups:
App Group, 39 ± 6 years; App + Discount Group, 34 ± 6 years; and
TAU Group, 34 ± 7 years. The average family income was 6.600
reais (SD = 9.300). Although there was a difference between the
App Group (8.890 reais, SD = 13.810), App + Discount group
(5.300 reais, SD = 3.380), and the TAU Control Group (5.310 reais,
SD = 6.130), this difference was not statistically significant, given a
large amount of variability within groups (p = 0.787). Almost half of
the participants were female (49.3%), and more than 80% of the
participants indicated their race as white. Most participants were
single (64.3%), and 62% had completed higher education. All other
Figure 1. CONSORT flowchart.
4
Carvalho et al.
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

sample characteristics and sociodemographic factors did not sig-
nificantly differ among groups (Supplementary Table 1).
Almost all participants had symptoms of ADHD before 12 years
of age (98.6%). The most prevalent ADHD presentations were
combined (54.8%) and inattentive (38.4%). The most commonly
identified comorbid disorder was past Major Depressive Disorder
(65.8%). Social Anxiety (12.3%) and Generalized Anxiety Disorder
(8.3%) were also prevalent. The three experimental groups did not
significantly differ in the prevalence of ADHD presentation or
comorbid conditions (see Supplementary Table 2 for all details).
Symptom response to medication
Figure 2 illustrates that clinician-administered ASRS scale scores in
all participants had a significant decrease in ADHD symptoms
(i.e., improvement) over Time (F = 71.24, p < 0.001, ES = 0.341).
There was no statistically significant main effect of Group (F = 2.55,
p = 0.087, ES = 0.027) and no significant Group X Time interaction
(F = 1.82, p = 0.129, ES = 0.017).
Adherence to pharmacological treatment of ADHD
All groups had a mean MPR above 100%, and there was no
statistically significant difference between groups (KW = 1.00,
p = 0.606, ES = 0.0176) (see Figure 3). Even when establishing an
80% cutoff, there was no significant association between the
pharmacological treatment adherence and groups (X2 = 0.96,
p = 0.677).
Participants’ knowledge of ADHD
When evaluating the results regarding the ADHD knowledge
assessment, all groups initially presented with high scores – above
the cutoff score used to classify a “good” ADHD understanding/
knowledge. Even though participants had a marginal increase in
their scores, this was not statistically significant (F = 1.22, p = 0.275,
ES = 0.007) and the result was independent of randomized group
status (Interaction effect: F = 0.45, p = 0.641, ES = 0.005; Group
effect: F = 1, p = 0.376, ES = 0.034) (see Figure 4).
Adoption, engagement, retention, and the quality of the FOCUS
ADHD App
Forty-nine participants were eligible to use the FOCUS ADHD App
(see Figure 1).
Adoption
Among these 49, 5 (10.2%) did not download the App. All five
participants that did not download the App were from the App
Group. Thus, the adoption of the App for this group (75%) was
significantly lower compared with the App + Discount Group
(100%) (X2 = 5.35, p = 0.05).
Engagement
Engagement was defined as the number of times (as a %) that users
registered their medication intake into the App (see Figure 5).
There was a significant effect of Group as the App + Discount
Group had significantly higher engagement compared with the
App Group (F = 7.19, p = 0.010, ES = 0.123); and a significant
effect of Time, as App engagement significantly decreased over time
(F = 10.17, p < 0.001, ES = 0.028). An observed significant Time X
Group interaction (F = 4.68, p = 0.012, ES = 0.013) demonstrates
that the increased medication intake registrations exhibited by the
Figure 2. Attention-deficit/hyperactivity disorder symptoms evolution.
Note. Mean (IC – 95%).
Figure 3. Average medication possession rate.
Note. Mean (IC – 95%).
European Psychiatry
5
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

App + Discount Group were significantly greater early in the trial,
returning to levels similar to the App Group by the third clinical
evaluation (Week 12).
Retention
Retention is defined as user engagement (i.e., the percentage of
subjects who registered their medication intake at least 80% of the
time) being consistent throughout the study (i.e., at 4 and 8 weeks).
Only four participants, all from the App + Discount Group (16.7%),
fulfilled this retention criterion throughout the entire study duration.
Quality
The FOCUS ADHD App received a positive quality evaluation by
25 study participants obtaining a total uMARS score of 3.45
(SD 0.63) out of 5 (see Supplementary Table 3). There was a trend
toward greater ratings of usability in the App + Discount Group
(Mean = 3.62; SD = 0.61) compared to the App Group
(Mean = 3.16; SD = 0.59), but this effect was not significant
(U = 40, p = 0.074, ES = 0.444). Analyzing the results from the
uMARS subscales, the “Information Quality” rating received the
highest mean score at 3.84 (SD 0.85), followed by Functionality at
3.67 (SD 0.88) and Esthetics at 3.49 (SD 0.66). The only rating
variable with a score lower than 3 was Engagement at 2.89
(SD 0.62). Comparing the differences between the groups, the
only significantly different subscale score was “Information
Quality” in favor of the App + Discount Group (U = 28.5,
p = 0.043, ES = 0.525).
Discussion
The present study assessed the effect of using digital technologies
for enhancing pharmacological treatment adherence in ADHD.
We found no significant difference in treatment adherence
(as measured by MPR) between participants receiving pharma-
cological TAU compared to participants who received TAU
combined with the use of the FOCUS ADHD mobile health
App. Though not consistent with previous studies showing a
positive effect of mHealth solutions on treatment adherence
[34–37], we note that these studies used different interventions
and methodologies. However, our measure of App engagement
(i.e., participant registration of medication intake) provides some
initial evidence of the positive effect of combining a financial
incentive with the use of the FOCUS ADHD App on treatment
adherence.
Treatment adherence and App engagement
Though no effect on adherence was observed, it is important to
highlight that all groups exhibited an MPR above 100% in our
study. These values are much higher than the established cutoff
value used to classify participants as adherent and non-adherent in
previous studies [22]. One possibility is that our study parameters,
including weekly calls to participants from research assistants as
well as scheduled assessments with the study psychiatrist, might
have led to an inflated MPR in all groups. Furthermore, MPR does
not guarantee that the prescribed and dispensed medication was
taken in the correct manner by the patient. Since there is no “gold
standard” for measuring adherence, the use of combinations of
measurement tools is recommended for a more accurate assess-
ment [16, 22, 32].
Regarding the FOCUS ADHD App, we observed a high adop-
tion rate, especially by those receiving a reinforcement for using
the App (100% for the App + Discount vs. 75% for the App
Group). User engagement has proved to be a key element in
determining the effectiveness of digital interventions [46–48],
and here we show that financial incentives improved engagement.
Indeed, registrations of medication intake were significantly
higher in the App + Discount Group compared with the App
Group, especially during the early phase of the trial (see Figure 5).
This was expected since those in the App + Discount Group had a
greater incentive for registering medication intake during the first
two study assessment periods (i.e., between Baseline and the first
month, and then between the first and second months) because
only after these two time periods were they eligible for the dis-
count. Since they would not receive the discount after the third
assessment, this motivation decreased between the second and the
third assessments, and their engagement decreased to levels simi-
lar to the non-reinforced App Group.
Related to the complexities of using MPR as a measure of
adherence, it is interesting to consider that our participants’
engagement with the App and their registration of medication
intake could be considered a reasonable measure of adherence.
The naturalistic nature of the present study prevented us from
documenting that these registrations were associated with actual
medication intake. That said, the fact that our participants made an
effort to get onto the App and registered an intake and confirmed to
our research assistants (via weekly contacts) that these registrations
were associated with actual medication intakes, provides reasonable
evidence that they took the medication. This suggests initial
Figure 5. FOCUS ADHD – users’ engagement.
Note. Mean (IC – 95%).
Figure 4. Attention-deficit/hyperactivity disorder knowledge assessment summary.
Note. Mean (IC – 95%).
6
Carvalho et al.
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

evidence that a financial incentive paired with the FOCUS ADHD
App increased medication adherence, at least initially.
In terms of retention of App use, no participant continued to
register medication intake more than 80% of the time during the
entire study in the App Group, while four participants in the
App + Discount Group (16.7%) fulfilled this criterion. Though
these numbers are low, they are consistent with the available
mHealth literature [46–48], highlighting that when evaluating digi-
tal health solutions, user engagement and adherence to treatment in
chronic disorders like ADHD remain central challenges in the
health field.
When evaluating ADHD knowledge assessment, no significant
differences emerged among groups. However, the average baseline
scores for the three groups were already above the adopted cutoff
for indicating a good understanding of ADHD, which may have
created a ceiling effect where only a small improvement could be
observed in all groups. Creating a more interactive educational
experience by customizing the available psychoeducational con-
tents with specific patient characteristics or needs could improve
the FOCUS ADHD App.
When we analyzed the FOCUS ADHD App performance based
on the uMARS questionnaire, the total average score of the evalu-
ations from both groups of App users was above 3, a cutoff that has
been utilized in previous studies as evidence of a positive evaluation
of App quality [49–52]. In particular, the information quality of the
App was recognized as its greatest strength, which may be related to
the video and text psychoeducational contents. That said, digital
health design requires a remarkable speed of innovation where
developers must constantly update their products to align with user
demands for novel design features that encourage user engagement.
Continuing with this approach might increase both adoption and
continued engagement with the App during treatment [53].
The literature also highlights the importance of designing and
implementing personalized approaches to promote users’ engage-
ment with digital mental health solutions [54]. The FOCUS ADHD
App engages the user according to their specific registered treat-
ment plan. In the present study, we did not personalize, for
example, different psychoeducational contents based on the user’s
profile and preferences, or present information using friendly
interfaces, such as AI chatbots. These new approaches might
increase users’ intrinsic motivation to engage within the App.
As clinical trials using digital technology continue, it will be
important to consider the conditions for optimal implementation
in clinical practice [55, 56]. Specifically, it will be critical to consider
the involvement and engagement of healthcare personnel, as a care
line in the deployment of such technology [57–61]. These changes
will facilitate technology development, implementation, and assess-
ment of mental health.
There are some limitations to consider when evaluating the
present results. The sample size was relatively small, and the
protocol did not have a digital control group. There were income
differences among study groups that were non-significant probably
due to the large amount of variability within groups. Although this
difference was not statistically significant, income could affect the
salience of financial incentives.
The current findings demonstrated that the FOCUS ADHD
App obtained a high adoption rate and received positive evalu-
ations from its users. In the present study, the use of the App did not
increase treatment adherence as measured by MPR. That said, for
those using the App, evidence of increased medication intake
registrations in the group that received a discount on their medi-
cation offers initial evidence supporting the utility of combining
financial incentives with mHealth solutions to increase treatment
adherence. These data, together with prior evidence showing that
App use can increase treatment adherence [34–37], call for future
pragmatic trials designed to evaluate the effectiveness of the current
proposed mHealth solution in real-life routine practice conditions.
Supplementary material. The supplementary material for this article can be
found at http://doi.org/10.1192/j.eurpsy.2023.2422.
Financial support. This study was financed in part by the Coordenação de
Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code
001. Partial financial support for the development and maintenance of the
FOCUS ADHDApp was received from Shire/Takeda as sponsorship. Also, partial
financial support to fund the discounts provided during this study was received by
a local Pharmacy Chain Retailer – Panvel/Dimed Group as sponsorship.
Competing interest. L.A.R. has received grant or research support from,
served as a consultant to, and served on the speakers’ bureau of Abbott, Aché,
Bial, Medice, Novartis/Sandoz, Pfizer/Upjohn, and Shire/Takeda in the last
3 years. The ADHD and Juvenile Bipolar Disorder Outpatient Programs chaired
by Dr. Rohde have received unrestricted educational and research support from
the following pharmaceutical companies in the last 3 years: Novartis/Sandoz and
Shire/Takeda. Dr. Rohde has received authorship royalties from Oxford Press and
ArtMed. E.H.G. has served on the speaker’s bureau for Shire/Takeda for the last
3 years. He also received travel awards (air tickets and hotel accommodations) for
participating in three psychiatric meetings from Shire/Takeda. M.M.V. declared a
commercial agreement between his mobile App Moodr for Bipolar Disorder with
Daiichi Sankyo. The remaining authors declare no competing interest.
Ethical aspects. The project was submitted and approved by the Research
Ethics Committees of the Faculty of Medicine of the University of São Paulo
(FMUSP) with the Certificate of Presentation of Ethical Appreciation CAAE
No. 22984819.8.0000.0065 and the Hospital de Clínicas de Porto Alegre
(HCPA) CAAE No. 22984819.8.3001.5327. The clinical trial was also registered
on
ClinicalTrials.gov
(https://clinicaltrials.gov/)
with
identification
(ID) NCT05551689. Ethical parameters were used in the development and
implementation of the project, in accordance with resolution 466/2012 of the
Guidelinesand Regulatory StandardsforResearchInvolving Human Beingsofthe
National Health Council applied the terms of free and informed consent. The
requirements of the General Data Protection Law (Law No. 13,709 of August
14, 2018) were also followed for the execution of this research project, and the
privacy rights of the participants were guaranteed.
References
[1] Polanczyk G, de Lima MS, Horta BL, Biederman J, Rohde LA. The
worldwide prevalence of ADHD: a systematic review and metaregression
analysis.AmJPsychiatry.2007;164(6):942–8.doi:10.1176/ajp.2007.164.6.942.
[2] Faraone SV, Banaschewski T, Coghill D, Zheng Y, Biederman J, Bellgrove
MA, et al. The World Federation of ADHD International Consensus
Statement: 208 evidence-based conclusions about the disorder. Neurosci
Biobehav Rev. 2021;128:789–818. doi:10.1016/j.neubiorev.2021.01.022.
[3] Battle DE. Diagnostic and Statistical Manual of Mental Disorders (DSM).
CoDAS. 2013;25(2):191–2. doi:10.1590/s2317-17822013000200017.
[4] Posner J, Polanczyk GV, Sonuga-Barke E. Attention-deficit hyperactivity
disorder. Lancet. 2020;395(10222):450–62. doi:10.1016/S0140-6736(19)
33004-1.
[5] Faraone SV, Glatt SJ. A comparison of the efficacy of medications for adult
attention-deficit/hyperactivity disorder using meta-analysis of effect sizes.
J Clin Psychiatry. 2010;71:754–63. doi:10.4088/JCP.08m04902pur.
[6] Rohde LA, Barbosa G, Tramontina S, Polanczyk G. Transtorno de déficit
de atenção/hiperatividade. Braz J Psychiatry [Internet]. 2000;22:7–11. doi:
10.1590/S1516-44462000000600003.
[7] Shaw M, Hodgkins P, Caci H, Young S, Kahle J, Woods AG, Arnold LE. A
systematic review and analysis of long-term outcomes in attention deficit
hyperactivity disorder: effects of treatment and non-treatment. BMC Med.
2012;10:99. doi:10.1186/1741-7015-10-99.
European Psychiatry
7
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

[8] Boland H, DiSalvo M, Fried R, Woodworth KY, Wilens T, Faraone SV,
Biederman J. A literature review and meta-analysis on the effects of ADHD
medications on functional outcomes. J Psychiatr Res. 2020;123:21–30. doi:
10.1016/j.jpsychires.2020.01.006.
[9] Biederman J. Advances in the neurobiology of ADHD. CNS Spectr. 2007;
12(S6):6–7.
[10] Kessler RC, Adler L, Barkley R, Biederman J, Conners K, Demler O, et al.
The prevalence and correlates of adult ADHD in the United States: results
from the National Comorbidity Survey Replication. Am J Psychiatry. 2006;
163(4):716–23. doi:10.1176/ajp.2006.163.4.716.
[11] Victor MM. Factors associated with clinical outcomes with the use of methyl-
phenidate in adults with attention-deficit/hyperactivity disorder (ADHD)
[dissertation]. Porto Alegre: Universidade Federal do Rio Grande do Sul,
Programa de Pós-Graduação em Psiquiatria; 2014. Available from: https://
lume.ufrgs.br/bitstream/handle/10183/106731/000944290.pdf;jsessionid=
90A1B90ED213D490FABEE3E6DD225CD4?sequence=1.
[12] Frank E, Ozon C, Nair V, Othee K. Examining why patients with attention-
deficit/hyperactivity disorder lack adherence to medication over the long
term: a review and analysis. J Clin Psychiatry. 2015;76(11):e1459–e1468.
doi:10.4088/JCP.14r09478.
[13] Matza LS, Paramore C, Prasad M. A review of the economic burden of
ADHD. Cost Eff Resour Alloc. 2005;3:5. doi:10.1186/1478-7547-3-5.
[14] Cutler RL, FernandezLlimos F, Frommer M, Frommer M, Benrimoj C,
Garcia-Cardenas V. Economic impact of medication non-adherence by
disease groups: a systematic review. BMJ Open. 2018;8:e016982. doi:
10.1136/bmjopen-2017-016982.
[15] Elliott J, Johnston A, Husereau D, Kelly SE, Eagles C, Charach A, et al.
Pharmacologic treatment of attention deficit hyperactivity disorder in
adults: a systematic review and network meta-analysis. PLoS One. 2020;
15(10):e0240584. doi:10.1371/journal.pone.0240584.
[16] Khan MU, Aslani P. A review of measures used to examine medication
adherence in people with ADHD at initiation, implementation and dis-
continuation of pharmacotherapy. Res Social Adm Pharm. 2020 Mar;
16(3):277–89. doi:10.1016/j.sapharm.2019.06.001.
[17] Adler LD, Nierenberg AA. Review of medication adherence in children
and adults with ADHD. Postgrad Med. 2010;122(1):184–91. doi:10.3810/
pgm.2010.01.2112.
[18] Kooij SJ, Bejerot S, Blackwell A, Caci H, Casas‐Brugue M, Carpentier PJ,
et al. European consensus statement on diagnosis and treatment of adult
ADHD: the European Network Adult ADHD. BMC Psychiatry. 2010;10:
67. doi:10.1186/1471-244x-10-67.
[19] Solberg BS, Haavik J, Halmøy A. Health care services for adults with
ADHD: patient satisfaction and the role of psycho-education. J Atten
Disord. 2019 Jan;23(1):99–108. doi:10.1177/1087054715587941.
[20] Iuga AO, McGuire MJ. Adherence and health care costs. Risk Manag
Healthc Policy. 2014;7:35–44. doi:10.2147/RMHP.S19801.
[21] Zolnierek KB, Dimatteo MR. Physician communication and patient
adherence to treatment: a meta-analysis. Med Care. 2009;47(8):826–34.
doi:10.1097/MLR.0b013e31819a5acc.
[22] Parkin R, Nicholas FM, Hayden JC. A systematic review of interventions to
enhance adherence and persistence with ADHD pharmacotherapy. J
Psychiatr Res. 2022;152:201–18. doi:10.1016/j.jpsychires.2022.05.044.
[23] Lukens EP, McFarlane WR. Psychoeducation as evidence-based practice:
considerations for practice, research, and policy. Brief Treat Crisis Interv.
2004;4(3):205–25. doi:10.1093/brief-treatment/mhh019.
[24] Oliveira CT, Dias ACG. Psychoeducation for attention deficit/hyperactiv-
ity disorder: what, how and who shall we inform? Temas Psicol. 2018 Mar;
26(1):243–61. doi:10.9788/TP2018.1-10Pt.
[25] Wiggins D, Singh K, Getz HG, Hutchins DE. Effects of brief group
intervention for adults with attention deficit/hyperactivity disorder. J
Ment Health Couns. 1999;21(1):82–92.
[26] Hoxhaj E, Sadohara C, Borel P, D’Amelio R, Sobanski E, Müller H, et al.
Mindfulness vs psychoeducation in adult ADHD: a randomized controlled
trial. Eur Arch Psychiatry Clin Neurosci. 2018;268(4):321–35. doi:
10.1007/s00406-017-0839-5.
[27] Vidal R, Bosch R, Nogueira M, Gómez-Barros N, Valero S, Palomar G,
et al. Psychoeducation for adults with attention deficit hyperactivity
disorder vs. cognitive behavioral group therapy: a randomized controlled
pilot study. J Nerv Ment Dis. 2013 Oct;201(10):894–900. doi:10.1097/
NMD.0b013e3182a5c2c5.
[28] Hirvikoski T, Waaler E, Lindström T, Bölte S, Jokinen J. Cognitive
behavior therapy-based psychoeducational groups for adults with ADHD
and their significant others (PEGASUS): an open clinical feasibility trial.
Atten Defic Hyperact Disord. 2015 Mar;7(1):89–99. doi:10.1007/s12402-
014-0141-2.
[29] Hirvikoski T, Lindström T, Carlsson J, Waaler E, Jokinen J, Bölte S.
Psychoeducational groups for adults with ADHD and their significant
others (PEGASUS): a pragmatic multicenter and randomized controlled
trial. Eur Psychiatry. 2017;44:141–52. doi:10.1016/j.eurpsy.2017.04.005.
[30] Selaskowski B, Steffens M, Schulze M, Lingen M, Aslan B, Rosen H, et al.
Smartphone-assisted
psychoeducation
in
adult
attention-deficit/
hyperactivity disorder: a randomized controlled trial. Psychiatry Res.
2022;317:114802. doi:10.1016/j.psychres.2022.114802.
[31] Khazanov GK, Morris PE, Beed A, Jager-Hyman S, Myhre K, McKay JR,
et al. Do financial incentives increase mental health treatment engage-
ment? A meta-analysis. J Consult Clin Psychol. 2022;90(6):528–44. doi:
10.1037/ccp0000737.
[32] García-Pérez L, Linertová R, Serrano-Pérez P, Trujillo-Martín M,
Rodríguez-Rodríguez L, et al. Interventions to improve medication adher-
ence in mental health: the update of a systematic review of cost-
effectiveness. Int J Psychiatry Clin Pract. 2020;24(4):416–27. doi:10.108
0/13651501.2020.1782434.
[33] Armitage LC, Kassavou A, Sutton S. Do mobile device apps designed to
support medication adherence demonstrate efficacy? A systematic review
of randomized controlled trials, with meta-analysis. BMJ Open. 2020;
10(1):e032045. doi:10.1136/bmjopen-2019-032045.
[34] Philippe TJ, Sikder N, Jackson A, Koblanski ME, Liow E, Pilarinos A,
Pilarinos A, Vasarhelyi K. Digital health interventions for delivery of
mental health care: systematic and comprehensive meta-review. JMIR
Ment Health. 2022;9(5):e35159. doi:10.2196/35159.
[35] Biederman J, Fried R, DiSalvo M, Woodworth KY, Biederman I, Noyes E,
et al. A novel text message intervention to improve adherence to stimu-
lants in adults with attention deficit/hyperactivity disorder. J Clin Psycho-
pharmacol. 2019;39(4):351–6. doi:10.1097/JCP.0000000000001055.
[36] Biederman J, Fried R, DiSalvo M, Driscoll H, Green A, Biederman I, et al. A
novel digital health intervention to improve patient engagement to stimu-
lants in adult ADHD in the primary care setting: preliminary findings
from an open label study. Psychiatry Res. 2020;291:113158. doi:10.1016/j.
psychres.2020.113158.
[37] Tran S, Smith L, El-Den S, Carter S. The use of gamification and incentives
in mobile health apps to improve medication adherence: scoping review.
JMIR Mhealth Uhealth. 2022;10(2):e30671. doi:10.2196/30671.
[38] Wiecek E, Torres-Robles A, Cutler RL, Benrimoj SI, Garcia-Cardenas V.
Impact of a multicomponent digital therapeutic mobile app on medication
adherence in patients with chronic conditions: retrospective analysis. J
Med Internet Res. 2020;22(8):e17834. doi:10.2196/17834.
[39] Huckvale K, Nicholas J, Torous J, Larsen ME. Smartphone apps for the
treatment of mental health conditions: status and considerations. Curr
Opin Psychol. 2020;36:65–70. doi:10.1016/j.copsyc.2020.04.008.
[40] Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert
DD. Standalone smartphone apps for mental health – a systematic review
and meta-analysis. NPJ Digit Med. 2019;2:118. doi:10.1038/s41746-019-
0188-8.
[41] Mattos P, Segenreich D, Saboya E, Louzã M, Dias G, Romano M. Adapta-
ção Transcultural para o Português da Escala Adult Self-Report Scale
(ASRS-18, versão1.1) para avaliação de sintomas do Transtorno de Déficit
de Atenção/Hiperatividade (TDAH) em adultos. Rev Psiquiatr Clín. 2006;
33(4):188–94. doi:10.1590/S0101-60832006000400004.
[42] Grevet EH, Bau CA, Salgado CA, Fischer A, Victor MM, Garcia C, et al.
Interrater reliability for diagnosis in adults of attention deficit hyperactiv-
ity disorder and oppositional defiant disorder using K-SADS-E. Arq
Neuropsiquiatr. 2005;63(2B):307–10. doi:10.1590/S0004-282X20050002
00019.
[43] Gomes M, Palmini A, Barbirato F, Rohde LA, Mattos P. Conhecimento
sobre o transtorno do déficit de atenção/hiperatividade no Brasil.
8
Carvalho et al.
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press

J. bras. psiquiatr. 2007;56(2):94–101. doi:10.1590/S0047-208520070
00200004.
[44] StoyanovSR, Hides L,KavanaghDJ, Wilson H.Development and validation
of the User Version of the Mobile Application Rating Scale (uMARS). JMIR
mHealth uHealth. 2016;4(2):e72. doi:10.2196/mhealth.5849.
[45] The jamovi project. jamovi. Version 2.3 [Computer software]. 2022.
Available from: https://www.jamovi.org.
[46] Nwosu A, Boardman S, Husain MM, Doraiswamy PM. Digital therapeut-
ics for mental health: is attrition the Achilles heel? Front Psychiatry. 2022;
13:900615. doi:10.3389/fpsyt.2022.900615.
[47] Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with
mental health apps: systematic search and panel-based usage analysis. J
Med Internet Res. 2019;21(9):e14567. doi:10.2196/14567.
[48] Ng MM, Firth J, Minen M, Torous J. User engagement in mental health
apps: a review of measurement, reporting, and validity. Psychiatr Serv.
2019;70(7):538–44. doi:10.1176/appi.ps.201800519.
[49] Serlachius A, Schache K, Kieser A, Arroll B, Petrie K, Dalbeth N. Associ-
ation between user engagement of a mobile health app for gout and
improvements in self-care behaviors: randomized controlled trial.
JMIR Mhealth Uhealth. 2019;7(8):e15021. doi:10.2196/15021.
[50] Fijačko N, Masterson Creber R, Gosak L, Kocbek P, Cilar L, Creber P, et al.
A review of mortality risk prediction models in smartphone applications. J
Med Syst. 2021;45(12):107. doi:10.1007/s10916-021-01776-x.
[51] LeBeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by
occupational therapists: evaluation using the User Version of the Mobile
Application Rating Scale. JMIR Mhealth Uhealth. 2019;7(5):e13019. doi:
10.2196/13019.
[52] Potzel AL, Gar C, Banning F, Sacco V, Fritsche A, Fritsche L, et al. A novel
smartphone app to change risk behaviors of women after gestational
diabetes: a randomized controlled trial. PLoS One. 2022;17(4):e0267258.
doi:10.1371/journal.pone.0267258.
[53] Birnbaum F, Lewis D, Rosen RK, Ranney ML. Patient engagement and the
design of digital health. Acad Emerg Med. 2015;22(6):754–6. doi:10.1111/
acem.12692.
[54] Saleem M, Kühne L, De Santis KK, Christianson L, Brand T, Busse H.
Understanding engagement strategies in digital interventions for mental
health promotion: scoping review. JMIR Ment Health. 2021;8(12):e30000.
doi:10.2196/30000.
[55] Connolly SL, Hogan TP, Shimada SL, Miller CJ. Leveraging implementa-
tion science to understand factors influencing sustained use of mental
health apps: a narrative review. J Technol Behav Sci. 2021;6(2):184–96. doi:
10.1007/s41347-020-00165-4.
[56] Jakob R, Harperink S, Rudolf AM, Fleisch E, Haug S, Mair JL, et al. Factors
influencing adherence to mHealth apps for prevention or management of
noncommunicable diseases: systematic review. J Med Internet Res. 2022;
24(5):e35371. doi:10.2196/35371.
[57] Hilty D, Chan S, Torous J, Luo J, Boland R. A framework for competencies
for the use of mobile technologies in psychiatry and medicine: scoping
review. JMIR mHealth uHealth. 2020;8(2):e12229. doi:10.2196/12229.
[58] Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, et al.
Barriers to and facilitators of user engagement with digital mental health
interventions: systematic review. J Med Internet Res. 2021;23(3):e24387.
doi:10.2196/24387.
[59] Connolly SL, Kuhn E, Possemato K, Torous J. Digital clinics and mobile
technology implementation for mental health care. Curr Psychiatry Rep.
2021;23(7):38. doi:10.1007/s11920-021-01254-8.
[60] Wisniewski H, Torous J. Digital navigators to implement smartphone and
digital tools in care. Acta Psychiatr Scand. 2020;141(4):350–5. doi:10.1111/
acps.13149.
[61] Funnell EL, Spadaro B, Benacek J, Martin-Key NA, Metcalfe T, Olmert T,
et al. Learnings from user feedback of a novel digital mental health assess-
ment. Front Psychiatry. 2022;13:1018095. doi:10.3389/fpsyt.2022.1018095.
European Psychiatry
9
https://doi.org/10.1192/j.eurpsy.2023.2422 Published online by Cambridge University Press