# Smartphone apps for mental health and wellbeing: A usage survey and machine learning analysis of psychological and behavioral predictors

Germano Vera Cruz, Elias Aboujaoude, Riaz Khan, Lucien Rochat, Farah Ben Brahim, Robert Courtois, Yasser Khazaal

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Germano Vera Cruz, Elias Aboujaoude, Riaz Khan, Lucien Rochat, Farah Ben Brahim, et al. Smartphone apps for mental health and wellbeing: A usage survey and machine learning analysis of psychological and behavioral predictors. Digital Health, 2023, 9, pp.205520762311521. 10.1177/20552076231152164. hal-03969856

HAL Id: hal-03969856

https://u-picardie.hal.science/hal-03969856v1

Submitted on 8 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Original research

# Smartphone apps for mental health and wellbeing:
A usage survey and machine learning analysis of
psychological and behavioral predictors

Digital Health
Volume 9: 1-12
© The Author(s) 2023
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/20552076231152164
journals.sagepub.com/home/dhj

<figure><img src="image_3.png" /></figure>

SAGE

Germano Vera Cruz<fnref n="1" /> ID, Elias Aboujaoude<fnref n="2" />, Riaz Khan<fnref n="3" />, Lucien Rochat<fnref n="4" />, Farah Ben Brahim<fnref n="5" />, Robert Courtois<fnref n="5" /> and Yasser Khazaal<fnref n="6" />,<fnref n="7" />,<fnref n="8" />

## Abstract

**Objective:** Despite the availability of thousands of mental health applications, the extent to which they are used and the factors associated with their use remain largely unknown. The present study aims to (a) assess in a representative US-based population sample the use of smartphone apps for mental health and wellbeing (SAMHW), (b) determine the variables predicting the use of SAMHW, and (c) explore how a set of variables related to mental health, smartphone use, and smartphone "addiction" may be associated with the use of SAMHW.

**Methods:** Data was collected via online questionnaire from 1989 adults. The data gathered included information on smartphone use behavior, mental health, and the use of SAMHW. Latent class analysis was used to categorize participants. Machine learning and logistic regression analyses were used to determine the most important predictors of SAMHW use and associations between predictors and outcome variables.

**Results:** While two-thirds of participants had a statistically high probability for using SAMHW, nearly twice more had high probability for using them to improve wellbeing compared to using them to address mental health problems (43% vs. 18%). In both groups, these participants were more likely to be female and in the younger adult age bracket than male and in the adult or older adult age bracket. According to the machine learning model, the most important predictors for using the relevant smartphone apps were variables associated with smartphone problematic use, COVID-19 impact, and mental health problems.

**Conclusion:** Findings from the present study confirm that the use of SAMHW is growing, particularly among younger adult and female individuals who are negatively impacted by problematic smartphone use, COVID-19, and mental health problems. These individuals tend to bypass traditional care via psychotherapy or psychopharmacology, relying instead on smartphones to address mental health conditions or improve wellbeing. Advising users of these apps to also seek professional help and promoting efforts to prove the efficacy and safety of SAMHW would seem necessary.

## Keywords

Smartphone apps, mental health, addiction/problematic use

Submission date: 22 June 2022; Acceptance date: 3 January 2023

## Corresponding author:

Yasser Khazaal, CHUV, Département de Psychiatrie, Service de médecine des addictions, Rue du Bugnon 23, 1011 Lausanne, Switzerland.

Email: Yasser.khazaal@chuv.ch

<figure><img src="image_4.png" /></figure>

<footnote n="1">¹Department of Psychology, University of Picardie Jules Verne, Amiens, France</footnote>

<footnote n="2">²Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA</footnote>

<footnote n="3">³Addiction Psychiatry, Foederatio Medicorum Helveticorum, Geneva, Switzerland</footnote>

<footnote n="4">⁴Addiction Division, Department of Psychiatry, University Hospitals of Geneva, Geneva, Switzerland</footnote>

<footnote n="5">⁵Department of Psychology, University of Tours, Tours, France</footnote>

<footnote n="6">⁶Addiction Medicine, Lausanne University Hospital, Lausanne, Switzerland</footnote>

<footnote n="7">⁷Department of Psychiatry, Lausanne University, Lausanne, Switzerland</footnote>

<footnote n="8">⁸Department of Psychiatry and Addictology, Montreal University, Montreal, QC, Canada</footnote>
## Introduction

The dissemination of smartphones (more than 2 billion users worldwide)¹ has been accompanied by the rapid dissemination of digital smartphone apps, including for health and wellbeing purposes. This phenomenon can potentially reduce the gap in mental health services delivery. As such, the World Health Organization launched the 2020–2025 global strategy on digital health “to improve health for everyone, everywhere by accelerating the development and adoption of appropriate, accessible, affordable, scalable, and sustainable person-centric digital health solutions” (p. 3).² The opportunity to provide wide scale screening and treatment tools for mental health conditions has led to a rapid development of the field (over 10,000 mental health apps available through Google play and App stores, in addition to many wellbeing-related apps).³,⁴ Furthermore, several smartphone apps have been launched specifically targeting problematic smartphone use.⁴,⁵ Despite important concerns related to the quality of health-related apps,⁶,⁷ there is some evidence from meta-reviews and meta-analyses that some of the mental health apps may be efficacious at reducing symptoms including anxiety, depression, and “stress.”⁸⁻¹¹

Mixed results were reported on general health-related app adoption. For instance, a study on a representative German sample reported low-to-moderate acceptance of such apps.¹² However, previous studies utilizing small convenience samples suggested interest in smartphone apps for mental health and wellbeing.¹³,¹⁴ Several studies described possible predictors of acceptance of mental health-related apps, including a positive attitude toward the apps, stress-related symptoms,¹⁴ and the perceived utility of a given app.¹⁵ Studies conducted in Saudi Arabia and the United States on convenience samples of individuals with mental health disorders showed that 25% to 40% of people were using mental health apps.¹⁶,¹⁷

The stress brought about by the COVID-19 pandemic (e.g. confinement, mental and general health difficulties, healthcare access challenges, financial anxiety, etc.) has led to a dramatic increase in the use of digital health tools, particularly for video-enabled treatment.¹⁸,¹⁹ This phenomenon was also possibly associated with an increase in wellbeing app downloads (e.g. meditation apps).²⁰ However, despite the wide availability of apps for mental health, wellbeing, and smartphone use control, and despite a broadening evidence base for their potential benefit, there is limited precise knowledge on the penetrance of these tools and the user characteristics linked to them.

## Study purpose

The present study aims to assess in a representative sample of the US population the use of smartphone apps for mental health and wellbeing (SAMHW). The study also aims to

explore how a set of variables related to mental health, smartphone use, and smartphone “addiction” may be associated with the use of SAMHW, and what the most important predictors for the use of SAMHW are.

## Research questions

Five research questions guided this study.

(a) First question (RQ1): In a representative sample of the US population, what is the prevalence of use of SAMHW?

(b) Second question (RQ2): In a representative sample of the US population, can we identify and describe underlying classes of users of SAMHW?

(c) Third question (RQ3): If underlying classes of smartphone users are identified, what are the associations between participants’ sociodemographic characteristics (age and sex) and the uncovered latent classes?

(d) Fourth question (RQ4): Among variables such as age, sex, average daily duration of smartphone use, perceived level of social problems due to smartphone use in 12 months preceding the study, alcohol use disorder, smartphone “addiction,” and mental health disorders experienced in 2 weeks preceding the study, what are the most important features (covariates) predicting smartphone users’ uncovered latent class membership?

(e) Fifth question (RQ5): What are the associations between the covariates (age, sex, average daily duration of smartphone use, perceived level of social problems due to smartphone use, alcohol use disorder, smartphone “addiction,” and mental health disorders experienced in 2 weeks preceding the study) and the unveiled latent classes of users of SAMHW?

As this was designed as an exploratory study, no hypotheses associated with these research questions were made.

## Methods

### Participants

A representative sample of the US adult population was recruited. Overall, 1989 individuals participated in the study and answered an online questionnaire. Participants’ descriptive statistics of the two considered sociodemographic variables are: age 19–76 years old (M = 45; SD = 16); age categories 19–35 (691/35%), 36–60 (802/40%), and 61–76 years old (496/25%); sex: male (965/48.51%), female (1006/50.57%), and nonbinary (18/0.9%).

### Recruitment and sampling procedures

The crowdsourcing platform Prolific (https://www.prolific.ac/) was used to recruit study participants online and anonymously. Prolific has advantages over other similar
platforms in that it is exclusively dedicated to research studies, and its participants are considered more naïve to experimental research and more ethnically and geographically diverse.²⁰ Prolific offers nationwide representative samples of adults in two countries (the United States and the United Kingdom). Based on the US base of one of the authors (EA) and that he was among the first to publish a scholarly assessment of smartphone apps to treat problematic smartphone use,²¹ the authors opted for using the Prolific platform to recruit a US representative sample in terms of age, sex, and ethnicity. A stratification method of sampling was used: from the Prolific US community of more than 42,000 users active on the platform in the past 3 months preceding the data collection, Prolific extracted a sample of 1989 individuals with a similar distribution of age, sex, and ethnicity as the national population as determined by US Census Bureau data.

Participants received approximately USD 1.3 for completing the questionnaire.

## Data collection material

The online questionnaire included items on two sociodemographic characteristics in participants (age and sex); smartphone operation system (OS); smartphone use in the 12 months preceding the study (e.g. daily use duration, average daily use duration for non-essential activities, social problems due to smartphone use); use of smartphone apps in the 12 months preceding the study to diagnose a mental health problem and/or to assess wellbeing (Yes/No); use of smartphone apps in the 12 months preceding the study to treat a mental health problem and/or to improve wellbeing (Yes/No); psychiatric diagnoses, including alcohol use disorder, over the course of the participant's lifetime (Yes/No); and smartphone use during the COVID-19 pandemic (7-point response scale). Participants also completed the Smartphone Application-Based Addiction Scale,²² which is comprised of 6 items and a 6-point response scale ranging from *strongly disagree* to *strongly agree*; the DSM (Diagnostic and Statistical Manual of Mental Disorders)-5 Self-Rated Level 1 Cross-Cutting Symptom Measure²³ (24 items, 5-point response scale ranging from *Not at all* to *Nearly every day*), assessing symptoms, during the 2 weeks preceding the survey completion, across 14 domains: anxiety, depression, anger, personality functioning, mania, sleep problems, somatic symptoms, psychosis, suicidal ideation, memory, dissociation, repetitive thoughts and behaviors, and substance use. In the present study, these dimensions were labeled “DSM Anxiety,” “DSM Depression,” etc. Although, outside of the treatment of smartphone addiction, no specific app names were included in the survey or queried about, survey questions were structured to communicate that “mental health” apps are meant for the diagnosis or treatment of a particular mental health diagnosis, whereas “wellbeing” apps are meant for the improvement of overall health through lifestyle modifications (exercise, meditation, etc.).

## Ethics

The survey was conducted in accordance with the Swiss Human Research Act (Chapter 1, Section 1, Article 2, Scope: 2c) as it relates to research involving anonymously collected health-related personal data (https://www.fedlex.admin.ch/eli/cc/2013/617/en). Additionally, the company who collected the data in the United States obtained the participants digital informed consent for participation in the survey. The data collection was conducted in compliance with the US law and ethical norms. Participation was voluntary and restricted to those aged 18 years and over. All data was collected anonymously.

## Data analysis

To answer RQ1, we constructed a crosstab with the smartphone health app variables x the participants’ sociodemographic characteristics and conducted a Pearson’s chi-square of independence.

To answer RQ2, we conducted a latent class analysis (LCA) using the “poLCA” R package on the relevant data. Using participants’ item responses, LCA algorithms divide participants into subgroups based on unobservable constructs (latent variables). For a more detailed description of LCA algorithms, see Hagenaars and McCutcheon²⁴ and Weller et al.²⁵ In total, we ran 6 LCA models, with first model nclass = 2, second model nclass = 3, and so on. In each model, the other poLCA function parameter settings were as follows: nrep = 10, na.rm = F, graphs = T, maxiter = 100,000. After making sure that the six models built were well identified (through maximum likelihood estimation), we proceeded to the comparison and model selection. We selected the model with three latent classes because it was the one with best performance metrics (Bayesian Information Criterion [BIC] metrics (where the lowest value is considered better), entropy metrics (where the highest value is considered better)) and interpretability.

To answer RQ3, we computed a crosstab with age x sex x latent-class and calculated the Pearson’s X² of independence.

To answer RQ4, we used the “randomForest” package for R to build a machine learning classification model. For more information on the random forest (RF) algorithm, see Breiman et al.²⁶ and Vera Cruz et al.²⁷ The RF classification models yield, among other outputs, the relative importance of each predictor variable by relying on the mean decrease accuracy (MDA) measure. More specifically, the MDA metric expresses how much accuracy the model would lose by excluding each variable, thus indicating the importance of each variable for the success of the classification. RF are nonparametric and as such are able to capture nonlinear relationships, including interactions among predictors.²⁶ To select the best option, we built three classification models with different fitting parameters. In machine learning, the original dataset has to be split into at least two sets: one to train
the model; the other to estimate the performance of the model in making predictions. In the current study, the dataset was ran- domly subdivided as follows: train-set = 70% of the sample; test-set = 30% of the sample. The fitting parameters for the selected model were as follows: “ntree” = 500, “mtry” = 4. The performance metrics of the selected model on the test-set data were as follows: accuracy score = 0.71 (95% confidence interval 0.65–0.77); No-Information Rate [NIR] = 75; p value (Accuracy [ACC]>NIR) = <0.001; kappa = 0.68.

The use of machine learning algorithms rather than stand-
ard methods can be explained by the fact that these algorithm
hyperparameters allow us to build and test different models
in terms of prediction capabilities and choose the best predic-
tion models according to specific metrics.²⁶ In particular, the
algorithm we used in our analysis (RF) is considered among
the best for the prediction and ranking of the most important
predictor variables.²⁶,²⁸⁻³⁰ It must be noted, however, that
machine learning classification and regression algorithms
do not make inference statistics; this explains why we used
standard methods such as logistic regression to obtain infer-
ence information.

To answer RQ5, we built a multinomial logistic regres-
sion model using the SPSS software (version 28.0).
According to the Likelihood Ratio chi-square test, the full
model was a significant improvement in fit over a null
model (χ²(44) = 647.748, p < 0.001). Pearson’s chi-square
test indicated that the model fit the data well (χ²(3930) =
3776.614, p = 0. 960), as did the deviance chi-square
(χ²(3930) = 3492.038, p = 1.000) (for both, nonsignificant
test results are indicators of good fit³¹,³²).

Results

Descriptive statistics on participants'
smartphone use

The statistics are reported as number of participants (per-
centage of participants).

Participants' smartphone OS distribution was as follows: Android = 1097 (55%); iOS = 874 (44%); other = 6 (03%); don't know = 12 (06%). Daily average smartphone use duration was as follows: Min = 0.27h; Max = 17h; M = 3.33h; standard deviation (SD) = 2.27h; median = 3h. Daily average smartphone use for non-essential activities (12 months preceding the study) was as follows: <1h = 279 (14%); 1–3h = 780 (39%); 3–5h = 527(26%); 5–7h = 188(9%); 7–9h = 113 (6%); >9h = 102 (6%). Participants' experience of social problems attributable to smartphone use (12 months preceding the study) was as follows: definitely no = 1109 (56%); probably no = 557 (28%); probably yes = 262 (13%); definitely yes = 61 (3%). Participants seeking professional help to reduce smartphone use: Yes = 26 (1%); No = 1963 (99%). Participants seeking professional help to deal with social problems due to smartphone problematic use: Yes = 38 (2%); No = 1951 (98%). The

existence of a mental health diagnosis over the participant
life course: Yes = 587 (30%); No = 1402 (70%). And, the
existence of a diagnosis of alcohol use disorder over the
participant life course: Yes = 102 (5%); No = 1887 (95%).

Descriptive statistics of participants' use of apps
to limit smartphone use

As many as 263 (13%) of participants reported having down-
loaded at least one app to limit smartphone use. Among them,
53 (3% of the total sample) downloaded more than one app.
Participants' use by app was as follows: Antisocial (60/
18%), Freedom (25/8%), ZenScreen (21/6%), Moment
(20/6%), BreakFree (20/6%), Space (19/6%), AppDetox
(18/5%), Social Fever (15/5%), Cold Turkey (12/4%),
innerMode (9/3%), F.lux (8/2%), and RealizD (6/2%). In
addition, 95(29%) of participants who downloaded apps to
limit smartphone use reported using "other apps" not
queried about in the survey.

The popularity of SAMHW

Table 1 shows a crosstab of participants’ health app use
variables × sociodemographic variables.

As shown in Table 1, 15% (297 out of 1989) of participants reported having used apps to diagnose mental health conditions such as depression, anxiety, and substance use disorders in the 12 months preceding the study; female and younger adults (19–35 years old) were significantly more likely to be part of this group compared to males, adults (36–60 years), and older adults (61–76 years). Also, 13% (255) of participants reported having used apps to treat mental health conditions such as depression, anxiety, and substance use disorders in the 12 months preceding the study; female and younger adults were more likely to be part of this group compared to males, adults, and older adults. Additionally, 60% (1193) of participants reported having used apps to improve their wellbeing (e.g. meditation apps, relaxation apps, mindfulness apps, exercise apps); younger adults were more likely to be part of this group compared to adults and older adults.

Composition of SAMHW users' latent classes

Table 2 displays the composition of the SAMHW users'
latent classes by age category and sex.

The first latent class (770 participants, 39% of the sample) was labeled *no-app-users* (NAU), because members of this group have low or non-existent probabilities of using any of the relevant smartphone apps (apps to diagnose or treat mental health conditions; apps to improve wellbeing; and apps to limit smartphone use). Adult and older adult participants were significantly more likely to be part of this group compared to young participants.
<table><caption>**Table 1.** SAMHWB: use by age and sex in US adult population.</caption><thead><tr><th rowspan="3">Demographics</th><th colspan="6">Using SAMHWB</th></tr><tr><th colspan="2">ACMH</th><th colspan="2">ACTMH</th><th colspan="2">AIWB</th></tr><tr><th>No</th><th>Yes</th><th>No</th><th>Yes</th><th>No</th><th>Yes</th></tr></thead><tbody><tr><td>Total</td><td>1692 (85)</td><td>297 (15)</td><td>1734 (87)</td><td>255 (13)</td><td>796 (40)</td><td>1193 (60)</td></tr><tr><td>Age</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>19-35</td><td>512 (30)<sup>a</sup></td><td>179 (60)<sup>a</sup></td><td>543 (31)<sup>a</sup></td><td>148 (58)<sup>a</sup></td><td>195 (24)<sup>a</sup></td><td>496 (42)<sup>a</sup></td></tr><tr><td>36-60</td><td>707 (42)<sup>a</sup></td><td>95 (32)<sup>a</sup></td><td>715 (41)<sup>a</sup></td><td>87 (34)<sup>a</sup></td><td>310 (39)<sup>a</sup></td><td>492 (41)<sup>a</sup></td></tr><tr><td>61-76</td><td>473 (28)<sup>a</sup></td><td>23 (8)<sup>a</sup></td><td>476 (28)<sup>a</sup></td><td>20 (8)<sup>a</sup></td><td>291 (37)<sup>a</sup></td><td>205 (17)<sup>a</sup></td></tr><tr><td>Sex</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Male</td><td>848 (50)<sup>a</sup></td><td>117 (39)<sup>a</sup></td><td>869 (50)<sup>a</sup></td><td>96 (38)<sup>a</sup></td><td>407 (51)<sup>a</sup></td><td>558 (47)<sup>a</sup></td></tr><tr><td>Female</td><td>831 (49)<sup>b</sup></td><td>175 (59)<sup>a</sup></td><td>853 (49)<sup>b</sup></td><td>153 (60)<sup>a</sup></td><td>386 (48)<sup>b</sup></td><td>620 (52)<sup>b</sup></td></tr><tr><td>Nonbinary</td><td>13 (1)<sup>c</sup></td><td>5 (2)<sup>a</sup></td><td>12 (1)<sup>c</sup></td><td>6 (2)<sup>a</sup></td><td>3 (1)<sup>c</sup></td><td>15 (1)<sup>c</sup></td></tr></tbody></table>

Data are shown as n(%).

Note: figures with the same exponent in each column are significantly/statistically different (p < 0.05). For example: regarding age in column ACMH, 30% is significantly different from 42% and from 28%, they have the same exponentent (the letter "a"); 28% and 30% are also significantly different. Figures with different exponents in each column are not significantly different.

ACMH: applications to check mental health; ACTMH: applications to treat mental health; AIWB: applications to improve wellbeing; SAMHWB: smartphone applications for mental health and wellbeing.

The second latent class (856 participants, 43% of the sample) was labeled *wellbeing-app-users* (WAU), because members of this group have high probabilities of using smartphone apps to improve wellbeing but low probabilities of using other relevant smartphone apps. Younger adult and adult participants were significantly more likely to belong to this group compared to older adults.

The third latent class (363 participants, 18% of the sample) was labeled *all-app-users* (AAU), because members of this group have high probabilities of using all the smartphone health apps surveyed (apps to diagnose or treat mental health conditions; apps to improve wellbeing; and apps to limit smartphone use). Females and younger adults were significantly more likely to be part of this group compared to males, adults, and older adults.

## The best predictor variables of latent class membership

Figure 1 shows the RF machine learning selected model MDA plot, that is, the 22 predictor variables (in decreasing order of importance) of the three uncovered SAMHW users' latent classes.

As shown in Figure 1, among the 22 predictors included in this model, the 10 most important were:

experienced social problems due to smartphone use (MDA = 14.76); COVID-19 impact on daily smartphone use duration (MDA = 14.11); age (MDA = 12.93); DSM anxiety (MDA = 11.03); DSM memory (MDA = 10.95); daily average smartphone use duration (MDA = 10.92); DSM anger (MDA = 10.62); DSM depression (MDA = 9.90); experienced smartphone "addiction" (MDA = 9.04); and DSM repetitive thoughts-behaviors (MDA = 8.49). The three least important predictor variables were: DSM somatic symptoms (MDA = 2.42); sex (MDA = 2.73); and DSM psychosis (MDA = 3.50). As stated, the value associated with the MDA (e.g. MDA = 14.76 for experienced social problems due to smartphone use) means the model loses that value (14.76 points) on its total accuracy score if the corresponding predictor variable is removed.

The importance of the RF machine learning algorithm for assessing predictor variables is not only that it allows ranking them in descending/ascending order: it can also be used for variable selection and variable multicollinearity checks. Thus, only the predictor variables with a MDA > 2 and the nonmulticollinearity ones were selected for inclusion in the multinomial logistic regression model, the results of which are presented below.
<table><caption>**Table 2.** Composition of the SAMHWB user latent classes by age and sex.</caption><thead><tr><th rowspan="2">Demographics</th><th colspan="4">Latent classes</th></tr><tr><th>NAU</th><th>WAU</th><th>AAU</th><th>Total NBP</th></tr></thead><tbody><tr><td>Total</td><td>770</td><td>856</td><td>363</td><td>1989 (100)</td></tr><tr><td>Age</td><td></td><td></td><td></td><td></td></tr><tr><td>19-35</td><td>176 (23)<sup>a</sup></td><td>306 (36)<sup>a</sup></td><td>209 (58)<sup>a</sup></td><td>691 (35)</td></tr><tr><td>36-60</td><td>304 (29)<sup>a</sup></td><td>373 (44)<sup>b</sup></td><td>125 (34)<sup>a</sup></td><td>802 (40)</td></tr><tr><td>61-76</td><td>290 (38)<sup>a</sup></td><td>177 (21)<sup>a,b</sup></td><td>29 (8)<sup>a</sup></td><td>496 (25)</td></tr><tr><td>Sex</td><td></td><td></td><td></td><td></td></tr><tr><td>Male</td><td>396 (51)<sup>a</sup></td><td>427 (50)<sup>a</sup></td><td>142 (39)<sup>a</sup></td><td>965 (48)</td></tr><tr><td>Female</td><td>372 (48)<sup>b</sup></td><td>420 (49)<sup>b</sup></td><td>214 (59)<sup>a</sup></td><td>1006 (51)</td></tr><tr><td>Non-binary</td><td>2 (1)<sup>c</sup></td><td>9 (1)<sup>c</sup></td><td>7 (2)<sup>a</sup></td><td>18 (1)</td></tr></tbody></table>

Data are shown as n(%).

Note: figures with the same exponent in each column are significantly/ statistically different (p < 0.05). For example: regarding age in column WAU, 36% is not significantly different from 44% (they have different exponents); 21% is significantly different from 44% (they have the same exponent “b”), 36% is not significantly different from 44% (they have different exponents “a” and “b”).

AAU: all-app users; NAU: non-app users; NBP: number of participants; SAMHWB: smartphone applications for mental health and wellbeing; WAU: wellbeing-app users.

## Associations between covariates and the three uncovered SAMHW latent classes

Table 3 shows the summary of the multinomial logistic regression model. In this model, the NAU latent class was set as the reference class and coded “0,” which means that the model was designed to predict the probability of an individual belonging to the WAU latent class (coded “1”) and the AAU latent class (coded “2”).

In this table, the first part (top) represents comparisons between NAU latent class, which is the reference class (baseline), and the WAU latent class. In this part of the table, the covariates significantly associated with the latent classes are: *Covid-19 impact on smartphone daily use duration* (b = 0.199; odds ratio (OR) = 1.220; p < 0.001); *experienced social problems due to problematic smartphone use* (b = 0.338; OR = 1.402; p < 0.001); *DSM anger* (b = 0.145; OR = 1.156; p = 0.049); *age* (b = -0.014; OR = 0.986; p < 0.001); *DSM substance abuse* (b = -0.209; OR = 0.811; p = 0.021). A positive b-coefficient means that an increase in the concerned covariate increases the probability of belonging to the WAU group; a negative b-coefficient means that an increase in the concerned

covariate increases the probability of belonging to the NAU group. In the context of this analysis, the OR values can be interpreted as effect-size statistics. For example, for the *experienced social problems due to problematic smartphone use* covariate, which has an OR of 1.402, for each unit increase in participants’ *experienced social problems due to problematic smartphone use* score, the odds of belonging to the WAU group is 40% greater, after controlling for other predictors.

The second part of the table (bottom) represents comparisons between NAU and AAU latent classes. In this part, the covariates significantly associated with the latent classes are: *Covid-19 impact on smartphone daily use duration* (b = 0.321; OR = 1.378; p < 0.001); *experienced social problems due to problematic smartphone use* (b = 0.531; OR = 1.701; p < 0.001); *DSM mania* (b = 0.238; OR = 1.269; p = 0.022); *DSM anxiety* (b = 0.115; OR = 1.239; p = 0.044); *age* (b = -0.038; OR = 0.963; p < 0.001); *sex* (b = 0.357; OR = 1.429; p = 0.022) (this means female participants are significantly more likely to belong to the AAU latent class); *diagnosed mental health condition* (in the 12 months preceding the study) (b = 1.053; OR = 2.865; p < 0.001).

## Discussion

### Using apps against apps

In total, 13% of participants reported a smartphone “addiction.” Interestingly, the same percentage reported having downloaded an app to limit or control smartphone use. In contrast, only 1% reported having sought professional help to reduce smartphone use, and only 2% to deal with social problems from problematic smartphone use. These findings suggest that individuals who are negatively impacted by smartphone use are opting to bypass traditional care via psychotherapy or psychopharmacology, relying instead on their smartphones. This would be in concordance with other studies and theories related to the potential for smartphone apps in addressing addictive smartphone use³³,³⁴ and is consistent with the move to use “apps to wean us off apps,”²¹ and can be considered another manifestation of how central digital platforms have become to the treatment provision landscape, including when it comes to problems that digital platforms themselves may have created. The telepsychiatry revolution, already unfolding pre-COVID-19 pandemic and greatly aided by the pandemic-related shift to remote delivery,³⁵ can now be seen to encompass problematic smartphone/internet use and internet gaming disorder as target diagnoses. There are other potential reasons for the relatively heavy reliance on apps: A downloadable app can be seen as a relatively easy solution given the structural barriers complicating access to traditional treatment, including availability of relevantly trained providers, affordability, time constraints, and other attitudinal factors.³⁶ The literature
<figure><img src="image_5.png" /><figcaption>**Figure 1.** MDA plot of the random forest machine learning model. It shows, in decreasing order, the importance of each predictor variable for latent class membership. For instance, among the 22 predictors included, the most important is the level of "experienced social problems due to smartphone use" (MDA = 14.76). DSM: Diagnostic and Statistical Manual of Mental Disorders; MDA: mean decrease accuracy; SA: smartphone "addiction"; SPDSU: social problems due to smartphone use; SUD: smartphone use duration; USNA: using smartphone on non-essential activities.</figcaption></figure>

suggests real limits to how much benefit can be drawn from standalone interventions, however, including due to app engagement issues. As such, evidence-based studies indicate the need for professional accompaniment for durable effectiveness,³⁷⁻⁴¹ making blended treatment and stepped-care models (combining app and human clinical support) reasonable recommendations, including for technology-related health issues.⁴²,⁴³

## The popularity of mental health and wellbeing apps

A rather moderate proportion—almost half (44%)—of participants reported using a smartphone mental health or wellbeing

app in the 12 months preceding the study. Their reported goals were to diagnose (15%) or treat (13%) mental health conditions or to enhance wellbeing (16%). The present study findings on prevalence of use are consistent with previous studies that indicated low-to-moderate adoption.¹²,¹⁶,¹⁷ The fact that females and younger adults (19–35 years) were statistically more likely to be part of this group than males, adults (36–60 years), and older adults (61–76 years), is also consistent with other studies,⁴⁴ and has been explained as a function of the high levels of interaction by this group with lifestyle apps and social media.⁴⁴ It is also consistent with the comparative reluctance by males to disclose, and seek treatment for, mental health problems, as widely reported.⁴⁵,⁴⁶
<table><caption>**Table 3.** The estimated beta coefficients of the associations between the covariates and the SAMHWB user latent classes.</caption><thead><tr><th>Latent classes</th><th>Covariates</th><th>b</th><th>OR</th><th>SE</th><th>p</th><th>95% CI</th></tr></thead><tbody><tr><td>WAU</td><td>COVID-19 impact on smartphone daily use duration</td><td>0.199</td><td>1.220</td><td>0.050</td><td>&lt;0.001*</td><td>1.107-1.345</td></tr><tr><td>WAU</td><td>Daily average smartphone use duration</td><td>-0.013</td><td>0.988</td><td>0.031</td><td>0.687</td><td>0.929-1.050</td></tr><tr><td>WAU</td><td>Daily average smartphone use duration for non-essential activities</td><td>0.045</td><td>1.046</td><td>0.053</td><td>0.398</td><td>0.942-1.161</td></tr><tr><td>WAU</td><td>Experienced social problems due to problematic smartphone use</td><td>0.338</td><td>1.402</td><td>0.086</td><td>&lt;0.001*</td><td>1.183-1.661</td></tr><tr><td>WAU</td><td>Experienced smartphone "addiction"</td><td>0.016</td><td>1.016</td><td>0.012</td><td>0.193</td><td>0.992-1.041</td></tr><tr><td>WAU</td><td>DSM depression</td><td>-0.145</td><td>0.865</td><td>0.089</td><td>0.104</td><td>0.727-1.030</td></tr><tr><td>WAU</td><td>DSM anger</td><td>0.145</td><td>1.156</td><td>0.073</td><td>0.049*</td><td>1.001-1.335</td></tr><tr><td>WAU</td><td>DSM mania</td><td>0.083</td><td>1.087</td><td>0.087</td><td>0.337</td><td>0.917-1.289</td></tr><tr><td>WAU</td><td>DSM anxiety</td><td>0.009</td><td>1.009</td><td>0.090</td><td>0.922</td><td>0.846-1.203</td></tr><tr><td>WAU</td><td>DSM somatic symptoms</td><td>0.051</td><td>1.053</td><td>0.082</td><td>0.529</td><td>0.897-1.235</td></tr><tr><td>WAU</td><td>DSM suicidal ideation</td><td>-0.191</td><td>0.826</td><td>0.121</td><td>0.117</td><td>0.651-1.049</td></tr><tr><td>WAU</td><td>DSM psychosis</td><td>0.006</td><td>1.006</td><td>0.198</td><td>0.977</td><td>0.683-1.482</td></tr><tr><td>WAU</td><td>DSM sleep problems</td><td>-0.012</td><td>0.988</td><td>0.054</td><td>0.819</td><td>0.889-1.098</td></tr><tr><td>WAU</td><td>DSM memory</td><td>-0.040</td><td>0.961</td><td>0.077</td><td>0.605</td><td>0.826-1.118</td></tr><tr><td>WAU</td><td>DSM repetitive thoughts behaviors</td><td>-0.109</td><td>0.897</td><td>0.122</td><td>0.373</td><td>0.706-1.139</td></tr><tr><td>WAU</td><td>Age</td><td>-0.014</td><td>0.986</td><td>0.004</td><td>&lt;0.001*</td><td>0.978-0.993</td></tr><tr><td>WAU</td><td>DSM dissociation</td><td>0.046</td><td>1.047</td><td>0.107</td><td>0.665</td><td>0.849-1.292</td></tr><tr><td>WAU</td><td>DSM personality functioning</td><td>0.170</td><td>1.186</td><td>0.090</td><td>0.059</td><td>0.994-1.414</td></tr><tr><td>WAU</td><td>DSM substance abuse</td><td>-0.209</td><td>0.811</td><td>0.090</td><td>0.021*</td><td>0.680-0.968</td></tr><tr><td>WAU</td><td>Sex</td><td>0.056</td><td>1.057</td><td>0.107</td><td>0.601</td><td>0.858-1.303</td></tr><tr><td>WAU</td><td>Diagnosed alcohol use disorder</td><td>-0.165</td><td>0.848</td><td>0.264</td><td>0.531</td><td>0.505-1.423</td></tr><tr><td>WAU</td><td>Diagnosed mental health condition</td><td>0.178</td><td>1.195</td><td>0.140</td><td>0.205</td><td>0.907-1.574</td></tr><tr><td>AAU</td><td>COVID-19 impact on smartphone daily use duration</td><td>0.321</td><td>1.378</td><td>20.819</td><td>&lt;0.001*</td><td>1.201-1.582</td></tr><tr><td>AAU</td><td>Daily average smartphone use duration</td><td>0.018</td><td>1.018</td><td>0.207</td><td>0.649</td><td>0.942-1.101</td></tr><tr><td>AAU</td><td>Daily average duration using smartphone for non-essential activities</td><td>0.127</td><td>1.136</td><td>2.855</td><td>0.091</td><td>0.980-1.316</td></tr><tr><td>AAU</td><td>Experienced social problems due to problematic smartphone use</td><td>0.531</td><td>1.701</td><td>24.379</td><td>&lt;0.001*</td><td>1.378-2.100</td></tr><tr><td>AAU</td><td>Experienced smartphone "addiction"</td><td>0.033</td><td>1.033</td><td>3.701</td><td>0.054</td><td>0.999-1.069</td></tr><tr><td>AAU</td><td>DSM depression</td><td>0.067</td><td>1.069</td><td>0.335</td><td>0.562</td><td>0.853-1.341</td></tr></tbody></table>

(continued)
<table><caption>**Table 3. Continued.**</caption><thead><tr><th>Latent classes</th><th>Covariates</th><th>b</th><th>OR</th><th>SE</th><th>p</th><th>95% CI</th></tr></thead><tbody><tr><td>AAU</td><td>DSM anger</td><td>0.119</td><td>1.127</td><td>1.578</td><td>0.209</td><td>0.935-1.357</td></tr><tr><td>AAU</td><td>DSM mania</td><td>0.238</td><td>1.269</td><td>5.267</td><td>0.022*</td><td>1.035-1.556</td></tr><tr><td>AAU</td><td>DSM anxiety</td><td>0.115</td><td>1.239</td><td>3.699</td><td>0.044*</td><td>0.996-1.542</td></tr><tr><td>AAU</td><td>DSM somatic symptoms</td><td>0.099</td><td>1.104</td><td>0.980</td><td>0.322</td><td>0.908-1.341</td></tr><tr><td>AAU</td><td>DSM suicidal ideation</td><td>-0.143</td><td>0.867</td><td>1.243</td><td>0.265</td><td>0.675-1.114</td></tr><tr><td>AAU</td><td>DSM psychosis</td><td>0.096</td><td>1.101</td><td>0.216</td><td>0.642</td><td>0.735-1.648</td></tr><tr><td>AAU</td><td>DSM sleep problems</td><td>0.011</td><td>1.011</td><td>0.022</td><td>0.883</td><td>0.877-1.165</td></tr><tr><td>AAU</td><td>DSM memory</td><td>0.001</td><td>1.001</td><td>0.000</td><td>0.993</td><td>0.838-1.195</td></tr><tr><td>AAU</td><td>DSM repetitive thoughtsbehaviors</td><td>-0.155</td><td>0.856</td><td>1.270</td><td>0.260</td><td>0.653-1.122</td></tr><tr><td>AAU</td><td>Age</td><td>-0.038</td><td>0.963</td><td>41.321</td><td>&lt;0.001*</td><td>0.952-0.974</td></tr><tr><td>AAU</td><td>DSM dissociation</td><td>-0.006</td><td>0.994</td><td>0.003</td><td>0.956</td><td>0.789-1.251</td></tr><tr><td>AAU</td><td>DSM personality functioning</td><td>0.266</td><td>1.305</td><td>6.361</td><td>0.012</td><td>1.061-1.605</td></tr><tr><td>AAU</td><td>DSM substance abuse</td><td>0.002</td><td>1.002</td><td>0.000</td><td>0.985</td><td>0.793-1.267</td></tr><tr><td>AAU</td><td>Sex</td><td>0.357</td><td>1.429</td><td>5.207</td><td>0.022*</td><td>1.052-1.941</td></tr><tr><td>AAU</td><td>Diagnosed alcohol use disorder</td><td>-0.062</td><td>0.940</td><td>0.036</td><td>0.850</td><td>0.496-1.781</td></tr><tr><td>AAU</td><td>Diagnosed mental health condition</td><td>1.053</td><td>2.865</td><td>33.954</td><td>&lt;0.001*</td><td>2.011-4.082</td></tr></tbody></table>

The top part of the table represents comparisons between NAU latent class, which is the reference class (baseline), and the WAU latent class. The bottom part represents comparisons between NAU latent class and the AAU latent class.

AAU: all-app users; b: beta coefficient; CI: confidence interval; DSM: Diagnostic and Statistical Manual of Mental Disorders; NAU: non-app users; OR: odds ratio; p: p-value; SAMHWB: smartphone applications for mental health and wellbeing; SE: standard error; WAU: wellbeing-app users.

## Latent classes of SAMHW users

While latent class analysis suggests that about two-thirds of participants had a high probability for using the relevant smartphone apps, the group of participants with high probability for using them to improve wellbeing (43%) was more than double the size of that with high probability for using them for mental health problems (18%). One explanation may lie in the fact that wellbeing is a broader concept that encompasses physical and mental health. Another explanation may be the desire to avoid the stigma still attached to psychiatric diagnoses such as major depressive disorder that would require “treatment,” and an easier time acknowledging the need to improve overall “wellbeing” through more “normalized” interventions such as “yoga” and “mindfulness.” Younger adults and adults were statistically more likely to belong to the first group compared to older adults, while females and younger adults were

statistically more likely to belong to the second group compared to males, adults, and older adults. This is unsurprising given that females and younger individuals are more likely to interact with social media and digital interfaces, and, therefore, may have fewer psychological and proficiency barriers when it comes to using them to address health problems.

## Predictors and relationships with participants categories

According to the machine learning model, the two most important predictors for using the relevant smartphone apps were variables associated with smartphone problematic use and COVID-19 impact. This suggests that pandemic-related stress (e.g. confinement, threatened health, professional and financial anxiety, etc.) and the sudden move to an even more heavily smartphone-

<footnote>* Significant at p < 0.05.</footnote>
dependent lifestyle may have contributed to an increase in mental health symptoms, smartphone use, and openness to finding help online, since almost all else was being pursed remotely.⁴⁷ Perhaps predictably, among the remaining important predictors, the large majority was linked to mental health problems (e.g. DSM anxiety, DSM anger, DSM depression, DSM memory, and DSM repetitive thoughts behaviors) and smartphone “addiction.” The level of insight into mental health symptoms could have influenced the desire to use the relevant apps in our study.

As for latent class covariates, individuals with high values or scores on age and DSM substance use disorder covariates were statistically more likely to belong to the NAU category of participants than the two other categories. As previously noted, older adults are less likely to interact with non-utilitarian mobile technology, perhaps due to technology knowhow barriers or being more accustomed to conventional ways.⁴⁴,⁴⁸ Also, several studies have shown strong evidence for comorbidity between mental health problems and substance use disorders,⁴⁹ suggesting a shared etiological origin or a causality relationship. For instance, it might be that some individuals experiencing mental health problems gravitate to substance use for coping purposes instead of seeking treatment, including treatment via smartphone apps.

There was no clear pattern regarding use of mental health versus wellbeing apps. Participants with high scores on the COVID-19 impact, social problems from smartphone use, and DSM anger covariates were statistically more likely to belong to the WAU and the AAU categories than to the NAU one. Individuals with high scores on the DSM mania and DSM anxiety covariates, individuals diagnosed with a mental health condition, and females were statistically more likely to be part of the AAU category than the other two. Individuals with a high score on DSM anger were statistically more likely to belong to WAU category than the other two. It would make sense that individuals experiencing social problems and mental health problems, broadly defined, would be more likely to seek help, including through mobile wellbeing and mental health apps.

## Limitations

The current study did not include several variables known to influence digital literacy and that might impact the use of smartphone wellbeing and mental health apps⁵⁰,⁵¹ and this represents a limitation in terms of results interpretability. App language was also not covered, representing another limitation.

Also, the sample representativeness was based only on age, sex, and ethnicity. Furthermore, information regarding the participants' education level, socioeconomic classifications, and geographic location was not available. That these details were not covered in our survey represents a limitation in terms of the interpretability and generalizability of our results. However, it is important to note that previous

studies reported high-quality sample representativeness using the Prolific crowdsourcing platform, in comparison to probabilistic survey conducted by the Pew Research Center for questions about user perceptions and experiences.⁵²,⁵³

A further limitation is linked to the fact that 95(29%) participants who downloaded apps to limit smartphone use reported using “other apps” not queried about in the survey. In the absence of any comprehensive scholarly survey of this new field, the list of smartphone apps to limit smartphone use that were included in the study was based on media and app-company reports of new or popular apps at the time the study was conceived. In this fast-moving field, it is possible that new apps had come to light by the time the study was conducted. It is also possible that respondents were using other apps (e.g. news or wellbeing apps) as “healthier” options to replace or limit the use of more problematic ones. A more exhaustive list of apps to limit smartphone use might have led to a better understanding of participants’ behavior regarding their smartphone use control.

## Conclusion

Findings from the present study confirm that the use of SAMHW is growing, particularly among younger adults and females who are negatively impacted by smartphone problematic use or COVID-19, or who might be experiencing psychological problems. The range of “symptoms” users appear to be targeting through their smartphones is broad and includes problems related to the smartphone itself. These individuals tend to bypass traditional care via psychotherapy or psychopharmacology, relying instead on smartphones to address mental health conditions or improve wellbeing. However, as studies on effectiveness of SAMHW recommend coupling these digital tools with professional help,³⁷⁻³⁹ it would make sense to ensure that these apps advise their users to seek professional help in parallel with using them or if they do not experience improvement in wellbeing or mental health. More studies are required to explore the efficacy and safety of standalone apps as a potentially important component of mobile telepsychiatry; their differential uptake across age, sex, and other demographics; the role of stigma in their adoption; and the possibility of enhancing efficacy through hybrid models.

## Abbreviations

<table><tr><td>AAU</td><td>all-app users</td></tr><tr><td>ADMH</td><td>apps to diagnose mental health</td></tr><tr><td>AIW</td><td>apps to improve wellbeing</td></tr><tr><td>Apps</td><td>applications</td></tr><tr><td>ATMH</td><td>apps to treat mental health</td></tr><tr><td>DSM</td><td>Diagnostic and Statistical Manual of Mental Disorders</td></tr><tr><td>LCA</td><td>latent class analysis</td></tr></table>
<table><tr><td>MDA</td><td>mean decrease accuracy</td></tr><tr><td>NAU</td><td>non-app users</td></tr><tr><td>OR</td><td>odds ratio</td></tr><tr><td>RF</td><td>random forest</td></tr><tr><td>RQ</td><td>research question</td></tr><tr><td>SA</td><td>smartphone "addiction"</td></tr><tr><td>SAMHW</td><td>smartphone apps for mental health and wellbeing</td></tr><tr><td>WAU</td><td>wellbeing-app users.</td></tr></table>

**Acknowledgments:** The authors wish to acknowledge the Prolific team for their help in conducting the survey.

**Contributorship:** EA and YK were involved in initial conception of the study. EA, YK, LR, FBB, RC, and RK were involved in conception of the survey and writing of the questions. RC, RK, FBB, and YK were involved in recruitment procedure and GVC in data analysis. EA, GVC, and YK prepared the first draft and all the authors gave the final draft.

**Declaration of Conflicting Interests:** The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

**Funding:** The author(s) received no financial support for the research, authorship, and/or publication of this article.

**Ethics approval and consent to participate:** Participants gave digital informed consent for their survey contribution. Participation was voluntary and restricted to those aged ≥18 years. All data was anonymously collected. The survey was conducted in accordance with the Swiss Human Research Act.¹⁴

**Availability of data and materials:** The material used in this study and data supporting this finding is available at: https://gitlab.huma-num.fr/gveracruz/smartphone_problematic_use_behavior/-/tree/main/Material%20and%20data

**Guarantor:** YK.

**ORCID iD:** Germano Vera Cruz https://orcid.org/0000-0002-8297-6933

**References**

1. Poushter J, Bishop C and Chwe H. Social media use continues to rise in developing countries but plateaus across developed ones. http://www.pewglobal.org/2018/06/19/social-media-use-continues-to-rise-in-developing-countries-but-plateaus-across-developed-ones/ (2018, accessed 20 April 2022).

2. World Health Organization. Global strategy on digital health 2020-2025. WHO. https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf (2021, accessed 21 April 2022).

3. Almoallim S and Sas C. Functionalities review of digital well-being apps: towards research-informed design implications for interventions limiting smartphone use. *JMIR Form Res* 2022; 6: e31730. PMID: 35188897.

4. Torous J and Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. *JAMA Psychiatry* 2017; 74: 437–438.

5. Kwon M, Lee JY, Won WY, et al. Development and validation of a smartphone addiction scale (SAS). *PLoS One* 2013; 8: e56936.

6. Penzenstadler L, Chatton A, Van Singer M, et al. Quality of smartphone apps related to alcohol use disorder. *Eur Addict Res* 2016; 22: 329–338. PMID: 27598779.

7. Van Singer M, Chatton A and Khazaal Y. Quality of smartphone apps related to panic disorder. *Front Psychiatry* 2015; 6: 96. DOI: 10.3389/fpsyt.2015.00096.

8. Firth J, Torous J, Nicholas J, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. *J Affect Disord* 2017; 218: 15–22.

9. Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. *World Psychiatry* 2017; 16: 287–298.

10. Heber E, Ebert DD, Lehr D, et al. The benefit of web- and computer-based interventions for stress: a systematic review and meta-analysis. *JMIR* 2017; 19: e32.

11. Lecomte T, Potvin S, Corbière M, et al. Mobile apps for mental health issues: meta-review of meta-analyses. *JMIR mHealth uHealth* 2020; 8: e17458.

12. Paslakis G, Fischer-Jacobs J, Pape L, et al. Assessment of use and preferences regarding internet-based health care delivery: cross-sectional questionnaire study. *JMIR* 2019; 21: e12416.

13. Stawarz K, Priest C and Coyle D. Use of smartphone apps, social media, and web-based resources to support mental health and well-being: online survey. *JMIR Ment Health* 2019; 6: e12546.

14. Apolinário-Hagen J, Hennemann S, Fritsche L, et al. Determinant factors of public acceptance of stress management apps: survey study. *JMIR* 2020; 6: e15373. DOI: 10.2196/15373.

15. Szinay D, Jones A, Chadborn T, et al. Influences on the uptake of and engagement with health and well-being smartphone apps: systematic review. *JMIR* 2020; 22: e17572.

16. Atallah N, Khalifa M, El Metwally A, et al. The prevalence and usage of mobile health applications among mental health patients in Saudi Arabia. *Comput Methods Programs Biomed* 2018; 156: 163–168.

17. Iliescua R, Kumaravela A, Smurawskaa L, et al. Smartphone ownership and use of mental health applications by psychiatric inpatients. *Psychiatry Res* 2021; 299: 113806.

18. Dave S, Abraham S, Ramkisson R, et al. Digital psychiatry and COVID-19: the Big Bang effect for the NHS? *BJ Psych Bull* 2021; 45: 259–263. PMID: 33081867.

19. Wang X, Markert C and Sasangohar F. Investigating popular mental health mobile application downloads and activity during the COVID-19 pandemic. *Int J Hum Factors Ergon* 2021; 0: 001872082199811. DOI: 10.1177/0018720821998110.

20. Peer E, Samat S, Brandimarte L, et al. Beyond the Turk: alternative platforms for crowdsourcing behavioral research. *J Exp Soc Psychol* 2017; 70: 153–163.
21. Aboujaoude E. Problematic Internet use two decades later: apps to wean us off apps. *CNS Spectr* 2019; 24: 371–373.

22. Csibi S, Griffiths MD, Cook B, et al. The psychometric properties of the Smartphone Application-Based Addiction Scale (SABAS). *Int J Ment Health Addict* 2018; 16: 393–403.

23. Mahoney MR, Farmera C, Sinclaira S, et al. Utilization of the DSM-5 self-rated level 1 cross-cutting symptom measure-adult to screen healthy volunteers for research studies. *Psychiatry Res* 2020; 286: 112822. PMID: 32086029.

24. Hagenaars JA and McCutcheon AL. *Applied latent class analysis*. Cambridge, UK: Cambridge University Press, 2002.

25. Weller BE, Bowen NK and Faubert SJ. Latent class analysis: a guide to best practice. *J Black Psychol* 2020; 46: 287–311.

26. Breiman L. Random forests. *Mach Learn* 2001; 45: 5–32.

27. Vera Cruz G, Bucourt E, Réveillère C, et al. Machine learning reveals the most important psychological and social variables predicting the differential diagnosis of rheumatic and musculoskeletal diseases. *Rheumatol Int* 2022; 42: 1053–1062.

28. Piryonesi SM and El-Diraby TE. Role of data analytics in infrastructure asset management: overcoming data size and quality problems. *J Transp Eng Part B Pavements* 2020; 146: 04020022.

29. Pirneskoski J, Tamminen J, Kallonen A, et al. Random forest machine learning method outperforms prehospital National Early Warning Score for predicting one-day mortality: a retrospective study. *Resusc Plus* 2020; 4: 100046.

30. Smith PF, Ganesh S and Ping L. A comparison of random forest regression and multiple linear regression for prediction in neuroscience. *J Neurosci Methods* 2013; 220: 85–91.

31. Field A. *Discovering statistics using IBM SPSS statistics*. 5th ed. Los Angeles: Sage, 2018.

32. Petrucci CJ. A primer for social worker researchers on how to conduct a multinomial logistic regression. *J Soc Serv Res* 2009; 35: 193–205.

33. Van Velthoven MH, Powell J and Powell G. Problematic smartphone use: digital approaches to an emerging public health problem. *Digital Health* 2018; 4: 205520761875916.

34. Monney G, Penzenstadler L, Dupraz O, et al. mHealth app for cannabis users: satisfaction and perceived usefulness. *Front in Psychiatry* 2015; 6: 120. DOI: 10.3389/fpsyt.2015.00120.

35. Reeves JJ, Ayers JW and Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. *J Med Internet Res* 2021; 23: e24785.

36. Weisel KK, Fuhrmann LM, Berking M, et al. Standalone smartphone apps for mental health—a systematic review and meta-analysis. *NPJ Digit Med* 2019; 2: 118.

37. Bychkov D, Young SD. Facing up to nomophobia: a systematic review of mobile phone apps that reduce smartphone usage. In: Sekhar S, Samui RP, Deo R, et al. (eds) *Big data in engineering applications*. Sage, 2018. DOI: 10.1007/978-981-10-8476-8_8.

38. Loid K, Täht K and Rozgonjuk D. Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. *Comput Hum Behav* 2020; 102: 22–30.

39. Zimmermann L. Your screen-time app is keeping track: consumers are happy to monitor but unlikely to reduce smartphone usage. *J Consum Res* 2021; 6: 377–382.

40. Whitehead L and Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. *J Med Internet Res* 2016; 18: e97.

41. Fleming TM, De Beurs D, Khazaal Y, et al. Maximizing the impact of e-therapy and serious gaming: time for a paradigm shift. *Front Psychiatry* 2016; 7: 65. DOI: 10.3389/fpsyt.2016.00065.

42. LaMonica HM, Roberts AE, Davenport TA, et al. Evaluation of the usability and acceptability of the InnoWell platform as rated by older adults: survey study. *J Med Internet Res Aging* 2021; 4: e25928.

43. Benarous X, Edel Y, Consoli A, et al. Ecological momentary assessment and smartphone application intervention in adolescents with substance use and comorbid severe psychiatric disorders: study protocol. *Front Psychiatry* 2016; 7: 157. DOI: 10.3389/fpsyt.2016.00157.

44. Noëa B, Turnera LD, Lindenb DEJ, et al. Identifying indicators of smartphone addiction through user-app interaction. *Comput Hum Behav* 2019; 99: 56–65.

45. Mental Health Foundation. Survey of people with lived experience of mental health problems reveals men less likely to seek medical support. MHF, https://www.mentalhealth.org.uk/news/survey-people-lived-experience-mental-health-problems-reveals-men-less-likely-seek-medical (2016, accessed 21 April 2022).

46. Matheson FI, Smith KLW, Fazli GS, et al. Physical health and gender as risk factors for usage of services for mental illness. *J Epidemiol Community Health* 2014; 68: 971–978.

47. Cui J, Lu J, Weng Y, et al. COVID-19 impact on mental health. *BMC Med Res Methodol* 2022; 22: 1–11. DOI: 10.1186/s12874-021-01411-w.

48. Den Haan M, Brankaert R, Kenning G, et al. Creating a social learning environment for and by older adults in the use and adoption of smartphone technology to age in place. *Front Public Health* 2021; 9: 568822. DOI: 10.3389/fpubh.2021.568822.

49. National Institute on Drug Abuse. Why is there comorbidity between substance use disorders and mental illnesses? Research Report, https://nida.nih.gov/download/1155/common-comorbidities-substance-use-disorders-research-report.pdf?v=5d6a5983e0e9353d46d01767fb20354b (2020, accessed 21 April 2022).

50. Svendsen MT, Bak CK, Sørensen K, et al. Associations of health literacy with socioeconomic position, health risk behavior, and health status: a large national population-based survey among Danish adults. *BMC Public Health* 2020; 20: 1–12.

51. Manganello J, Gerstner G, Pergolino K, et al. The relationship of health literacy with use of digital technology for health information: implications for public health practice. *J Public Health Manag Pract* 2017; 23: 380–387.

52. Peer E, Rothschild D, Gordon A, et al. Data quality of platforms and panels for online behavioral research. *Behav Res* 2022; 54: 1643–1662.

53. Tang J, Birrell E and Lerner A. How well do my results generalize now? The External validity of online privacy and security surveys. arXiv preprint arXiv:2202.14036, 2022. DOI: 10.48550/arXiv.2202.14036.