JMIR FORMATIVE RESEARCH

Rutter et al

Original Paper“I Haven’t Been Diagnosed, but I Should Be”—Insight Into
Self-diagnoses of Common Mental Health Disorders:
Cross-sectional Study
Lauren A Rutter1, PhD; Jacqueline Howard1, BA; Prabhvir Lakhan1; Danny Valdez2, PhD; Johan Bollen3, PhD;
Lorenzo Lorenzo-Luaces1, PhD
1

Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN, United States

2

Department of Applied Health Science, School of Public Health, Indiana University Bloomington, Bloomington, IN, United States3

Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN, United States

Corresponding Author:
Lauren A Rutter, PhD
Department of Psychological and Brain Sciences
Indiana University Bloomington
1101 E Tenth Street
Bloomington, IN, 47405
United States
Phone: 1 8128569953
Email: larutter@iu.edu

Abstract
Background: In recent years, social media has become a rich source of mental health data. However, there is a lack of web-based
research on the accuracy and validity of self-reported diagnostic information available on the web.
Objective: An analysis of the degree of correspondence between self-reported diagnoses and clinical indicators will afford
researchers and clinicians higher levels of trust in social media analyses. We hypothesized that self-reported diagnoses would
correspond to validated disorder-specific severity questionnaires across 2 large web-based samples.
Methods: The participants of study 1 were 1123 adults from a national Qualtrics panel (mean age 34.65, SD 12.56 years; n=635,
56.65% female participants,). The participants of study 2 were 2237 college students from a large university in the Midwest (meanage 19.08, SD 2.75 years; n=1761, 75.35% female participants). All participants completed a web-based survey on their mental
health, social media use, and demographic information. Additionally, the participants reported whether they had ever been
diagnosed with a series of disorders, with the option of selecting “Yes”; “No, but I should be”; “I don’t know”; or “No” for eachcondition. We conducted a series of ANOVA tests to determine whether there were differences among the 4 diagnostic groups
and used post hoc Tukey tests to examine the nature of the differences.
Results: In study 1, for self-reported mania (F3,1097=2.75; P=.04), somatic symptom disorder (F3,1060=26.75; P<.001), andalcohol use disorder (F3,1097=77.73; P<.001), the pattern of mean differences did not suggest that the individuals were accurate
in their self-diagnoses. In study 2, for all disorders but bipolar disorder (F3,659=1.43; P=.23), ANOVA results were consistent
with our expectations. Across both studies and for most conditions assessed, the individuals who said that they had been diagnosed
with a disorder had the highest severity scores on self-report questionnaires, but this was closely followed by individuals who
had not been diagnosed but believed that they should be diagnosed. This was especially true for depression, generalized anxiety,and insomnia. For mania and bipolar disorder, the questionnaire scores did not differentiate individuals who had been diagnosed
from those who had not.
Conclusions: In general, if an individual believes that they should be diagnosed with an internalizing disorder, they areexperiencing a degree of psychopathology similar to those who have already been diagnosed. Self-reported diagnoses correspondwell with symptom severity on a continuum and can be trusted as clinical indicators, especially in common internalizing disorders
such as depression and generalized anxiety disorder. Researchers can put more faith into patient self-reports, including those in
web-based experiments such as social media posts, when individuals report diagnoses of depression and anxiety disorders.
However, replication and further study are recommended.

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Rutter et al(JMIR Form Res 2023;7:e39206) doi: 10.2196/39206
KEYWORDS
assessment; depression; anxiety; self-report; social media

Mental disorders are one of the leading public health problems
worldwide [1-3], yet they are often underdetected or
misdiagnosed, particularly in underrepresented groups (sexual,
gender, racial, and ethnic minorities) and young adults [4-8].
The stigma of mental illness [9], concerns about confidentiality,
shame, or a variety of other reasons lead some individuals to
avoid explicit help-seeking behaviors that could alleviate thesymptoms of mental disorders [10]. Some turn to social media
and web-based resources to share and better understand theirmental health symptoms and experiences [11].

variety of mental health concerns, ranging from “daily grind”
to highly specific questions about their diagnoses and treatment.
An analysis of subreddits on “mental health,” that is, Reddit
channels on specific topics, showed that individuals who are apart of subreddits related to mental health used more negative
emotion words and more frequently disclosed challenges ofdaily activities than participants (“redditors”) in more general
Reddit channels. In addition, anonymous redditors tended to be
more disinhibited and share more details about their personal
relationships and health issues in casual terms that did not reflect
their current mental health status. In summary, anonymous
Reddit accounts (“throwaway Reddit accounts”) showed greaterlevels of self-disclosure related to mental health and decreased
feelings of vulnerability [18].

Prior Work

Goal of This WorkIn recent years, social media has become a rich source of data
for studies of mental health communication, disclosure,prevalence rates, and trajectories [12]. In the medical field, moregenerally, data science approaches have allowed researchers tomine large health care data sets from social media [13]. In asystematic review of how social media data have been used toclassify and predict the mental health state of users [14], text
analysis was found to be the most common analytic method
across 48 studies, in particular, the use of natural language
processing and unsupervised machine learning algorithms toanalyze patterns of term use. For example, ten Thij et al [15]searched on Twitter (Twitter Inc) for individuals who hadreported receiving a diagnosis of depression on the platform
and compared the timing of their posts with those of a random
sample of Twitter users, finding differences in the pattern ofactivity by time of day. Specifically, the individuals whoreported depression diagnoses showed more web-based activity
during later hours of the day. Similarly, Bathina et al [16]showed the frequency of occurrence of cognitive distortion
schema, sequences of 1 to 5 words associated with theexpression of cognitive distortions, in the language ofindividuals who reported a clinical diagnosis of depression on
Twitter versus in the language of a random sample ofindividuals.

Although text analysis is the most commonly used method tostudy mental disorders at scale, the reliability and validity of
web-based data have been called into question, particularly withrespect to self-reported diagnoses. Although we have reason tobelieve that individuals tend to be relatively accurate reportersof their mental health experiences [19], relying on text analysis
alone may be perceived as insufficient to establish ground truth.
Although clinician reports may provide an alternative toself-reports, they are more time intensive to collect at scale,require access to care, and are costly [20,21]. These concerns
warrant a multimodal approach that combines language
modeling, sentiment, clinical diagnostics, and user engagement
to reliably study common psychiatric disorders from social
media data [22,23]. Crucial to many of these efforts is the ability
to triangulate self-reported diagnostic information with clinicalindicators. This is the idea that motivated our study, which asked
the following question: Do self-reported diagnoses correspondto validated disorder-specific severity questionnaire scores?

Introduction
Background

Analyses of social media data, such as the works of ten Thij etal [15], Bathina et al [16], and Coppersmith et al [17], havehistorically relied on individuals’ self-disclosure of mental
health status or search terms related to mental health disorders,for example, someone tweeting “I was just diagnosed withdepression by my doctor.” A key limitation of these approaches
is self-selection bias, which results, in part, from relying on
people’s willingness to reveal stigmatized identities. In addition,
people who are willing to self-disclose their mental health status
on a public platform may be less inhibited or more likely toseek social support but may not be representative of all those
who struggle with their mental health. For example, in 2014,
De Choudhury and De [18] studied the self-disclosure of mental
health problems on Reddit. On Reddit, individuals discuss a
https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

A better understanding of the degree of correspondence between
self-reported diagnoses and clinical indicators will afford
researchers and clinicians higher levels of trust in self-reported
diagnoses on social media. This is particularly the case for acommon approach that relies on web-based statements ofself-reported diagnoses to establish clinically relevant samples[15-18].
The primary aim of this study was to examine whether people
are “accurate” when they report that they are or should be
diagnosed with a mental disorder. In other words, does anindividual’s belief that they should be diagnosed with a disordercorrespond to their scores on validated self-report measurementsof their symptoms. We addressed this research question using
two large-scale samples of thousands of individuals: (1) anationally representative sample of adults and (2) a large sampleof university undergraduates. The participants were asked tofill out self-report questionnaires on a variety of mental disorder
symptoms and asked whether they had ever been diagnosed
with any conditions or felt that they should be diagnosed withJMIR Form Res 2023 | vol. 7 | e39206 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
those. We first hypothesized that there would be a positive
relationship between self-reported disorder-specific severity
scores and having received a diagnosis. Second, we expectedthat there would be significant differences between theparticipants who believed that they should receive a diagnosis
and those who reported that they had never been diagnosed.
Finally, we hypothesized that the group of participants whobelieved that they should be diagnosed would look more similar
to the participants who had been diagnosed than to those who
had never been diagnosed, lending initial credibility and support
for relying on self-diagnoses in future research.

Rutter et al

This study was approved by the institutional review board ofIndiana University (#2002549202 and #2005948214).

national Qualtrics panel of Twitter users (study 1) and (2) apsychology student pool at a large Midwestern university in the
United States, the participants from which received credit forparticipation (study 2). In both studies, the participants
completed a battery of questionnaires entirely on the web. Our
inclusion criteria consisted of passing reCAPTCHA andaffirming “Yes” to the question, “I will provide my bestanswers.” After excluding the individuals who could not pass
the reCAPTCHA or did not commit to providing truthfulanswers, a total of 3460 participants responded to our surveys.
Although the procedural elements and questionnaires completed
by the participants were nearly identical, we have first presented
the nationally representative sample of Qualtrics participants(study 1) and then presented the college student sample (study
2). Data were collected between July 2020 and April 2022.
Additional data collection is ongoing for facets of a larger
project unrelated to this study.

Recruitment

Participants

The participants were recruited for a broader study called theSurvey Online Cohorts for Internalizing Symptoms andLanguage. The aim of the Survey Online Cohorts forInternalizing Symptoms and Language studies was to triangulatesocial media data with self-reported mental health status. In thismanuscript, we focused on the latter source of data. The
participants were sampled from two distinct sources: (1) a

The demographic information of the participants of studies 1
and 2 is presented in Table 1. For study 1 (n=1123), our sample
was well represented, with the participants’ racial demographics
mapped onto census data and a large distribution of age. As forstudy 2 (n=2337), it can be assumed that the sample was notnationally representative, as most participants were Indiana
residents who were college aged.

Methods
Ethics Approval

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Rutter et al

Table 1. Demographic information of a web-based panel (study 1; n=1123) and university students (study 2; n=2337) who responded to a web-based
survey on mental health and social media use (Survey Online Cohorts for Internalizing Symptoms and Language study).
Variable

Value
Study 1

Study 234.65 (12.56)

19.08 (2.75)

White

826 (73.68)

1645 (70.3)

Black

128 (11.42)

170 (7.26)

American Indian or Alaska Native

7 (0.62)

2 (0.08)Asian

62 (5.53)

240 (10.27)

Native Hawaiian or Pacific Islander

1 (0.09)

2 (0.08)

Hispanic

86 (7.67)

123 (5.26)

Other

11 (0.98)

39 (1.67)

Male

484 (43.18)

491 (21.01)

Female635 (56.65)

1761 (75.35)

Prefer not to answer

2 (0.18)

2 (0.09)Man

492 (43.89)

491 (21.01)

Woman

615 (54.86)

1732 (74.11)

Nonbinary

12 (1.07)

25 (1.07)

Other

2 (0.18)

2 (0.08)

Prefer not to answer

0 (0)4 (0.17)

Heterosexual

921 (82.16)

1896 (81.13)

Homosexual

42 (3.75)

52 (2.23)

Bisexual or pansexual or both124 (11.06)

226 (9.67)

Other

18 (1.61)

62 (2.65)

Prefer not to answer

16 (1.43)

18 (0.77)<10,000

99 (8.83)

116 (4.96)

10,000-19,999

95 (8.47)

68 (2.91)

20,000-29,999

141 (12.58)

94 (4.02)

30,000-39,999

103 (9.19)

105 (4.49)

40,000-49,999

65 (5.8)102 (4.36)

50,000-59,999

79 (7.05)

106 (4.53)

60,000-69,999

51 (4.55)

113 (4.83)

70,000-79,999

73 (6.51)

107 (4.58)

80,000-89,999

38 (3.39)

130 (5.56)

90,000-99,999

64 (5.71)

164 (7.01)

100,000-149,999

171 (15.25)

469 (20.04)>150,000142 (12.67)

656 (28.07)

Age (years), mean (SD)Race, n (%)

Sex, n (%)

Gender, n (%)

Sexual orientation, n (%)

Annual household income (US $), n (%)

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Measures
Overview
As noted above, the measures of relevance in this research were
the same for studies 1 and 2. All the study measures wereself-report instruments administered on the web. Most measures,detailed below, were selected from the American Psychiatric
Association (APA)’s “emerging measures” for research andclinical evaluation and are in the public domain to be used byresearchers and clinicians. These web-based assessment
measures are included in Section III of the Diagnostic andStatistical Manual for Mental Disorders (DSM)–5th Edition [24].

Insomnia Severity Index
The Insomnia Severity Index [25] is a 7-item self-report measurethat assesses insomnia symptom severity over the previous 2
weeks. The items are rated on a 5-point Likert scale ranging
from 0 (“none”) to 4 (“very severe”), yielding total scores of 0
to 28. The total score is interpreted as follows: 0 to 7 indicate
the absence of insomnia, 8 to 14 indicate subthreshold insomnia,
15 to 21 indicate moderate insomnia, and 22 to 28 indicate
severe insomnia. Prior studies have found adequate
psychometric properties for the English and French versions[25-27] and indicate that the measure is likely clinically usefulas a screening device [25]. In our samples, Cronbach alphas
were .87 for study 1 and .86 for study 2, suggesting that thismeasure was internally consistent in both studies.

Severity Measure for Somatic Symptoms—Adult (APA,
2013)

Rutter et al

Severity Measure for Social Anxiety Disorder (Social
Phobia)—Adult (APA, 2013)
The severity measure for social phobia symptoms consists of
10 items rated on a 5-point Likert scale ranging from 0 (“never”)
to 4 (“all of the time”). The participants are asked to rate thefrequency of fear and avoidance of social situations over the
past 7 days. The total scores range from 0 to 40, with higherscores indicating a greater severity of social anxiety disorder(social phobia). The average score (total raw score/number of
items answered) can be used as a proxy for social phobia
severity: 0 indicates none, 1 indicates mild, 2 indicates moderate,
3 indicates severe, and 4 indicates extreme. In our samples,
Cronbach alphas were .95 for study 1 and .93 for study 2,
suggesting that this measure was internally consistent in bothstudies.

Severity Measure for Panic Disorder—Adult (APA, 2013)
The severity measure for panic disorder consists of 10 items
rated on a 5-point Likert scale ranging from 0 (“never”) to 4(“all of the time”). The participants are asked to rate thefrequency of fear and avoidance of panic attacks over the past
7 days. The total scores range from 0 to 40, with higher scoresindicating a greater severity of panic disorder symptoms. The
average score (total raw score/number of items answered) canbe used as a proxy for panic disorder severity: 0 indicates none,
1 indicates mild, 2 indicates moderate, 3 indicates severe, and
4 indicates extreme. In our samples, Cronbach alphas were .96
for study 1 and .94 for study 2, suggesting that this measure
was internally consistent in both studies.

The severity measure for somatic symptoms was adapted fromthe Patient Health Questionnaire Physical Symptoms—15 [28]and consists of 15 items rated on a 3-point Likert scale ranging
from 0 (“not bothered at all”) to 2 (“bothered a lot”). The
participants are asked to rate their somatic symptoms over the
past 7 days. The total scores range from 0 to 30, with higherscores indicating more severe somatic symptoms. The scores
are interpretated in terms of how they relate to the level ofsomatic symptom severity: 0 to 4 indicate a minimal level ofseverity, 5 to 9 indicate a low level of severity, 10 to 14 indicate
a medium level of severity, and 15 to 30 indicate a high levelof severity. In our samples, Cronbach alphas were .89 for study
1 and .81 for study 2, suggesting that this measure was internallyconsistent in both studies.

Severity Measure for Generalized Anxiety
Disorder—Adult (APA, 2013)

Severity Measure for Depression—Adult (APA, 2013)

Severity Measure for Agoraphobia—Adult (APA, 2013)

The severity measure for depression was adapted from thePatient Health Questionnaire-9 (PHQ-9) [29] and consists of 9
items rated on a 4-point Likert scale ranging from 0 (“not atall”) to 3 (“nearly every day”). The participants were asked torate their symptoms of depression over the last 7 days. Of note,
we also used the PHQ-9 in examining the depressive symptomsof bipolar disorder. The PHQ-9 has excellent psychometricproperties [29]. A score of 5 is considered mild, and a score of
10 has shown to maximize the sensitivity and specificity forscreening for the diagnosis of depression [30]. In our samples,
Cronbach alphas were .91 for study 1 and .88 for study 2,
suggesting that this measure was internally consistent in bothstudies.

The severity measure for agoraphobia consists of 10 items rated
on a 5-point Likert scale ranging from 0 (“never”) to 4 (“all of
the time”). The participants are asked to rate the frequency offear and avoidance in agoraphobic situations over the past 7
days. Total scores range from 0 to 40, with higher scoresindicating a greater severity of agoraphobia symptoms. The
average score (total raw score/number of items answered) canbe used as a proxy for agoraphobia severity: 0 indicates none,
1 indicates mild, 2 indicates moderate, 3 indicates severe, and
4 indicates extreme. In our samples, Cronbach alphas were .96
for study 1 and .93 for study 2, suggesting that this measure
was internally consistent in both studies.

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

The severity measure for generalized anxiety disorder (GAD)
consists of 10 items rated on a 5-point Likert scale ranging from0 (“never”) to 4 (“all of the time”). The participants are asked
to rate the frequency of worry and associated symptoms overthe past 7 days. Total scores range from 0 to 40, with higherscores indicating a greater severity of GAD symptoms. The
average score (total raw score/number of items answered) canbe used as a proxy for GAD severity: 0 indicates none, 1
indicates mild, 2 indicates moderate, 3 indicates severe, and 4
indicates extreme. In our samples, Cronbach alphas were .95
for study 1 and .91 for study 2, suggesting that this measure
was internally consistent in both studies.

JMIR Form Res 2023 | vol. 7 | e39206 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Alcohol Use Disorders Identification Test
The Alcohol Use Disorders Identification Test (AUDIT) is a
10-item self-report measure developed by the World HealthOrganization (1989) to assess alcohol use and misuse. The
participants answer the first 8 items on a 5-point Likert scaleranging from 0 (“never”) to 4 (“daily or almost daily”) and thelast 2 items on a 3-point scale (“No”’ “Yes, but not in the lastyear”; “Yes, during the last year”). The AUDIT assesses 2
domains of alcohol use: consumption (first 3 items) andalcohol-related problems (last 7 items). Previous studies haveshown that this measure has good reliability and validity amongcollege students [31]. A score of ≥8 indicates hazardous orharmful use of alcohol. In our sample, Cronbach alphas were
.93 for study 1 and .80 for study 2, suggesting that this measure
was internally consistent in both studies.

Substance Use—Adult (APA, 2013)This measure was adapted from the National Institute on Drug
Abuse-Modified Assist to assess the misuse of prescription
medications and illicit substance use in adults aged ≥18 years.
Each item asks the participant to rate the severity of theirsubstance use during the past 2 weeks. The scale asks about 10
categories of drugs, and each item is rated on a 5-point scaleranging from 0 (“not at all”) to 4 (“nearly every day”). Rating
multiple items >0 indicates a greater severity and complexityof substance use problems.

Rutter et al
our samples, Cronbach alphas were .84 for study 1 and .71 forstudy 2, suggesting that this measure was internally consistentin both studies, with good consistency in study 1 and acceptableconsistency in study 2.

Diagnosis Questionnaire
We assessed the diagnoses of each of the disorders listed above
and some additional diagnoses (insomnia, substance use
disorder, depression, specific phobia, social anxiety disorder,
panic disorder, posttraumatic stress disorder, GAD, agoraphobia,
somatic symptom disorder, alcoholism, and mania) with theprompt, “Have you ever been clinically diagnosed with [blank]
disorder?” Response options were as follows: (1) “Yes”; (2)“No, but I should be”; (3) “I don’t know”; and (4) “No.”

Statistical Analysis
The data were analyzed using R Studio (version 1.0.153,
RStudio, Inc). For each diagnosis (eg, depression, anxiety, andsubstance use) we conducted an ANOVA test to determine
whether the disorder-specific severity questionnaire scores weredifferent among the 4 diagnostic groups (ie, “Yes”; “No, but I
should be”; “I don’t know”; and “No”). We used Tukey post
hoc tests to determine whether the differences in means between
the groups were statistically significant.

Results

Altman Self-rating Mania Scale

Descriptive Statistics

The Altman Self-rating Mania Scale (ASRM; [32]) is a 5-item
self-report scale that assesses the presence and severity of mania
symptoms over the past week. Items are rated on a 5-point
Likert-style scale, with response categories having different
anchors depending on the symptoms. The scores range from 5
to 25, with higher scores indicating more severe manic
symptoms. The interpretation of the ASRM is as follows: (1) ascore of ≥6 indicates a high probability of a manic or hypomanic
condition, (2) a score of ≥6 may indicate the need for treatment
or further diagnostic assessment, and (3) a score of ≤5 is lesslikely to be associated with significant symptoms of mania. In

Table 2 presents the number of participants from each study
who selected (1) “Yes”; (2) “No, but I should be”; (3) “I don’tknow”; and (4) “No” to the question, “Have you ever beenclinically diagnosed with [blank] disorder,” along with thepercentage of the sample they represented. As shown in Table
2, study 1 had higher prevalence rates of each disorder assessed,
indicating more clinical diagnoses in the national sample thanin the college student sample. Missing data are not included inTable 2, but approximately 1% to 2% (from 11 to 19 participantsin study 1 and 30 to 31 participants in study 2) did not answer
the questions regarding diagnoses.

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Rutter et al

Table 2. Total number and percentage of participants who reported being diagnosed with mental disorders in a web-based panel (study 1; n=1123) andamong university students (study 2; n=2337) who responded to a web-based survey on mental health and social media use (Survey Online Cohorts forInternalizing Symptoms and Language study).
Study and diagnosis

Yes, n (%)

No, but I should be, n (%)

I don't know, n (%)

No, n (%)

Insomnia

265 (23.64)

69 (6.16)

20 (1.78)

767 (68.42)

Somatic symptom disorder58 (5.17)

5 (0.45)

31 (2.77)

1027 (91.61)

Depression

472 (42.11)

66 (5.89)

29 (2.59)

554 (49.42)

Social anxiety

292 (26.05)

64 (5.71)

39 (3.48)

726 (64.76)

Panic

207 (18.47)

43 (3.84)

26 (2.32)

845 (75.38)

Generalized anxiety

333 (29.71)

55 (4.91)

40 (3.57)

693 61.82)

Agoraphobia

65 (5.8)

20 (1.78)

37 (3.3)

999 (89.12)

Alcohol use disorder

114 (10.17)

16 (1.43)

26 (2.32)

965 (86.08)

Substance use disorder

117 (10.44)

12 (1.07)

26 (2.32)

966 (86.17)

Bipolar disorder

121 (10.79)

20 (1.78)

24 (2.14)956 (85.28)

Insomnia

138 (5.9)

208 (8.89)

46 (1.97)

1874 (80.09)

Somatic symptom disorder

23 (0.98)

14 (0.6)45 (1.92)

2183 (93.29)

Depression589 (25.17)

248 (10.6)

90 (3.85)

1340 (57.26)

Social anxiety

302 (12.91)

240 (10.26)

74 (3.16)

1649 (70.47)

Panic

140 (5.98)

65 (2.78)

56 (2.39)

2004 (85.64)

Generalized anxiety

663 (28.33)

324 (13.85)

75 (3.21)1204 (51.45)

Agoraphobia

10 (0.43)

18 (0.77)

31 (1.32)

2206 (94.27)

Alcohol use disorder

12 (0.51)

16 (0.68)

11 (0.47)

2226 (95.12)

Substance use disorder35 (1.5)

23 (0.98)

15 (0.64)

2192 (93.68)

Bipolar disorder41 (1.75)

49 (2.09)

40 (1.71)

2135 (91.24)

Study 1 (n=1123)

Study 2 (n=2337)

Overview

who have been diagnosed (“Yes”) and those who selected “No,but I should be” compared with those who selected “No” or “I
don’t know.”In Table 3, we have presented the responses from theparticipants in studies 1 and 2 regarding the diagnoses of eachdisorder we assessed and whether they have been diagnosed:(1) “Yes”; (2) “No, but I should be”; (3) “I don’t know”; and(4) “No.” These have been presented along with the mean and
SD of the disorder-specific severity questionnaire scores
associated with each disorder. Table 3 also presents the resultsof a series of ANOVAs, followed by post hoc Tukey tests to
examine potential differences in scores between the individuals

If the disorder-specific severity questionnaire scores ofindividuals were “accurate” or consistent with their reported
diagnostic status (“Yes”; “No, but I should be”; “I don’t know”;
or “No”), we expected that the individuals who had not received
a diagnosis would have lower mean scores than (1) those who
had a diagnosis (“Yes”) and (2) those who said that they had
no diagnosis but that they should be diagnosed (“No, but I
should be”) and expected 3) that the “Yes” group and the “No,but I should be” group would be roughly comparable.

Self-diagnosis Accuracy

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Rutter et al

Table 3. Descriptive statistics and ANOVA results comparing reports of diagnosis across disorders.
Diagnosis (questionnaire: Yes,
score range)
mean
(SD)

NBa,
mean
(SD)

IDKb,
mean
(SD)

No,
mean
(SD)

Model,
F test(df)

P valueYes versus
No, Tukey
difference(range)P value

NB versus
No, Tukey
difference(range)P valueYes versus
NB, Tukey
difference(range)P value

Study 1 (n=1123)
Insomnia (ISIc: 0 to
28)

16.39
(5.22)

15.61(5.34)

13.25(6.48)

9.56(5.84)

107.2<.001(3,1105)−6.84(−7.89 to−5.73)<.001

6.05 (4.21
to 7.89)

<.001

−0.80
(−2.77 to
1.19)

.73

Somatic symptom

15.92(7.64)

7.77(9.29)

14.00(5.66)

8.64(6.39)

26.75<.001(3,1060)−7.28(−9.64 to−4.92)

<.001

−0.98
(−10.56 to8.61);.99

.99

−8.26
(−18.09 to1.58);.14.14(SOMd: 0 to 30)
Depression (PHQe:
0 to 27)

12.13(6.73)

11.98(6.21)

10.38(5.44)

5.85(5.69)

92.23<.001(3,1098)−6.27(−7.27 to−5.27)<.001

6.13 (4.06
to 8.20)

<.001

−0.14
(−2.22 to1.95);.99.99

Social anxiety

14.72(8.78)

7.81(8.65)

90.39<.001(3,1092)−10.57(−12.26 to−8.88)<.001

5.77 (2.66
to 8.88)

<.001

−4.80
(−8.10 to−1.50).001(SAD : 0 to 40)

18.38 13.58(10.91) (9.34)

Panic (PANg: 0 to40)

17.74 11.07(10.56) (8.17)

13.62 6.42
(10.71) (8.83)

85.07
<.001
(3,1093)−11.33(−13.19 to−9.47)<.001

4.65 (0.95
to 8.35)

.006

−6.68
(−10.65 to−2.70)<.001

Generalized anxiety 16.67 13.93(10.46) (7.62)(GADh: 0 to 40)

16.65 8.52
(10.36) (9.41)

57.48
<.001
(3,1099)−8.15(−9.82 to−6.47)<.001

5.41 (1.94
to 8.87)

<.001

−2.74
(−6.35 to
0.86)

.20

Alcohol use (AU-f

15.97 8.44
(10.07) (5.60)

14.15 4.96
(11.41) (7.29)

77.73
<.001
(3,1097)−11.33(−13.00 to−9.03)<.001

3.48 (−1.52 .28
to 8.8)−7.54(−12.84 to−2.23).002

DITi: 0 to 40)
Drug use (SUDj: 0
to 40)

11.05(9.80)

7.08(6.79)

14.46 2.65
(11.52) (5.59)

84.36
<.001
(3,1096)−8.40(−10.00 to−6.79)<.001

4.43 (−.14
to 8.98)

.06

−3.97
(−8.75 to
0.80)

.14

Bipolar (PHQ: 0 to
27)

15.06
(7.03)

12.95(5.22)

11.92(6.49)

8.04(6.48)

44.88<.001(3,1092)−7.02(−8.66 to−5.38)<.001

4.91 (1.12
to 8.70)

.004

−2.11
(−6.16 to
1.94)

.54

Bipolar (ASRMk: 0
to 20)

11.79(5.54)

9.95(4.39)

13.43(4.88)

11.06(5.54)

2.75.04(3,1097)−0.72(−1.97 to
0.52)

.43

−1.11
(−3.95 to
1.73)

.75

−1.84
(−4.89 to
1.20)

.40

Insomnia (ISI: 0 to
28)

14.20
(5.61)

14.56(4.61)

12.96(5.99)

8.07(4.89)

170.4<.001(3,2261)−6.13(−7.25 to
−501)

<.001

6.49 (5.56
to 7.42)<.001

0.36 (−1.03 .91
to 1.75)

Somatic symptom(SOM: 0 to 30)

12.95(6.56)

11.59(6.08)

11.45(5.42)

6.92(4.68)

31.00<.001(3,2164)−6.04(−8.60 to−3.48)<.001

5.51 (2.24
to 8.78)

<.001

−0.52
(−4.66 to
3.60)

.98

Depression (PHQ: 0 10.58
to 27)(6.08)

11.59(6.08)

7.46(4.94)

5.00(4.34)

232.40 <.001(3,2259)−5.58(−6.22 to−4.93)<.001

6.59 (5.9 to <.001
7.49)

1.01 (0.03
to 2.00).04

Social anxiety(SAD: 0 to 40)

15.74(9.68)

15.15(8.96)

10.29(6.08)

5.63(6.18)

265.30 <.001(3,2255)−10.11(−11.24 to−8.97)<.001

9.51 (8.25
to 10.77)

<.001

−0.60
(−2.17 to
0.98)

.76

Panic (PAN: 0 to 40) 11.89(8.84)

10.98(8.38)

9.22(8.68)

2.74(4.68)

194.60 <.001(3,2253)−9.15(−10.34 to−7.96)<.001

8.24 (6.53
to 9.96)

<.001

−0.91
(−2.95 to
1.13)

.66

Generalized anxiety 13.21(GAD: 0 to 40)(8.57)

11.57(7.25)

11.03(7.28)

5.53(5.26)

208.30 <.001(3,2048)−7.67(−8.51 to−6.83)<.001

6.03 (4.94
to 7.12)

<.001

−1.64
(−2.82 to−0.46).002

Study 2 (n=2337)

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
NBa,
mean
(SD)

IDKb,
mean
(SD)

No,
mean
(SD)

Model,
F test(df)

P valueYes versus
No, Tukey
difference(range)P value

NB versus
No, Tukey
difference(range)P valueYes versus
NB, Tukey
difference(range)P value

11.50(8.54)

16.88(9.81)

11.03(7.28)

5.53(5.26)

54.51<.001(3,2132)−6.61(−10.02 to−2.99)<.001

11.98 (8.85 <.001
to 15.12)

5.38 (0.61
to 10.14).02

Drug use (SUD: 0 to 3.14
40)
(2.05)

3.43(1.83)

2.40(1.76)

0.70(1.45)

63.89<.001(3,2260)−2.44(−3.08 to−1.80)<.001

2.74 (1.95
to 3.52)<.001

0.29 (−0.72 .88
to 1.30)

Bipolar (PHQ: 0 to
27)

11.80
(5.78)

12.98(6.24)

11.69(5.57)

6.97(5.65)

35.33<.001(3,2257)−4.84(−7.13 to−2.54)<.001

6.01 (3.91
to 8.12)<.001

1.17 (−1.91 .76
to 4.26)

Bipolar (ASRM: 0
to 20)

4.86(2.54)

6.83(2.59)

4.80(3.29)

4.80(3.40)

1.43(3,659)−0.05(−2.39 to
2.29)

.99

2.03 (−0.50 .16
to 4.56)

Diagnosis (questionnaire: Yes,
score range)
mean
(SD)
Alcohol use (AUDIT: 0 to 40)aRutter et al.23

1.98 (−1.43 .44
to 5.39)

NB: “No, But I should be.”b

IDK: “I don’t know.”c

ISI: Insomnia Severity Index.

dSOM: severity measure for somatic symptoms—adult.

e

PHQ: Patient Health Questionnaire (Severity Measure of Depression).

f

SAD: Severity Measure for Social Anxiety Disorder.

g

PAN: severity measure for panic disorder.

h

GAD: severity measure for generalized anxiety disorder.

i

AUDIT: Alcohol Use Disorders Identification Test.

j

SUD: level 2—Substance Use—Adult.

k

ASRM: Altman Self-rating Mania Scale.

Study 1
For most conditions assessed, the results were consistent with
our expectations, indicating that the mean disorder-specificseverity questionnaire scores for the four diagnostic
conditions—(1) “Yes”; (2) “No, but I should be”; (3) “I don’tknow”; and (4) “No”—were different between the groups. For
self-reported mania, somatic symptom disorder, and alcohol
use disorder, the pattern of mean differences did not suggestthat the individuals were accurate in their assessment. However,as indicated by ANOVAs and post hoc Tukey tests, for social
anxiety disorder, panic disorder, alcohol use disorder, substanceuse disorder, and bipolar disorder, there were significant
differences (P<.05) in questionnaire scores between theindividuals who had been previously diagnosed by a clinician
and those who believed that they should be diagnosed. We were
expecting the individuals who selected “Yes” and those who
selected “No, but I should be” to present similar scores, but,instead, the individuals who reported they had a diagnosis had
higher questionnaire scores, indicting greater disorder severity.
The ASRM appears to be a poor measurement tool for the
self-diagnosis of bipolar disorder: it did not differentiate theindividuals who had been diagnosed from those who had not.
However, when using the PHQ-9 to assess the depressive
symptoms in bipolar disorder, the scores on the PHQ-9 weredifferent in the expected directions between the groups.

disorder-specific severity questionnaire scores between the 4diagnosis conditions—(1) “Yes”; (2) “No, but I should be”; (3)“I don’t know”; and (4) “No”—were different between thegroups. Upon further investigation with post hoc Tukey tests,
there were no differences in disorder-specific severity
questionnaire scores between the individuals who had beenpreviously diagnosed by a clinician and those who believed thatthey should be diagnosed with insomnia, somatic symptom
disorder, social anxiety disorder, panic disorder, or GAD. There
were significant differences between previously diagnosed
individuals and those who believed that they should be
diagnosed with depression, GAD, or alcohol use disorder. Here,the depression and alcohol use scores were actually higher inthe group of people who believed that they should be diagnosed,
potentially indicating stigma around reporting to a professional
or difficulty accessing care. Of note, a smaller portion of oursample in study 2 completed the ASRM, as it was removed
partway through the study owing to its poor ability to captureself-reported mania. The ASRM appeared to poorly capture thedifferences in mania symptoms among the individuals who havebeen diagnosed, those who believed that they should be
diagnosed, and those who were not been diagnosed, suggestingthat a self-report assessment of mania may not be a goodbarometer of bipolar-spectrum psychopathology in college
students. However, the PHQ-9 was able to detect differencesamong the groups in the expected directions for bipolar disorder.

Study 2In all disorders but bipolar disorder, the ANOVA results wereconsistent with our expectations, indicating that the meanhttps://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Discussion
Principal Findings
This study investigated the scores on self-reported
disorder-specific severity questionnaires of individuals who
indicated that they had been previously diagnosed (“Yes”),
believed that they should be diagnosed (“No, but I should be”),
were unsure (“I don’t know”), and had never been diagnosed(“No”) across a variety of internalizing and externalizing
disorders. Given the misdiagnosis, underdetection, and stigma
associated with mental disorders, this study was a novel way
to explore the “accuracy” of self-diagnosis in a national survey
panel and a sample of college students. Overall, the results
indicate that people who say they have been diagnosed with adisorder have the highest severity scores in general, but this is
closely followed by individuals who have not been diagnosed
but believe that they should be. This is especially true for clinicalconditions that are relatively nonspecific and whose names are
relatively descriptive such as depression, generalized anxiety,and insomnia. The results suggest that self-reported mania
questionnaires do not align with reported diagnoses, which isconsistent with prior studies [33-35], although there is someevidence that self-reported measures can correlate with a lifetime
diagnosis of bipolar disorder [36]. Additional differences inpatterns observed between our samples contribute to ourinterpretation of these findings; with some exceptions, college
students tended to be more accurate than the national sample
in self-diagnosis.
Although study 2 consisted of college students in the
Midwestern United States, which was a relatively homogeneous
population in terms of age and ethnicity, our national sample(study 1) was more variable in age, ethnicity, and otherdemographic characteristics, including age and gender. Our
national sample from study 1 also reported more severe
psychopathology. The differences between our samples, withcollege students being slightly more accurate in self-diagnosis,
yield additional support for the validity of university samples,
which are often criticized for lacking generalizability [37].
Initially, we had expected that college students may not know
as much about their diagnoses, but it seems that college students
are reasonably accurate in self-diagnosis, reflecting trends of
reduced stigma and increased awareness of the mental healthconcerns that are present in early adulthood [38,39]. In addition,
it is possible that the college students were more accurate inself-diagnosis because they were psychology students and,
therefore, were aware of various mental health complaints. A
secondary interpretation is that younger adults are generally
better at web-based help-seeking behaviors and thus more
attuned to their personal mental states [40]. Indeed, younger
adults may be more prone to and more savvy with engaging in
web-based information seeking, including the use of social
media support networks to understand their mental health status[41]. This makes social media a useful tool for studying mental
health as well as disseminating support resources, treatment
options, and research.

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

Rutter et alComparison With Prior Work
Overall, our hypotheses were supported. Our findings showedthat self-diagnosis is accurate for internalizing disorders,meaning that most individuals with disorders such as major
depression and GAD can accurately detect that their symptoms
are significant, even if they have not received a formal diagnosis.
Our findings also reveal that self-reported diagnoses are moreaccurate for more commonly occurring conditions; for example,
depression and GAD showed high accuracy in self-diagnosis,
but they are also among the most common mental disorders[42,43], unlike other anxiety disorders like panic that occur at
lower frequencies [43,44]. It is possible that the accuracy ofself-diagnosis corresponds best to disorders that occur at highbase rates, which could partially explain our poor self-diagnosis
accuracy for mania, panic, and somatic symptom disorders.
Indeed, prior research has demonstrated that self-reporting ismore accurate for people who are more familiar with a disorderand its symptoms [45]. Social media may play an importantrole in disseminating information about common mental healthdisorders to communities of interest such as adolescents andyoung adults.
Interestingly, consistent with prior work, the participants in both
samples showed poor abilities in detecting mania symptoms via
self-report compared with diagnoses from professionals. We
acknowledge this is not the first study to find that mania is moredifficult to detect than depressive symptoms in a self-reportform [33]. It is likely that bipolar disorder, specifically manic
symptoms, cannot be screened via self-report, and a multistage
screener with clinician report may be necessary [33-35]. It isalso possible that the time frame of the ASRM assessment, thatis, the past week, does not align well with the diagnosis ofbipolar disorder, for which a manic episode could have occurredyears ago [46]. Indeed, for most patients with a bipolar-spectrum
illness, mania symptoms are rarely present at the time of theassessment.
Alcohol and substance use disorders followed bipolar disorderin terms of lowest self-diagnosis accuracy. We offer several
explanations for this finding. First, in college students, alcohol
consumption, particularly binge drinking, is much moreprevalent than in the general public [47,48], and the behavior
is normalized and encouraged in many social groups [49,50].
This may lead to a low likelihood of reporting interference fromdrinking, despite a high level of alcohol use. However, this is
likely not the case in our studies, as in our college student
sample, the participants who believed that they should be
diagnosed actually had higher AUDIT scores than those who
had already been diagnosed, indicating good insight. However,given that the rates were higher in those who believed that theyshould be diagnosed than in those who had received diagnoses,
it is possible that college students who are struggling withalcohol and drug problems receive diagnoses for other comorbid
conditions (depression) and do not receive substance use
disorder diagnoses until they are out of college when heavy
drinking behaviors are not normalized. Indeed, in our college
student sample, <1% had been diagnosed with an alcohol use
disorder, whereas in the national sample, approximately 10%
had previously received a diagnosis, which is in line withepidemiological data [51]. In our national sample, which had
JMIR Form Res 2023 | vol. 7 | e39206 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
lower socioeconomic status and may have had lower educational
attainment overall compared with our university sample, there
was a lower correspondence between self-diagnosis andself-report disorder-specific severity questionnaire scores,
indicating a need for further research into drinking and substanceuse behaviors, perhaps via more daily monitoring of use.

Limitations
Despite the strengths of our study, including its novelty and theuse of 2 large samples, it is not without its limitations. First, we
did not have clinicians interview the participants to assess theirdisorders at the time of the study. The reported disorders werebased on prior clinical diagnoses and relied on participants’honesty and awareness of their mental health conditions.
Moreover, we did not include an assessment of all disorders,
including those that may resemble mood disorders early in theircourse, including psychotic disorders, personality disorders,and medical or substance-induced disorders, as we focused mostthoroughly on the internalizing spectrum. Future research should
consider assessing the full spectrum of psychopathology. In
addition, we note that the timeline of disorder onset is not clear
from our questionnaires. Some participants may have received
their diagnoses years before the current assessment period, andour questionnaires asked about their symptoms within the last
week or 2 weeks. Second, differential diagnosis was nearly
impossible in our sample, given the wide variety of disorders
and lack of clinical reports. This means that the participants
may have appeared to have a higher degree of psychopathology
than was truly representative, given their diagnostic
comorbidities, DSM hierarchy rules, and rule-outs that clinicians
would be aware of but patients may not be aware of. Third, it
is possible that in our study, the participants appeared to begood at yielding accuracy in the self-diagnoses of some disorders
such as social anxiety disorder, but this is a reflection of themeasure being highly sensitive but not specific. Further testingof the DSM severity measures in a wide variety of samples iswarranted before their extended use.

Conclusions
Our results support ongoing and future research that uses patient
self-report of symptoms, diagnoses, and disorders on a more
solid footing. In particular, for some conditions of common
mental disorders such as depression, anxiety, and insomnia,

Rutter et al
relying on textual reports may not bias samples, at least in termsof symptom severity: when individuals say that they struggle
with a problem, they in fact seem to do so. The next steps ofthis research involve examining these results in conjunction
with relevant features of individuals’ social media accounts,
including sentiment analyses, text analysis of themes associatedwith internalizing disorders, and other computational informatics
analyses of social media data. Indeed, we have previously shownthat individuals who self-disclose depression diagnoses use
more cognitive distortions [16], have different circadian rhythms[15], and show more negative affect based on sentiment analysis
according to unpublished data from Rutter et al (Rutter, LA,
unpublished data, August 2022). However, the validity of resultsfrom Twitter have been called into question because
self-disclosure is not well understood and is often stigmatized
as a nonefficacious source of data [52]. It is also possible that
individuals who self-disclose their diagnoses on the web aredifferent from those who report their diagnoses and symptoms
via survey. On the basis of the results of this study, we can now
more definitively say that self-reported survey diagnoses
correspond well with symptom severity on a continuum andcan, on balance, be trusted as clinical indicators, especially ininternalizing disorders such as depression and GAD. However,more research on the validity and modalities of web-based
self-reports is needed, and research should be expanded tocapture other related mental disorders. Gaps in researchnotwithstanding, if more studies ask about self-reported
diagnoses and compare self-reported symptoms, the field could
benefit by receiving more accurate detection rates. In general,
if a person believes that they should be diagnosed with aninternalizing disorder, they may be experiencing a degree ofpsychopathology that is similar to that experienced by someone
who has been diagnosed, warranting clinical attention.
Psychologists and providers can put more confidence into thepatient’s perspective of their disorders and when time is limited,
reduce questionnaire questions and query more specifically if
the person believes that they have a disorder. In addition,
researchers may be able to place more trust in social media datawhen individuals report diagnoses of anxiety and depression;
however, replication and further study are needed. This
perspective may yield additional insight into the burden ofmental illness and the associated distress and impairment.

Acknowledgments
The authors thank the participants for contributing to this study.

Conflicts of Interest
None declared.

References
1.

2.

Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable tomental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013 Nov
09;382(9904):1575-1586. [doi: 10.1016/S0140-6736(13)61611-6] [Medline: 23993280]
Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291
diseases and injuries in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet
2012 Dec 15;380(9859):2197-2223. [doi: 10.1016/S0140-6736(12)61689-4] [Medline: 23245608]

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 11(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
3.

4.
5.
6.

7.

8.

9.

10.
11.

12.

13.
14.
15.

16.
17.
18.
19.

20.
21.

22.

23.

24.
25.

Greenberg PE, Fournier AA, Sisitsky T, Pike CT, Kessler RC. The economic burden of adults with major depressive disorderin the United States (2005 and 2010). J Clin Psychiatry 2015 Feb;76(2):155-162 [FREE Full text] [doi:10.4088/JCP.14m09298] [Medline: 25742202]
Bailey RK, Mokonogho J, Kumar A. Racial and ethnic differences in depression: current perspectives. Neuropsychiatr Dis
Treat 2019 Feb 22;15:603-609 [FREE Full text] [doi: 10.2147/NDT.S128584] [Medline: 30863081]
Liang J, Matheson BE, Douglas JM. Mental health diagnostic considerations in racial/ethnic minority youth. J Child FamStud 2016 Jun;25(6):1926-1940 [FREE Full text] [doi: 10.1007/s10826-015-0351-z] [Medline: 27346929]
McKay T, Watson RJ. Gender expansive youth disclosure and mental health: clinical implications of gender identity
disclosure. Psychol Sex Orientat Gend Divers 2020 Mar;7(1):66-75 [FREE Full text] [doi: 10.1037/sgd0000354] [Medline:
33855103]
Newman-Toker DE, Moy E, Valente E, Coffey R, Hines AL. Missed diagnosis of stroke in the emergency department: across-sectional analysis of a large population-based sample. Diagnosis (Berl) 2014 Jun;1(2):155-166 [FREE Full text] [doi:10.1515/dx-2013-0038] [Medline: 28344918]
Suite DH, La Bril R, Primm A, Harrison-Ross P. Beyond misdiagnosis, misunderstanding and mistrust: relevance of thehistorical perspective in the medical and mental health treatment of people of color. J Natl Med Assoc 2007
Aug;99(8):879-885. [Medline: 17722664]
Pescosolido BA, Halpern-Manners A, Luo L, Perry B. Trends in public stigma of mental illness in the US, 1996-2018.
JAMA Netw Open 2021 Dec 01;4(12):e2140202 [FREE Full text] [doi: 10.1001/jamanetworkopen.2021.40202] [Medline:
34932103]
Overton SL, Medina SL. The stigma of mental illness. J Couns Dev 2008;86(2):143-151. [doi:10.1002/j.1556-6678.2008.tb00491.x]
Moreno MA, Jelenchick LA, Egan KG, Cox E, Young H, Gannon KE, et al. Feeling bad on Facebook: depression disclosuresby college students on a social networking site. Depress Anxiety 2011 Jun;28(6):447-455 [FREE Full text] [doi:10.1002/da.20805] [Medline: 21400639]
Birnbaum ML, Rizvi AF, Confino J, Correll CU, Kane JM. Role of social media and the Internet in pathways to care foradolescents and young adults with psychotic disorders and non-psychotic mood disorders. Early Interv Psychiatry 2017
Aug;11(4):290-295 [FREE Full text] [doi: 10.1111/eip.12237] [Medline: 25808317]
Kaur H, Wasan SK. Empirical study on applications of data mining techniques in healthcare. J Comput Sci 2006 Feb
1;2(2):194-200. [doi: 10.3844/jcssp.2006.194.200]
Wongkoblap A, Vadillo MA, Curcin V. Researching mental health disorders in the era of social media: systematic review.
J Med Internet Res 2017 Jun 29;19(6):e228 [FREE Full text] [doi: 10.2196/jmir.7215] [Medline: 28663166]
Ten Thij MT, Bathina K, Rutter LA, Lorenzo-Luaces L, van de Leemput IA, Scheffer M, et al. Depression alters thecircadian pattern of online activity. Sci Rep 2020 Oct 14;10(1):17272 [FREE Full text] [doi: 10.1038/s41598-020-74314-3][Medline: 33057099]
Bathina KC, Ten Thij M, Lorenzo-Luaces L, Rutter LA, Bollen J. Individuals with depression express more distorted
thinking on social media. Nat Hum Behav 2021 Apr;5(4):458-466. [doi: 10.1038/s41562-021-01050-7] [Medline: 33574604]
Coppersmith G, Harman C, Dredze M. Measuring post traumatic stress disorder in Twitter. Proc Int AAAI Conf Web Soc
Media 2014 May 16;8(1):579-582. [doi: 10.1609/icwsm.v8i1.14574]
De Choudhury M, De S. Mental health discourse on Reddit: self-disclosure, social support, and anonymity. Proc Int AAAI
Conf Web Soc Media 2014 May 16;8(1):71-80. [doi: 10.1609/icwsm.v8i1.14526]
Kaplan ML, Asnis GM, Sanderson WC, Keswani L, De Lecuona JM, Joseph S. Suicide assessment: clinical interview vs.
self-report. J Clin Psychol 1994 Mar;50(2):294-298. [doi:10.1002/1097-4679(199403)50:2<294::aid-jclp2270500224>3.0.co;2-r] [Medline: 8014256]
Zimmerman M, Mattia JI. A self-report scale to help make psychiatric diagnoses: the Psychiatric Diagnostic Screening
Questionnaire. Arch Gen Psychiatry 2001 Aug;58(8):787-794. [doi: 10.1001/archpsyc.58.8.787] [Medline: 11483146]
Federici A, Summerfeldt LJ, Harrington JL, McCabe RE, Purdon CL, Rowa K, et al. Consistency between self-report andclinician-administered versions of the Yale-Brown Obsessive-Compulsive Scale. J Anxiety Disord 2010 Oct;24(7):729-733.
[doi: 10.1016/j.janxdis.2010.05.005] [Medline: 20561767]
Yazdavar AH, Al-Olimat HS, Ebrahimi M, Bajaj G, Banerjee T, Thirunarayan K, et al. Semi-Supervised approach tomonitoring clinical depressive symptoms in social media. Proc IEEE ACM Int Conf Adv Soc Netw Anal Min
2017;2017:1191-1198 [FREE Full text] [doi: 10.1145/3110025.3123028] [Medline: 29707701]
Correia RB, Wood IB, Bollen J, Rocha LM. Mining social media data for biomedical signals and health-related behavior.
Annu Rev Biomed Data Sci 2020 Jul;3:433-458 [FREE Full text] [doi: 10.1146/annurev-biodatasci-030320-040844][Medline: 32550337]
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). 5th edition. Washington,
DC, USA: American Psychiatric Association; 2013.
Bastien CH, Vallières A, Morin CM. Validation of the insomnia severity index as an outcome measure for insomnia research.
Sleep Med 2001 Jul;2(4):297-307. [doi: 10.1016/s1389-9457(00)00065-4] [Medline: 11438246]

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

Rutter et al

JMIR Form Res 2023 | vol. 7 | e39206 | p. 12(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
26.
27.

28.
29.
30.

31.

32.
33.
34.
35.
36.

37.
38.

39.

40.
41.

42.

43.

44.

45.

46.

47.
48.

Blais FC, Gendron L, Mimeault V, Morin CM. Evaluation de l'insomnie: validation de trois questionnaires. Encephale
1997;23(6):447-453. [Medline: 9488928]
Morin CM, Belleville G, Bélanger L, Ivers H. The Insomnia Severity Index: psychometric indicators to detect insomnia
cases and evaluate treatment response. Sleep 2011 May 01;34(5):601-608 [FREE Full text] [doi: 10.1093/sleep/34.5.601][Medline: 21532953]
Kroenke K, Spitzer RL, Williams JB. The PHQ-15: validity of a new measure for evaluating the severity of somatic
symptoms. Psychosom Med 2002;64(2):258-266. [doi: 10.1097/00006842-200203000-00008] [Medline: 11914441]
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001
Sep;16(9):606-613 [FREE Full text] [doi: 10.1046/j.1525-1497.2001.016009606.x] [Medline: 11556941]
Levis B, Benedetti A, Thombs BD, DEPRESsion Screening Data (DEPRESSD) Collaboration. Accuracy of Patient Health
Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis. BMJ 2019
Apr 09;365:l1476 [FREE Full text] [doi: 10.1136/bmj.l1476] [Medline: 30967483]
García Carretero MÁ, Novalbos Ruiz JP, Martínez Delgado JM, O'Ferrall González C. Validation of the alcohol use
disorders identification test in university students: AUDIT and AUDIT-C. Adicciones 2016 Mar 02;28(4):194-204 [FREEFull text] [doi: 10.20882/adicciones.775] [Medline: 26990260]
Altman EG, Hedeker D, Peterson JL, Davis JM. The Altman self-rating Mania scale. Biol Psychiatry 1997 Nov
15;42(10):948-955. [doi: 10.1016/S0006-3223(96)00548-3] [Medline: 9359982]
Zimmerman M, Ruggero CJ, Chelminski I, Young D. Is bipolar disorder overdiagnosed? J Clin Psychiatry 2008
Jun;69(6):935-940. [doi: 10.4088/jcp.v69n0608] [Medline: 18466044]
Zimmerman M. Screening for bipolar disorder: confusion between case-finding and screening. Psychother Psychosom
2014;83(5):259-262 [FREE Full text] [doi: 10.1159/000362564] [Medline: 25116428]
Zimmerman M. Using screening scales for bipolar disorder in epidemiologic studies: lessons not yet learned. J Affect
Disord 2021 Sep 01;292:708-713. [doi: 10.1016/j.jad.2021.06.009] [Medline: 34161888]
Achtyes ED, Halstead S, Smart L, Moore T, Frank E, Kupfer DJ, et al. Validation of computerized adaptive testing in an
outpatient nonacademic setting: the VOCATIONS trial. Psychiatr Serv 2015 Oct;66(10):1091-1096 [FREE Full text] [doi:10.1176/appi.ps.201400390] [Medline: 26030317]
Ward EA. Generalizability of psychological research from undergraduates to employed adults. J Soc Psychol 1993
Aug;133(4):513-519. [doi: 10.1080/00224545.1993.9712176]
Clement S, Jarrett M, Henderson C, Thornicroft G. Messages to use in population-level campaigns to reduce mental
health-related stigma: consensus development study. Epidemiol Psichiatr Soc 2010;19(1):72-79. [doi:10.1017/s1121189x00001627] [Medline: 20486426]
Thornicroft G, Mehta N, Clement S, Evans-Lacko S, Doherty M, Rose D, et al. Evidence for effective interventions to
reduce mental-health-related stigma and discrimination. Lancet 2016 Mar 12;387(10023):1123-1132. [doi:10.1016/S0140-6736(15)00298-6] [Medline: 26410341]
Pretorius C, Chambers D, Coyle D. Young people's online help-seeking and mental health difficulties: systematic narrative
review. J Med Internet Res 2019 Nov 19;21(11):e13873 [FREE Full text] [doi: 10.2196/13873] [Medline: 31742562]
Kruzan KP, Williams KD, Meyerhoff J, Yoo DW, O'Dwyer LC, De Choudhury M, et al. Social media-based interventions
for adolescent and young adult mental health: a scoping review. Internet Interv 2022 Dec;30:100578 [FREE Full text] [doi:10.1016/j.invent.2022.100578] [Medline: 36204674]
Hasin DS, Sarvet AL, Meyers JL, Saha TD, Ruan WJ, Stohl M, et al. Epidemiology of adult DSM-5 major depressive
disorder and its specifiers in the United States. JAMA Psychiatry 2018 Apr 01;75(4):336-346 [FREE Full text] [doi:10.1001/jamapsychiatry.2017.4602] [Medline: 29450462]
Kessler RC, Chiu WT, Jin R, Ruscio AM, Shear K, Walters EE. The epidemiology of panic attacks, panic disorder, andagoraphobia in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2006 Apr;63(4):415-424 [FREE Fulltext] [doi: 10.1001/archpsyc.63.4.415] [Medline: 16585471]
Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders:a systematic review and meta-analysis 1980-2013. Int J Epidemiol 2014 Apr;43(2):476-493 [FREE Full text] [doi:10.1093/ije/dyu038] [Medline: 24648481]
Vermani M, Marcus M, Katzman MA. Rates of detection of mood and anxiety disorders in primary care: a descriptive,
cross-sectional study. Prim Care Companion CNS Disord 2011;13(2):PCC.10m01013 [FREE Full text] [doi:10.4088/PCC.10m01013] [Medline: 21977354]
Kogan JN, Otto MW, Bauer MS, Dennehy EB, Miklowitz DJ, Zhang HW, STEP-BD Investigators. Demographic anddiagnostic characteristics of the first 1000 patients enrolled in the Systematic Treatment Enhancement Program for Bipolar
Disorder (STEP-BD). Bipolar Disord 2004 Dec;6(6):460-469. [doi: 10.1111/j.1399-5618.2004.00158.x] [Medline: 15541061]
Knight JR, Wechsler H, Kuo M, Seibring M, Weitzman ER, Schuckit MA. Alcohol abuse and dependence among U.S.
college students. J Stud Alcohol 2002 May;63(3):263-270. [doi: 10.15288/jsa.2002.63.263] [Medline: 12086126]
O'Malley PM, Johnston LD. Epidemiology of alcohol and other drug use among American college students. J Stud Alcohol
Suppl 2002 Mar(14):23-39. [doi: 10.15288/jsas.2002.s14.23] [Medline: 12022728]

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

Rutter et al

JMIR Form Res 2023 | vol. 7 | e39206 | p. 13(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
49.
50.
51.

52.

Rutter et al

Wombacher K, Matig JJ, Sheff SE, Scott AM. "It just kind of happens": college students' rationalizations for blackout
drinking. Health Commun 2019 Jan;34(1):1-10. [doi: 10.1080/10410236.2017.1384351] [Medline: 29048236]
Barry AE, Goodson P. Contextual factors influencing U.S. college students' decisions to drink responsibly. Subst Use
Misuse 2012 Aug;47(10):1172-1184 [FREE Full text] [doi: 10.3109/10826084.2012.690811] [Medline: 22662910]
Grant BF, Goldstein RB, Saha TD, Chou SP, Jung J, Zhang H, et al. Epidemiology of DSM-5 alcohol use disorder: resultsfrom the national epidemiologic survey on alcohol and related conditions III. JAMA Psychiatry 2015 Aug;72(8):757-766[FREE Full text] [doi: 10.1001/jamapsychiatry.2015.0584] [Medline: 26039070]
Valdez D, Unger JB. Difficulty regulating social media content of age-restricted products: comparing JUUL’s official
Twitter timeline and social media content about JUUL. JMIR Infodemiology 2021 Dec 7;1(1):e29011. [doi: 10.2196/29011]

Abbreviations
APA: American Psychiatric Association
ASRM: Altman Self-rating Mania Scale
AUDIT: Alcohol Use Disorders Identification Test
DSM: Diagnostic and Statistical Manual for Mental Disorders
GAD: generalized anxiety disorder
PHQ-9: Patient Health Questionnaire-9

Edited by G Eysenbach; submitted 03.05.22; peer-reviewed by E Achtyes, C Chakouch; comments to author 10.10.22; revised version
received 02.12.22; accepted 05.12.22; published 13.01.23
Please cite as:
Rutter LA, Howard J, Lakhan P, Valdez D, Bollen J, Lorenzo-Luaces L“I Haven’t Been Diagnosed, but I Should Be”—Insight Into Self-diagnoses of Common Mental Health Disorders: Cross-sectional
Study
JMIR Form Res 2023;7:e39206
URL: https://formative.jmir.org/2023/1/e39206
doi: 10.2196/39206
PMID:©Lauren A Rutter, Jacqueline Howard, Prabhvir Lakhan, Danny Valdez, Johan Bollen, Lorenzo Lorenzo-Luaces. Originallypublished in JMIR Formative Research (https://formative.jmir.org), 13.01.2023. This is an open-access article distributed under
the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted
use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, isproperly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as wellas this copyright and license information must be included.

https://formative.jmir.org/2023/1/e39206

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e39206 | p. 14(page number not for citation purposes)