Gender and Racial Fairness in Depression Research using Social Media (2021)

Abstract: Multiple studies have demonstrated that behaviors expressed on online social media platforms can indicate the mental health state of an individual. The widespread availability of such data has spurred interest in mental health research, using several datasets where individuals are labeled with mental health conditions. While previous research has raised concerns about possible biases in models produced from this data, no study has investigated how these biases manifest themselves with regards to demographic groups in data, such as gender and racial/ethnic groups. Here, we analyze the fairness of depression classifiers trained on Twitter data with respect to gender and racial demographic groups. We find that model performance differs for underrepresented groups, and we investigate sources of these biases beyond data representation. Our study results in recommendations on how to avoid these biases in future research.

## introduction work from de @xcite and @xcite , showing that an individual's mental health can be evaluated based on the language they generate on social media platforms, has served as the basis for a substantial amount of computational research over the last decade. subsequent studies have examined an even wider range of mental health conditions, social media platforms, and types of online behavior at both the individual and population level @xcite @xcite . vast potential for societal benefits underlies this work, as conservative estimates suggest that 8.1% of american adults suffer from major depressive disorder at any given time and up to 16.2% of individuals will experience at least one major depressive episode during their lifetime @xcite @xcite . mental health services are transitioning to online mediums at a rapid pace, with the recent covid-19 pandemic dramatically further accelerating this trend @xcite . thus, analysis of online language may play a key role in mental health treatment in the future. nonetheless, care must be taken to understand potential biases inherent in this research before any technologies are deployed in a clinical setting. for instance, previous work has found that black and hispanic/latinx individuals are less likely to be treated for depression than white individuals @xcite . possibly a result of this underlying bias, recent studies of the us population have concluded that baseline rates of depression vary depending on demographics @xcite -major depressive disorder was found to be more prevalent in females and white adults. yet, it remains unclear whether these supposed differences in depression prevalence between gender and racial/ethnic demographic groups are the result of measurement error or other confounders. various psychological studies have found mental health disorders, including depression, may manifest differently depending on cultural background and thus make uniform diagnosis a difficult proposition @xcite . these ambiguities were highlighted by recent computational research from @xcite , which found predictive rates of depression inferred using classifiers for social media data to not match previous us depression estimates. indeed, the authors actually find that black and hispanic/latinx individuals are more likely to be affected by depression than white individuals. additionally, nlp and other data-driven algorithms have been shown to suffer from content biases; that is, undesirable group-wise differences with respect to protected groups, such as race/eth-nicity or gender @xcite @xcite @xcite . therefore, in consideration of the social impact of nlp research @xcite , in the area of mental health content analysis it stands to reason that we should also look for population biases, as they pertain to protected groups, and the way these might affect nlp algorithms' fairness. previous research has utilized user demographics within social media mental health studies to construct control groups @xcite , to enhance classifier performance through additional features @xcite , and to analyze trends amongst specific populations @xcite . in an attempt to preemptively address population biases, @xcite proposed a cohort-based sampling approach to collect representative measures of wellness amongst the general population. however, as noted in a recent literature reviews @xcite , no previous computational mental-health study has accounted for differences in population-level depression rates nor explored performance variations across demographic subgroups at training time. therefore, little is known about the fairness of these automated systems. are models trained for mental health fair across demographic groups? are current datasets demographically representative? if bias exists, what is its source? in this study, we analyze two common depression-inference datasets and explore the susceptibility of different computational methods to demographic biases. we find that existing datasets are not demographically representative and that, without accounting for this, we find degradation in model performance for underrepresented groups. we explore the possible sources of this bias and conclude with recommendations for future research that may address these issues. ## mental health and social media challenges obtaining mental health annotations for social media data have thus far constrained the size and quality of existing datasets. for instance, manual annotation of mental health status generally requires expert domain knowledge, while the sensitive nature of such annotations limit multiinstitutional data sharing @xcite . consequently, most datasets rely on labels based on behavioral proxies or self-reported diagnoses, which more easily scale, but introduce problematic self-disclosure bias and label noise. furthermore, as our understanding of mental health is continually evolving, studies have used different and sometimes conflicting guidelines for annotation @xcite . with these challenges at the forefront of dataset curation, issues surrounding demographic balance and representation have been largely kicked down the road of the research domain. challenges accounting for demographics go beyond the computational research space and are wellillustrated by disparities between two recent surveys of depression prevalence. the centers for disease control and prevention (cdc) found depression prevalence between race/ethnicity groups did not differ @xcite , while a study using the results of the national epidemiologic survey on alcohol and related conditions iii (nesarc-iii) found depression to be more prevalent in white americans versus minorities @xcite . ## ethical considerations the sensitive nature of mental health research and individual demographics requires us to consider possible benefits of this study alongside its potential harms. specifically, we must evaluate the cost-benefit trade-off of inferring and/or securing three highly-personal individual attributes: (depression diagnoses: @xcite ; gender identity: larson, 2017; race/ethnicity identity: wood-doughty et al., 2020). the potential immediate benefit of this study is a better understanding of demographic bias in computational mental health research. a potential secondary benefit is the mitigation of extant clinical treatment disparities @xcite . as mental health treatment increasingly adopts an online delivery mechanism, this research is uniquely situated to inform the development of new ai systems and public policy in the area. however, we are cognizant of the potential harms from our work. mental health status and demographic identities are both sensitive personal attributes that could be used to maliciously target individuals on publicly-facing online platforms. therefore, we follow the guidelines of @xcite and @xcite on data use, storage, and distribution. all analysis was conducted on deidentified versions of data, with any identifiable information being used only during intermediate data-processing subroutines that were hidden from researcher interaction and approved by the original dataset distributors. our study was exempted from review by our institutional review board under 45 cfr § 46.104. to facilitate any form of statistical analysis, we also need to formalize gender and race/ethnicity. we seek a balance between the limitations of demographic inference systems 1 and alignment to demographic categories conventions used by the mental health literature @xcite , versus propagating demographic definitions that exacerbate existing biases towards gender and racial/ethnic minorities. we consider the 'folk conception' of gender as described in @xcite and prominently leveraged in traditional depression research in the united states -we use the sex categories male and female to denote the corresponding gender categories masculine and feminine. however, many individuals do not fit in these gender categories, some present a gender online inconsistent to their true identity @xcite , and they often experience depression and other mental health conditions at a higher rate @xcite . for race/ethnicity labels, we consider the mutually-exclusive labeling conventions invoked by @xcite and wood-doughty et al. (2020): non-hispanic white, non-hispanic black, non-hispanic asian and hispanic/latinx, as they are representative of the majority of racial and ethnic identities in the us. our racial/ethnic categories do not capture multiracial individuals or those with a race/ethnicity outside this group. we acknowledge these important limitations, but at the same time, there is an urgency to the questions we pose. computational methods for monitoring mental health have already been deployed by digital surveillance companies @xcite , while analytics dashboards based on these methods are gradually making their way into patients' @xcite and providers' hands @xcite . the question is: should we avoid asking these questions about current datasets because we cannot produce clear answers, or should we conduct analyses with acknowledged limitations to learn what we can about research that is already being moved into products? we firmly believe the latter. our hope is that this paper causes re1 we infer gender and race/ethnicity labels using a content classifier and explore additional limitations in section 5 searchers to carefully consider these issues, elevate the need for further work, and produce studies that go beyond our study's limitations with new data and methods. this should not be the last study on this topic; rather, we hope it is the first step which can inform further critical analyses of work in this area. ## datasets we select the task of depression inference for this study, as it is the most widely studied mental health condition in social media research @xcite . we consider two twitter datasets: clpsych @xcite and mul-titask @xcite . ## analysis we conduct an analysis of these datasets and depression models trained on these datasets to answer the following questions: before we can empirically measure if these datasets are demographically representative, we must first establish the expected distribution of a representative dataset. while the demographic groups distribution should match the true population (twitter users with depression), there are no estimates of depression prevalence on twitter. thus, we use the twitter us population as our baseline, and combine it with estimated prevalence of depression among us demographic groups. 2 methods. @xcite found in a study of adults (>20yrs.) that women are almost twice as likely to be diagnosed with depression compared to men (1.89×) using patient health questionnaires (phq-9). we refer to this study as 'cdc.' similarly, @xcite used a national survey of adults (>18yrs.) and the dsm-5 standard for major depressive disorder (mdd) to estimate that women are almost twice as likely to be diagnosed with depression compared to men (1.86×). we refer to this study as 'nesarc.' while there were only small incongruencies between these studies in estimated prevalence of depression as a function of gender, there were significant discrepancies between studies with respect to estimated prevalence as a function of race/ethnicity. specifically, cdc found that rates of depression were not statistically different between groups, whereas nesarc found a greater prevalence of depression among whites compared to african americans, hispanics/latinx and asian americans (1.25x). we project these estimates of depression prevalence to the general twitter population, where both males and females are estimated to participate equally (approximately matching the us population). while there is a slight under-representation of white individuals in twitter compared to us population (60% vs 64%), black and hispanic/latinx individuals are well represented @xcite . thus, barring slight variations, twitter roughly mimics the demographic composition of the united states demographics fairly with respect to gender and race/ethnicity. we combine the twitter population estimates with depression rates in the us to get the target distributions of demographic users that we expect to observe in our datasets. figure 1 shows differences between the expected, representative distribution and our complete twitter datasets. results. based on these estimates, are the depression datasets demographically representative? figure 1 shows that clpsych and multitask are not demographically representative with respect to either gender or race/ethnicity. white individuals are over-represented, while hispan-2 our study uses mental health statistics from the united states since they are extensive and widely available. however, due to data anonymization we could not filter our data based on residence in the us. since these datasets are filtered to focus on english accounts, us accounts likely dominate. ic/latinx individuals are the most underrepresented. in fact, there are no male h/l individuals represented in the train split of clpsych. multi-task exhibits a larger population bias against minorities compared to clpsych; white individuals are over-represented and black individuals are under-represented. with respect to gender, both clpsych and multitask have similar distributional skews -females are over-represented compared to the depression adjusted general us population. at the user level, we found no major differences on number of tweets and vocabulary size between demographics: see details in appendix b. overall, clpsych and multitask are not demographically representative with respect to us depression rates projected on twitter demographic estimates. ## limitations depression and control groups. the method used to curate the depression group in these datasets is susceptible to self-selection bias, as noted by @xcite and @xcite , as it likely over-represents individuals who are more vocal about their condition. therefore, differences in use of social media and cultural perceptions around mental health may introduce biases in these datasets. further, while expert annotators identified non-genuine disclosures depression and removed these individuals from the clpsych and multitask datasets, they did not verify the authenticity of the diagnosis. similarly, individuals in the control group may have been actually diagnosed with depression, but did not disclose their condition anywhere in their public timeline. thus, labels for both the the depression and control groups are bound to be noisy. representation. we balance the multitask dataset to match depression rates in the us, which may not be representative of non-us populations. additionally, we preserve the even depression/control splits for class balance in model training, in-stead of using the true depression/control population rates (about 1/10). these splits are not reflective of the true depression prevalence and their use may need to be modified depending on downstream classifier use. demographic labels. due to dataset limitations, the ethnicity and gender labels for this study were inferred using the unigram model from @xcite . this model considers only the four largest race/ethnicity groups in the us, which aligns with conventions from @xcite but ignores smaller populations and multi-racial categories. further, in some of our analyses, we combine asian, black and hispanic/latinx individuals into people of color due to a lack of data. with respect to gender, the male and female labels used by this model do not consider individuals who fall outside of traditional binary gender. as our experiments rely on upstream demographic label inference, we cannot fully rule out confounding factors due to e.g. noisy labels in our experimentation, but we perform a high-confidence filter on demographic labels and statistical testing on results to strengthen our conclusions. ## conclusion and recommendations we examine whether datasets and the resulting trained classifiers for depression prediction are fair across demographic groups. our analysis finds that (1) depression datasets are not demographically representative, in some cases excluding entire intersectional groups and (2) the resulting classifiers perform worse on people of color in general. in examining the reason for these differences, we find that performance difference could be improved after accounting for (3) the size of the dataset and balance across demographic groups. (4) finally, we show that signals of depression found by previous work using e.g. liwc features are not equally representative for all demographics. these findings should give pause to researchers in this area. since datasets and the resulting models are not demographically representative, advances in methods may be furthering biases towards some groups. worse, since some intersectional demographic groups are not even represented in the data, compounded by the fact that most datasets do not have labels for demographic groups, we currently lack the means to even check how new methods perform on each group. going forward, research in this area should include demographic analyses so that improvements on the overall dataset can be contextualized by how they perform on each demographic group. at the same time, there is reason for optimism. our data balancing and dataset size experiments reduced demographic disparities of trained models. this suggests that research can continue with existing datasets, but with the modifications we proposed. we release the demographically balanced dataset from our experimentation upon appropriate terms of usage agreement. ultimately, the best approach will be to construct new datasets that better represent the population, especially underrepresented minorities who are most at risk from systematic bias. this may necessitate changes to the data collection methods themselves, which may bias collection against certain groups. for example, self-reports may be problematic as they rely on cultural attitudes towards the expression of mental health information. further research is needed to understand if self-reports and other proxy-based methods for obtaining labels can be successfully adapted to include a more diverse population, e.g. do keywords used to collect tweets skew resulting user populations? further, to produce more conclusive insights with respect to demographics, language-based classifiers for demographic labels need to be further improved. alternatively, other data collection strategies, such as the cohort method of @xcite , may be more successful at ensuring representative datasets.