Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback (2022)

Abstract: Though approximately 50% of medical school graduates today are women, female physicians tend to be underrepresented in senior positions, make less money than their male counterparts and receive fewer promotions. There is a growing body of literature demonstrating gender bias in various forms of evaluation in medicine, but this work was mainly conducted by looking for specific words using fixed dictionaries such as LIWC and focused on global assessments of performance such as recommendation letters. We use a dataset of written and quantitative assessments of medical student performance on individual shifts of work, collected across multiple institutions, to investigate the extent to which gender bias exists in a day-to-day context for medical students. We investigate differences in the narrative comments given to male and female students by both male or female faculty assessors, using a fine-tuned BERT model. This allows us to examine whether groups are written about in systematically different ways, without relying on hand-crafted wordlists or topic models. We compare these results to results from the traditional LIWC method and find that, although we find no evidence of group-level gender bias in this dataset, terms related to family and children are used more in feedback given to women.

## introduction female physicians and trainees have advanced considerably in the medical field within recent years, and approximately 50% of medical school graduates are now women @xcite . however, female physicians lag their male counterparts in salary, promotions, and positions in senior leadership @xcite @xcite . a mechanism that perpetuates this inequality may be unequal evaluations of male and female physicians. past work has revealed gender bias in several forms of evaluation. evaluations of recommendation letters in academia found that women tended to be described in communal traits (caring, nurturing) whereas men were described in agentic terms (ambitious and self-confident) @xcite . the same trend holds in direct observation comments given to emergency medicine (em) residents, with feedback themes varying by gender, particularly around the domains of authority and assertiveness @xcite . in the same context, women were also found to receive more contradictory and polarized assessments on their skills as compared to men @xcite . if there are systemic differences in evaluations for different genders, it may be possible that these differences arise early in a student's career and snowball into fewer opportunities in late career, when they are quantitatively detectable through metrics such as salary and number of promotions. it is important to understand at what phases of a student's career inequities arise, so that interventions can be targeted toward supporting women or other underrepresented minorities at these stages. focus groups of female physicians in the field find different experiences at early, mid, and late career stages, with older women experiencing more overt discrimination, and younger women reporting more implicit bias, though it is unknown if this is due to decreased discrimination in recent years, or due to younger physicians not yet recognizing signs of discrimination @xcite . findings on gender differences in language are mixed for students earlier in their careers. a qualitative analysis of surgical residency letters of recommendation, collected from before the students applied for residency, found that male applicants' letters contained more achievement-oriented terms, whereas female applicants' letters contained more care-oriented terms @xcite . however, a similar analysis on the em standardized letter of evaluation found no such difference @xcite . to investigate this question more thoroughly, we use a new dataset of written assessments on medical students' work based on individual shift performance before their residencies. most previous work from the medical community has used relatively simple linguistic methods such as the linguistic inquiry and word count dictionary (liwc) @xcite @xcite , but using pretrained language models may allow us to investigate bias in a more fine-grained manner @xcite . additionally, existing work on medical bias within the nlp community mainly focuses on patients, rather than physicians themselves @xcite . we fine-tune a pretrained bert model and use its predictions as a tool to try to identify group-level prediction residuals. if such a difference exists on a systematic level, it may indicate that assessors are writing about students in different ways based on their gender, given the same objective performance. caution should be taken when using similar methods as language models can also come imbued with biases of their own, but we outline the method in this work and highlight its use in comparing model predictions and human judgments when both text data and quantitative data are available. although we can replicate past work showing a significant difference in social-communal terms used to describe women, we do not find as clear a relationship between comments written about a student and the global score given on a shift. we do not find a systemic difference between male and female students when comparing group-wide residual differences. this indicates that although male and female students may be written about differently, no gender is written about in a systemically worse way. due to privacy concerns, the dataset is not available online, but the full dataset can be obtained through emailing our medical co-author: kmhiller@iu.edu. ## bias statement we study the relationship between text comments and numeric ratings of performance given to male and female medical students. we introduce the method of comparing language model residual predictions to numeric data to find group-wide differences in language use. we fine-tune a language model to predict the rating associated with a given comment about a student, and ask if there is a cross-group difference in the residual error that the trained model makes. for instance, are female students given less positive-sounding comments than their male counterparts for the same level of clinical skills (as measured by their numeric evaluation scores)? feedback from supervisors is used to make decisions on whether a student receives a residency, or later on whether they get promoted to a higher position within medicine. this has potential to address allocational harms to women within medicine. the under-representation of women in senior positions in medicine could also lead to wider harms in inequity as a result. there are shortcomings in presenting gender as a binary, and in this dataset gender information was not collected based on self-identification. we hope that future work will explore a wider diversity of gender identification, but we present this analysis as a first step. ## dataset the dataset consists of evaluations of undergraduate medical students conducted with the national clinical assessment tool (ncat-em), the first standardized assessment based on direct observation of skills in a clinical setting (of american @xcite . the ncat-em was developed by em educators, and has been implemented at 13 institutions in the united states. data was collected from departments participating in the ncat-em consortium from 2017-2019 @xcite . the dataset contains short free text comments on a student's performance, categorical assessments on multiple skill areas, a global competency score (lower third, middle third, top third, and top 10%), as well as demographic information about students and assessors: gender, age, rank of assessor (junior vs senior faculty). these attributes are outlined in figure 1 . examples of free-text comments and associated scores are given in after excluding samples with missing data, there were 3162 individual assessments, where 1767 were evaluations of male students and 1395 were evaluations of female students. because students may work multiple shifts, and the same supervisor may supervise multiple students, there are some students and assessors who are repeated, although each sample represents a different shift. names and named entities were removed from comments using the spacy entity recognizer and replaced with the letter "x". gendered pronouns were removed and replaced with the gender-neutral pronoun "they". this dataset consists mostly of short comments focused on student performance. the mean number of words in a comment was 28.4, and the maximum number of words in a comment was 187. the overall distribution of assessment ratings was: 5% in bottom third, 35% in middle third, 45% in top third, and 15% in the top 10%. a slightly higher density of female students received the top rating compared to male students. we convert these to integer values from 0-3. we use two main methods to identify possible biases in this dataset: prediction residual analysis and word/topic based analysis. previous work has focused on word-level analysis, but since we have access to both comments as well as a competency score, we investigate to what extent we can reconstruct the mapping from text comment to the score a student receives by applying a language model, and if there are differences in this mapping between male and female students. we used 70% of the dataset to train, 15% to evaluate, and 15% as the test set. ## conclusion gender bias in medical education is a major barrier to women in the field, and it is important to know in what circumstances and career stages it occurs in order to create targeted training and intervention. previous work has found that there may be potential bias in medical student recommendation letters, but we investigate whether there is systemic bias in an everyday setting in feedback given to male and female medical students. we collect data using ncat-em evaluations to answer this question, and use language model residuals to investigate the relationship between free text comments and integer ratings given to students. we find no evidence of bias using the residual definition, although we find that there is a statistically significant difference in the percentage of comments that mention social-communal themes, with women receiving more mentions of family-oriented words in their evaluations. one limitation of this dataset is that the mapping between text comment and global score is quite noisy, as neither a fine-tuned language model nor human judges were able to achieve a high score in classifying the text based on the global rating. however, the prediction residual method can be used in any dataset with both text data and outcome data, for instance applications to educational programs, or employee evaluations. one caveat is that language models themselves can be biased, so this method is best applied after sensitive attributes have been obfuscated. additionally, this dataset is quite small and limited to a relatively small set of samples. it is possible that biases could be found in a larger dataset of shift evaluations, or in data collected from a different set of institutions. however, we leave such data collection to future work, and hope that this encourages the collection and analysis of similar data on a wide scale. we hope that this work will inspire further research into how bias manifests or does not manifest at different stages of professionals' careers, and how we can combine multiple sources of information together with text to form a wider view of bias and fairness.