Professional Presentation and Projected Power: A Case Study of Implicit Gender Information in E nglish CV s (2022)

Abstract: Gender discrimination in hiring is a pertinent and persistent bias in society, and a common motivating example for exploring bias in NLP. However, the manifestation of gendered language in application materials has received limited attention. This paper investigates the framing of skills and background in CVs of self-identified men and women. We introduce a data set of 1.8K authentic, English-language, CVs from the US, covering 16 occupations, allowing us to partially control for the confound occupation-specific gender base rates. We find that (1) women use more verbs evoking impressions of low power; and (2) classifiers capture gender signal even after data balancing and removal of pronouns and named entities, and this holds for both transformer-based and linear classifiers.

## introduction in this paper, we study word choice and implied power and agency in curriculum vitae (cvs) authored by men and women, combining lines of research that emerged from a long research tradition in both the social sciences and, more recently, natural language processing @xcite @xcite . from a sociology perspective, it has been suggested that choices of words are influenced by the social status of the respective genders at a given moment in society @xcite . women are known to use more communal forms of words and emotional connotations than men @xcite @xcite , and that such choices reflect the different levels of power and influence both politically and economically @xcite . conversely, the choice of language impacts how the reader perceives the entity described in the text. in particular, the choice of * equal contribution ♥ corresponding author verbs has been suggested as an indicator of the perceived levels of power and agency of the described entity @xcite . organisational scholars have long documented gender discrimination in employment @xcite @xcite . sociological studies have repeatedly shown that women are evaluated more harshly than men especially in recruitment @xcite @xcite . men tend to be assessed for their competence, while women are assessed based on characteristics ('likeability'), even when they demonstrate the same levels of qualifications, experience and education @xcite . @xcite studied the impact of "gendered wording" in job advertisements on gender inequality in traditionally male-dominated occupations via content analysis, while @xcite showed how gender signal in online biographies lead to disparate performance in the task of occupation classification. experience has shown that leaving hiring decisions to supposedly objective algorithms did not remove bias from the process -both in real-world applications like amazon's gender-biased automatic hiring tool @xcite , as well as a surge in research on showing and alleviating bias in nlp models @xcite . we present a data set of 1.8k human-written cvs and study differences in word choice and framing between men and women, and the extent to which classifiers are susceptible to gendered language. unlike prior studies which were either occupation-specific @xcite or used proxy data like online biographies (de-arteaga et al., 2019), we inspect application materials directly and cover 16 occupations (appendix b) which allows us to study gender differences while partially controlling for the confound of occupation-specific base rates. however, we find that even within occupations, confounds remain as women tend to occupy lower ranking positions and men and women cluster in different types of finetrained jobs within an occupation category (section 3). our cv authors provide self-identified gender as part of our screening questions. ## dataset collection on prolific 2 , we hired 2,000 participants (50% women) who were (1) us american and live and work in the us; and (2) in full-time employment. after answering a number of screening questions, participants composed a cv "pretending that you were applying for your next promotion". we specifically asked our participants to copy from their existing cv, instead of write an entirely new cv to mimic real-world cvs as closely as possible. it was encouraged to anonymize information wherever possible, but otherwise craft a cv as realistic as possible given their current situation. for a uniformed structure, we segmented the cv submission into five parts, each as a free-text box: (1) an optional professional summary/career objective, (2) professional experience, (3) education; (4) skills and attributes; (5) optional certifications/qualifications. ## power and agency in cvs do men and women differ in the way they present themselves in a cv? we compare the extent of power and agency implied in the verbs used by male and female applicants. we apply @xcite 's connotation frames of power and agency, which associate verbs with the reactions they evoke the reader @xcite . by focussing on verbs, we abstract away from (named) entities with a strong occupation association and focus on self-presentation @xcite . we consider all transitive verbs in cvs. given the content (primarily focused on the author) and style (listings, incomplete sentences) of cvs, we assume that the agent of every verb is the author. the power dimension distinguishes verbs where the agent (subject) has more (@xmath0; 'lead'), less (@xmath1; 'assist'), or equal (@xmath2; 'care') power to the theme (object). the agency dimension categorizes verbs as high (+; 'support'), low (-; 'wait') or neutral (neu; 'access') agency. we use @xcite 's framelabeled data set of 2k english verbs. 48% of verb types in our cvs are in the labeled data set (conversely, 57% of frame-labeled verbs occur in our cvs). the numbers are comparable across genders. ## gender prediction from cv text sections 3 and 4 explored gender-speciric content framing differences cvs which may impact judgment the reader (or hirer). we next quantify the susceptibility of representative predictive models to gender information in cvs. we use the task of binary author gender prediction based on the text of the cv as a diagnostic tool to assess the extent to which models can infer gender information from cvs. we explicitly caution against using this task as a ml benchmark (cf., ethics statement). we test the following binary classifiers: a linear svc with l1 regularization, which by design learns sparse and interpretable features; a lo- ## discussion we presented a data set of 1.8k authentic, us-english cvs across 16 occupations, aligned with self-reported binary gender of the author. this data set allowed us to inspect features of men-and women-specific language in cvs, while controlling for the confounding factor of occupation: most occupations are heavily gender-skewed. this paper connects the concept of framing, i.e., influencing readers of a document through careful choice of words @xcite , with existing power discrepancies between men and women in western society in general, and the job market specifically @xcite . we showed that women use verbs that imply lower power significantly more often than men, even after controlling for occupation. subtle changes in word choice have been shown to impact human perception, reaction and choice @xcite . in the context of human hiring, this suggests that (a) removing explicit gender indicators is insufficient; and (b) further support for sensitizing both hirers and applicants to subconscious bias. we further trained classifiers to predict binary author gender based on cv text, in scenarios where gender proxy information was removed by genderbalancing the training data and/or removing named entities. we show that classifiers perform significantly above chance across all settings, confirming that subtle gender signal remains. this result is expected, and in line with prior research @xcite , but for the first time shown directly on data more akin to application materials presented to human and automatic hirers. our experiments retain a confounding factor of job type within an occupational group: within an occupation, women tend to have lower-ranking jobs; and within our 16 broad occupational groups, different specific occupations will exhibit different gender skews. in section 3, we inspected genderassociated word choice in 6 most frequent occupations in our data set, finding that across occupations, 'scientistic' terms (engineer, developer, database) and leadership terms (leadership, administration, planning) are more associated with male cvs; while women are more likely to mention interpersonal skills, support, or teamwork @xcite , typically associated with administrative roles. consequently, gender signals in cvs not only originate from lexical choice, but also also reflect real-world differences in work tasks and position levels. disentangling these factors is an important direction for future work. in sum, we maintain that perpetuated gendered patterns embedded in cvs can bias both human and automated hiring, and that the naive use of ml methods bears the risk of exacerbating bias: by picking up spurious associations on different levels from explicit gender information (names, hobbies) to subtler word choice (the level 'power' or 'agency'). suggestions for further work include usability studies and social-psychological interventions for users of recruitment software, for both job applicants and decision makers. interventions could include 'nudges' in the user experience flow, informing users about potential gender signals being encoded in their data, and suggestions of strategies to mitigate or minimise this. as our findings suggest, scrubbing names and entities off the cvs is not effective in de-gendering cvs for fairer recruitment decisions, and should not be used as the be-all-and-end-all in promoting fair hiring, as often is the wont of current initiatives.