Regex in a Time of Deep Learning: The Role of an Old Technology in Age Discrimination Detection in Job Advertisements (2022)

Abstract: Deep learning holds great promise for detecting discriminatory language in the public sphere. However, for the detection of illegal age discrimination in job advertisements, regex approaches are still strong performers. In this paper, we investigate job advertisements in the Netherlands. We present a qualitative analysis of the benefits of the ‘old’ approach based on regexes and investigate how neural embeddings could address its limitations.

## introduction age discrimination is often related to work and it starts in the pre-hiring phase with job advertisements. each year, thousands of job descriptions in the netherlands contain age discrimination, which is illegal under dutch law @xcite . the state of the art in detection of illegal age discrimination in dutch job ads uses regular expressions (regex) @xcite . this 'old' approach works surprisingly well because illegal age discrimination uses predictable vocabulary, and keywords such as 'age' are quite reliable indicators. however, individual sentences from job ads suggest that neural embedding approaches, with their ability to capture semantics, could also be helpful, e.g., 'given our own advancing years, it would be just lovely to have a younger soul join us. @xcite the contribution of this paper is a qualitative analysis of the role that regex should continue to play in detecting illegal age discrimination, now that the language technology community has moved towards deep learning approaches. since regexes offer explainable decisions, we do not seek to abandon the regex approach, but rather to understand its potential compared with the potential of neural embeddings. because it is known that the regex approach can suffer from low recall @xcite , our main focus is on understanding false positives (i.e., cases of discrimination that the detector misses). in this paper, we report the essential findings on illegal age discrimination detection in dutch job ads of a larger study @xcite , which contains further analysis. after a brief introduction to age discrimination (sec. 2) and the regex approach of @xcite (sec. 3), we present two analyses. the first (sec. 4) investigates the regex approach, which is currently the state of the art. the second (sec. 5) looks at whether and how neural embeddings could complement regexes in the future. our analyses make use of the job digger dataset, which contains 1.2 million dutch job advertisements collected by a dutch company, job digger, and made available to us for use in our study. job digger had created the dataset by carrying out a large scale crawl of internet job postings in the netherlands in 2014. the comprehensiveness of this crawl ensures that our dataset is representative of the full spectrum of possible dutch job ads. our investigation reveals that the regex approach is more difficult to improve upon than one might think. the final section of the paper (sec. 6) provides an outlook and discusses how researchers in the future should seek to leverage both regexes and neural embeddings for explainable detection of illegal age discrimination. ## background and related work age discrimination is defined as bias and prejudice against people based on their age and ageism is one of the three big 'isms', next to sexism and racism @xcite . in practice, age discrimination predominantly targets older people @xcite . ageism is in this sense unique among 'isms' because, in the natural course of life, in-group members become out-group members @xcite . however, despite the fact that it threatens everyone, ageism is difficult to fight. it is culturally acceptable @xcite and people are unaware of it @xcite . in the netherlands, con-cern about age discrimination has grown recently, mainly in employment @xcite . age discrimination occurs in two main forms @xcite . objective ageism is defined through legal frameworks that protect the vulnerable group from discrimination. subjective ageism (or perceived ageism) is bias and discrimination that does not fall under a legal definition. in the netherlands, the dutch equal treatment act regarding age discrimination at the workplace prohibits discrimination in the context of work, including job advertisements. the law defines two forms of discrimination: direct discrimination involves an explicit mention of the age of the candidate, e.g., 'you are younger than 30 years'. indirect discrimination, involves formulations that imply age, e.g., specifically recruit students (who, in the netherlands, characteristically are young). the literature on age discrimination detection in job ads is surprisingly limited. the work closest to ours studied the relationship between stereotypes in english-language job ads and in hiring @xcite . it implemented an age discrimination detector for job ads, but focused on stereotypes, which are not necessarily illegal. in contrast, we study detection of discriminatory statements that are explicitly defined, and prohibited, by law. ## regex baseline the state of the art in the detection of age discrimination in dutch job ads @xcite uses a list of keywords to detect objective ageism. the keywords were identified by manually reading a large number of job ads. they were selected because they were judged to be indicative of illegal discrimination when used in certain contexts. appendix a contains the keyword list with a sample sentence from a job ad for each keyword. the keywords form the basis of a set of regular expressions, which @xcite constructed with the aim of covering all possible contexts in which each keyword could be discriminatory. the importance of context is illustrated by the following example. the sentences, 'you will be responsible for young students' contains both the words 'young' and 'student', but is not discriminatory because the words describe the job and not the candidate. @xcite published a set of these regexes on github foot_0 . they discovered that regexes perform best if they allow a certain amount of flexibility by including the white card character .{0,30}, e.g., 'you\s+are\s+a\s+.{0,30}student'. they report that such flexible regexes achieve a high precision (94.5%), but a somewhat low recall (75.7%) on their test set. ## role of the regex baseline in this section, we discuss our first qualitative analysis, which aimed to reveal both the potential and the inherent weaknesses of the regex approach. ## role of neural embeddings in this section, we discuss our second qualitative analysis, which aimed to discover how neural embeddings can potentially complement regex. since the issue of missing keywords was already raised by @xcite , we focus on another property of regexes that sec. 4 revealed to be an issue for detection of illegal age discrimination: they cannot capture discrimination when it is phrased using different syntax but expresses similar semantics. this inflexibility becomes particularly important when we consider the importance of modeling the broader context of a keyword within a sentence. ## conclusion and outlook in this paper, we have investigated the contribution of regexes to the task of automatically detecting illegal age discrimination in dutch job ads. we have found there is potential to improve the recall of the regex lists of @xcite , which constitute the current state of the art, not only by adding keywords, but also by creating additional regexes. future work should investigate a simple approach based on rule mining, which was not explored by @xcite . in @xcite , we report an exploration of automating the generation of regular expressions using active learning and genetic programming, but more work is necessary if these directions are to yield fruit. the results of our analysis suggest that there is little to be gained in using neural embeddings directly in age discrimination detectors. instead, neural embeddings could have a role in the discover of new keywords and new regexes, extending a simple rule mining approach. using neural embeddings in this way would allow us to continue to benefit from the explainability of the regex approach. the results of our qualitative study are not dependent on particular keywords, writing styles, or special properties of the dutch language. for this reason, we expect that our findings can be reproduced using other datasets and in other languages. in fact, regex has been successfully used for general discrimination detection in indonesian job ads @xcite . reproduction of our study will confirm and extend our findings, ensuring that the 'old' technology of regex is not discarded for a task for which it is well suited.