Extracting position titles from unstructured historical job advertisements (2024)

Abstract: This paper explores the automated extraction of job titles from unstructured historical job advertisements, using a corpus of digitized German-language newspapers from 1850-1950. The study addresses the challenges of working with unstructured, OCR-processed historical data, contrasting with contemporary approaches that often use structured, digitally-born datasets when dealing with this text type. We compare four extraction methods: a dictionary-based approach, a rule-based approach, a named entity recognition (NER) mode, and a text-generation method. The NER approach, trained on manually annotated data, achieved the highest F1 score (0.944 using transformers model trained on GPU, 0.884 model trained on CPU), demonstrating its flexibility and ability to correctly identify job titles. The text-generation approach performs similarly (0.920). However, the rule-based (0.69) and dictionary-based (0.632) methods reach relatively high F1 Scores as well, while offering the advantage of not requiring extensive labeling of training data. The results highlight the complexities of extracting meaningful job titles from historical texts, with implications for further research into labor market trends and occupational history.

## introduction historical job advertisements provide unique information about the history and development of the labor market. analyzing the positions offered and sought over time offers insights into temporal and regional differences and development, as well as into social aspects, such as gender-specific job offers. the first step in such an analysis is the extraction of job titles. however, when using historical data from digitized newspapers, rather than digitally-born structured data, the automatic extraction of job titles proves to be a non-trivial task. in the jobads project (fwf p35783), we study historical job advertisements from digitized newspapers from the anno corpus @xcite . the advertisements are predominantly in german, and our defined time span is 1850-1950. the newspaper pages were initially obtained in the form of images and transformed into textual data by conducting the processes of page segmentation, optical character recognition (ocr), and automatic post-correction based on manually transcribed ground truth. afterwards, the job advertisements were extracted. a result of such a pipeline is a corpus containing tens of thousands machine-readable yet unstructured job advertisements. contemporary research often works with modern, digitally-born data, and usually benefits from their structure, such as html tags, to identify position titles. modern research in the context of job advertisements addresses challenges such as extracting or grouping requirements in the job ads, e.g. @xcite @xcite @xcite , automated matching process between the position requirements and the skills of a candidate written in their cv, e.g. @xcite @xcite @xcite , job advertisements classification/categorization, e.g. @xcite @xcite , and job title classification, e.g. @xcite @xcite ; with some works covering more than one research focus. in contrast, we face the challenge of working with unmarked text, and need to find an automated way to extract this information. in this paper, we present a comparison of four approaches to identification of position titles in historical job advertisements. on the one hand, we consider dictionary-based and rule-based approaches, which do not require a time-consuming creation of an annotated training dataset. on the other hand, we use a machine-learning (ml) approach, specifically named entity recognition (ner) and textgeneration models training. this requires manually annotated training data, but yields better results and can recognize position titles beyond those seen in the training data. please note that in this paper, we aim to extract position titles appearing inside the advertisement, in contrast to some of the aforementioned research works which aimed to classify jobs into the occupation categories. the following section summarizes existing research related to job title identification and information retrieval from unstructured text. section 3 describes our dataset and the process of job titles annotation. section 4 presents in detail various approaches we used to identify the job titles and the evaluation methodology, while section 5 presents and discusses the results. section 6 concludes this paper. ## related work while most of the modern research benefits from the structure of ads to extract job titles, as exemplified above, some research addresses the challenge of their extraction nevertheless. one reason for this can be the noisy information included within the job title tags, such as the name of the company, or the need to work with ads from newspapers that were obtained in the form of the image. @xcite develop a methodology for matching a job ad with its standardized occupation in french language. although the job title is included within a dedicated html tag, the authors further process it because other words, such as 'looking for', are sometimes included. they remove these extraneous words based on a manually created to-delete list to obtain cleaned job titles. @xcite work with unstructured text, as they scrape job ads in the form of images from job web portals and newspapers. they apply an ocr process to convert them into machinereadable text. aiming to create a structured dataset, they extract information such as the position name, skills, company name, and contact information. they use a rule-based approach, matching regex patterns or phrases. however, the accuracy of job title identification is only 56% @xcite . the authors do not explicitly state which specific regex patterns were used for job titles extraction. @xcite focus on job title normalization, where job titles are normalized according to a predefined set of occupations. researchers dealing with this question also need to extract job titles; however, they often rely on external taxonomies, structured text, or manual labeling. in @xcite , "the job titles were manually and semiautomatically collected from resumes and vacancy postings." not having found a suitable approach in the existing literature, we focus on other ways to extract job titles from the advertisements. information extraction (ie) is the process of automatically extracting entities, objects, and their roles from text, often within a specific domain or topic @xcite , with named entity recognition (ner) being one of the common technologies used @xcite @xcite . while the standard entities in ner models are typically proper names, locations, or dates, custom ner models can be trained to recognize new entities. therefore, we can consider job titles entities, and having created appropriate training data, use the same approach to extract job titles from the unstructured text. the emergence of large language models (llms) has also opened up the possibility to approach ner as a text generation or translation task @xcite . while this approach has, to the best of our knowledge, not been evaluated on historical data, it has seen promising results in a variety of fields, e.g., @xcite , and we shall include this method in our evaluation. while no work specifically addresses position extraction from historical data, several works deal with historical data and ner. @xcite addressed recognition of person and place names within a digitized corpus of british parliamentary proceedings from 1685-1691 and 1814-1817 using a rule-based approach. working with the output of an ocr software, they reach total f-scores from 70.35 to 76.94 on individual datasets @xcite . the main challenge were the ocr mistakes, namely the noise, misrecognition of characters and issues with separating the text from the marginal notes. @xcite focus on toponyms in two collections of historical letters, one collection in early-modern english, another in modern english. as obstacles they mention e.g. "language changes over time, spelling variations, ocr errors, sources written in multiple languages, and general ambiguity in language use" @xcite 2). the authors do not train new models but evaluate the performance of existing modern tools for the task of location extraction, including two different types of pre-processing and experimenting with re-writing early-modern english text into modern english. they evaluate five different ner systems and an ensemble method which works based on a voting system among the individual ner models. all experiments reached a best minimum f1 score of about 70 @xcite . @xcite trained a pre-trained bert model for a ner task in contemporary and historical german corpora, containing entities of location, organisation and person. working with historical corpora is hindered through the less standardized languages and errors in the ocr. combining unsupervised pre-training on historical german corpus with supervised pre-training using contemporary german ner ground-truth, they achieve the highest f1 score of 84.6 on historical data for 5-fold cross validation. ## dataset using digitized historical newspapers from the anno corpus, we manually annotated, ocred and corrected several thousands job advertisements spanning the period 1850-1950 from 14 different newspapers. within a subset of these job ads, we annotated position names using the doccano software @xcite , yielding 1,486 job advertisements as training data and 637 as testing data. the split into training and testing dataset was random. ads that did not explicitly mention a position or contained errors, such as inconsistent spacing, were mostly not included within the datasets for the evaluation task (tab. 1), however, after the training and evaluation during a human control we discovered a small number of ads containing errors that were included in the dataset by mistake. additionally, we standardized the text by replacing the long s with the letter s in all ads. the training data was used to train the ner and text-generation models and also served as a basis for creating partof-speech (pos) and syntax rules, as well as for adding entries to the dictionary, both established through human observations. the testing data was consistently used for evaluation purposes. although all duplicate ads were removed and the training and testing datasets are mutually exclusive, we cannot exclude the possibility that very similar ads appear in both datasets, given the homogeneity of ads and their structure defined by their genre. each advertisement could contain more than one position offered/sought for. ## methods based on preliminary results, four approaches were selected and compared: a dictionary-based approach, a rule-based approach, a ner and a text generation approach. ## results and discussion the ner approach (fig. 1 ) reached the highest f1 score of 0.944 using the transformer architecture, resp. 0.884 when trained on cpu on the testing dataset, indicating that this method is more flexible than the rule-based and dictionary-based. however, a significant drawback of this approach is its reliance on a manually created training set. the generative approach shows comparable performance with an f1 score of 0.920 when evaluated based on segments and 0.902 when evaluated on the aggregated results for entire ads. since the only difference between the two evaluation approaches is the exclusion of "none" predictions when aggregating, the decrease from the segment based score to the ad based score indicates that the model is slightly better suited to determine that no positions are mentioned in a line of text instead of extracting all mentioned positions. a preliminary check of prediction mistakes has, however, shown that some false positives occur from identifying ambiguous words such as mann the rule-based approach appears to be wellsuited for this task, given the strong linguistic structure of job advertisements. it is also immune to certain spelling variations or typographic mistakes, as long as they do not include spacing errors, or do not hinder the correct pos classification. the problematic aspect of this approach is when more than one position is advertised within an ad, leading to either incomplete detection of all positions or the introduction of false positives in other ads. the dictionary-based approach reached the lowest f1 score. while lemmatizing/standardizing of the tokens mainly solves the issue of spelling variants, it is time-consuming to lemmatize the entire text of the advertisement. however, this approach without lemmatization reaches lower score as usually only one form is present in the dictionary, while the text contains several variants (e.g., commis and kommis [assistance], kontoristin and comptoiristin [clerk f.]). additionally, some professions are simply not present in the dictionary, especially those that were less common or highly specialized. this approach also fails when typographic errors are present in the text (kö chin with line-breaker in the original text instead of köchin [cook f.]). moreover, all mentioned positions are identified, even if they appear as part of a name of a street, or if they are not the primary focus of the advertisement but just mentioned within it, e.g.: kinderarzthilfe. mit zweijähriger praxis, sucht stelle bei arzt oder in einem laboratorium. [pediatric assistant with two years of experience is looking for a position with a doctor or in a laboratory.] our dataset contained advertisements in their ideal form, i.e., manually corrected with only occasional typographic errors. in reality, the thousands of ads can only be post-corrected automatically rather than manually, and more errors will thus be present in the data. this will mostly affect the dictionary-based approach, which needs exact matches with correct spelling, and partially the rule-based approach, which is in theory immune to the spelling variations but fails to correctly identify the pos and syntax dependencies if too many errors occur. the last consideration is the ambiguous nature of what a job title in historical job advertisements is. many ads contain words like mann [man], mädchen [girl] without specifying a clear job title, although it is implicitly understood that e.g., the girl is wanted to help in the household. another example is französinnen [french women], mentioned alongside other job titles for women, where it is implied that they are sought to teach the french language or to provide companionship while conversing in french (fig. 2 ). these advertisements were not included in our dataset for this evaluation task, however, they appear commonly in the corpus. further research could help to address these limitations by focusing on the semantic context of the advertisements, perhaps using techniques like average embeddings to represent the average meaning of the whole advertisement. this approach could help overcome the challenges posed by the ambiguous and context-dependent nature of job titles in historical advertisements. also, the potential of text-generation methods shall be further investigated. ## conclusion historical job advertisements offer many opportunities to study the transformation of the labor market. the first step in analyzing these ads is the extraction of job titles, which allows for the exploration of positions offered and sought, their frequency analysis, and temporal and regional variations. in this study, we compared four approaches for job title extraction: a dictionary-based approach, a rule-based approach, a ner approach, and a text-generation approach. the ner approach achieved the highest f1 score of 0.944 when a transformer-based model was trained on gpu, resp. 0.884 when a model was trained on cpu, as we treated position names as entities and trained a model specifically for their identification. the generative model follows closely with a f1 score of 0.920 and offers potential viability for identifying ambiguous positions. the rule-based (0.69) and dictionary-based (0.632) approaches had lower f1 scores but offered the advantage of not requiring the time-consuming creation of a training dataset. while extracting position names is a crucial first step, two important considerations remain: