Recognising Occupational Titles in G erman Parliamentary Debates (2024)

Abstract: The application of text mining methods is becoming more and more popular, not only in Digital Humanities (DH) and Computational Social Sciences (CSS) in general, but also in vocational education and training (VET) research. Employing algorithms offers the possibility to explore corpora that are simply too large for manual methods. However, challenges arise when dealing with abstract concepts like occupations or skills, which are crucial subjects of VET research. Since algorithms require concrete instructions, either in the form of rules or annotated examples, these abstract concepts must be broken down as part of the operationalisation process. In our paper, we tackle the task of identifying occupational titles in the plenary protocols of the German Bundestag. The primary focus lies in the comparative analysis of two distinct approaches: a dictionary-based method and a BERT fine-tuning approach. Both approaches are compared in a quantitative evaluation and applied to a larger corpus sample. Results indicate comparable precision for both approaches (0.93), but the BERT-based models outperform the dictionary-based approach in terms of recall (0.86 vs. 0.77). Errors in the dictionary-based method primarily stem from the ambiguity of occupational titles (e.g., ‘baker’ as both a surname and a profession) and missing terms in the dictionary. In contrast, the BERT model faces challenges in distinguishing occupational titles from other personal names, such as ‘mother’ or ‘Christians’.

## text mining in vet research thanks to the spread of the internet in general and social media in particular, more and more communication is now taking place in written form. and communication that takes place outside the internet also finds its way into web archives and repositories, for example in the form of minutes or transcripts. text documents can therefore be a valuable source for (social science) research, from which conclusions about social processes and contexts can be derived. the challenge often lies in building bridges between what is said or written and the extra-textual phenomena, i.e. the actual objects of research (see @xcite ). in qualitative content analysis, this is done by including a small number of texts in the manual analysis, in which the theoretical concepts are then related to specific text passages, for example using a hermeneutic approach. however, as soon as larger volumes of text are to be analysed, manual methods reach their limits. like other disciplines from the social science or humanities, research on the labour market and vocational training deals with fuzzy concepts, such as occupations, skills or job tasks, which are often characterised by the fact that they are defined in an abstract and complex way @xcite . as interdisciplinary research field, vet research connects many disciplines, like social sciences, education sciences, economics or psychology, and deals with research questions on apprenticeships, learning designs or labour market needs. within the intersection of vet research and text mining, many studies analyse skills, competencies or qualifications based on job advertisements as data source @xcite . recent studies also include training regulations and curricula in the quantitative text analysis @xcite . in many cases, skills extraction is operationalised by using dictionary-based approaches. initial approaches are also testing the use of machine learning to recognise skills in job advertisements @xcite . @xcite provide an extensive overview on methods for skills extraction. alongside skills, a central concept in vet research is the one of occupation. at first glance, this concept seems easy to grasp, but it harbours various challenges when it comes to (automated) identification of references to occupations in texts. we will use the example of identifying occupational titles automatically in the plenary protocols of the german bundestag to demonstrate these challenges. the plenary protocols are well suited to this demonstration as they contain around 900,000 speeches, which would be difficult to process manually. the corpus is briefly described in the following section. section 3 then introduces the concept of occupational titles. section 4 is devoted to the concrete implementation by describing two different approaches to operationalisation and comparing their results on a small data basis. section 5 finally describes the application of both operationalisations on a larger data basis in order to compare the varying results. the code is available on github: https://github.com/johannabi/profrec_bundestag 2 plenary debates as research object parliamentary debates have been in focus for recent research in multiple disciplines, like political sciences, corpus linguistics or computational linguistics. research interest ranged from network analysis @xcite over rhetorical analysis @xcite to argument mining (eide, 2019) or sentiment analysis @xcite . to our best knowledge, there has not yet been any research in the context of parliamentary debates that has dealt with the identification of personal nouns, i.e. common nouns referring to (groups of) persons, in general or occupational titles in particular. for vet research, the identification of occupational titles in plenary debates could provide valuable insights into how politicians -as representatives of society -talk about certain professions. for example, it could be analysed to what extent the interests of certain professional groups are addressed and asserted in the speeches. or, for research into inequality and stereotypes, occupational titles could be analysed to determine the extent to which the mention of certain professions within debates on inequality is characterised by stereotypes. the presented experiments are based on the open discourse corpus @xcite , but the experiments could also be adopted to other corpora of german parliamentary debates, such as germaparl @xcite . the open discourse corpus was developed by using the transcriptions of plenary sessions that are published by the german parliament in an xml format. in addition to the german-language speech content, the corpus contains metadata such as the date, the legislative period as well as various information on the speaker such as their name or fraction. documents are separated by speakers and interjections are stored in a different structure. the open discourse corpus contains 907,644 different speeches from the period 1949 to 2021. sections 4 and 5 use samples from the open discourse corpus to show how occupational titles can be automatically identified in the plenary transcripts and what challenges arise in the process. so, in order to better describe these challenges, the following section will first introduce the concept of occupational title. ## occupational titles as research object from a linguistic point of view, occupational titles are common nouns that represent a subgroup of personal nouns. these in turn are linguistic expressions that refer to individual persons or groups of persons. in this context, both personal and occupational nouns stand in contrast to proper names. while proper names refer to concrete persons, personal and occupational nouns refer to more abstract concepts. in the case of occupational titles, the terms are characterised by the fact that the person or group of persons performs a profession. while other research have already focused on identifying personal nouns in german texts automatically @xcite , the focus on occupational titles as a separate group was only set for the task of classifying spanish tweet to whether they contain occupational titles. @xcite . occupational titles can refer to a specific person or group (examples 1 & 2), they can be more generic (example 4 & 5), or they can be attributes of (specific or generic) individuals (example 3). my hairdresser worked a miracle. the teachers of our school are preparing for the school festival. the new neighbour is a carpenter. as a nurse today, you have to be very resilient. i am speaking here on behalf of all bus drivers. further characteristics of occupational titles, which are mainly related to the german language, concern the formation of compounds and gender forms. since compounding is very productive in german, occupational titles can either be part of other lexemes that do not refer to occupations (as in bauernverband (farmers' association)) or the occupational title can be a specification of another occupational title (as versicherungsmaklerin (insurance broker) being a specification of maklerin (broker)). productivity of occupational titles in compounds is a first indicator that dictionary-based approaches might be insufficient to identify all occupational title within a text. with regard to the gender forms of occupational titles, dictionary-based approaches can be adapted to the extent that the word list is enriched with masculine, feminine and neutral forms (see section 4.2). as the use of gendered forms and gender-inclusive language can itself be of research interest ( see carla sökefeld (2021 occupational titles can semantically cover different aspects of the profession. in some cases, the activity is emphasised, as in dachdecker/in (roofer), lehrer/in (teacher) or verkäufer/in (salesperson). other occupational titles originate from the subject that the person deals with during work (stahlarbeiter/in (steelworker), immobilienmakler/in (real estate agent)) or the place where the profession is usually practised (grundschullehrer/in (primary school teacher), bankkaufmann/frau (bank clerk)) @xcite . like many other expressions, occupational titles can also be affected by ambiguities. on the one hand, many (german) surnames originate from traditional professions, such as bäcker (baker), müller (miller) or fischer (fisherman). in example 6, the two meanings of the lexeme müller can be clearly separated from each other, as it belongs to the group of proper names mentioned above and not to the profession of miller. another type of ambiguity arises particularly with occupational titles that are derived from the verb that describes the activity, such as pfleger/in (caregiver) or verkäufer/in (salesperson) (see example 7). here, the resolution of the ambiguity between professional and non-professional activity on the basis of a sentence is not always clear. in example 7, however, the lexeme verkäufer refers to the legal meaning of a seller, so that all persons who sell something are meant here, regardless of whether they do so professionally or non-professionally foot_0 . finally, another type of ambiguity has emerged from a rather metonymic use of occupational titles. in example 8, the occupational title is used to refer to the business type rather than the person because beim bäcker could also be replaced by bei der bäckerei without changing the meaning of the sentence. this concludes a contract between buyer and seller. the bakery around the corner has the best bread rolls. in addition to these ambiguities, where the textual context -i.e. the sentence -determines whether a particular lexeme is an occupational title, there are other personal nouns where the decision whether it is an occupational title depends heavily on the definition of occupation. as with many concepts in the humanities and social sciences, this definition varies greatly depending on the discipline and the research question being asked. sailmann (2018) provides a detailed overview of the conceptual history of the profession. @xcite , for example, divides the term into an objective meaning, which focuses on the social function, and a subjective meaning, which emanates from the individual. he defines the social function of occupations very broadly, so that, for example, husband also becomes a profession. in comparison, two other criteria are decisive for weber: the gainful character of work and a minimum level of qualification required @xcite . according to this definition, voluntary activities, such as working as a lay judge, would be excluded from the definition of an occupation. as weber's definition often forms the basis for occupational statistics today, the operationalisation here will also be based on it. ## operationalising for automatic language processing after introducing the concept of occupations, this next section will now compare two ways to identify occupational titles automatically, namely a dictionary-based approach and a machine-learning approach. as we have stated above, occupational titles can be affected by ambiguities and lexical productivity, which might make it difficult to apply a simple keyword search (see @xcite ). nevertheless, dictionary-based analysis are often applied in dh and css, because they are feasible if a dictionary already exists (see calanca et al. (2019); stops et al. (2020); djumalieva et al. (2018) ). so, we aim to evaluate such an approach in order to investigate error sources of word lists in a more detailed way. for both approaches, the identification of occupational titles takes place at token level, i.e. for each word in a sentence, the algorithm decides whether it is an occupational title. it is also possible for an occupational title to be represented by a multiword expression, such as medical assistant. ## application on further parliamentary protocols as the test corpus is comparatively small at 163 sentences, we applied the best dictionary-based and the best bert-based model foot_3 to further 2,000 speeches from 2010 in order to compare the extractions where both models disagree. since we do not have manually annotated sentences here, we can not report evaluation scores in this section. the most common types that were annotated by each model are reported in figures 1 & 2. in total, the phrasematcher identified 3,285 tokens (397 types) and the bert model 2,509 tokens (866 types) as occupational titles. of these, both models agree with their decision for 1,185 tokens (272 types). the diverging type-token-ratio indicates that the phrasematcher mostly annotated more frequent types, such as präsident/präsidentin (president), which are usually part of the salutation at the beginning of the speech. meanwhile, bert annotates more different types, which indicates that the context is taken into account and the lexical variance of the occupational titles themselves is not an obstacle for the model. if we look at the cases where only the dictionarybased model has marked an occupational title, it is noticeable -as in the previous section describedthat these are often words that are not occupational titles in the respective context, such as surnames, or where an incorrect lemmatisation has led to the annotation. for example, koch (cook) is marked seven times by the phrasematcher, although the occupation is only meant once and the surname in all other cases. in contrast, the bert model was able to correctly distinguish between the two meanings. there are also some valid occupational titles, such as milchbauer (dairy farmer), banker (banker) or pfleger (carer), that were only found by the bert model because they were missing from the keyword list. in contrast, the bert-based model often annotates other personal nouns that do not refer to professions, such as christen (christians), spekulant (speculator), rentner (pensioner) or schöffen (lay judge) (see figure 1 ). in addition, in some cases, company types, such as banken (banks), are annotated as occupations. to prevent these false extractions, the training data for a future model should include more personal nouns or company types as negative examples, so that the distinction between occupational groups and other personal nouns or types of organizations becomes clearer. finally, the bert model annotates many generic terms such as spezialist (specialists) or akademiker (academics) as occupational titles. these terms are (for the most part) not included in the keyword list because they do not reflect the specialised nature of an occupation like the dictionary of the federal employment agency does. nevertheless, these terms could be a starting point for subsequent analysis, since they group together various professions. in this case, the decision whether these terms are occupational terms again depends heavily on the occupational definition and the subsequent content analysis. ## conclusion the evaluation using the test data (section 4.4) as well as the application of the best models on further debates (section 5) have shown that both approaches have various strengths and weaknesses. the weakness of dictionary-based approaches lies in the context-free consideration of keywords and the treatment of unknown occupational titles. with regard to the false negatives, the word list could first be enriched by searching for similar words in a type embedding, such as word2vec. regarding false positives, some errors could be minimised by setting up additional rules. for example, surnames could be excluded by not annotating a potential token if the word kollege (colleague) precedes it. however, it is clear that these rules reach their limits as soon as a distinction has to be made between profession and businesses, like for friseur (hairdresser). the bert model presented here also has problems with these distinctions, regardless of which aggregation strategy was chosen. in addi-tion, the bert model still seems to overgeneralise the concept of occupational title, as many of the annotated terms refer to other groups of people. as stated in the previous section, this overgeneralisation could be prevented by adding more examples of personal nouns to the training data in order to improve the distinction between occupational groups and other groups. in addition, the bert base model could be compared to more specified models, such as jobbert-de 7 , which is domainadapted on german-language job advertisements from switzerland @xcite . this model may be able to depict occupational concepts more clearly than the bert base model. finally, other hyperparameters should be varied in training to determine the best possible configuration for finetuning. one point that usually follows the identification of expressions in texts is the grouping of the expressions on the basis of existing classifications. this is because it is often not the term as such that is to be analysed, but the referenced concept or the terms in relation to the referenced concept. the classification of occupations (kldb) is often used for statistical analyses on occupational activities. however, @xcite have already shown that occupational titles are only suitable for coding according to the kldb to a limited extent, because occupational activities, as defined by the kldb, and occupational titles do not fully correspond to each other. for example, when a handwerker (craftsman) is mentioned in the plenary protocols, it is often not recognisable which occupational activity is meant since labour market research often distinguishes between for example carpenters and mechanics. alternatively, the extracted data could be used for analyses that are based on structuring features other than occupational specialisation. initial ideas for groupings would be, for example, gender forms (lehrer vs. lehrerin) or industry affiliation (employee in the automotive industry). finally, lexical change of occupational titles could be examined at the level of occupational titles, for example by comparing whether a renaming of certain occupations or corresponding apprenticeships also led to a change in language usage or whether a preceding change in language usage has finally led to a change in official training regulations. in addition, occupational titles could be grouped according to profession and gender in or-7 https://huggingface.co/agne/jobbert-de der to investigate whether changes in the gender distribution in the profession have also led to a change in language @xcite .