Var3n/jobads-position

JobAds Position Model

The JobAds Position Model finds the names of job positions (e.g. Köchin, Magazineur, Lehrerstelle) in job advertisements from historical German-language newspapers (1850-1950). It is a spaCy named entity recognition (NER) pipeline with a single custom entity class, position, built on the ELECTRA model zeitungs-lm-v1 (Schweter, 2024). On manually corrected text it reached an entity-level F1 of 0.956, on raw OCR text 0.855 (see Evaluation).

This model is part of the JobAds processing pipeline. The JobAds Identification Model finds the job advertisements in the newspaper text, and this model then extracts the position names from them.

Intended use

The model marks every mention of a job position in the text of one job advertisement. It is intended for the analysis of historical labour markets, for example to count which occupations were offered or sought in different periods.

• Language: German
• Period: trained for newspapers from 1850 to 1950
• Input: the text of one job advertisement
• Output: spans of the entity class position

Label	Meaning
position	The name of a job position or occupation, including inflected forms and compounds such as Lehrerstelle ("teacher position")

 

The annotation excluded ambiguous words such as Mann ("man") or Mädchen ("girl") which are not considered positions.

Important: the model assumes that the input already is a job advertisement. Use the JobAds Identification Model first to filter job advertisements.

Out-of-scope use

The model has not been validated for newspapers outside 1850-1950, for languages other than German, or for born-digital text.

How to use

import spacy
from huggingface_hub import snapshot_download

model_path = snapshot_download("Var3n/jobads-position")
nlp = spacy.load(model_path)

text = "Tüchtiger Gärtner wird aufgenommen."
text = text.replace("ſ", "s")  # the model was trained on text with long s (ſ) replaced by s

doc = nlp(text)
print([(ent.text, ent.label_) for ent in doc.ents])

 

To get the character offsets of the found positions, use [(ent.start_char, ent.end_char, ent.label_) for ent in doc.ents].

Install the requirements first: pip install "spacy>=3.7.5,<3.8" spacy-transformers huggingface_hub (see Training procedure for the versions used in training). A GPU is optional but speeds up processing considerably: call spacy.require_gpu() before loading the model.

Training data

The training data are manually annotated job advertisements from 14 newspaper titles of the ANNO corpus of the Austrian National Library (1850-1950). The annotators marked each position name as one entity in the annotation tool doccano (Nakayama, 2018). The data are described in Venglarova, Adam and Vogeler (2024) and available at JobAds-FWFProject/PositionsExtraction (train_data.json, test_data.json).

Preprocessing.

• The long s (ſ) was replaced by s.
• Advertisements without any annotated position were removed before the split.

Split. The remaining advertisements were divided with a random 70/30 split:

Subset	Advertisements
Training	1,486
Test	637

 

The split is by advertisement, not by page or newspaper. The published train_data.json and test_data.json contain the exact split used.

Training procedure

The model was trained with spaCy's standard NER training procedure and the default settings of its GPU/transformer configuration. No custom training procedure was implemented. The pipeline has two components: a transformer (stefan-it/zeitungs-lm-v1, via spacy-transformers) and a spaCy ner component on top of it (TransitionBasedParser, hidden width 64).

• Transformer input: strided spans of 128 tokens with a stride of 96, so long advertisements are processed in windows
• Optimizer: Adam, peak learning rate 5e-5, linear warm-up for 250 steps, linear decay over 20,000 steps, weight decay (L2) 0.01, gradient clipping 1.0
• Dropout: 0.1
• Batching: batches of about 2,000 padded tokens (batch_by_padded), gradients accumulated over 3 steps
• Steps: at most 20,000, evaluated every 200 steps, early stopping with a patience of 1,600 steps
• Precision: full (no mixed precision)
• Seed: 0 (the data split was not seeded)
• Hardware: NVIDIA RTX A4000
• Libraries: spaCy 3.7.x (the model requires spacy>=3.7.5,<3.8.0, as recorded in its meta.json). The versions of spacy-transformers and torch used in training were not recorded.
• Model used: model-best, the checkpoint with the highest entity F1 on the test set during training. It is the checkpoint at step 1,200 (epoch 30), after which no improvement followed within the patience of 1,600 steps, and training stopped at step 2,800.

Note on model selection. No separate validation set was used. The test set served as the development set during training, so the checkpoint was chosen on the test set. The reported scores can therefore be slightly optimistic.

The full configuration is available in config.cfg in this repository.

Evaluation

The held-out test set contains 637 advertisements. The F1 scores in the table below were computed on 558 of them (after dropping ads not containing any positions), the test set of Adam, Venglarova and Vogeler (2025). All 558 advertisements belong to the test set and none is in the training data. They were evaluated in three versions of the same text: manually corrected, automatically post-corrected, and raw OCR. The model was trained on the manually corrected training set only. The scores are micro-averaged F1 values as reported in the paper.

Text version	F1
Manually corrected text	0.956
Automatically post-corrected text	0.907
Raw OCR text	0.855

 

How predictions are compared with the annotations.

The scores do not require exact span matches. For each advertisement, the predicted and the annotated position strings are compared as follows:

• A trailing -stelle is removed (Lehrerstelle is compared as Lehrer).
• Both strings are lemmatized with the DTA::CAB web service for historical German (Jurish, 2012).
• Two strings count as the same position if their normalized Levenshtein similarity is at least 0.9.
• Each distinct string per advertisement is counted once.

The scores are therefore more lenient than strict span matching. For comparison, the strict span-based scores of the model on the full test set of 637 advertisements (spaCy's ents_p, ents_r, ents_f, recorded in meta.json) are precision 0.932, recall 0.960 and F1 0.946. The earlier paper on the same task (Venglarova et al., 2024) used a similar lenient comparison (lemmatization and a Levenshtein tolerance of 0.1) and reported an F1 of 0.944 for the transformer model on corrected text. The test set contains only advertisements with at least one position, and the results come from a single training run. Results broken down by newspaper or time period are not available.

Error analysis

The following error types appear in the test-set predictions (examples from the test set, and findings of Venglarova et al., 2024):

1. Missed positions in lists. In advertisements with several positions, the model sometimes finds the first ones and misses later ones, e.g. Markörin after Kellnerin and Alleinkellnerin, or Arbeiter after Wagenschmied.
2. Words split by OCR or hyphenation. Words split by a space, e.g. Forst adjunkt, Hand schuhschneider or Ge schäftsgänger, are often marked only in part or not at all.
3. Different span boundaries. The model and the annotation disagree on the exact form, e.g. annotated Magazineurs vs. predicted Magazineurstelle, or annotated Arztesposten vs. predicted Arzt.
4. Ambiguous or related words. Words such as Mann, Mädchen or Französinnen are marked as positions although the annotators did not mark them, and related words such as Agentur ("agency") can be mistaken for positions.

Limitations and bias

• The model was trained on manually corrected text (some OCR spacing errors such as split words remain). Its performance drops on raw OCR text (F1 0.855 instead of 0.956).
• It was trained and evaluated only on advertisements that contain at least one position. It has not learned to leave advertisements without a position unmarked, so it may produce false positives on such texts.
• The test set was used for selecting the model checkpoint. Performance on unseen newspapers, pages or periods can be lower than reported.
• Historical job ads reflect the social norms of their time, e.g. gendered job titles and discriminatory requirements. The model does not correct for this.

Citation

Method and data

@inproceedings{venglarova2024positions,
  title     = {Extracting position titles from unstructured historical job advertisements},
  author    = {Venglarova, Klara and Adam, Raven and Vogeler, Georg},
  booktitle = {Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities},
  pages     = {75--84},
  year      = {2024},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2024.nlp4dh-1.8/}
}

@article{adam2025exploring,
  title   = {Exploring Historical Labor Markets: Computational Approaches to Job Title Extraction},
  author  = {Adam, Raven and Venglarova, Klara and Vogeler, Georg},
  journal = {Journal of Data Mining \& Digital Humanities},
  year    = {2025},
  doi     = {10.46298/jdmdh.15038}
}

 

There is no citation for this model itself yet. If you use it, please cite the papers above and link to this repository.

Lemmatization tool: Jurish, B. (2012). Finite-State Canonicalization Techniques for Historical German. Potsdam: Universität Potsdam.urn:nbn:de:kobv:517-opus-55789.

Annotation tool: Nakayama, H. et al. (2018). Doccano: Text Annotation Tool for Human.https://github.com/doccano/doccano.

Base model

@misc{schweter2024zeitungslm,
  author = {Schweter, Stefan},
  title  = {zeitungs-lm-v1},
  year   = {2024},
  url    = {https://huggingface.co/stefan-it/zeitungs-lm-v1}
}

 

Acknowledgements

The ground truth was created in the project JobAds (FWF P35783), which was supported by the Austrian Science Fund (FWF). We thank the Austrian National Library for providing the data from the ANNO corpus. We thank our project colleague Wiltrud Mölzer and the project PI Jörn Kleinert for their support with ground-truth data annotation.

License

This model is released under the CC BY 4.0 license.

Authors and contact

University of Graz, Austria (JobAds project)

• Klara Venglarova (klara.venglarova@uni-graz.at) (annotations and model training)
• Raven Adam (raven.adam@uni-graz.at) (integrating model into a processing pipeline)
• Georg Vogeler (georg.vogeler@uni-graz.at) (supervision)