Var3n/jobads-salary

JobAds Salary Model

The JobAds Salary Model finds information about pay in job advertisements from historical German-language newspapers (1850-1950). It is a spaCy named entity recognition (NER) pipeline with four custom entity classes (salary, board, salary period and unspecific salary), built on the ELECTRA model zeitungs-lm-v1 (Schweter, 2024). On the test set it reached a strict entity-level F1 of 0.76 (between 0.69 and 0.79 for the individual classes), see Evaluation.

This model is part of the JobAds processing pipeline. The JobAds Identification Model finds the job advertisements in the newspaper text, and this model then extracts the information about pay from them. It was trained in the same way as the JobAds Position Model.

Intended use

The model marks every mention of pay in the text of one job advertisement. It is intended for the analysis of historical labour markets, for example to study which advertisements mention a salary, how often wages came with board and lodging, or for which period a wage was quoted.

• Language: German
• Period: trained for newspapers from 1850 to 1950
• Input: the text of one job advertisement
• Output: spans of the four entity classes

Label	Meaning	Example
salary	A quantitative salary	12 fl.
verpflegung (board)	Payment in kind, including board and lodging	gegen Koſt und Quartier
salary_period	The time period to which the salary refers	fünf Gulden C. M. monatlich (only monatlich is annotated)
unspecific_salary	A qualitative statement about pay without any numbers	gegen billiges Honorar

 

The label of the board class is the German word verpflegung (provisions, board); the other labels are English.

The salary period is only the word that names the period (monatlich, weekly), not the amount.

Important: the model assumes that the input already is a job advertisement. Use the JobAds Identification Model first to filter job advertisements.

Out-of-scope use

The model has not been validated for newspapers outside 1850-1950, for languages other than German, or for born-digital text.

How to use

import spacy
from huggingface_hub import snapshot_download

model_path = snapshot_download("Var3n/jobads-salary")
nlp = spacy.load(model_path)

text = "Köchin gesucht, fünf Gulden monatlich gegen Kost und Quartier."

doc = nlp(text)
print([(ent.text, ent.label_) for ent in doc.ents])

 

To get the character offsets of the found entities, use [(ent.start_char, ent.end_char, ent.label_) for ent in doc.ents].

Install the requirements first: pip install "spacy>=3.7.5,<3.8" spacy-transformers huggingface_hub (see Training procedure for the versions used in training). A GPU is optional but speeds up processing considerably: call spacy.require_gpu() before loading the model.

Training data

The training data are manually annotated job advertisements from newspaper titles of the ANNO corpus of the Austrian National Library (1850-1950). The annotators marked each mention of pay with one of the four labels above.

Training procedure

The model was trained in the same way as the JobAds Position Model, with spaCy's standard NER training procedure and the default settings of its GPU/transformer configuration. The pipeline has two components: a transformer (stefan-it/zeitungs-lm-v1, via spacy-transformers) and a spaCy ner component on top of it (TransitionBasedParser, hidden width 64).

• Transformer input: strided spans of 128 tokens with a stride of 96
• Optimizer: Adam, peak learning rate 5e-5, linear warm-up for 250 steps, linear decay over 20,000 steps, weight decay (L2) 0.01, gradient clipping 1.0
• Dropout: 0.1
• Batching: batches of about 2,000 padded tokens (batch_by_padded), gradients accumulated over 3 steps
• Steps: at most 20,000, evaluated every 200 steps, early stopping with a patience of 1,600 steps
• Precision: full (no mixed precision)
• Seed: 0 (the data split was not seeded)
• Hardware: NVIDIA RTX A4000
• Libraries: spaCy 3.7.x (the model requires spacy>=3.7.5,<3.8.0).
• Model used: model-best, the checkpoint with the highest entity F1 on the test set during training.

Evaluation

The scores are the strict entity-level scores of spaCy's scorer on the held-out test set. A prediction counts as correct only if its span and its label both match the annotation exactly.

Label	Precision	Recall	F1
salary	0.800	0.727	0.762
salary_period	0.714	0.833	0.769
unspecific_salary	0.825	0.750	0.786
verpflegung	0.688	0.688	0.688
All entities (micro average)	0.781	0.740	0.760

 

Results broken down by newspaper or time period are not available.

Limitations and bias

• The scores are clearly lower than those of the Position Model (strict F1 0.946 vs. 0.760). The class verpflegung (F1 0.688) is the least reliable one, so extracted values should be checked before they are used for quantitative conclusions.
• The split is not by page or newspaper, so performance on unseen newspapers, pages or periods can be lower than reported.
• The model reflects the newspapers, regions and years in the training data. Newspapers cover only a part of the historical labour market, and wages quoted in advertisements are not necessarily the wages actually paid.
• Historical job ads reflect the social norms of their time, e.g. gendered job titles and unequal pay offers. The model does not correct for this.

Citation

There is no citation for this model itself yet. If you use it, please cite the link to this repository.

Base model

@misc{schweter2024zeitungslm,
  author = {Schweter, Stefan},
  title  = {zeitungs-lm-v1},
  year   = {2024},
  url    = {https://huggingface.co/stefan-it/zeitungs-lm-v1}
}

 

Acknowledgements

The ground truth was created in the project JobAds (FWF P35783), which was supported by the Austrian Science Fund (FWF). We thank the Austrian National Library for providing the data from the ANNO corpus. We thank our project colleague Wiltrud Mölzer and the project PI Jörn Kleinert for their support with ground-truth data annotation.

License

This model is released under the CC BY 4.0 license.

Authors and contact

University of Graz, Austria (JobAds project)

• Klara Venglarova (klara.venglarova@uni-graz.at) (annotations and model training)
• Raven Adam (raven.adam@uni-graz.at) (integrating model into a processing pipeline)
• Georg Vogeler (georg.vogeler@uni-graz.at) (supervision)