README.md · daelba/vital2wikibase at 55bd5a0711b5b4e1c511faa19cf3aeed8f32eafd

metadata

license: cc-by-nc-sa-4.0
language:
  - cs
  - de
pipeline_tag: text2text-generation
tags:
  - code
  - wikibase
  - vital records
base_model: google/mt5-small
inference:
  parameters:
    max_new_tokens: 100

A model for connverting vital records to Wikibase. Trained on a transcription dataset obrained with HTR (eScriptorium) from 150 pages of the Jewish registry book HBMa 2501 (Wikidata). The dataset contains 1071 vital records, which were uploaded to Wikibase Cloud after processing using this model.

Based on Google's mT5.

Processing ditto marks

mt5prefix = "dto"
upperCell = "18 Juny 1858 H. Barb. Kohn"
lowerCell = "dto dto H. Ros. Plowitz"
dataset_input = f"{mt5prefix}:{upperCell}|{lowerCell}"
dataset_target = "18 Juny 1858 H. Ros. Plowitz"

Processing a table cell

mt5prefix = "wbPre"
header = "Namen des Vaters mit Anführung des Standes, ob Familiant od. Gewerbsmann, u. seines Wohnortes, Schutzdom. Haus-Nr. und bei ehelichen Geburten seines Gubernial-Ehekonsenses Datum und Zahl"
cell = "Markus Gibianer, Lederhändler, S. d. Seligmann Gibianer u. der Magdalena geb. Motz"
dataset_input = f"{mt5prefix}:{header}|{cell}"
dataset_target = "[[P23:Markus Gibianer|P26:Lederhändler]] [[P27:Seligmann Gibianer]] [[P28:Magdalena Gibianer geb. Motz]]"

Processing a whole record

mt5prefix = "wbDef"
input_string = "[[P5:60]] [[P6:1865-05-05]] [[P7:H. Charlotte Löw]] [[P12|DD:23]] [[P12|MM:04]] [[P12|YY:1865]] [[P14|P8:1061/2]] [[P14:Q8]] [[P15|DD:06]] [[P15|MM:05]] [[P15|YY:1865]] [[P16:Q22]] [[P17:Friederika]] [[P20:4]] [[P21:Q13]] [[P22:Q15]] [[P23:Jakob Raubitschek|P26:Eisenhändler]] [[P27:Josef Raubitschek]] [[P28:Theresia Raubitschek geb. Latus]] [[P24:Pauline XXX]] [[P29:Markus Raudnitz]] [[P30:Rachel Raudnitz geb. Kuh]]"
dataset_input = f"{mt5prefix}:{input_string}"
dataset_target = "[[P10:Q5]] [[P12:1865-04-23]] [[P14:Q8|P8:1061/2]] [[P15:1865-05-06]] [[P16:Q22]] [[P17:Friederika]] [[P18:Raubitschek]] [[P19:Friederika Raubitschek]] [[P1:Q1]] [[P20:4]] [[P21:Q13]] [[P22:Q15]] [[P23:Jakob Raubitschek|P26:Eisenhändler]] [[P24:Pauline Raubitschek]] [[P27:Josef Raubitschek]] [[P28:Theresia Raubitschek geb. Latus]] [[P29:Markus Raudnitz]] [[P30:Rachel Raudnitz geb. Kuh]] [[P33:Charlotte Löw]] [[P38:Q170]] [[P5:60]] [[P6:1865-05-05]] [[P7:H. Charlotte Löw]]"