ebible_m2m-ie-big-shareable

A multilingual, from scratch encoder decoder transformer that translates verses of the Bible from a fixed Koine Greek source into many languages. It reproduces and extends the closed text translation experiment first carried out by Sami Liedes in 2018, replacing his convolutional sequence to sequence model with a modern transformer and adding quantitative evaluation. This checkpoint comes from the ie_big_shareable experiment.

The model is trained on verse aligned Bible translations from the eBible corpus. Whole books are withheld from certain languages during training, and the model then generates those withheld books. The practical aim is to draft scripture in a language for which parts of the Bible do not yet exist, using the many translations the model has already seen as context.

How to use

from transformers import MarianMTModel, MarianTokenizer

model = MarianMTModel.from_pretrained("DavidCBaines/ebible_m2m-ie-big-shareable")
tokenizer = MarianTokenizer.from_pretrained("DavidCBaines/ebible_m2m-ie-big-shareable")

# Prepend the target language tag (for example <2spa> for Spanish) to the
# Koine Greek source verse.
source = "<2spa> Ἐν ἀρχῇ ἐποίησεν ὁ Θεὸς τὸν οὐρανὸν καὶ τὴν γῆν"
batch = tokenizer([source], return_tensors="pt")
generated = model.generate(**batch, num_beams=5, max_length=192)
print(tokenizer.decode(generated[0], skip_special_tokens=True))

Model details

• Architecture: MarianMT style encoder decoder, randomly initialised (no pretrained weights).
• Parameters: approximately 210 million.
• Source language: composite Koine Greek (Brenton Septuagint for the Old Testament, Tischendorf for the New Testament), in Greek script.
• Target languages: indicated by an atomic <2xxx> tag prepended to the source verse.
• Tokeniser: SentencePiece, trained on the training split only.

Held out books

The following books were withheld from training and generated by the model.

Translation	Held out
engbsb	OT
deutkw	OT
hin2017	OT

Evaluation

Scores are corpus level per held out book, following the silnlp and sacreBLEU conventions. chrF3 is the headline metric. Two baselines are reported for context. The source copy baseline (copy_chrF3) returns the Greek source verse unchanged, and sits near zero because the source is in a different script. The other language baseline (other_chrF3, with the language in other_lang) is the score of the single most similar training language's own text for the same verses, which is a far more demanding floor because a close relative shares script and vocabulary with the target.

translation	language	books	verses	chrF3	spBLEU	BLEU	copy_chrF3	other_chrF3	other_lang
deutkw	deu	36	20823	37.03	12.58	11	0.32	21.22	nld
engbsb	eng	36	20833	47.01	26.42	25.23	0.34	19.57	fra
hin2017	hin	36	20833	43.82	26.25	20.71	0.25	32.79	urd

Scores are verse-weighted across each language's held-out books. Per-book metrics for every book are in generated/metrics.csv.

Training data and licensing

This model was trained only on translations whose licences permit derivative works, so that the model may itself be shared. The published model is released under cc-by-sa-4.0. Every source translation and its licence is listed below.

translation	language	licence
deutkw	deu	Public Domain
engbsb	eng	Public Domain
hin2017	hin	by-sa
asmfb	asm	by-sa
benobcv	ben	by-sa
ces1613	ces	Public Domain
ckb	ckb	by-sa
dan1931	dan	Public Domain
frasbl	fra	Public Domain
guj2017	guj	by-sa
hne	hne	by-sa
hrv	hrv	Public Domain
isl	isl	by-sa
ita1927	ita	Public Domain
latVUC	lat	Public Domain
lit	lit	by-sa
mar	mar	by-sa
nld1939	nld	Public Domain
noblb	nob	by-sa
npiulb	npi	by-sa
ory	ory	by-sa
pan	pan	by-sa
pesOPV	pes	Public Domain
polsz	pol	by-sa
poronbv	por	by-sa
ronbtf	ron	Public Domain
russyn	rus	Public Domain
slk	slk	by-sa
spabll	spa	Public Domain
srp1868	srp	Public Domain
ukrfb	ukr	Public Domain
urd	urd	by-sa

Reproducibility

• Experiment: ie_big_shareable
• Git commit: ef0c7ccec0d79f213e93ceb4426e22398676995b
• Random seed: 13

The full training and evaluation code, configuration and selection list are in the project repository.

Acknowledgement

This work reproduces the closed text Bible translation experiment described by Sami Liedes in his 2018 blog post "Machine translating the Bible into new languages". The reproduction and this model are independent work and are not endorsed by him.