Rahvusarhiiv/fi_summariser

Finnish summariser

Rahvusarhiiv/fi_summariser is an experimental Finnish summarization model fine-tuned from facebook/mbart-large-50-many-to-many-mmt.

This is a research release. It is not claimed to be state of the art; the model should be evaluated for the target domain before production use.

Intended Use

• Summarization of Finnish news, legal/administrative, encyclopedic, or academic-style text.
• Batch/offline summarization, evaluation, and retrieval-assisted workflows where outputs can be checked.
• Not intended as a sole source of factual truth, legal advice, medical advice, or archival description without human review.

How to Use

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo_id = "Rahvusarhiiv/fi_summariser"
text = "Tampereen yliopistollinen sairaala on ottanut käyttöön uuden toimintatavan, jossa osa lapsipotilaista voi saada antibioottihoitoa kotona pumpun avulla. Hoitajan käynti kotona riittää vaihtamaan liuoksen kerran päivässä."

tokenizer = AutoTokenizer.from_pretrained(repo_id, src_lang="fi_FI")
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)

inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=768)
language_token_id = tokenizer.convert_tokens_to_ids("fi_FI")
summary_ids = model.generate(
    **inputs,
    forced_bos_token_id=language_token_id,
    max_new_tokens=512, num_beams=4, no_repeat_ngram_size=3,
)
print(tokenizer.batch_decode(summary_ids, skip_special_tokens=True)[0])

Recommended Generation Settings

• max_new_tokens: 512
• num_beams: 4
• no_repeat_ngram_size: 3

Training Data

Fine-tuned from mBART-50 on a rebalanced Finnish mix: Yle news/RSS/archive rows, Theseus theses, filtered synthetic FinePDFs summaries, and downweighted EurLex/legal data. The Yle archive benchmark rows were explicitly removed from training before this checkpoint was trained.

• Yle RSS and archive articles collected locally; held-out Yle archive benchmark rows excluded from training.
• Theseus thesis metadata/abstract examples.
• Filtered subset of MultiSynt/finepdfs-summaries.
• Finnish EurLex/legal examples after summary-section extraction and boilerplate filtering.

Known public Hub datasets used or partially used where applicable:

• https://huggingface.co/datasets/MultiSynt/finepdfs-summaries

Training metadata:

• Base/initial model recorded by trainer: facebook/mbart-large-50-many-to-many-mmt
• Train rows: 2741
• Validation rows: 246
• Max source tokens: 768
• Max target tokens: 512
• Seed: 20260611

Evaluation

benchmark	n	ROUGE-L	ROUGE-1	ROUGE-2	ref coverage	src precision	judge factuality	judge coverage	judge overall	words	repeat 4g	legalese	max hit
News-supported v2 automatic	253	0.221	0.247	0.140	0.251	0.856				26.1	0.0000	0.0000	0.0000
Wiki4 final automatic	100	0.366	0.407	0.288	0.394	0.882				53.9	0.0000	0.0000	0.0000
GDELT final automatic	100	0.116	0.135	0.044	0.123	0.831				23.4	0.0000	0.0000	0.0000

The automatic metrics include ROUGE, coverage proxies, repetition and artifact checks. The judge metrics use a local Ollama LLM judge on sampled examples and should be treated as comparative, not absolute.

Limitations and Failure Modes

• The model can hallucinate or over-generalize details, especially when the input is long or under-specified.
• Named entities, numbers, dates, and legal references should be verified against the source.
• Some evaluation datasets reward lead-like summaries; performance may differ on non-news or conversational text.
• The training data mixes sources with different writing styles and upstream terms; users are responsible for checking suitability for their use case.
• This is the no-leak Finnish archive-balanced checkpoint; the earlier archive-v1 benchmark score should not be used because it had held-out benchmark leakage.

Bias, Risks, and Ethical Considerations

The model reflects the distributions and editorial choices of its source corpora. It may reproduce biases present in news, legal, academic, or synthetic summaries. Human review is recommended for public-facing summaries and high-impact decisions.

CO2 Emissions

Training emissions were not instrumented with CodeCarbon or an equivalent tracker. No numeric CO2 estimate is reported. Training was local fine-tuning of an mBART-50 checkpoint rather than pretraining from scratch.

Model Tree

• Base model: facebook/mbart-large-50-many-to-many-mmt
• Relationship: fine-tuned sequence-to-sequence summarization checkpoint.

License

The model card uses license: other because the training mix contains multiple upstream data sources with different terms. Treat this as a research-use release and verify upstream data/license constraints before redistribution or commercial use.