Mitos-Qwen3-8B-Process-LoRA

Qwen3-8B adapted for Greek e-government question answering. This is Mitos-Qwen-Process: fine-tuned on question-answer records derived from the structured text of every procedure of the National Registry of Administrative Procedures (ΕΜΔΔ / Mitos) with templates and no LLM (90,145 records over all 4,407 procedures). Its answers are longer and follow the structure of the registry entries.

This repository holds LoRA adapters, in two places:

Location	Trained on	Use
repository root	all procedures (4,407; dataset config `json-all`)	the final adapter; merged into the merged model
`holdout/`	4,187 procedures, without the 220 held-out procedures (dataset config `json-train`)	the model behind the benchmark numbers of the paper; for evaluation and reproduction only

Do not evaluate the root adapter on the benchmark's unseen split: it has seen those procedures. Use holdout/.

Training

Setting	Value
Base model	Qwen/Qwen3-8B
Method	LoRA, rank 16, alpha 32, all attention and MLP projections
Optimization	learning rate 2e-4, cosine schedule, 2 epochs, effective batch 32, bfloat16
Maximum sequence length	4096 tokens
Loss	assistant tokens only, Qwen3 chat template with thinking disabled, standard cross-entropy
Data	all procedures of the registry snapshot (d-michail/mitos-qa, config `json-all`)

Evaluation of the `holdout/` adapter (judge-free metrics, 1,933 questions)

System	Key-fact recall (base)	Key-fact recall (this model)	Token F1 (this model)	Words (this model)
Closed-book	0.043	0.119	0.157	135
RAG (field-level chunks)	0.442	0.554	0.348	83
RAG-split (windows)	0.260	0.344	0.270	116
RAG-process (whole procedures)	0.236	0.296	0.223	139

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "d-michail/Mitos-Qwen3-8B-Process-LoRA")                       # the final adapter
# model = PeftModel.from_pretrained(base, "d-michail/Mitos-Qwen3-8B-Process-LoRA", subfolder="holdout")  # the benchmark (held-out) adapter
tok = AutoTokenizer.from_pretrained("d-michail/Mitos-Qwen3-8B-Process-LoRA")

Prompting is the same as for the merged model (no system prompt, thinking disabled; retrieval prompt):

Απάντησε στην ερώτηση του πολίτη χρησιμοποιώντας μόνο τις παρακάτω πληροφορίες από το Εθνικό Μητρώο Διοικητικών Διαδικασιών (ΜΙΤΟΣ). Αν οι πληροφορίες δεν επαρκούν για να απαντήσεις, πες το ξεκάθαρα.

Πληροφορίες:
{contexts}

Ερώτηση: {question}

Intended use and limitations

Intended use: research on Greek public-administration question answering, closed-book and with retrieval-augmented generation. It is not legal advice: answers are not legally authoritative, the registry changes, and a deployed system must show its source and the retrieval date and link to the official procedure.
The model was trained without a system prompt and with thinking disabled (enable_thinking=False). Use the Qwen3 chat template and put the question, or the retrieval prompt below, in the user message.
Closed-book, the model learns the register and format of the domain but hardly its facts (key-fact recall stays low); retrieval adds most of the accuracy. It can state concrete data (amounts, deadlines, documents) that are wrong.
Small models sometimes degenerate into repetition loops that run to the token limit; use a repetition penalty or a stop rule in production.
The training data are synthetic or template-generated from one registry snapshot and may reproduce the generator's errors and phrasing.
The registry itself does not guarantee the completeness or correctness of its content (terms of use of mitos.gov.gr, A.6), so this model cannot either. Verify every answer against the live procedure at https://id.mitos.gov.gr/<Mitos ID> (the 6-digit ID of the procedure). The data are a snapshot (see below) and procedures change, so the model goes stale.

License and attribution

These weights are released under CC BY-SA 4.0 (license), the license of the registry content they were trained on (a cautious choice for the share-alike condition). The base model Qwen3 is released under Apache-2.0; its license text is included in this repository (LICENSE-Qwen3-Apache-2.0.txt, Copyright 2024 Alibaba Cloud) and base_model names it. The training dialogues were generated with Qwen3.8-27B (Apache-2.0).

Attribution: trained on data from the Εθνικό Μητρώο Διοικητικών Διαδικασιών – Μίτος (mitos.gov.gr; open data API: https://api.mitos.gov.gr/info), content licensed under CC BY-SA 4.0; according to the terms of use of mitos.gov.gr the data belong to, or are supervised by, the Hellenic Ministry of Interior (the site names no copyright holder explicitly). Changes: the registry texts were split into topics and processed into dialogues or question-answer records and used to fine-tune Qwen3. Data snapshot: downloaded on 1-2 October 2026 (records last updated between 26 June and 1 October 2026).