gemma-maia-e4b

LoRA adapter of maia, a model specialised in Andorra built from Gemma 4. Experimental: not a final model yet.

This card describes the latest version, 20261102-1437-e4b-k-canon. Every training run is a tagged version of this repo:

• 20261102-1437-e4b-k-canon
• 20261028-2208-e4b-k
• 20261027-2015-e4b-k

Load a specific version with revision="<tag>".

• Base model: unsloth/gemma-4-E4B-it-unsloth-bnb-4bit
• Data: variant K-canon of the training-data export (conversations in Andorran Catalan)
• Code: github.com/ericrisco/maia (training/train.py)

Evaluation

Both models were run as Q4_K_M GGUF through Ollama, at temperature 0, under the same conditions.

Fact recall on 338 held-out questions about facts of the augmented canon (asked in wording never seen in training; a correct answer must contain the expected keywords):

	Gemma 4 E4B (base)	maia (20261102-1437-e4b-k-canon)
Facts recalled	25 / 338 (7 %)	121 / 338 (36 %)

maia recalls 103 facts the base model misses and misses 7 the base model knew.

maia-bench (926 questions about Andorra, judge Gemini 3.8 Flash; the benchmark questions and the articles it reserves are never trained on):

Question type	Gemma 4 E4B (base)	maia	Difference (95 % CI, paired bootstrap)
All	30.3 %	32.9 %	+2.6 pts [−0.6, +5.9]
Multiple choice (604)	42.4 %	46.2 %	+3.8 pts [−1.0, +8.6]
Open (231)	9.1 %	9.3 %	+0.2 pts [−3.2, +3.7]
Trap (91)	3.8 %	5.0 %	+1.1 pts [−3.8, +6.0]

maia improves on every question type and regresses on none, but the benchmark gain alone is not statistically significant (p ≈ 0.05): the benchmark deliberately asks about facts that are not in the training data, so most of the gain shows up in fact recall.

Training

QLoRA with Unsloth on 1 NVIDIA L4. The loss is computed on the model's responses only.

Parameter	Value
LoRA r / alpha	32 / 32
Epochs	2
Learning rate	0.0002 (cosine, 3 % warmup)
Effective batch	16 (4 x 4)
Max length	2048 tokens
Chat template	gemma-4
Seed	3407

Final validation loss: 1.6060

Step	Epoch	Validation loss
200	0.11	1.8957
400	0.22	1.7958
600	0.33	1.7506
800	0.45	1.7123
1000	0.56	1.6821
1200	0.67	1.6637
1400	0.78	1.6394
1600	0.89	1.6271
1800	1.00	1.6266
2000	1.12	1.6423
2200	1.23	1.6382
2400	1.34	1.6218
2600	1.45	1.6203
2800	1.56	1.6143
3000	1.67	1.6076
3200	1.78	1.6036
3400	1.90	1.6059
3586	2.00	1.6060

System message

The model was always trained with this system message (in Catalan). Use it at inference time too:

Ets maia, un assistent que coneix a fons Andorra: la seva història, les institucions, la geografia, la cultura i la gent. Respons en català, amb naturalitat i amb la parla d'Andorra.

Samples

Generated at the end of training (greedy decoding).

Què és el Consell de la Terra?

És el nom que el Consell General tenia abans de la Constitució del 1993.

Quines són les set parròquies d'Andorra?

Canillo, Encamp, Ordino, la Massana, Andorra la Vella, Sant Julià de Lòria i Escaldes-Engordany.

Explica'm què era la Querimònia.

Era un acte de gràcia que el Consell General concedia a un veí que havia fet una feina útil per a la comunitat. No era un salari: el Consell pagava una quantitat simbòlica, i el beneficiari podia fer servir aquella quantitat per pagar el seu propi deute. Per això, la recompensa ajudava a alliberar-lo d’una obligació, no pas a guanyar diners per a ell.

Què es menja per Sant Jordi a Andorra?

Per Sant Jordi es menja la coca massegada, una coca dolça amb llet, farina, sucre, ous i mantega.

Com va anar la reforma constitucional del 1993?

El Consell General va aprovar la Constitució el 2 de febrer de 1993, i el poble la va ratificar en referèndum el 14 de març.

Usage

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained("<this repo>", revision="20261102-1437-e4b-k-canon", load_in_4bit=True)

Quantized GGUF (Q4_K_M) for llama.cpp or Ollama: maia-e4b-Q4_K_M.gguf. With Ollama, use a Modelfile with FROM ./maia-e4b-Q4_K_M.gguf, RENDERER gemma4 and PARSER gemma4, and keep the system message above.