emese-tech/er

Emese-Ér (517M)

Ér ("brook/rill" — the smallest flowing water in the family name scheme) is the edge/embedded/research tier of the Emese Hungarian model family. Unlike the other three tiers, Ér is not built on EuroLLM — it's a from-scratch Hungarian foundation model, pretrained on ~4.5B tokens of Hungarian text with a custom 32k SentencePiece tokenizer.

Parameters	516.8M
Base	none — from-scratch pretraining
Architecture	LLaMA-style decoder (RoPE + YaRN, RMSNorm, SwiGLU, tied embeddings)
Hidden / layers / heads	1280 / 24 / 20
FFN size	3456
Vocabulary	32,000 (custom Hungarian SentencePiece Unigram)
Max context length	32,768 tokens (trained at 1,024, YaRN-extended to 8,192 during CPT, RoPE scale 4.0 rated to 32,768)
Precision	bfloat16
License	MIT

Formats in this release

Folder	Format	Notes
er/ (this repo)	bf16, MLX-native safetensors	not a standard transformers repo — see below
er-mlx/	MLX q8	mlx_lm-loadable, quantized

⚠️ Not transformers-compatible out of the box

Ér predates the EuroLLM/Llama-arch pipeline used for Csermely/Patak/Folyó. Its config.json uses a custom schema (d_model, n_layers, n_heads, d_ff — not HF's hidden_size/num_hidden_layers/...) and its weight tensor names are custom (embed.weight, layers.N.attn.wq.weight, ...), not the standard model.embed_tokens.weight / model.layers.N.self_attn.q_proj.weight naming. AutoModelForCausalLM. from_pretrained will fail on this repo. Use the project's own model.py (an MLX Emese class) or load er-mlx/ via mlx_lm, which does understand this checkpoint despite the non-standard config.

Usage (MLX, via mlx_lm)

from mlx_lm import load, generate
model, tok = load("er-mlx")  # or "er" — both load via mlx_lm
print(generate(model, tok, prompt="A magyar nyelv", max_tokens=100))

Usage (project's own loader)

python generate.py --model models-release/er/ --prompt "A magyar nyelv"

Training

• Pretraining (from scratch): ~4.5B tokens of Hungarian text (the same corpus family documented in corpus/cpt/README.md: Wikipedia + HPLT web text, quality-filtered). No separate "CPT" stage — this is the base pretraining, there is no upstream foundation model underneath it.

Benchmarks

Not applicable — emese-bench (and its Ultimate/BlindSpot predecessors) are chat-instruction benchmarks (ChatML turns, persona/safety/instruction-following categories); Ér has no chat template or instruction-tuning, so scoring it on those benchmarks would not be a meaningful comparison against the other three tiers. During development it was evaluated on HuCoLA (Hungarian grammatical-acceptability classification) instead, not on chat-style benchmarks.

• No SFT / no DPO. Ér ships as a base/research model only — it has no chat template, no persona training, no instruction-tuning. It is not conversational; treat it as a research/embedding artifact for downstream fine-tuning experiments, not an assistant.

Limitations

• Base model only — expects raw text continuation, not chat-formatted prompts.
• Smallest tier by a wide margin (517M vs. 1.7B+ for the rest of the family) — limited world knowledge and reasoning capacity even relative to Csermely.
• Requires custom loading code (model.py) for anything beyond mlx_lm.