emese-tech/folyo-mlx

Emese-Folyó (22B) — MLX q8

MLX q8 — the native training/serving precision for Folyó (CPT, SFT, and DPO were all trained on top of a q8-quantized base). See the folyo/ repo's README for full architecture, CPT/SFT/DPO training details, and benchmarks — this file covers only the q8-specific notes.

Quantization	q8, group size 64
Size on disk	~22 GB (vs. ~42 GB bf16)
Quality	This is the model's native precision — the folyo/ bf16 repo is dequantized from this, not the other way around.
Max context length	32,768 tokens (EuroLLM-22B's native context)

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/folyo-mlx")
p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))

Decode: temperature 0.2, no repetition penalty, eos {2, 4}, ChatML template.

⚠️ This repo is mlx_lm-only — MLX's q8 quantization packs weights into uint32 + per-group scales/biases tensors with a quantization block in config.json that plain transformers does not understand. Use the folyo/ (bf16) repo for transformers/vLLM/TGI.

Training

This is the primary artifact of the CPT+SFT+DPO training chain — see folyo/README.md for the full q8-native CPT (~6M tokens/6,000 iters, rank64), SFT (instruct_v18b, 1 epoch, rank16/scale16/lr5e-6, iteration 3,600 selected), and DPO (36 alfa pairs, 120 iters, rank16/scale16/lr2e-6) recipe, plus the 211/250 Ultimate · 310/376 BlindSpot benchmark results.

Benchmarks

This exact q8 artifact scored 410/500 (82%) on emese-bench v1 (200 questions + 10 chat sessions, consolidated Ultimate+BlindSpot) — a close second to Patak (413/500) and far ahead of Csermely (211/500). Strongest on safety, reading, translation, and code; weakest on multi-step math and logic puzzles. Notably the best of the three models on in-context memory recall and roleplay (both 10/10), but it failed to stay in English during the English-chat session, answering in Hungarian throughout. See emese-bench/results/folyo-mlx.md for the full transcript and category breakdown.