Lafa-0.5B

Lafa (Large flow language model) is a continuous flow language model that generates text block by block and can verify its own flow drafts autoregressively in self-speculative decoding.

EMA weights in BF16. Block size: B = 4.

Paper: arXiv:2610.28559 · Code: GitHub

Results

Paper results; model size is shown in each plot.

Lafa 7B reaches 2.54× AR throughput at matched or better quality.

Flow decoding accuracy, averaged over GSM8K, HumanEval, and IFEval.

Self-speculative decoding accuracy vs. tokens per forward pass (TPF), averaged over six benchmarks.

Load

pip install -r requirements-inference.txt

import torch
from transformers import AutoModel, AutoTokenizer

MODEL_ID = "lafa-hf/lafa-0.5B"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModel.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Explain flow matching briefly."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)
inputs = {k: v.to(model.device) for k, v in inputs.items()}

Flow decoding

output_ids = model.generate(
    **inputs,
    gen_mode="flow",
    max_new_tokens=128,
    num_steps=4,
    grid_mode="warp",
    noise_scale_override=0.1,
    cfg_scale=1.0,
    sde_gamma=0.0,       # 0 = ODE
    sde_alpha=0.0,
    sde_vp=False,
    sde_noiseless=False,
    euler_alpha=1.0,
)

Parameter	Notation	Default
num_steps	S	4
noise_scale_override	noise multiplier	0.1
cfg_scale	w_CFG	1.0
sde_gamma	gamma	0.0
sde_alpha	alpha	0.0
grid_mode	warp, uniform, uniform_range	warp
euler_alpha	alpha_Euler	1.0

AR decoding

output_ids = model.generate(
    **inputs,
    gen_mode="ar",
    max_new_tokens=128,
)

Self-speculative decoding

output_ids = model.generate(
    **inputs,
    gen_mode="speculative",
    max_new_tokens=128,
    grid_mode="warp",
)

All modes return token IDs. Print the generated text with:

text = tokenizer.decode(
    output_ids[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(text)

Generation is greedy. Use batch size 1 for flow and self-speculative decoding.

Citation

@article{chavan2026largeflow,
  title   = {Large Flow Language Models},
  author  = {Chavan, Arnav and Lee, Chanhyuk and Bamba, Udbhav and Yoo, Jaehoon
             and Hong, Seunghoon and Raghunathan, Aditi and Kim, Jinwoo},
  journal = {arXiv preprint arXiv:2610.28559},
  year    = {2026}
}