mys/laya-multilingual-GGUF

Laya Multilingual GGUF (ggmlc)

Multilingual System 1 decision model compiled from convaiinnovations/laya-multilingual (mmBERT-base, 322M, context 1024, 100+ languages).

These files are not llama.cpp / llama-cli GGUFs. They are produced by ggmlc, a neural network compiler that lowers PyTorch, JAX, Flax, and Keras models to high-performance GGML execution. Loading them in llama.cpp will fail.

Typed questions (choice / score / noul) are scored in one encoder pass. There is no autoregressive token generation.

Source, CLI, and binaries: examples/laya

Other families: laya-GGUF (English) · laya-typed-decisions-GGUF

Files

File	Quant	Size	Notes
laya_multilingual_f16.gguf	F16	~633 MB	Full precision.
laya_multilingual_q8_0.gguf	Q8_0	~345 MB	Usually the best size / quality trade.
laya_multilingual_ud_q4_k_m.gguf	UD_Q4_K_M	~500 MB	Larger than Q8_0 here because embeddings stay F16.

Tokenizer is Gemma BPE + Metaspace (▁), specials <bos>/<eos>/<pad>/<mask> (ids 2/1/0/4), not ModernBERT [CLS]/[SEP]/[MASK].

huggingface-cli download mys/laya-multilingual-GGUF laya_multilingual_f16.gguf --local-dir .

Run with laya

Download a binary from ggmlc releases (laya.exe / laya). --device defaults to auto (CUDA or Metal if present, else CPU).

laya help
laya list-presets
laya info laya_multilingual_f16.gguf

laya decide laya_multilingual_f16.gguf --preset email --text "二重に請求されました" --device auto --cuda-graph
laya decide laya_multilingual_f16.gguf --preset triage --text "Bitte erstatten Sie die doppelte Abbuchung." --json
laya serve laya_multilingual_f16.gguf --port 8080 --device auto --cuda-graph
laya bench laya_multilingual_f16.gguf --preset email --device auto --cuda-graph

Put English + multilingual GGUFs in one directory to route before the forward (non-Latin script → multilingual; otherwise English function-word count):

laya detect-lang --text "二重に請求されました"
laya decide --models-dir . --preset email --text "二重に請求されました"

serve starts Decision Studio (GET /) and POST /api/decide. daemon is newline JSON-RPC on stdin/stdout.

What this is

Laya is the open reproduction of TypeSafe Jev: given a state and typed questions, it returns calibrated probabilities instead of generating tokens. The English checkpoint stays confident while collapsing off English; use this family for non-English (or mixed) text.

License

Apache 2.0, same as the upstream Laya weights. Compiler: ggmlc (MIT).