tory-labs/vica-laya

Vica Laya

Vica Laya is a Korean-focused judgment fine-tune of Laya multilingual. This repository contains the exact Q8_0 model installed for local general judgment in Vica on 2026-11-05.

한국어 문장을 읽고 주어진 선택지에 따라 분류하는 Laya 튜닝 버전입니다. 현재 사용 중인 모델을 보관한 저장소이며, 독립 공개기사 시험 결과는 **93/130건(71.5%)**입니다. 85% 정확도 목표는 아직 달성하지 못했습니다.

Files and compatibility

• laya_multilingual_q8_0.gguf: 361,797,216 bytes; includes the tokenizer and compiled decision graph.
• topic-questions.json: the four-way news taxonomy used with this model.
• example-state.json and example-request.jsonl: an authored Korean security example.
• evaluation.json: aggregate results and evaluation limits.
• model-manifest.json and SHA256SUMS: provenance and file integrity.

Use the Laya runtime from ggmlc v0.9.7. This is a compiled laya.cpp artifact; it is not a generative LLM GGUF for llama.cpp or Ollama. The exported tokenizer is embedded, so no separate tokenizer download is needed.

Model SHA-256:

4a00ab531b5a6c7a1ef009c9617137a4a76adfd8a5877cd35b50928030169bb2

Run

Download the model and example JSON files from this repository. With the compatible laya executable available, run:

laya decide ./laya_multilingual_q8_0.gguf \
  --family multilingual \
  --state-file example-state.json \
  --questions-file topic-questions.json \
  --device auto --json

For repeated requests, keep one daemon alive and write one JSON request per line to its standard input:

laya daemon ./laya_multilingual_q8_0.gguf --family multilingual --device auto

example-request.jsonl contains a complete request. Responses contain a selected category and scores. The scores are model estimates, not a measured probability that each answer is correct.

The news choices are security, ai, development, and other (business and unrelated topics). The question instructions define the boundaries; changing them can change accuracy. Use the provided questions when reproducing the reported news task.

Evaluation

The model was selected using calibration data before one source-separated final evaluation of 130 public-news excerpts. Reference labels were authored by an independent assistant and were not human-adjudicated. The test measures this specific classification task, not general Korean fluency or all computer-use decisions. It is now a consumed test set.

News classification result	Value
Overall accuracy	93/130 (71.5%)
Mean per-class recall	71.1%
AI recall	13/22 (59.1%)
Development recall	11/14 (78.6%)
Business/other recall	41/56 (73.2%)
Security recall	28/38 (73.7%)

The preceding local tuned version scored 71/130 (54.6%) on the same final test. That comparison is not against the untouched upstream model.

Existing judgment regression checks were also retained:

Task	Correct / cases
Workflow mode	253/272
Next action	106/138
Work/chat label	136/138
Message kind	90/98
Needs action	85/98
Evidence status	88/96

These regression sets were previously used during development and do not constitute independent final tests. A separate paired Korean/English diagnostic scored 26/32 in each language; it is not a general language-proficiency benchmark.

On the local Apple Silicon machine, a fixed calibration workload of 20 inputs repeated three times measured warm daemon round-trip latency of 19.51 ms median / 27.52 ms p95. These are local runtime measurements, not end-to-end application latency or a guarantee on other hardware.

Later experimental models reached 83.2% on a calibration set and 78.5% on a previously consumed news set, but failed the 85% acceptance target. Those models and results do not describe the weights distributed here.

Training and provenance

• Upstream: convaiinnovations/laya-multilingual, revision 1720e3e3357cfe1e281542e223f8273b0890ca34.
• Tuning used Korean judgment examples and public-news excerpts, with prior-task replay and separate calibration data.
• The last accepted stage used 1,565 training rows and 415 calibration rows; epoch 3 was selected. It tuned 22 encoder layers with encoder learning rate 3e-6, head learning rate 8e-6, and seed 20261105.
• Q8_0 export: monatis/ggmlc, commit 5490a13661d95add025ec1b111adbc88234a9b4f (v0.9.7).
• Raw training/evaluation examples, local logs, and the separate Vica routing model are not included.

Intended use and limits

The model classifies supplied text with typed questions. It does not read screenshot pixels or infer screen coordinates. A computer-use observer must supply text and current screen bounds; the host agent handles labels, colors, placement, scrolling, and stale observations.

Ambiguous articles and the boundary between AI applications, general development, technical releases, and business announcements remain error sources. Confidence scores can be high on wrong answers. Validate the model on the specific tasks and inputs where it will be used.

Local development

Model training and evaluation now live in this repository, independently of the Vica application checkout. scripts/laya-tuning contains the training, Q8 export, evaluation and acceptance tools. The experiment directories preserve successive data preparation and fixed candidate-selection workflows. Optional Vica application checks are in integrations/vica and require an explicit application checkout.

Private datasets, training checkpoints, runtime binaries and detailed experiment outputs remain outside Git. The existing local environment is reused:

LAYA_RUN="$HOME/.local/state/vica/laya/tuning/topic-boundary-20261105"
python3.12 experiments/topic-boundary/prepare-data.py --root "$LAYA_RUN"
python3.12 experiments/topic-boundary/train-stages.py --root "$LAYA_RUN"

These commands require the existing private experiment inputs and training environment; a public clone alone does not contain them. Outputs are preserved: do not rerun preparation or training into a completed experiment directory. Only a calibration-selected, frozen candidate is eligible for the final test. Model installation and Hugging Face publication are separate steps.

If cloned with GIT_LFS_SKIP_SMUDGE=1, the root GGUF is only an LFS pointer. Use an actual downloaded or installed GGUF for inference.

Tuning history — 2026-11-06

Aggregate experiment results preserve four attempts, their selection criteria, model hashes and measured failures. None of these attempts replaced the distributed model.

The latest candidate was selected from four Q8 exports before a new 160-example test. Compared with the distributed model on that same test, AI recall rose from 65% to 70%, business/other recall fell from 80% to 72.5%, and overall accuracy stayed at 114/160 (71.25%). Warm p95 latency was 27.78 ms versus 27.80 ms. The business/other loss exceeded the declared three-percentage-point limit, so the candidate was rejected. These are assistant-labeled examples, with 40 cases per class; they do not establish production accuracy or achievement of the 85% target.

The frozen verifier also reported a balanced-accuracy loss of about 1e-16 from floating-point rounding. With equally sized classes and identical total correct counts, balanced accuracy was mathematically unchanged. This numerical issue is recorded for a future verifier correction; it does not remove the separate business/other regression or change the preserved rejection.

The 275- and 160-example final tests are now consumed and must not be reused for training or candidate selection. The source archive requires private local inputs to reproduce full experiments. Publishing this history does not deploy a model.

License and attribution

Apache License 2.0, following the upstream model license. See LICENSE and NOTICE. Vica Laya is a modified derivative, not an official upstream Laya release.