aungthuhein-dev/laya-burmese-sib200

Laya Burmese SIB-200

laya-burmese-sib200 is a fine-tuned version of the multilingual Convai Innovations Laya checkpoint for seven-way Burmese topic classification.

The model was trained on all 701 Burmese training examples from SIB-200 using plain cross-entropy. It produces typed topic decisions and calibrated class probabilities through the Laya Agent interface.

This is an independent research release. It is not an official model from Convai Innovations or TypeSafe AI, and it does not contain official Jev weights.

Model details

Property	Value
Base model	Laya multilingual checkpoint
Training dataset	SIB-200, mya_Mymr
Training examples	701
Task	Seven-way topic classification
Objective	Soft cross-entropy
Optimizer steps	264
Random seed	1
Input representation	Unicode Burmese
Calibration	Temperature scaling on the validation split
Temperature	6.018

Topics

The model predicts one of seven SIB-200 topics:

• entertainment
• geography
• health
• politics
• science/technology
• sports
• travel

Evaluation results

The model was evaluated on the official 204-example Burmese SIB-200 test split.

Metric	Result
Test accuracy	0.716
Macro-F1	0.677
ECE after temperature scaling	0.051
Zero-shot Laya baseline accuracy	0.441

The released checkpoint was trained separately from the three runs used in the associated scaling study. GPU training was not bit-reproducible, so its metrics may differ slightly from the reported three-seed aggregate.

Calibration

A temperature of 6.018 was fitted on the held-out SIB-200 validation split. The value is stored in rl_agent_config.json and is applied automatically by Laya's Agent when the model is loaded.

Do not apply temperature scaling a second time.

The reported ECE describes calibration on the 204-example SIB-200 test split. It does not guarantee calibration on natural Burmese text or other domains.

Intended use

The model is intended for:

• Research on Burmese text classification
• Reproduction of the associated SIB-200 experiments
• Demonstrations of typed, non-autoregressive topic decisions
• Evaluation of probability calibration
• Comparison with the zero-shot Laya multilingual checkpoint

Out-of-scope use

The model should not be treated as:

• A general-purpose Burmese language model
• A classifier for topics outside the seven SIB-200 labels
• A safety, moderation, medical, political, or legal decision system
• A replacement for human review in consequential applications
• A model validated on naturally occurring Burmese documents
• A model validated on Zawgyi or mixed Unicode–Zawgyi input

Limitations

Translated evaluation data

SIB-200 is based on translated FLORES sentences. Its text is primarily news-like and may not represent conversational Burmese, social media, customer-support messages, or other naturally occurring domains.

Unicode only

The model was trained and evaluated using Unicode Burmese. Performance on Zawgyi, mixed encodings, other Myanmar-script languages, and corrupted text has not been established. Applications receiving legacy Zawgyi text should detect and convert it to Unicode before inference.

Limited test size

The test split contains 204 examples. Small differences in accuracy or calibration should therefore be interpreted cautiously.

Restricted label space

The model can only choose among the seven listed topics. Inputs that do not belong clearly to one of these topics will still receive a probability distribution over the available options.

Domain-specific calibration

The stored temperature was fitted on the SIB-200 validation distribution. Probabilities may become miscalibrated under domain or language shift. Applications should evaluate and, if necessary, recalibrate the model using representative held-out data.

Training procedure

The checkpoint was initialized from the Laya multilingual model and fine-tuned using:

Hyperparameter	Value
Objective	Soft cross-entropy
Batch size	16
Epochs	6
Optimizer steps	264
Encoder learning rate	2.5e-5
Decision-head learning rate	1.0e-4
Weight decay	0.01
Gradient clipping	1.0
Learning-rate schedule	Cosine
Random seed	1

After fine-tuning, a single temperature was fitted on the official 99-example Burmese validation split. The model was then evaluated on the untouched 204-example test split.

Recommended inference practice

Use the probability assigned to the predicted label when implementing a selective prediction or human-review threshold. Choose the threshold using held-out data from the intended deployment domain.

Do not interpret Laya's normalized-entropy confidence field as a calibrated probability that the prediction is correct.

Dataset

SIB-200 contains topic-classification data for more than 200 languages and dialects. This checkpoint uses the Burmese mya_Mymr configuration.

• Dataset: https://huggingface.co/datasets/Davlan/sib200
• SIB-200 paper: https://aclanthology.org/2024.eacl-long.14/

Base model and architecture

The checkpoint is derived from the multilingual Laya model:

• Base model: https://huggingface.co/convaiinnovations/laya
• Architecture context: https://typesafe.ai/blog/introducing-system-one-models-and-jev

Laya is a non-autoregressive decision model that maps an input state and a typed question schema directly to a structured probability distribution.

Live demo

An interactive demonstration is available on Hugging Face Spaces:

https://huggingface.co/spaces/aungthuhein-dev/laya-burmese-sib200-demo

The demo reports calibrated probabilities and optionally compares the fine-tuned checkpoint with the zero-shot multilingual checkpoint.

License

This fine-tuned checkpoint is released under the Apache License 2.0. Users must also comply with the licenses and terms applicable to the base model and SIB-200 dataset.

Citation

If you use this checkpoint before the accompanying paper receives a permanent identifier, cite the model repository:

@misc{hein2026layaburmese,
  author       = {Aung Thu Hein},
  title        = {Laya Burmese SIB-200},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/aungthuhein-dev/laya-burmese-sib200}}
}