sallout/laya-multitask-intents

Laya Multitask Intent Classifier

This is a supervised fine-tune of Laya Multilingual for single-label intent classification. It is not a zero-shot model and was not trained with the upstream RLCD recipe. The repository contains the PyTorch checkpoint at its root and a float16 Core ML export in coreml/. inference.json stores the supported task labels and instructions. Training data is not included.

Training

The model was trained jointly on the training splits of Banking77, ArBanking77, and CLINC150. ArBanking77 training used Modern Standard Arabic and Palestinian Arabic examples. The training objective was supervised cross-entropy over shuffled choices. The source checkpoint was convaiinnovations/laya-multilingual at revision 052592a15d198d9ad47da779604259b10b47b7aa (weights SHA-256 9d628fd971b700382ac6f65920a86f149777b2e748e0c955fb3b19695aa8f204). The selected checkpoint is update 3,500 from a 5,400-update run, chosen using validation data. The model uses a 1,024-token context and a maximum of 896 tokens for the question and options.

Evaluation

The original and fine-tuned PyTorch checkpoints were evaluated with identical prompts, labels, input budgets, and examples. Exact text overlaps with this model's training and validation data were excluded from the external datasets.

Dataset	Examples evaluated	Original	Fine-tuned	Difference
MInDS-14, English banking intents	1,654	75.27%	84.89%	+9.61 pp
HWU64, English assistant intents	1,071	47.71%	50.79%	+3.08 pp

MInDS-14 has 14 banking intent labels. Its en-US, en-GB, and en-AU train splits were used only as an external evaluation corpus for this model; they were not part of training or checkpoint selection. The comparison removed 148 duplicate or conflicting rows and 7 exact overlaps with training or validation text. The HWU64 comparison used 64 intent labels and removed 5 exact overlaps. These evaluations measure transfer to different label sets, not performance on every possible banking or assistant intent.

On the held-out test splits associated with the training tasks, the fine-tuned model scored 89.94% on Banking77, 73.57% on ArBanking77, and 86.05% on CLINC150. These test scores should not be described as zero-shot. They were measured separately from the external comparisons above.

Use

Install the companion laya-classifier-ft project, download this repository to a local directory, and run:

hf download sallout/laya-multitask-intents --local-dir ./laya-multitask-intents
uv run laya-ft predict \
  --checkpoint ./laya-multitask-intents \
  --task banking77 \
  --text "My card was declined at the shop"

The Core ML export requires a compatible Apple platform (macOS 15 or iOS 18 or later). On macOS, install laya-coreml==0.1.0 and use the same task metadata:

import json
from pathlib import Path

import laya_coreml

root = Path("laya-multitask-intents")
task = next(
    task
    for task in json.loads((root / "inference.json").read_text())["tasks"]
    if task["name"] == "banking77"
)
agent = laya_coreml.load(str(root / "coreml"), local_files_only=True)
result = agent.predict(
    "My card was declined at the shop",
    {
        "intent": {
            "type": "choice",
            "instructions": task["instruction"],
            "criteria": dict.fromkeys(task["labels"]),
        }
    },
)
print(result["answers"]["intent"]["choice"])

The checkpoint supports the trained label sets in inference.json; use the training project and your own labeled CSV data to create a classifier for another set of intents. The returned softmax probabilities are not calibrated. The evaluation figures above are for PyTorch; the Core ML export may differ slightly.

License and attribution

The upstream Laya Multilingual model is released under Apache 2.0 by Convai Innovations. This fine-tuned derivative is released under the same license. Dataset licenses remain with their respective publishers; no dataset text is included in this repository.