laya-indic

A multilingual typed-decision model with a focus on Indian languages and Hinglish. You give it a text (a message, ticket, review, headline or document) and one or more questions written in plain English. It answers every question in a single forward pass, with a calibrated probability for each option. Question schemas are defined at inference time, so it handles new labels and new tasks without retraining.

laya-indic is an independent personal release. It is not affiliated with or endorsed by HAL-X AI. It uses the THX-01 decision architecture and the open-source thx01 library (Apache-2.0, HAL-X AI), but it is built on the jhu-clsp/mmBERT-base encoder (MIT), not on THX-01's weights. The encoder's pretraining is first continued on public Indian-language web text (Sangraha, CC-BY-4.0), and then the decision model is trained. All training data is public and permissively licensed, plus labels generated by open-weights Qwen3 models.

Parameters	about 322M (mmBERT-base encoder plus a 2-layer decision head)
Input	up to 1,024 tokens per question (question, options and text)
Question types	choice, noul (yes/no), score (ordinal)
Training	continued pretraining on 457M tokens of public text, then 2 epochs over 521,752 decisions: 285,078 from 17 public datasets, 173,226 verified Qwen3 synthetic questions and 63,448 verified Qwen3 questions about real Indian-language text
Languages	trained on 30+ languages; strongest on English, Hindi, Hinglish and 10 other Indian languages
Licence	Apache-2.0

What's new in v2

• Continued pretraining on Indian languages. Before decision training, the mmBERT-base encoder ran one masked-language-modelling pass over 457M tokens of Sangraha (11 Indian languages plus English) and Hinglish replay. Held-out MLM loss dropped 7–25% per language.
• Real Indian-language text. 23,955 snippets of real Sangraha web text in 11 languages, with 63,448 questions about them written and verified by open-weights Qwen3. v1 saw only synthetic Indic text.
• Results. Overall 0.789 → 0.800. SIB-200 Indic topic 0.626 → 0.670, up in 9 of 10 languages. SIB-200 in THX-01's languages 0.751 → 0.779. Other families are flat, except IndicSentiment, which dipped (0.892 → 0.884). The earlier weights are kept at revision v1.0.

Question types

type	returns	criteria
choice	one of N options, with a probability for each	{"key": "description", ...} (descriptions may be null)
noul	P(yes)	none
score	an ordinal level and its distribution	an ordered list of level descriptions

THX-01 also supports number, excerpt and "cite": true. This model was not trained for those and does not support them.

Quickstart

import thx01   # pip install thx01

agent = thx01.load("arunp0/laya-indic")          # or a local directory; GPU if available
# v1 weights: thx01.load(huggingface_hub.snapshot_download("arunp0/laya-indic", revision="v1.0"))

text = "Kal se mera UPI payment fail ho raha hai, paise kat gaye par merchant ko nahi mile"
out = agent.system_one(text, {
    "topic":  {"type": "choice", "instructions": "What is the customer writing about?",
               "criteria": {"failed_payment": "payment failed or money debited but not received",
                            "refund": "asking for a refund", "account": "login or account access", "other": None}},
    "urgent": {"type": "noul", "instructions": "Does the customer need help urgently?"},
    "anger":  {"type": "score", "instructions": "How frustrated is the customer?",
               "criteria": ["calm", "mildly annoyed", "clearly frustrated", "furious"]},
})
# topic  -> failed_payment (0.97)
# urgent -> P(yes) 0.55
# anger  -> "clearly frustrated" 0.85

The response's model field says "THX-01". That is the thx01 library's default label; the weights are this model's.

For many texts, use agent.predict_batch(texts, questions, batch_size=64). questions can be one schema shared by every text or a list with one schema per text.

Write questions in English and leave the text in its own language and script. That is how the model was trained.

Benchmarks

Zero-shot accuracy on 67 tasks, compared with THX-01 (322M, same architecture, roughly 2.1M training decisions) and with laya-indic v1. Both models were scored with the same harness, the same prompts and the same test samples. The harness is adapted from THX-01's open benchmark code (bench/tasks.py).

Task family	Tasks	THX-01	laya-indic v1	laya-indic v2
Overall (all tasks)	67	0.757	0.789	0.800
English general (AG News, Emotion, Banking77, SMS, MASSIVE en)	7	0.813	0.851	0.851
MASSIVE scenario, THX's 14 languages	14	0.871	0.879	0.878
MASSIVE intent@20 + yes/no (az, hi)	4	0.916	0.914	0.919
SIB-200 topic, THX's 10 languages	10	0.766	0.751	0.779
SIB-200 topic, Indic (hi + 9)	10	0.628	0.626	0.670
MASSIVE scenario, Indic (bn ta te kn ml ur)	6	0.670	0.848	0.849
IndicSentiment (6 languages)	6	0.879	0.892	0.884
IndicCOPA (6 languages)	6	0.525	0.497	0.513
Hinglish-TOP (domain, intent@20)	2	0.790	0.935	0.940
Qwen3 synthetic held-out schemas	4	0.717	0.878	0.872

Per task: v2 is ahead of THX-01 on 47 of 67 tasks, behind on 19, tied on 1; ahead of v1 on 40, behind on 24, tied on 3.

Read these numbers with the training data in mind. Neither model saw these test sets. Some tasks, however, have their training splits in one model's data and not in the other's:

Eval family	This model trained on	THX-01
MASSIVE	train split, 20 languages (incl. hi bn ta te kn ml ur, az ka tr fa)	train splits ("public benchmarks' training splits")
Hinglish-TOP	train + validation splits	not stated
Banking77, SMS spam	Banking77 train; SMS messages outside the 1,000-message test sample	not stated
AG News	nothing (licence unclear, so excluded)	not stated individually
SIB-200	nothing (CC-BY-SA, so excluded); synthetic topic sentences and, in v2, Qwen3-labelled topics on real Sangraha text	not stated
IndicSentiment, IndicCOPA, DAIR Emotion	nothing; synthetic reviews and emotion datasets instead	not stated

All training texts, including the continued-pretraining corpus (line by line), were deduplicated against every evaluation set, using normalised exact match.

Per-task results (accuracy and expected calibration error)

Task	n	THX-01 acc	v1 acc	v2 acc	THX-01 ECE	v2 ECE
AG News (4)	1000	0.931	0.878	0.864	0.008	0.043
DAIR Emotion (6)	1025	0.425	0.518	0.492	0.264	0.317
Banking77 (77)	1001	0.658	0.811	0.819	0.050	0.052
SMS spam (yes/no)	1000	0.881	0.957	0.964	0.098	0.024
MASSIVE-intent@20 (en)	1000	0.928	0.923	0.925	0.063	0.044
Intent yes/no (en)	1000	0.961	0.959	0.967	0.037	0.020
MASSIVE-intent@20 (az)	1000	0.908	0.876	0.881	0.053	0.037
Intent yes/no (az)	1000	0.948	0.947	0.945	0.028	0.024
MASSIVE-scenario (en)	500	0.906	0.914	0.926	0.090	0.059
MASSIVE-scenario (az)	500	0.878	0.844	0.862	0.059	0.038
MASSIVE-scenario (ru)	500	0.896	0.892	0.900	0.071	0.047
MASSIVE-scenario (tr)	500	0.874	0.882	0.866	0.089	0.080
MASSIVE-scenario (de)	500	0.884	0.894	0.902	0.068	0.054
MASSIVE-scenario (fr)	500	0.892	0.914	0.902	0.068	0.057
MASSIVE-scenario (es)	500	0.868	0.894	0.902	0.072	0.059
MASSIVE-scenario (ar)	500	0.832	0.846	0.844	0.073	0.056
MASSIVE-scenario (hi)	500	0.856	0.894	0.892	0.058	0.071
MASSIVE-scenario (zh-CN)	500	0.872	0.904	0.898	0.068	0.055
MASSIVE-scenario (fa)	500	0.894	0.892	0.890	0.077	0.047
MASSIVE-scenario (ka)	500	0.762	0.752	0.728	0.061	0.067
MASSIVE-scenario (ja)	500	0.910	0.910	0.922	0.066	0.058
MASSIVE-scenario (ko)	500	0.868	0.876	0.856	0.049	0.075
SIB-200 (az)	204	0.779	0.721	0.770	0.102	0.105
SIB-200 (en)	204	0.833	0.794	0.819	0.075	0.081
SIB-200 (ru)	204	0.779	0.774	0.848	0.072	0.087
SIB-200 (tr)	204	0.779	0.779	0.745	0.088	0.132
SIB-200 (de)	204	0.819	0.774	0.819	0.088	0.072
SIB-200 (ar)	204	0.770	0.740	0.779	0.079	0.074
SIB-200 (hi)	204	0.721	0.740	0.730	0.136	0.083
SIB-200 (zh)	204	0.794	0.784	0.828	0.067	0.065
SIB-200 (kk)	204	0.691	0.735	0.760	0.133	0.083
SIB-200 (uz)	204	0.696	0.667	0.691	0.133	0.106
Hinglish-TOP domain	1000	0.889	0.963	0.969	0.062	0.041
Hinglish-TOP intent@20	1000	0.692	0.908	0.910	0.050	0.052
MASSIVE-intent@20 (hi)	1000	0.877	0.883	0.904	0.034	0.049
Intent yes/no (hi)	1000	0.933	0.951	0.945	0.032	0.022
BoolQ-hi (yes/no) ‡	1000	0.700	0.622	0.618	0.140	0.160
MASSIVE-scenario (bn)	500	0.700	0.836	0.846	0.074	0.069
MASSIVE-scenario (ta)	500	0.700	0.840	0.848	0.066	0.057
MASSIVE-scenario (te)	500	0.626	0.846	0.838	0.081	0.059
MASSIVE-scenario (kn)	500	0.614	0.854	0.854	0.065	0.045
MASSIVE-scenario (ml)	500	0.652	0.850	0.846	0.054	0.053
MASSIVE-scenario (ur)	500	0.728	0.862	0.862	0.061	0.044
SIB-200 (mr)	204	0.647	0.667	0.726	0.180	0.099
SIB-200 (bn)	204	0.618	0.652	0.662	0.179	0.135
SIB-200 (ta)	204	0.681	0.686	0.696	0.117	0.118
SIB-200 (te)	204	0.623	0.593	0.657	0.129	0.136
SIB-200 (gu)	204	0.618	0.544	0.672	0.128	0.101
SIB-200 (kn)	204	0.608	0.618	0.662	0.179	0.126
SIB-200 (ml)	204	0.529	0.593	0.598	0.238	0.187
SIB-200 (pa)	204	0.564	0.564	0.603	0.178	0.169
SIB-200 (ur)	204	0.672	0.603	0.691	0.137	0.137
IndicSentiment (hi)	998	0.918	0.923	0.927	0.096	0.033
IndicSentiment (bn)	998	0.866	0.888	0.878	0.065	0.061
IndicSentiment (mr)	998	0.896	0.891	0.908	0.082	0.046
IndicSentiment (ta)	998	0.871	0.900	0.847	0.065	0.088
IndicSentiment (te)	998	0.859	0.864	0.868	0.057	0.073
IndicSentiment (gu)	998	0.863	0.890	0.877	0.058	0.066
IndicCOPA (hi)	449	0.546	0.488	0.501	0.163	0.292
IndicCOPA (mr)	449	0.523	0.485	0.514	0.178	0.264
IndicCOPA (bn)	500	0.522	0.522	0.518	0.185	0.263
IndicCOPA (ta)	500	0.540	0.498	0.490	0.153	0.266
IndicCOPA (te)	500	0.530	0.478	0.528	0.177	0.258
IndicCOPA (gu)	448	0.489	0.509	0.529	0.221	0.239
Qwen3-synth held-out schemas (choice)	1008	0.727	0.845	0.845	0.118	0.065
Qwen3-synth held-out (Hinglish)	719	0.730	0.900	0.890	0.127	0.055
Qwen3-synth held-out schemas (noul)	1342	0.750	0.931	0.930	0.117	0.035
Qwen3-synth held-out schemas (score)	650	0.658	0.835	0.823	0.102	0.046

ECE is expected calibration error with 15 equal-width bins; lower is better. Mean ECE over the 67 tasks is 0.091 for v2, 0.087 for v1 and 0.098 for THX-01.

‡ BoolQ-hi is left out of every mean. Our harness had a label-parsing bug in the first runs of this benchmark. The row shows THX-01 rescored after the fix (0.643, n = 1000) next to v1 and v2, which were scored after the fix.

Where v2 is stronger than THX-01:

• Indian-language intent and domain (MASSIVE Indic +0.18).
• Hinglish (+0.15).
• Banking77 (+0.16), SMS spam and emotion.
• SIB-200 topic in Indian languages (+0.04) and Kazakh.
• New decision schemas it has never seen (synthetic held-out +0.16).

Where it is weaker:

• AG News (0.864 vs 0.931). THX-01 reports training on public benchmarks' training splits; AG News is excluded here.
• SIB-200 in English and German.
• IndicCOPA, a causal-reasoning task, where both models are close to chance.

v2 vs v1: v2 is ahead on 40 tasks and behind on 24. The gains are concentrated in SIB-200 topic classification (Indic +0.044, others +0.028). The largest drop is Tamil IndicSentiment (0.900 → 0.847).

Training

Method. RLCD (Reinforcement Learning for Calibrated Decisions), as introduced by THX-01 and implemented in thx01:

• Policy-gradient term: sampling from noisy logits, rewarded by a strictly proper scoring rule.
• Soft cross-entropy term: on label-smoothed targets.
• Calibration: a temperature for each question type, fitted after training on held-out data.

Stage 1: continued pretraining (new in v2). jhu-clsp/mmBERT-base ran one pass of masked-language modelling (15% masking, sequence length 512, learning rate 3e-5) over 457M tokens:

• Sangraha "verified" documents in Telugu, Gujarati, Kannada, Malayalam, Punjabi, Hindi, Bengali, Tamil, Marathi, Urdu, Odia and English.
• A small replay of Hinglish texts from the training data below.

Every line that matched an evaluation text was removed. Stage 2: decision training from that encoder, with a freshly initialised decision head.

Epochs	2
Batch	16 × 2 gradient accumulation = 32 decisions per update; 32,610 updates
Learning rates	encoder 5e-5, head 2e-4; AdamW, weight decay 0.01, 6% warmup, cosine decay
Precision	fp16 autocast, gradient checkpointing
Hardware / time	one NVIDIA H100 shared with other jobs: about 1.5 hours of continued pretraining and 2.8 hours of decision training
Question mix	233k choice, 195k yes/no, 93k score

Public data (285,078 decisions; every source is CC0, CC-BY, Apache-2.0 or MIT; full attribution in LICENSES.md):

Source	Decisions	Licence	Used for
MoritzLaurer/synthetic_zeroshot_mixtral_v0.1	59,998	Apache-2.0	general zero-shot choice / yes-no over diverse tasks
MASSIVE (20 languages, train)	54,875	CC-BY-4.0	intent and domain choice, intent yes/no
HuffPost News Category	28,552	CC-BY-4.0	news-topic choice, coarse and fine
HelpSteer2	20,000	CC-BY-4.0	response-quality score questions
BRIGHTER emotion	19,344	CC-BY-4.0	emotion yes/no, human-labelled in-language (Hindi and Marathi in full, plus 8 other languages)
Civil Comments	12,999	CC0-1.0	toxicity yes/no and score
GoEmotions	11,998	Apache-2.0	fine-grained emotion
Nemotron Safety Guard v3	11,000	CC-BY-4.0	multilingual safety categories
Aegis 2.0	10,999	CC-BY-4.0	safety categories
LEDGAR (LexGLUE)	10,000	CC-BY-4.0	contract-clause type
Banking77 (train)	9,964	CC-BY-4.0	77-way intent
WMT DA human evaluation	9,000	Apache-2.0	translation-quality score
Measuring Hate Speech	8,000	CC-BY-4.0	hate-speech facets
IBM Argument Quality 30k	8,000	CC-BY-3.0	argument-quality score
Hinglish-TOP (train + validation)	6,936	Apache-2.0	Hinglish domain and intent
Aya Red-teaming	2,685	Apache-2.0	multilingual harm categories
SMS Spam Collection	728	CC-BY-4.0	spam vs ham

Synthetic data (173,226 verified questions over 62,263 texts). The generator is Qwen3-235B-A22B-Instruct-2507 and the verifier is Qwen3-30B-A3B-Instruct-2507, both Apache-2.0. The pipeline is label-first:

1. The generator proposes a decision schema.
2. Gold answers are sampled.
3. The generator writes a text in a chosen language, script and register that makes those answers true.
4. The verifier answers blind, and a question is kept only if the verifier agrees with the gold answer (about 72% were kept).

About 68% of the texts are in Indian languages or Hinglish. No proprietary-model outputs were used anywhere. The dataset is published as arunp0/laya-indic-synthetic; see also SYNTHETIC_DATA.md.

Real Indian-language text (new in v2). 23,955 snippets of 1–3 sentences or a short paragraph were cut from Sangraha documents (AI4Bharat, CC-BY-4.0):

Languages	Snippets
Gujarati, Telugu, Kannada, Punjabi, Malayalam	about 2,850 each
Marathi, Bengali, Hindi, Tamil	about 2,000 each
Odia	1,464
Urdu	113

For each snippet:

1. Qwen3-235B picked the topic from a fixed 16-topic list. It also wrote 2–3 further questions specific to the snippet (choice, yes/no or score).
2. The topic options were assembled in code. In 7% of cases the correct topic was removed, so "none of the options" is the answer.
3. Qwen3-30B answered every question blind, and only questions where the two models agreed were kept: 63,448 in training.

The data is published as arunp0/laya-indic-real-text (CC-BY-4.0).

Limitations

• Text classification only. It does not do number, excerpt or citation questions, open-ended generation or multi-step reasoning. IndicCOPA results show it cannot do commonsense causal reasoning.
• Instructions in English. Questions and option descriptions should be in English; the text can be in any supported language.
• Long texts are truncated at 1,024 tokens per question.
• Calibration is good on average but not everywhere. Recalibrate on your own data before using probabilities as thresholds.
• About 45% of the training labels come from Qwen3. 33% are questions on synthetic texts and 12% are questions on real web text. Synthetic Indic text is cleaner and more formal than real user text. Sangraha is mostly news and web prose, not chat. Urdu and Odia have little real text.
• Content warning. The safety and toxicity datasets (Aegis 2.0, Nemotron Safety Guard, Measuring Hate Speech, Civil Comments, Aya Red-teaming) contain offensive and harmful text. The model can classify such content but is not a tested content-moderation system.

Acknowledgements and licence

• Architecture, training objective and inference library: THX-01 / thx01 by HAL-X AI (Apache-2.0).
• Benchmark harness: THX-01's bench/tasks.py, including its MASSIVE scenario descriptions and label humaniser, which were also used in some training instructions.
• Encoder: mmBERT-base, by JHU CLSP (MIT).
• Indian-language corpus: Sangraha, by AI4Bharat (CC-BY-4.0), used for continued pretraining and real-text decisions.

The model is released under the Apache License 2.0. See NOTICE for the required attributions and LICENSES.md for every dataset's authors and licence.