NeuronUz/NeuronAI-2B

NeuronAI 2B

NeuronUz/NeuronAI-2B is an Uzbek-first, text-only Qwen3.5 2B assistant model with English capability retained. The published weights are a merged checkpoint: no adapter is required at inference time.

This release updates the repository to the best overall checkpoint from the 2026-08-18 LoRA r64 checkpoint sweep (checkpoint-4749, one SFT epoch). It replaces the previous full-parameter SFT checkpoint.

Model lineage

1. Qwen/Qwen3.5-2B-Base
2. Uzbek tokenizer retrofit (same vocabulary size) and embedding re-initialization
3. Uzbek continued pretraining
4. Annealing
5. Assistant-only supervised fine-tuning with LoRA r64
6. LoRA merged into the base weights for deployment

The model is called NeuronAI 2B in its identity training examples.

Evaluation

The four retained checkpoints were evaluated on the same strict eight-task public suite using full test splits, greedy vLLM decoding, and COMET-primary translation scoring. Checkpoint 4749 achieved the best normalized weighted score. All format-scored tasks had a 0% invalid-output rate.

Benchmark	Metric	Checkpoint 4749
UzLiB	accuracy	0.49597
TUMLU-Uzbek	accuracy	0.32571
FLORES+ en→uz	COMET	0.87615
FLORES+ en→uz	BLEU	12.00
Uzbek news classification	accuracy	0.78549
MMLU (English)	accuracy	0.54066
MMLU (Uzbek)	accuracy	0.46852
FLORES+ uz→en	COMET	0.85347
FLORES+ uz→en	BLEU	27.21
Uzbek sentiment (binary)	accuracy	0.95500
Normalized weighted score	COMET-primary	0.59542

Suite weights were UzLiB 0.20, TUMLU 0.20, en→uz COMET 0.15, news 0.10, MMLU-en 0.10, MMLU-uz 0.10, uz→en COMET 0.05, and sentiment 0.05, normalized over their total weight of 0.95.

Checkpoint 1188 scored higher on UzLiB (0.54326), but checkpoint 4749 was selected because it was stronger overall, particularly in translation, classification, and sentiment.

Tokenizer fertility

The Uzbek tokenizer work in this model is a retrofit, not an extension: the vocabulary stays at Qwen3.5's 248,044 entries, and Uzbek merges replace low-value slots rather than being appended. The embedding matrix therefore does not grow, and the parameter count is unchanged from the base model -- the gain is purely in how Uzbek text segments.

Tokenizer	Vocabulary	Tokens per word (uz-crawl)
NeuronAI-2B	248,044	2.407
Qwen3.5-2B (stock)	248,044	3.426

Measured on 20,000 documents (4.8M whitespace words) reservoir-sampled with seed 42 from uz-crawl (tahrirchi web crawl), keeping documents of at least 250 characters. Stock Qwen3.5 needs 1.42x as many tokens for the same Uzbek text, so the retrofit is worth about 30% of the context window, the prefill cost, and the decode steps on Uzbek input.

Caveat: uz-crawl is part of the Uzbek continued-pretraining corpus, so it is in-domain for the tokenizer that was fitted on it. The fertility gap on unseen Uzbek web text is expected to be somewhat smaller.

Comparison with other models

Compared against the two open Uzbek instruction models in this size class (alloma-1B-Instruct and alloma-3B-Instruct, both Llama-3.2 fine-tunes) and against the stock base model, Qwen/Qwen3.5-2B.

Benchmark	Metric	Weight	NeuronAI-2B	Qwen3.5-2B	alloma-3B	alloma-1B
uzlib (uz MCQ)	accuracy	0.20	0.4960	0.2869	0.3208	0.2332
TUMLU-uz	accuracy	0.20	0.3257	0.3129	0.2771	0.2200
FLORES en->uz (COMET)	COMET	0.15	0.8762	0.7010	0.8673	0.7383
News classification	accuracy	0.10	0.7855	0.3675	0.1360	0.2541
MMLU (en)	accuracy	0.10	0.5407	0.5239	0.3873	0.2198
MMLU-uz	accuracy	0.10	0.4685	0.3710	0.3274	0.2111
FLORES uz->en (COMET)	COMET	0.05	0.8535	0.8072	0.7954	0.7636
Sentiment (binary)	accuracy	0.05	0.9550	0.7687	0.3885	0.7954
Weighted score		0.95	0.5657	0.4301	0.3940	0.3478

On score comparability. These four runs were produced by three different suite revisions. The alloma runs used the APST suite variant, stock Qwen used the plain variant, and NeuronAI-2B used the COMET-primary variant; the first two weight translation by BLEU and the third by COMET, so their archived weighted scores are not on one scale. Every run passed --compute-comet, so the table above recomputes all four from the per-task outputs under a single weighting (the COMET-primary weights, translation scored by COMET). The NeuronAI-2B weighted score therefore reads 0.5657 here versus 0.5954 in the section above, which normalizes over the 0.95 total weight; the ranking is unaffected.

Both alloma models were evaluated with the apostrophe-to-APST substitution their model card requires. Their runs tripped the suite's invalid-output gate on several MCQ tasks (alloma-1B on uzlib and TUMLU, alloma-3B on TUMLU and MMLU-en), meaning part of their MCQ gap is format compliance rather than knowledge.

NeuronAI-2B leads the weighted score and every one of the eight tasks, including against alloma-3B at over half again its parameter count. The margin is narrowest on FLORES+ en->uz (0.8762 against alloma-3B's 0.8673) and on TUMLU-Uzbek, which is the weakest task for every model tested, ours included -- at 0.3257 it is close enough to the 0.25 four-way chance level to treat as unsolved at this scale.

Training summary

• Base: Uzbek CPT + anneal checkpoint
• Method: LoRA, merged for release
• LoRA rank/alpha/dropout: 64 / 128 / 0.05
• LoRA targets: all linear layers
• Trainable adapter parameters: 67,276,800
• Training data: 152,152 grouped and globally deduplicated examples
• Validation data: 1,537 grouped examples
• Train/dev prompt-group overlap: 0
• Sequence length: 2,048
• Packing: disabled
• Epochs: 1
• Effective batch size: 32
• Learning rate: 1e-4, cosine schedule, 3% warmup
• Optimizer: AdamW, beta2 0.95, weight decay 0.01
• Precision: bf16 with selected recurrent parameters retained in fp32
• Loss: assistant turns only; <|im_end|> explicitly supervised

The training mixture is Uzbek-first and includes clean general assistant data, translation, Uzbek language and literature, spelling and orthography, MMLU-style tasks, classification, math, and English-retention examples. Training data is not distributed in this model repository.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NeuronUz/NeuronAI-2B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="cuda:0",
)

messages = [
    {"role": "system", "content": "Siz foydali AI yordamchisiz."},
    {"role": "user", "content": "O'zbekiston haqida qisqacha ma'lumot bering."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=512, do_sample=False)

reply = tokenizer.decode(
    output[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
).strip()
print(reply)

The model emits <|im_end|> and the generation metadata registers both <|im_end|> and <|endoftext|> as end-of-sequence tokens.

vLLM

from vllm import LLM, SamplingParams

llm = LLM(
    model="NeuronUz/NeuronAI-2B",
    max_model_len=4096,
)
params = SamplingParams(temperature=0.0, max_tokens=512)

messages = [
    {"role": "system", "content": "Siz foydali AI yordamchisiz."},
    {"role": "user", "content": "O'zbekiston haqida qisqacha ma'lumot bering."},
]
print(llm.chat(messages, params)[0].outputs[0].text)

Notes and limitations

• Use a Transformers/vLLM release with Qwen3.5 support.
• Keep the hybrid Qwen3.5 model on one device unless the serving stack has been validated for multi-device Gated-DeltaNet inference.
• TUMLU-Uzbek remains close to the 0.25 random baseline, so specialist factual knowledge should be independently verified.
• The model may produce inaccurate, biased, or fabricated information and has not been comprehensively evaluated for safety or high-stakes use.