pdnhan/Arcee-VyLinh-mobile

Arcee-VyLinh for mobile (ONNX, int4)

arcee-ai/Arcee-VyLinh converted to a 4-bit ONNX model that ONNX Runtime runs on Android and iOS, with ready-to-use Kotlin and Swift code: the Qwen tokenizer, the generation loop with a KV cache, and a speech-transcript refiner for Vietnamese.

The refiner turns raw speech-recognition output (for example PhoWhisper) into clean written text: it adds punctuation and capitals, fixes misheard words from context, removes fillers and stutters, writes numbers in digits, dates as day/month/year (20/11/2026) and units as symbols (5 kg), and writes spoken lists as bullet points. It is what the Thì Thầm app runs after PhoWhisper.

Transcript	Refined
ờ hôm nay chúng ta sẽ họp về ba vấn đề một là ngân sách quý tới hai là tuyển thêm nhân viên và ba là kế hoạch du lịch công ty	Hôm nay chúng ta sẽ họp về ba vấn đề: - Ngân sách quý tới. - Tuyển thêm nhân viên. - Kế hoạch du lịch công ty.
thủ tướng phạm minh chính đã tiếp bộ trưởng nông nghiệp mỹ tại hà lội vào sáng nay	Thủ tướng Phạm Minh Chính đã tiếp Bộ trưởng Nông nghiệp Mỹ tại Hà Nội vào sáng nay.
bạn có biết mấy giờ rồi không	Bạn có biết mấy giờ rồi không?
nhớ mua giúp mình mấy thứ nhé trứng sữa tươi bánh mì và một ít táo	Nhớ mua giúp mình mấy thứ nhé: - Trứng - Sữa tươi - Bánh mì - Một ít táo
các bước cài đặt như sau đầu tiên tải ứng dụng sau đó đăng nhập bằng số điện thoại cuối cùng là bật thông báo	Các bước cài đặt như sau: - Đầu tiên tải ứng dụng. - Sau đó đăng nhập bằng số điện thoại. - Cuối cùng là bật thông báo.
ừm thì là cuộc họp ngày mai được dời sang hai giờ chiều nha mọi người	Thì là cuộc họp ngày mai được dời sang 2 giờ chiều nha mọi người.

The weights are those of the original model; only the format and the precision differ.

Files

File	What	Size	SHA-256
onnx/model.onnx	int4 model graph	287.8 kB	bd4ff6ab8b1d287897bb2c160b3f25576cc2aa62702c71198faba3bdd5ac6d7e
onnx/model.onnx.data	int4 weights	1,946.3 MB	756bec774a581fc1a02ffa1c3a5ec09c13a2a02d15106976ef1593a88568c4a7
onnx/genai_config.json	model settings	1.5 kB	e04e766c5d733d97e910f98f93fc54ee1522d4f92bbaa7a13c4c5d59cd6b0b8f
vocab.txt	tokenizer vocabulary	1.5 MB	f79f19c08fbb0c35f098f503b219845c7fa03f42432e991fe82b52ff8f257014
merges.txt	tokenizer merges	1.7 MB	599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3
onnx/tokenizer.json	tokenizer for Python	11.4 MB	3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8

Mobile apps download onnx/model.onnx, onnx/model.onnx.data, onnx/genai_config.json, vocab.txt and merges.txt (about 1950 MB in total) and use the code in android/ or ios/.

• onnx/model.onnx + onnx/model.onnx.data: built with the ONNX Runtime GenAI model builder (-p int4 -e cpu): GroupQueryAttention with a KV cache, 4-bit MatMulNBits (symmetric, blocks of 32, accuracy_level 4) and the tied embedding as a 4-bit GatherBlockQuantized shared with the LM head, which only computes the last position's logits. Plain ONNX Runtime (1.22 or newer; tested with 1.31.1) runs it; the folder also works as is with ONNX Runtime GenAI.
• vocab.txt: the byte-level BPE vocabulary, one token per line, line number = token id (special tokens included).
• merges.txt: the BPE merges, one a b pair per line, by rank.
• onnx/tokenizer.json, onnx/tokenizer_config.json, onnx/chat_template.jinja: the original tokenizer files, for Python and ONNX Runtime GenAI.

Model interface

	Name	Type	Shape
input	input_ids	int64	[batch, tokens]
input	attention_mask	int64	[batch, past + tokens]: 1 for every token so far
input	past_key_values.{0..35}.key / .value	float32	[batch, 2, past, 128] (past = 0 for the first run)
output	logits	float32	[batch, 1, 151936]: next-token scores after the last input token
output	present.{0..35}.key / .value	float32	[batch, 2, past + tokens, 128]: the next run's past_key_values

Accuracy of the refiner

On 300 PhoWhisper transcripts of FLEURS Vietnamese test clips (the first 150 clips, transcribed by the int8 PhoWhisper tiny and small models of pdnhan/PhoWhisper-small-mobile and its siblings), against the original transcriptions:

Recognizer	Word error rate of the transcript	after the refiner
PhoWhisper small (int8)	12.3%	10.8%
PhoWhisper tiny (int8)	28.9%	27.4%

Word error rate is measured case- and punctuation-insensitively; FLEURS writes numbers as both digits and words, so a number the refiner writes in digits can count as an error. Punctuation (. , ? ! : on the words both texts share): precision 0.75, recall 0.69, F1 0.72. Time per transcript on an x86 desktop CPU (8 threads): 1.41 s; phones are several times slower.

The tokenizer gives the same ids as Hugging Face tokenizers on all 720 test texts, and the prompt is token for token the model's own chat template. The Kotlin code (JVM, ONNX Runtime 1.31.1) and the Swift code give the same text as the Python reference on the test cases.

How the refiner works

python/qwen_refine.py is the readable reference (numpy, onnxruntime and regex); the Kotlin and Swift code follow it step by step.

1. Tokenizer: NFC, Qwen2's pre-tokenizer regex, byte-level BPE. Special tokens are never parsed from the transcript: the prompt inserts their ids, so a transcript cannot start a new chat turn.
2. Numbers: spoken numbers of two or more words are written in digits (hai nghìn không trăm hai mươi sáu → 2026), then dates, decimals, percentages, units and phone numbers the Vietnamese way (ngày 20/11/2026, tháng 4/2025, 3,2, 20%, 12 km, 0912345678, 2 phòng), only where the words can mean nothing else. A small model adds numbers up wrong far more often than it copies digits.
3. Prompt: a system message and four example turns (fillers, a spoken list with a decimal and a unit, a question with a date that must not be answered, misheard words and a percentage, unknown names kept). They are the same for every transcript, so their KV cache is computed once when the model loads.
4. Decoding: greedy, until <|im_end|>, at most 1.5 × the transcript's tokens + 32.
5. Merge: small models reword, drop words and "correct" names they do not know. The output is aligned word by word with the transcript (longest common subsequence) and only trusted edits are kept: punctuation, capitals, line breaks and bullets; fillers (ờ, ừm, …), stutters (tôi tôi) and list markers (thứ hai là) left out; a word replaced by a similar-sounding one (hà lội → Hà Nội, thể giục → thể dục: gi/d/r, tr/ch and x/s count as the same sound, then at most one letter apart, two when both words are long, ignoring tone marks); a spoken number written in digits with the same value, Vietnamese style (2.300.000, 3,5, 20% for phần trăm, 0912345678, ngày 20/11/2026, tháng 4/2025); a spoken unit written as its symbol right after a number (ki lô gam → kg, mét vuông → m², độ c → °C). Anything else is undone with the transcript's own words, and when less than half of them come through (the model answered instead of correcting) the transcript is kept as it is. Last, hesitation sounds the model kept (ờ, ừm) are left out and a yes/no question the model closed with a full stop (có … không, được không, đã … chưa) gets a question mark.

Android (Kotlin)

Copy android/QwenRefiner.kt into the app. It uses only the Kotlin standard library and the ONNX Runtime Java API.

// build.gradle.kts: implementation("com.microsoft.onnxruntime:onnxruntime-android:1.22.0") or newer
// dir: a directory with vocab.txt, merges.txt, onnx/genai_config.json, onnx/model.onnx, onnx/model.onnx.data
val refiner = QwenRefiner.load(dir, threads = 4)   // keep one instance: loading takes a few seconds
val text = refiner.refine("thủ tướng phạm minh chính đã tiếp bộ trưởng nông nghiệp mỹ tại hà lội vào sáng nay")
// "Thủ tướng Phạm Minh Chính đã tiếp Bộ trưởng Nông nghiệp Mỹ tại Hà Nội vào sáng nay."

Call refine off the main thread. The model needs about 2.5 GB of RAM while it runs.

iOS (Swift)

Add the Swift package in ios/ (it depends on onnxruntime-swift-package-manager), or copy ios/Sources/QwenRefiner/*.swift into an app that already uses ONNX Runtime (SPM module OnnxRuntimeBindings or CocoaPods onnxruntime-objc, 1.22 or newer).

import QwenRefiner

let refiner = try QwenRefiner(directory: dir, threads: 4)
let text = try refiner.refine("thủ tướng phạm minh chính đã tiếp bộ trưởng nông nghiệp mỹ tại hà lội vào sáng nay")

The Swift code was compiled and checked on Linux (Swift 6.0) against a stand-in for the ONNX Runtime module that replays the Python reference's generated tokens, not yet on an iPhone.

Python

from huggingface_hub import snapshot_download
import sys

repo = snapshot_download("pdnhan/Arcee-VyLinh-mobile")
sys.path.insert(0, f"{repo}/python")
from qwen_refine import Refiner  # numpy, onnxruntime>=1.22, regex

refiner = Refiner(repo)
print(refiner.refine("bạn có biết mấy giờ rồi không"))  # Bạn có biết mấy giờ rồi không?

How it was made

The scripts are in scripts/: export.py runs the ONNX Runtime GenAI model builder and writes the tokenizer files, asr_hyps.py transcribes the FLEURS clips with PhoWhisper, validate.py checks the tokenizer and the prompt and measures the table above. The Kotlin and Swift checks are in scripts/kotlin and scripts/swift-check.

License and credit

Changes by ITGRATE Vietnam: the original weights were converted to ONNX and quantized to 4-bit (int4); nothing else in the model was changed. Apache 2.0, the license Arcee AI publishes arcee-ai/Arcee-VyLinh under (see LICENSE); the code in this repo is shared under the same license. All credit for the model goes to Arcee AI, qnguyen3 and the Qwen team (Built with Qwen). FLEURS (CC BY 4.0) and PhoWhisper (BSD-3-Clause) were used for evaluation only.