DontHash/neptrans-v4-lora

NepTrans v4 — Nepali–English Speech Recognition (LoRA Adapter)

This repository contains the original NepTrans v4 LoRA adapter, approximately 115.5 MB, for openai/whisper-large-v3. It is intended for Transformers/PEFT users and further adaptation. It is not a standalone model: loading it also requires the approximately 1.55B-parameter base model and its processor.

For straightforward self-hosted transcription, use the complete NepTrans v4 CTranslate2 model instead. That release already merges the adapter with the base weights.

Source code and self-hosting guide: DontHash/neptrans-asr. Includes the inference helper, CPU/GPU/offline instructions, and versioned release notes.

Quick start with Transformers and PEFT

python -m pip install torch "transformers==4.57.6" "peft==0.21.0" accelerate soundfile

The example below expects a mono or stereo 16 kHz WAV clip no longer than 30 seconds. It averages stereo channels before preprocessing. Use a suitable audio decoding/resampling pipeline and chunking for longer recordings. The original adapter and tokenizer are distributed without pickled training state.

import soundfile as sf
import torch
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor

BASE = "openai/whisper-large-v3"
BASE_REVISION = "06f233fe06e710322aca913c1bc4249a0d71fce1"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32

processor = WhisperProcessor.from_pretrained(BASE, revision=BASE_REVISION)
base = WhisperForConditionalGeneration.from_pretrained(
    BASE, revision=BASE_REVISION, torch_dtype=dtype,
)
model = PeftModel.from_pretrained(
    base, "DontHash/neptrans-v4-lora", revision="v4.0.0",
).to(device).eval()
model.generation_config.forced_decoder_ids = None

audio, sr = sf.read("clip_16khz.wav", dtype="float32")
if sr != 16000:
    raise ValueError("Resample the input to 16 kHz first.")
if audio.ndim == 2:
    audio = audio.mean(axis=1)
if len(audio) == 0 or len(audio) > 30 * sr:
    raise ValueError("Provide a nonempty clip no longer than 30 seconds.")
inputs = processor(audio, sampling_rate=sr, return_tensors="pt", return_attention_mask=True)
with torch.inference_mode():
    ids = model.generate(
        input_features=inputs.input_features.to(device=device, dtype=dtype),
        attention_mask=inputs.attention_mask.to(device),
        language=None, task="transcribe", return_timestamps=False,
        num_beams=1, max_length=448,
    )
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Download and cache both the base model and adapter before offline use, or load both from local directories. For reproducibility, retain the pinned base revision and the adapter release tag. Do not substitute a different Nepali Whisper checkpoint: this adapter was trained against the original openai/whisper-large-v3 weights.

The base model requires substantially more memory than the adapter file alone suggests. Full-size Transformers inference has different resource requirements from the quantized CT2 release. The included adapter configuration pins the original base revision; the adapter tensor file is byte-for-byte identical to the verified v4 export source. The packaging checks cover tensor loading and processor compatibility; the usage example's full inference check is recorded separately in the release verification report.

Model overview

NepTrans v4 is a Nepali–English automatic speech recognition model, fine-tuned from OpenAI Whisper large-v3 using expanded LoRA adaptation. It is intended for Nepali speech and conversations that switch between Nepali and English, including draft notes, interview transcripts, and subtitles that a person reviews before use.

The model is shared by DontHash. Download it once and run inference on your own computer or server. Audio processed locally stays within your chosen runtime; this model does not call a NepTrans transcription service. The initial model download requires internet access unless the files have already been transferred locally.

Property	Value
Task	Automatic speech recognition / transcription
Target languages	Nepali (ne) and English (en), including code-switching
Architecture	Whisper large-v3 encoder–decoder, approximately 1.55B base parameters
Base checkpoint	openai/whisper-large-v3
Base revision	06f233fe06e710322aca913c1bc4249a0d71fce1
Fine-tuning method	LoRA, rank 16, alpha 32, dropout 0.05
LoRA modules	q_proj, k_proj, v_proj, out_proj, fc1, fc2
Audio features	16 kHz audio, 128 mel bins
Release	v4.0.0, 20 October 2026

Training data and method

The v4 training manifest contains 21,470 clips / 49.094 hours:

Training component	Clips	Hours
Resegmented conversational Nepali and Nepali–English speech	8,075	approximately 34.09
Curated OpenSLR-54 Nepali read speech	13,395	approximately 15.00
Total	21,470	49.094

The training manifest has 14,473 Nepali-only, 6,613 mixed-language, and 384 English-only clips. Development uses 652 clips / 2.626 hours; the internal formal test uses 625 clips / 2.587 hours. These counts and durations were recomputed from the saved manifests when preparing this release.

The training recipe specifies three epochs, learning rate 5e-5, batch size 1, gradient accumulation 16, BF16 mixed precision, linear learning-rate scheduling, 50 warmup steps, AdamW, and seed 42. The saved final trainer state records three completed epochs and 4,026 optimizer steps, selecting checkpoint 4,026 using the lowest development target-weighted WER. The recipe and saved state are distinct sources: intermediate evaluation and save intervals changed during the run. The reported training runtime is approximately 7.89 hours; carbon emissions were not measured.

Training audio, transcripts, per-recording predictions, and private manifests are not distributed in these repositories. OpenSLR-54 is credited below. The conversational component is a project-curated corpus; this release does not claim that every label was independently human-verified or that the corpus represents all Nepali speakers, accents, or recording conditions. The full training dataset is not publicly downloadable from this model page.

Evaluation

The following are saved evaluation results from the v4 Transformers/PEFT training run, not a new evaluation of the CTranslate2 export. Lower values are better.

Split	Clips	WER (%)	CER (%)	Custom target-weighted WER (%)
Internal development, checkpoint 4,026	652	28.17	11.16	31.72
Internal formal test	625	36.47	19.24	44.15

WER is the corpus word error rate, (substitutions + deletions + insertions) / reference words; CER uses characters. The custom selection score is 0.60 × mixed WER + 0.20 × Nepali-only WER + 0.20 × English-only WER. It is a workload-priority score and must not be interpreted as pooled WER or word accuracy.

The formal test composition is:

Bucket	Clips	Reference words	Share of reference words
Mixed Nepali–English	360	17,137	62.56%
Nepali-only	75	1,141	4.17%
English-only	190	9,117	33.28%

The saved evaluator uses jiwer.wer and jiwer.cer directly on reference and decoded text, without an additional custom punctuation, case, or spelling normalization pass. Generation uses automatic language selection, the transcription task, greedy decoding (num_beams=1), maximum sequence length 448, and no timestamp tokens. The included evaluation/protocol.py preserves the relevant scoring and generation logic without publishing any recordings or transcripts.

Interpretation and evidence limits:

• These are internal project splits, not FLEURS, Common Voice, or public leaderboard results. No state-of-the-art ranking is claimed.
• Split composition is uneven. Overall WER alone does not establish Nepali-only or mixed-language quality; per-bucket v4 error rates were not present in the saved aggregate evaluation files.
• The full evaluation was not repeated during packaging. The CT2 int8_float16 export has separate functional smoke tests; its complete test-set WER/CER have not been established.
• The formal evaluation's adapter-file hash was not recorded in its metrics JSON. The release links the exported adapter and CT2 weights through saved export provenance and verified hashes; the metric-to-file linkage is at the recorded training-run level, not an independently reproduced benchmark.
• No independent source/speaker leakage audit, confidence intervals, or broad accent-fairness evaluation is asserted by this release.
• Runtime numbers from the training/evaluation environment do not establish streaming latency or performance on a recipient's machine.

Machine-readable copies of the saved metrics, manifest aggregates, and checkpoint selection are in evaluation/summary.json.

Intended uses and limitations

Use the model for personal or research transcription, draft subtitles, and local applications with human correction. Evaluate it on representative recordings before relying on it in a workflow.

Expected failure modes include incorrect names, numbers, English technical words, code-switch boundaries, punctuation, and script choice. Short recordings can trigger wrong language detection. Whisper-family models can repeat or hallucinate text over silence, music, noise, or unclear speech. Overlapping speakers and distant microphones may reduce quality.

This is a transcription model: it does not provide speaker identity, diarization, source separation, or verified factual understanding. Timestamp output from an inference library is an alignment estimate, not manually confirmed timing. Neither translation quality nor retention of all other Whisper languages has been evaluated for this fine-tune.

Review transcripts before publication or consequential use. The reported error rates do not support unattended high-stakes transcription. Users are responsible for obtaining permission to process recordings and for securing any application they host.

Provenance, licenses, and attribution

The released model artifacts use Apache-2.0, following the pinned Hugging Face base checkpoint's declared license. See LICENSE. The included helper code is MIT-licensed; see CODE_LICENSE. Upstream Whisper's original MIT notice is preserved in WHISPER_LICENSE, and the third-party sources are documented in NOTICE.

The OpenSLR-54 training component is Large Nepali ASR training data set, Copyright 2016, 2017, 2018 Google, Inc., distributed under CC BY-SA 4.0. This fine-tune used a curated approximately 15-hour subset. Original dataset: OpenSLR-54. Its source license is preserved in OPENSLR54_LICENSE; dataset licenses remain separate from the model and helper-code licenses. No training audio or text is included in this release.

release_manifest.json records SHA-256 checksums for the distributed files, the upstream revision, quantization where applicable, and package versions used for release verification. Export provenance excludes local account paths and private source locations.

Citation

@misc{donthash2026neptransv4,
  author = {DontHash},
  title = {NepTrans v4: Whisper Large-v3 Adaptation for Nepali-English Transcription},
  year = {2026},
  url = {https://huggingface.co/DontHash/neptrans-v4-ct2},
  note = {Version v4.0.0; companion LoRA adapter: DontHash/neptrans-v4-lora}
}

@inproceedings{kjartansson-etal-sltu2018,
  title = {Crowd-Sourced Speech Corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali},
  author = {Kjartansson, Oddur and Sarin, Supheakmungkol and Pipatsrisawat, Knot and Jansche, Martin and Ha, Linne},
  booktitle = {Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU)},
  year = {2018},
  pages = {52--55},
  doi = {10.21437/SLTU.2018-11}
}

For Whisper, cite Robust Speech Recognition via Large-Scale Weak Supervision, Radford et al. For the inference runtime, see faster-whisper and CTranslate2. Report issues through this repository's Hugging Face Community tab, including runtime versions and a reproducible example you have permission to share.