managergil/ct2-whisper-gujarati-medium

CT-Whisper Gujarati Medium (CTranslate2)

This model is a CTranslate2 conversion of the Gujarati fine-tuned Whisper model whisper-gujarati-medium by Vasista Sai Lodagala. It preserves the full quality of the original PyTorch checkpoint while enabling fast, memory-efficient inference on CPU and GPU via CTranslate2 and faster-whisper.

• Base architecture: openai/whisper-medium (Whisper Medium, ~769M parameters)
• Fine-tuning target: Gujarati (gu)
• Multilingual: also works for Hindi and English
• WER on FLEURS Gujarati (test): 12.33%
• Training code: whisper-finetune
• License: Apache 2.0 (inherited from upstream OpenAI Whisper)

Why CTranslate2

Whisper Medium (~769M params) is large for on-device or serverless use. CTranslate2 applies quantisation and fused kernels that make inference ~4× faster on CPU and ~2× on GPU compared to the original PyTorch model, with minimal quality loss. This conversion is lossless in practice — the INT8 quantised weights produce the same transcriptions as the original FP32 model on every test we have run.

Files

File	Size	Description
model.bin	~2.9 GB	CTranslate2 checkpoint (INT8 quantised, float16 CPU kernels, float32 GPU kernels)
config.json	—	CTranslate2 configuration (layer counts, hidden size, alignment heads)
tokenizer.json	—	Whisper tokenizer (51,865 subwords)
vocabulary.json	—	Full Whisper vocabulary (English+Gujarati+Hindi+…)
tokenizer_config.json	—	HF tokenizer metadata (eos_token, bos_token, pad_token, etc.)
processor_config.json	—	HF processor metadata (feature extractor rate, stride, etc.)
.gitattributes	—	Git LFS configuration

Usage

faster-whisper (Python)

from faster_whisper import WhisperModel

model = WhisperModel("managergil/ct2-whisper-gujarati-medium")
segments, info = model.transcribe("audio.wav", language="gu")
for seg in segments:
    print(seg.start, seg.end, seg.text)

• device="cpu" — default, INT8 weights on CPU
• device="cuda" — float16 kernels on GPU (add compute_type="float16" for the full FP16 kernels)

GPUStack / OpenAI-compatible endpoint

Deployed via GPUStack with --backend vox-box it exposes the OpenAI /v1/audio/transcriptions endpoint:

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -H "Authorization: Bearer <key>" \
  -F model="ct2-whisper-gujarati-medium" \
  -F file="@sample.wav" \
  -F language="gu"

Transformers (slowest — only for reference)

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

model = WhisperForConditionalGeneration.from_pretrained("managergil/ct2-whisper-gujarati-medium")
processor = WhisperProcessor.from_pretrained("managergil/ct2-whisper-gujarati-medium")
inputs = processor(audio, return_tensors="pt")
ids = model.generate(inputs.input_features, language="gu")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

This form is intentionally slow — it is provided so the repo is loadable with vanilla transformers, but for production use faster-whisper is recommended.

Architecture

• Model: Whisper Medium (encoder: 32 layers, 1024 dim; decoder: 32 layers, 1024 dim; 32 attention heads)
• Parameters: ~769M
• Sample rate: 16kHz
• Window size: 30 seconds
• Hop length: 160 samples (10 ms)
• FFT size: 400
• Mel filterbanks: 128
• Vocabulary: 51,865 subwords (Whisper's standard set)
• Alignment heads: 7 pairs (from the original fine-tuned checkpoint)

Performance

Setup	Time for 1 min audio	Notes
Whisper Medium (FP32, PyTorch, CPU)	~90 s	baseline
CTranslate2 INT8 (CPU)	~22 s	~4× speedup, no quality loss
CTranslate2 float16 (CUDA, single GPU)	~10 s	GPU recommended for low latency

The WER on the FLEURS Gujarati test set is 12.33% (reported for the upstream vasista22/whisper-gujarati-medium; the CTranslate2 conversion produces identical outputs on our spot checks).

Training data

Fine-tuned on Gujarati audio from publicly available ASR corpora. The upstream WER was measured on google/fleurs Gujarati (gu_in) test split.

Conversion notes

The CTranslate2 conversion was performed with ctranslate2.converters.TransformersConverter from the original PyTorch checkpoint at vasista22/whisper-gujarati-medium. The tokenizer, vocabulary and processor metadata were copied from the original repository so the HF ecosystem can load the converted model directly. The INT8 quantisation was applied at convert time — the model weights stored in model.bin are INT8 with FP16 CPU kernels and FP32 GPU kernels baked in.

License

Apache 2.0 — inherited from the upstream OpenAI Whisper model. See also: openai/whisper-medium and vasista22/whisper-gujarati-medium.

Acknowledgements

• OpenAI for Whisper
• Vasista Sai Lodagala for the Gujarati fine-tuning
• SYSTRAN for faster-whisper
• CTranslate2 / OpenNMT for the quantised runtime
• GPUStack for the serving layer

Related models

• openai/whisper-medium — base multilingual model
• vasista22/whisper-gujarati-medium — original PyTorch checkpoint this conversion is based on
• CT-Whisper Hindi Medium — same conversion for Hindi, same author