Translation — audio layer

Audio / video → Whisper transcript → Buthaina's text translation layer.

Built for an RTX 5090 on Vast.ai (large-v3, float16). A 4 GB laptop GPU auto-falls back to small or CPU.

Vast.ai (RTX 5090)

Rent a 5090 image with CUDA 12.8+. Then:

git clone https://huggingface.co/mohameddalii/translation-audio
cd translation-audio
chmod +x scripts/run_vast.sh
./scripts/run_vast.sh

Smoke test after the server is up:

python -m audio sample.wav --transcribe-only
# or
curl -s http://127.0.0.1:8000/health
curl -F "file=@sample.wav" -F "translate=false" http://127.0.0.1:8000/translate

Load test concurrent users (CLI):

python -m audio.loadtest --file sample.wav --users 8 --requests 40 --no-translate

Locust UI (keep uvicorn running in another terminal):

pip install locust
locust -f locustfile.py --host http://127.0.0.1:8000 --web-host 0.0.0.0 --web-port 8089

On Vast Jupyter, try https://<your-jupyter-host>/proxy/8089/. Start with 4 users, spawn rate 1. The GPU serializes ASR, so extra users wait or get 429.

Env vars (already set in scripts/run_vast.sh):

Variable	5090 value
`ASR_BACKEND`	`whisper` or `cohere`
`ASR_MODEL`	`large-v3` (Whisper only)
`ASR_DEVICE`	`cuda`
`ASR_COMPUTE_TYPE`	`float16` (do not use int8 on Blackwell)
`SKIP_TRANSLATION`	`true` until the text service is up

Whisper vs Cohere (restart the server between runs — one backend per process):

python -m audio sample.wav --transcribe-only --backend whisper > whisper.json
python -m audio sample.wav --transcribe-only --backend cohere > cohere.json
python -m audio.compare whisper.json cohere.json

Layout

audio/ — ASR, ffmpeg video→wav, FastAPI, GPU queue, load test
shared/contracts.py — payload sent to the text layer
text/stub.py — stand-in for Buthaina's translator

Downloads last month: -; Downloads are not tracked for this model. How to track