lEtoileNoir/Hausa_English_Direct_S2TT

Hausa → English direct speech-to-text translation (C1 development champion)

This is a research-only direct Hausa-speech→English-text model. It is a standalone Transformers SpeechEncoderDecoderModel: Hausa waveform → XLS-R encoder → internal cross-attention → mBART English decoder. It does not call an external Hausa ASR or machine-translation system.

The C1 package uses the same frozen weights as tested revision 571430cc00b0321b997bde01e98f1a3fc9e093c7, with one decoding change: no_repeat_ngram_size=3. No additional trained weights are claimed.

This release is intended to replace the small Whisper direct-pilot cell in tsuxalo/Spoken-Language-Translation-Model, but it is not a drop-in Whisper/PEFT adapter. C1 uses an XLS-R encoder and mBART decoder, so consumers must load the standalone model and processor as shown below.

Intended use and limitations

Use for research, diagnostics, and reproducibility only. Do not use for medical, legal, safety-critical, high-impact, or unattended translation. The model often produces fluent but semantically inaccurate English and remains weak on names, numbers, dates, negation, and very short audio. Automatic diversity improvements do not establish translation correctness.

Development evidence

All new C1 results use the complete 1,037-example internal validation set derived from NaijaS2ST project train. This set influenced model development; results are not independent generalization evidence.

Metric	C1	Baseline	Paired C1−baseline 95% CI
chrF++	16.6682	16.2896	+0.2412 to +0.5316
SacreBLEU	0.4992	0.5008	−0.1040 to +0.0600
SSA-COMET	0.2101	0.1885	+0.01745 to +0.02548

C1 has 0% within-output repetition, 3.95% generic-response reuse, 72.32% raw unique outputs, and 100% reference-adjusted unique outputs. chrF++ and BLEU are corpus metrics; SSA-COMET uses pinned McGill-NLP/ssa-comet-mtl@6e64e0a56ce69524c67f304b092725687a362ef8 with pinned Davlan/afro-xlmr-large-76L@14a5ee38911f7f9992cf7a1c5fcd1588d9d14319.

The historical 1,500-example official-dev evaluation was run before this decoding change and remains sealed. It is not presented as a C1 package result and must not be rerun for tuning.

Generation configuration

{
  "num_beams": 5,
  "max_new_tokens": 128,
  "early_stopping": true,
  "no_repeat_ngram_size": 3,
  "repetition_penalty": 1.0,
  "length_penalty": 1.0
}

Minimal inference

import soundfile as sf
import torch
from scipy.signal import resample_poly
from transformers import SpeechEncoderDecoderModel, Wav2Vec2Processor

model_id = "lEtoileNoir/Hausa_English_Direct_S2TT"
processor = Wav2Vec2Processor.from_pretrained(model_id)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = SpeechEncoderDecoderModel.from_pretrained(model_id).to(device).eval()
audio, sampling_rate = sf.read("hausa.wav", dtype="float32", always_2d=True)
audio = audio.mean(axis=1)
if sampling_rate != 16_000:
    audio = resample_poly(audio, 16_000, sampling_rate)
    sampling_rate = 16_000
inputs = processor(audio, sampling_rate=sampling_rate, return_tensors="pt").to(device)
with torch.inference_mode():
    tokens = model.generate(**inputs)
print(processor.batch_decode(tokens, skip_special_tokens=True)[0])

Do not pass Whisper-only arguments such as language="hausa" or task="translate", and do not wrap this release with PeftModel. The merged weights and C1 generation defaults load directly from this repository. For a stable project integration, pin the immutable Hub commit reported for the release instead of following main indefinitely.

Provenance and reproducibility

• Base: facebook/wav2vec2-xls-r-300m-21-to-en@0e06afa94bacd448b178007cb85b95a92a59a0da
• Dataset: McGill-NLP/NaijaS2ST@898f51582750fe244693794f22e3f4b32c5baf95
• Source adapter SHA-256: ea34c4f539e260807438c150cea330833d60d281d773590695345ab12d400800
• C1 config SHA-256: 27f1a0179990cc718485ccaf95c1cb863badc382a031b30a11ea170822d8c9b8
• Selection artifact SHA-256: e4c8be2aec56609d0aaf57189528a7019ea359c0f08a00974ba710c8d2cb6a3e
• Generation-object SHA-256: 5b2698adf98a883a5399a00e8253abd36ae0054d89cdb13a03fbfef0dfeb7f76

The package manifest records byte counts and SHA-256 hashes for every distributed file. No protected row-level evaluation text is included.

License and attribution

The model code and weights are distributed under Apache-2.0, matching the pinned base model. Training data came from McGill-NLP/NaijaS2ST, which is released under CC BY 4.0 and requires attribution. Dataset authors describe it as the African Celtic Speech Dataset; use the citation and attribution information in its dataset card. No NaijaS2ST audio or row-level translations are redistributed in this model repository.

frozen_evidence.json is an immutable selection-time snapshot, so its pre-publication upload status describes the moment of freezing rather than the current Hub state.