LelandHenry/Labrador-v0.1

Labrador v0.1

Encoder LoRA for English speech recognition in noisy and reverberant rooms.

Labrador is an experimental adaptation of Qwen3-ASR-1.7B-hf, developed by Leland Henry to investigate how a small amount of diverse speech, simulated room acoustics, and guidance from a frozen teacher can improve recognition under acoustic degradation.

On the original noisy development panel, the retained adapter reduced word errors from 196 to 182, a 7.14% relative reduction in word error rate (WER). Results on the additional development panels were mixed. The table below reports both improvements and regressions.

Status: experimental research checkpoint. Official FFASR evaluation has not been completed; no leaderboard score or rank is claimed.

What is included

This repository contains the retained v08 A adapter (e_strong_teacher, update 512), its inference loader, training provenance, and development results. It uses automatic language detection. Later notebook numbers describe subsequent experiments and diagnostics; they do not identify the released weights.

The base model is downloaded separately at revision bcd2b5b7f32b480ab5790554cfa8347f246a14f3. Adapter weights are stored in safetensors and are not merged into the base model.

Method and training

• Adaptation: LoRA on the audio encoder's query and value projections across 24 layers; the original model parameters remain frozen.
• Adapter size: 786,432 trainable parameters. Training metadata records 2,038,052,480 frozen parameters, for 2,038,838,912 parameters in the complete system; “1.7B” is the upstream model's name.
• LoRA settings: rank 8, alpha 16, dropout 0.05.
• Optimization: 512 updates, four examples per update, peak learning rate 1e-5, eight warmup updates, linear decay, seed 20261026.
• Objective: transcript cross-entropy plus a weight-1 KL term on noisy/reverberant examples, guiding the student toward the frozen base model's predictions on paired clean speech.
• Speech pool: 892 complete LibriSpeech train-clean-100 utterances from 80 speakers, totaling 2.006 hours; 883 distinct utterances from all 80 speakers were actually presented during the completed run.
• Room augmentation: a pool of 2,205 Treble10 RIRs across eight rooms; Room4 and Room8 were reserved for local development. The completed run sampled 912 distinct RIRs and used one speech-free room-noise recording supplied for the project.
• Training presentations: 407 clean, 1,428 noisy/reverberant, and 213 no-speech examples, totaling 2,048 presentations.

AMI SDM was used for development evaluation, not for training this checkpoint. Frequency masking tested in a later experiment is not part of the released checkpoint. Detailed provenance is in labrador_adapter_metadata.json.

Development results

Both models use the same local text normalization and automatic-language decoding. Lower WER is better; parentheses show word errors divided by reference words.

Reused development panel	Base Qwen	Labrador v0.1
Original public clean	1.59% (12/754)	1.59% (12/754)
Original public noisy	13.00% (196/1508)	12.07% (182/1508)
Treble noisy	8.62% (130/1508)	8.62% (130/1508)
AMI SDM	15.58% (74/475)	15.79% (75/475)
Broader probe, clean	1.85% (32/1726)	1.91% (33/1726)
Broader probe, noisy	11.07% (382/3452)	11.10% (383/3452)
Personally recorded speech	1.68% (6/357)	1.68% (6/357)
Noise-only clips producing speech	0/16	0/16

These are development results used during experiment selection. The broader probe includes 96 source utterances from 24 speakers, each rendered under three conditions. It has been inspected during development and is not a blind test. The local normalization differs from FFASR's evaluation procedure.

Six of eight original project gates pass. The remaining gates concern personally recorded speech (6 errors, target at most 4) and Treble noisy speech (130 errors, target at most 103). These project gates are separate from a leaderboard score.

The observed improvement is specific to the original public-noisy panel. The additional results do not establish general superiority over base Qwen or Tiro. Full aggregate results are in development_results.json.

Reproducibility and decoding

A real Tesla T4 reload test reproduced the saved generated tokens and parsed outputs on 12 of 12 selected development cases, including four noise-only clips that remained blank. Two difficult speech cases also reproduced their earlier blank transcripts; the check verifies consistent loading and inference, not perfect recognition. See release_validation.json.

The verified environment used Python 3.13.16, PyTorch 2.11.0+cu130, Transformers 5.21.0, PEFT 0.21.2, FP16 base weights, FP32 adapters, and SDPA attention. Inference uses greedy decoding, a 256-token limit, and the pinned model's two EOS IDs, 151643 and 151645.

The loader preserves the recorded audio level and applies no VAD, resampling, denoising, forced-language hint, or transcript cleanup. Malformed or truncated generations raise an error. Input must be a finite, nonempty, mono 16 kHz WAV, and the provided loader requires CUDA.

Usage

In a CUDA environment compatible with the verified Colab setup:

python -m pip install "huggingface-hub==1.33.0"
hf download LelandHenry/Labrador-v0.1 --local-dir ./labrador
python -m pip install -r ./labrador/requirements_colab.txt
python ./labrador/labrador_inference.py ./example_16khz_mono.wav

Or import the loader after downloading and installing its dependencies:

from pathlib import Path
import sys

repo = Path("./labrador").resolve()
sys.path.insert(0, str(repo))
from labrador_inference import LabradorASR

with LabradorASR(adapter_dir=repo) as asr:
    print(asr.transcribe("./example_16khz_mono.wav"))

The bundled loader explicitly loads and verifies this adapter with the pinned native Transformers base. Use this loader to reproduce the documented settings.

Limitations and benchmark status

Evaluation covers short English clips and limited speakers, rooms, noise sources, and microphones. Long-form and streaming audio, other languages, and overlapping speakers have not been validated for this adapter. The small noise-only panel does not establish a general hallucination rate. Full training reproduction also requires the original data-preparation workflow; this repository packages inference and provenance.

FFASR compatibility remains to be checked: its current worker uses Python 3.12, an NVIDIA L4, and PyTorch below 2.11. The proposed extra dependencies in requirements_ffasr_draft.txt respect that constraint; the successful Colab test does not establish performance or compatibility in that separate environment. See the official worker configuration.

Forced-English decoding was rejected after it generated false speech on all 16 noise-only development clips while removing only one net speech-word error across 500 speech-containing cases. Automatic language detection is retained. Further work will examine fresh speakers and rooms, broader noise diversity, and phoneme-level error diagnosis.

License and attribution

Labrador's adapter weights and this model card are released under CC BY-NC-SA 4.0, subject to its attribution, noncommercial, and ShareAlike terms. Original project code is licensed separately under Apache-2.0; third-party components retain their own licenses.

Upstream materials:

• Qwen3-ASR-1.7B-hf: Qwen team; model card, Apache-2.0. The base weights are downloaded separately.
• LibriSpeech: Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, ICASSP 2015; OpenSLR 12, CC BY 4.0.
• Treble10-RIR: Sarabeth S. Mullins, Georg Götz, Eric Bezzam, Steven Zheng, and Daniel Gert Nielsen, Treble10: A high-quality dataset for far-field speech recognition, dereverberation, and enhancement, 2025; dataset, CC BY-NC-SA 4.0. Training used revision 5755ba8b59f648265f91bf73f58e88be43c220d0 to synthesize reverberant mixtures.
• AMI SDM: AMI dataset, development evaluation only.

Source recordings and room impulse responses are not redistributed in this repository. These notices preserve the separate upstream terms; this release does not claim unrestricted commercial-use permission. Labrador is an independent research project; no endorsement by an employer, Qwen, Treble Technologies, or the corpus creators is implied.