hammertoe/BimOmni-30B-A3B

BimOmni 30B-A3B

BimOmni is a Barbados-adapted version of Qwen3-Omni-30B-A3B-Instruct.

It was developed to test a simple idea: when audio is ambiguous, can stronger knowledge of Barbados help a multimodal model choose the correct local name, place, institution, event, or phrase?

BimOmni keeps Qwen3-Omni's audio and visual input towers, but adapts the Thinker's language model through text-only domain-adaptive pretraining on a Barbados newspaper archive.

This repository contains the fused bf16 checkpoint. The trained LoRA has been merged into the base weights and unloaded.

BimOmni hears Barbados properly.

Project background

BimOmni was developed as part of Future Caribbean, a regional initiative connecting technology, talent, and opportunity across the Caribbean through a global agentic AI buildathon.

The model is one component of Pulse, a public-signal intelligence system for Barbados. Pulse ingests fragmented social, broadcast, and news signals, then resolves them into a graph of sources, places, events, people, and communities.

The model itself is not tied to Pulse. It can be used independently for Caribbean-domain transcription, information extraction, search, summarisation, research, and other multimodal applications.

Why BimOmni?

Automatic speech recognition works well until somebody mentions a local school, village, politician, festival, restaurant, performer, or cricket ground.

At that point, transcription is not purely acoustic. The model must choose between several plausible sequences.

For Barbados, useful context includes names such as:

• Kensington Oval
• Cave Hill
• Samuel Jackman Prescod Polytechnic
• Crop Over
• Oistins Fish Fry
• Richard Haynes Boardwalk
• Puddin' and Souse
• Animal Flower Cave

BimOmni was trained to strengthen that local language prior:

ambiguous audio + stronger Barbados context -> better local word choice

It is important to be precise about what this means. BimOmni was adapted using text, not paired Barbadian audio and transcripts. It does not have a newly trained acoustic encoder and should not be described as having learnt a Barbadian accent. The intended improvement is in how the Thinker interprets ambiguous multimodal evidence and selects its output tokens.

Training

BimOmni uses domain-adaptive pretraining, or DAPT. Rather than converting the corpus into question-and-answer pairs, the model continued its next-token training on cleaned Barbados newspaper text.

The working corpus was built from 1,286 extracted editions of the Barbados Advocate, covering 2013 to 2023. The initial prepared snapshot contained approximately 51.6 million tokens.

The corpus is not redistributed.

Architecture and recipe

The training adapted Qwen3-Omni's Thinker:

• Base model: Qwen/Qwen3-Omni-30B-A3B-Instruct
• Base revision: 26291f793822fb6be9555850f06dfe95f2d7e695
• Method: LoRA domain-adaptive pretraining
• LoRA targets: attention and MLP/MoE projections
• Trainable parameters: approximately 2.59 billion
• Trainable proportion: approximately 7.54%
• Context length: 4,096 tokens
• Precision: bfloat16
• Final schedule: 1,624 of 1,624 planned optimiser steps
• Audio and visual encoders: frozen
• Talker: disabled

The V4 schedule was completed on a rented NVIDIA H200 through Hugging Face Jobs. Checkpoints were persisted separately so that the run could resume after its original 12-hour training window ended at step 1,000.

Evaluation

Barbados knowledge evaluation

The text evaluation contains 60 four-choice completion probes:

• 20 local Barbados probes
• 20 rare-local probes
• 20 general-knowledge controls

Each answer is scored using its mean token log-probability rather than asking the model to generate a multiple-choice letter.

Model	Training recipe	Schedule	Overall accuracy
Qwen3-Omni base, 4-bit	No adaptation	-	61.7%
V2	Attention-only LoRA	500 / 801	55.0%
V3	Attention + MLP/MoE LoRA	1,000 / 1,624	70.0%
BimOmni (V4)	Attention + MLP/MoE LoRA	1,624 / 1,624	75.0%

Re-running the base, V3, and V4 artefacts on the same MLX evaluation stack reproduced their original results to zero percentage points.

BimOmni improved by 13.3 percentage points overall compared with the unadapted 4-bit base model.

Track	Qwen3 base	V3	BimOmni
Local	55.0%	65.0%	70.0%
Rare local	40.0%	55.0%	60.0%
General controls	90.0%	90.0%	95.0%
Overall	61.7%	70.0%	75.0%

This is a small, project-specific knowledge probe rather than a broad measurement of cultural understanding.

TikTok transcription comparison

We also compared three local transcription paths on two Barbados TikTok videos:

• Whisper: the Whisper transcription path used by Pulse
• Qwen3: unadapted Qwen3-Omni-30B-A3B-Instruct, 4-bit MLX
• BimOmni: BimOmni-30B-A3B-MLX-4bit

The test used 29-second audio windows with five seconds of overlap. Audio was normalised to 16 kHz mono before inference. Overlapping chunks were stitched after transcription.

Video	Whisper	Qwen3	BimOmni	Reported character similarity
Itinerary	386 words / 7.8 s	387 words / 30.7 s	384 words / 25.8 s	0.372
Harbour Lights	206 words / 2.8 s	208 words / 12.4 s	207 words / 11.2 s	0.564

Whisper was the fastest path on both samples. BimOmni was faster than the unadapted Qwen3 model, but its main advantage was the handling of Barbados-specific proper nouns.

Itinerary proper nouns

The following manually reviewed examples come from the itinerary video used in the comparison.

Reference	Whisper	Qwen3	BimOmni
Worthing Square	Wooding Square	Willing Square	Worthing Square
Chefette	Shafet	Chefette	Chefette
Richard Haynes	Richard Hines	Richard Hinds	Richard Haynes
Q.P. Bistro	Kupi Bistro	QPB Bistro	Q.P. Bistro
Oistins Fish Fry	Oistin's Fish Fry	Oyston's Fish Fry	Oistins Fish Fry
Puddin' and Souse	putting on south	Puddin' and Sauce	Puddin' and Souse
Farley Hill	Farley Hill	Carlisle Hill	Farley Hill
Animal Flower Cave	Animal Flower Cave	Animaux Flower Cave	Animal Flower Cave
Carlisle Bay	Carlyle Bay	Carlisle Bay	Carlisle Bay
Champers	Champas	Champers	Champers

Model	Correct examples
Whisper	2 / 10
Qwen3	3 / 10
BimOmni	10 / 10

These ten examples were selected to examine local proper-noun handling. They are not a complete word-error-rate benchmark, and the two-video comparison should not be interpreted as a general ranking of transcription systems.

The practical result is narrower:

• Whisper remained substantially faster.
• The unadapted Qwen3 model corrected some local names but was the slowest.
• BimOmni produced the strongest Barbados-specific proper nouns in these samples.

Model structure

Component	Status
Thinker language model	Adapted and fused
Audio input tower	Preserved
Visual input tower	Preserved
Text output	Preserved
Talker and speech output	Removed
enable_audio_output	false

The model accepts text, audio, image, and video inputs, but produces text only.

Usage

Download the fused bf16 checkpoint:

hf download hammertoe/BimOmni-30B-A3B \
    --local-dir BimOmni-30B-A3B

Use the same processor and prompt structure documented for the original Qwen3-Omni model.

For local Apple Silicon inference, use the 4-bit MLX snapshot:

hf download hammertoe/BimOmni-30B-A3B-MLX-4bit \
    --local-dir BimOmni-30B-A3B-MLX-4bit

For audio with mlx-vlm, keep individual audio windows below 30 seconds. The project's validated transcription path uses 29-second windows with a five-second overlap and 16 kHz mono audio.

Limitations

• BimOmni was adapted using newspaper text, not paired Barbadian speech data.
• It should not be treated as proof of improved accent or dialect recognition.
• The newspaper corpus contains OCR errors and reflects the editorial choices and historical biases of its source.
• The model may over-prioritise familiar Barbados entities when acoustic evidence is weak.
• The knowledge evaluation contains only 60 probes.
• The transcription comparison contains only two videos.
• The proper-noun table contains a deliberately selected set of ten local entities and is not a general ASR benchmark.
• Latency depends on hardware, software versions, quantisation, chunking, and generation settings.
• The model's speech-generation components were removed. It cannot produce audio output.
• As with the base model, generated information should be verified before use in consequential settings.

Training story

The project and its training process are documented in two articles:

1. Teaching an Audio Model More About Barbados
   The motivation, corpus preparation, first training recipe, initial knowledge evaluation, and the inference bug that initially made the adapter appear inactive.

2. From a half-finished training run to a reproducible eval
   The broader V3/V4 recipe, checkpoint recovery, completed 1,624-step schedule, reproducibility work, and text, radio, and TikTok evaluations.

Author

Created by Matt Hamilton.

• Website: dharach.com
• Email: matt@dharach.com

Provenance

• Project: Future Caribbean
• System: Pulse
• Author: Matt Hamilton
• Base model: Qwen/Qwen3-Omni-30B-A3B-Instruct
• Base revision: 26291f793822fb6be9555850f06dfe95f2d7e695
• Adapter: hammertoe/Qwen3-Omni-30B-A3B-Barbados-LoRA-v4
• Fused bf16: hammertoe/BimOmni-30B-A3B
• MLX 4-bit: hammertoe/BimOmni-30B-A3B-MLX-4bit

Licence

The base model identifies its licence as Apache-2.0. Users should review the licence and terms of the original Qwen3-Omni model before redistribution or deployment.