BiMTokenizer

BiMTokenizer is a bidirectional Mamba speech tokenizer for low-bitrate neural speech coding. This repository provides four 16 kHz checkpoints from the same model family.

• Code: ZhangXinWhut/BiMTokenizer
• Audio demo: BiMTokenizer Demo

Authors

Xin Zhang, Lin Li, Chuanbo Liu, Jianquan Liu, and Kong Aik Lee.

Available checkpoints

Model	Checkpoint	Codebook	Bitrate
BiMTokenizer-Whisper	whisper/bimtokenizer_whisper_librispeech.pt	5 x 196560	1100 bps
BiMTokenizer-SenseVoice	sensevoice/bimtokenizer_sensevoice_librispeech.pt	5 x 196560	1100 bps
BiMTokenizer-SenseVoice (32768+4096)	sensevoice-32768-4096/bimtokenizer_sensevoice_32768_4096_librispeech.pt	32768 + 6 x 4096	1087.5 bps
BiMTokenizer-SenseVoice (8x2048)	sensevoice-2048/bimtokenizer_sensevoice_2048_librispeech.pt	1 semantic + 7 acoustic, each 2048	1100 bps

Each checkpoint is stored next to its matching config.yaml. Do not mix a checkpoint with a configuration from another variant.

LibriSpeech test-clean results

Model	SIM ↑	STOI ↑	PESQ-NB ↑	PESQ-WB ↑	UTMOS ↑	WER ↓
Ground Truth	1.00	1.00	4.55	4.64	4.09	2.16
BiMTokenizer-Whisper	0.87	0.95	3.56	3.03	4.21	2.44
BiMTokenizer-SenseVoice	0.85	0.94	3.45	2.85	4.18	2.53
BiMTokenizer-SenseVoice (32768+4096)	0.86	0.943	3.459	2.893	4.20	2.48

Download

Install the Hub client:

pip install -U huggingface_hub

Download one model variant together with its configuration and the repository manifest. Including the root config.yaml also allows Hugging Face to count the download at repository level.

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ZhangXinWhut/BiMTokenizer",
    allow_patterns=["config.yaml", "whisper/*"],
    local_dir="./BiMTokenizer-Whisper",
)

Replace whisper/* with sensevoice/*, sensevoice-32768-4096/*, or sensevoice-2048/* for the other checkpoints.

The complete repository can also be downloaded with:

huggingface-cli download ZhangXinWhut/BiMTokenizer \
  --local-dir ./BiMTokenizer-weights

Inference

Clone the code repository and place the downloaded checkpoint under ./weights/. Example:

git clone https://github.com/ZhangXinWhut/BiMTokenizer.git
cd BiMTokenizer

python inference.py \
  --config_path /path/to/whisper/config.yaml \
  --checkpoint_path /path/to/whisper/bimtokenizer_whisper_librispeech.pt \
  --input_dir /path/to/input_wavs \
  --output_dir output_wavs \
  --device cuda \
  --batch_size 1

The fixed Leech codebooks required by the configurations are included in the GitHub code repository under bimtokenizer/modules/quantizer/cache/.

Notes

• All released checkpoints operate at a 16 kHz sample rate.
• The public download counter is repository-level and does not identify individual downloaders.
• Download statistics may take time to appear after the first downloads.