loom-ai-org/zipformer-small-loom

Zipformer small

Xiaomi's Next-gen Kaldi Zipformer (arXiv:2310.11230), small variant: a multi-rate transformer encoder with a stateless transducer, for Chinese and English, exported for loom.cpp in two precisions.

This is a loom.cpp export: a single self-describing GGUF that carries its own graph topologies, tokenizer (if any) and driver script, produced by loom-exporter.

Original model

Exported from pkufool/zipformer-small. The F32 file carries the same parameters unmodified, in loom.cpp's GGUF format; the others are quantized from them (see Files).

License

apache-2.0, inherited from the base model above.

Language(s)

zh, en

Usage

Run it with loom-py -- loom-py-rt on PyPI:

pip install -U "loom-py-rt[hub]"

import loom

# This repo holds more than one precision of the model; name the file you want (see "Files" below).
model = loom.Model.from_pretrained("loom-ai-org/zipformer-small-loom", "zipformer-small-f32.gguf")

# Audio is a mono float list at 16 kHz. This model decodes in the one language it was trained for and
# takes no `language=` argument -- passing one warns and is ignored, because nothing in its decode
# could act on it.
result = model.speech2text.infer(audio)
print(result.text)

The layer underneath

The call above is the high-level door: one per task, named for the modality pair it maps between, with the windowing, sampling and assembly this model needs already applied. Under it, model.infer(...) passes your arguments straight to the driver this GGUF embeds -- which is where you go for a knob the door does not name.

model.driver_source prints that driver, including a header comment documenting every argument it accepts for this model, and is the authority on it. See loom-py for the API and loom.cpp for what the engine does between the two.

Known limitations

The whole clip in one pass, not live streaming. This is the non-streaming checkpoint. pkufool/zipformer-small-streaming is a separate model and is not exported.

Cost grows faster than the clip. The first stack attends over every 20 ms frame. On a 4-thread laptop-class CPU, 16 s of speech took about 2.2 s; 60 s took 9.8 s and 1.8 GB; 120 s took 25 s and 3.4 GB. Split long audio at its pauses -- silero-vad-loom finds them.

On an M1 Pro, Metal is slower than the CPU (13x real time against 27x at F32); it is here for when the CPU is busy. Vulkan on a small integrated GPU (Radeon Vega 3) runs 3.8x at F32 and 6.4x at Q8_0, against 7x on that machine's CPU. CUDA has not been run.

The transducer head only. The checkpoint also has a CTC head, which upstream rates as less accurate (LibriSpeech test-clean 5.12% against 3.78%). It is not in these files.

Case and punctuation follow the training data. One model covers both languages and several corpora, so English comes back mostly upper-case and unpunctuated, and sometimes in mixed case with punctuation. Normalise before comparing. The language is not selectable: language= does nothing here.

No timestamps: segments is one span covering the whole clip. Mono 16 kHz.

Files

One model in several precisions; every file carries the same graph and driver. The quantized ones pack the convolution and projection weights (loom-export --quantize <type>), so their numbers differ from the original model's by the amount noted.

• zipformer-small-f32.gguf -- full precision, 140 MB. Token for token the reference's own greedy search on all 73 test clips, on CPU, Vulkan and Metal alike, and its encoder output to about 3e-5.
• zipformer-small-q8_0.gguf -- 58 MB. WER 7.30% to 7.39% against 7.22% on the same clips, depending on the backend, with 63 to 65 of 73 transcripts identical to F32, the rest mostly a spelling apart. Barely faster than F32 on an x86 CPU, about 1.2x on Apple silicon's CPU and Metal, and 1.7x on a small Vulkan iGPU. At most 7% less CPU memory: attention's working memory is the same size either way.