cstr/flova-omr-GGUF

Flova/omr_transformer — GGUF (for CrispEmbed)

GGUF conversions of Flova/omr_transformer, an end-to-end handwritten / whiteboard Optical Music Recognition model that transcribes monophonic "simple notes" staff images into LilyPond note strings (e.g. c'2 a''8 c''8 r4 c'1 …). It is the only permissively-licensed (Apache-2.0) end-to-end handwritten-music OMR model, and complements the printed-music engines (SMT, Polyphonic-TrOMR) already in CrispEmbed.

Architecture

A Donut Vision-Encoder-Decoder (~143M params):

• Encoder — DonutSwin (Swin-Base scale): patch 4, window 10, embed_dim 128, depths [2,2,14,2], heads [4,8,16,32], hidden 1024, input 583×409.
• Decoder — 4-layer pre-norm mBART: d_model 1024, 16 heads, ffn 4096, vocab 75, learned positions (offset +2), scale-embedding (×√1024), GELU. decoder_start/bos = 56 (<s>), eos = 54 (</s>), pad = 55.

Files

File	Size	Notes
flova-f32.gguf	573 MB	Full precision (reference).
flova-q8_0.gguf	162 MB	Recommended. Decodes byte-identically to the f32 / HF model on the sample images.

The Swin encoder is all-linear, so q8_0 needs no keep-guard: the patch-embed convolution and the relative-position index table are kept at F32 automatically.

Parity

Validated per-stage against the HF reference (CPU): every stage cosine 1.000000 (encoder stages + output, all 4 decoder blocks, logits), 40/40 teacher-forced argmax agreement, and byte-exact greedy decode matching the model card. q8_0 reproduces the exact LilyPond output on all three sample images.

Usage (CrispEmbed)

# auto-downloads flova-q8_0.gguf on first use
crispembed --model flova --ocr score.png
# → c'2 a''8 c''8 r4 c'1 e'8 c'8 c'8 a''8 f'4 a'8 c'8

or with an explicit path:

crispembed -m flova-q8_0.gguf --ocr score.png

CrispEmbed: https://github.com/CrispStrobe/CrispEmbed

License & attribution

Apache-2.0, inherited from the source model.

• Model: Flova/omr_transformer
• Blueprint / project: UHHRobotics22-23/robot_project (marimbabot_vision)

These are format conversions only; all model weights and credit belong to the original authors.