owenmorgan/bookforge-rubric

BookForge Rubric

A block-category model for book page layout. Given every text block on a page — its geometry, type size, line count and text — it labels each one with what it is: body text, a chapter opening, a running head, a footnote, a caption, a table fragment, and so on.

BookForge uses it to turn a scanned book into a clean EPUB: running heads and page numbers have to be dropped, footnotes separated from body prose, chapter openings found (they become the EPUB split points and the M4B chapter marks), and tables and indexes kept out of the narration.

The thirteen categories

body        title       chapter     heading     subheading
quote       caption     footnote    header      footer
image       table       list

list is entry-per-line structured content — a table of contents, an index, a bibliography, a glossary, a chronology, ordinary bullets. table stays table even when OCR has shredded it into fragments.

Files

File	Size	Notes
rubric-v3-4b-Q4_K_M.gguf	2.5 GB	Qwen3-4B QLoRA fine-tune, merged and quantized. The current model.

Running it

Serve with llama-server and generate on /completion — not the chat endpoint:

llama-server --model rubric-v3-4b-Q4_K_M.gguf -c 8192 -np 1

curl -s http://127.0.0.1:8080/completion -d '{
  "prompt": "<the fully-templated prompt>",
  "temperature": 0,
  "n_predict": 2048,
  "stop": ["<|im_end|>"],
  "cache_prompt": true
}'

The prompt must arrive verbatim. Training used Qwen3's chat template with thinking disabled, which inserts an empty <think>\n\n</think> block that stock templates omit — so a server that builds the prompt itself feeds the model a shape it never saw, and the answers degrade in a way that looks like a bad model rather than a bad client. /completion takes the prompt as given; chat endpoints do not. cache_prompt is worth setting: every page shares the same long system prefix.

Answers are one line per block, <block-id> <category>, terminated by <|im_end|>. Blocks the model does not answer for are left unlabelled rather than guessed.

Context

The longest real page measured 6,529 tokens, so -c 8192 covers a page with room for the answer. Set it explicitly: a host with a smaller default silently TRUNCATES an over-long prompt, and the model then answers about blocks it never saw with nothing reporting an error.

Versioning

The v3 in the filename is load-bearing. It selects the prompt format and the legal class list on the client side — v1 and v2 were trained on different prompt shapes and an earlier taxonomy (which included positional front_matter / back_matter classes, retired because they described where a page sat rather than what was on it). A v3 checkpoint served under a name without a version reads as v1 and gets a prompt advertising a taxonomy it never saw.

New versions are added as new files here; old ones are kept so a client pinned to one keeps working.

Training

QLoRA (NF4) on hand-labelled book pages, merged to fp16 and re-quantized to Q4_K_M. The labels come from books labelled a page at a time in BookForge's PDF editor. Coverage across books, not raw block count, is the binding constraint: the 4B is a clear win over a 0.6B on the same data (+0.10 macro-F1), but table and subheading still score near zero because too few books in the set contain them.