davanstrien/LightOnOCR-2-1B-no-blank-commentary

LightOnOCR-2-1B, no blank-page commentary (a weight-editing experiment)

This is a fun experiment, not a model for real use. I wanted to see whether a training-free weight edit could remove one specific OCR failure. It was tested on one benchmark, with two runs per setting, and it has known side effects (see Limits). If you need clean output from LightOnOCR-2-1B, use the base model and filter its output.

When served with vLLM (as its model card recommends), LightOnOCR-2-1B often writes "There is no content in the provided image to convert into Markdown format…" on blank pages instead of nothing. This copy has a small edit to its weights that removes that habit. No training, no prompt change.

Curiously, the same model run with transformers (for example, LightOn's demo Space) rarely does this: in a small check on 4 blank pages × 10 runs, vLLM 0.26 wrote the statement 40/40 times and transformers 5.22.1 1/40. The two engines build prompts of different lengths for the same image (2,322 vs 2,267 tokens); I have not checked why. This edit was only tested with vLLM.

What changed

• The direction: the mean hidden state at the first output position on blank pages where the model starts the statement, minus the mean on blank pages where it writes nothing. Fitted on blank pages from BHL books that are not in the benchmark.
• The neurons: in decoder layers 19–23, the 256 MLP neurons per layer that write most along that direction on statement pages compared with silent pages (score = activation × the neuron's output along the direction, ranked by the statement-minus-silent difference).
• The edit: for those neurons only, the part of their output along the direction is flipped: W[:, S] ← W[:, S] − 2·u·(uᵀ W[:, S]) on mlp.down_proj. 1,282,748 values change: 0.13% of the model.
• Everything except model.safetensors is the original release, revision c97bd377f04481830395218fa8951df9deaba756 (same weights and processor files as be7c4131, the current main at the time of writing).

Results: BHL OCR benchmark (2,165 pages, 26 models)

	overall CER	rank	text-page CER	words on blank pages
base (leaderboard row)	0.0408	14	0.0271	7,085
this edit, run 1 / 2	0.0333 / 0.0321	8 / 7	0.0265 / 0.0263	4,058 / 3,454

• The statement disappears from all ~100 pages where the base model wrote it.
• Text-page CER is unchanged within noise (paired bootstrap).
• A random direction of the same size did nothing in the dense version of this edit (all columns, layers 17–23). The sparse edit itself has no random control yet.

Limits

• It does not stop "Note: The image contains…" descriptions on bird plates. They become more common: 8 → 18 blank pages.
• A plain text filter for the statement would remove more blank-page text than this edit. The point is to show the fix can live in the weights, not to offer the best way to clean output.
• These are not leaderboard rows. Sampling at temperature 0.2 makes results vary between runs, and each setting ran only twice.

How it was run

All compute ran on Hugging Face Jobs: an interactive GPU Job (L40S) to collect hidden states and fit the direction, and a10g-small Jobs for the benchmark runs (about 48 minutes each). The whole experiment, including controls and a second edit not published here, cost about $11. To run your own: Jobs guide for the hf CLI and Python, pricing.

Use

Same as the base model: serve it with vLLM as shown in the base model card. The edit was only tested with vLLM 0.26.

Credit

The idea comes from "Sparks of Weight-Space Programming" by Takehi, Teoh, Zhang & Khattab. Sister experiment: a similar edit that stops OXR-1.0 looping on leader dots (X thread). The experiment was run by a Claude agent; setup and checks by Daniel van Strien.