anakhiu/ram-plus-onnx-int8

RAM++ ONNX — dynamic INT8

RAM++ (Recognize Anything Plus) quantized to INT8, so it loads in about a second and tags an image in about 700 ms on a plain CPU.

This is not an original model. It is a quantization of someone else's ONNX export of someone else's model:

Layer	Source	License
RAM++ weights	xinyu1205/recognize-anything	Apache-2.0
ONNX export (fp32)	CannotFindObject/RAM_ONNX	Apache-2.0
INT8 quantization	this repo	Apache-2.0

All credit for the model belongs to the RAM++ authors. Please cite their work, not this repo.

INT8 optimization, CPU benchmarks and threshold notes by anakhiu — 21 July 2026.

Files

File	Size	Notes
ram_plus_int8.onnx	873 MB	input [1,3,384,384] f32, output [1,4585] f32 logits
ram_tag_list.txt	4585 lines	tag i corresponds to output index i
ram_tag_list_threshold.txt	4585 lines	per-tag thresholds from the official repo

The tag and threshold files are byte-equivalent to the upstream ones and are included only so the package is self-contained — pairing these weights with a differently ordered tag list would silently produce wrong labels.

Why INT8 and not FP16

FP16 is the obvious first thing to try and it is the wrong answer on CPU. Measured on the same 5 images, same machine, ONNX Runtime 1.24, CPU EP:

Build	File	Peak RSS	Inference	Load
RAM++ fp32	1859 MB	2699 MB	1268–1433 ms	15.9 s
RAM++ fp16	926 MB	3568 MB	1472–1658 ms	3.9 s
RAM++ int8	873 MB	1770 MB	678–744 ms	1.3 s
RAM (not ++) fp32	901 MB	1412 MB	1038–1242 ms	~8 s

FP16 uses more memory than FP32 and runs slower, because the CPU execution provider has no native fp16 kernels — it converts the weights to fp32 at runtime and ends up holding both copies. INT8 has real kernels, so it wins on memory and speed at once.

Dropping to plain RAM halves memory too, but loses the tags that actually distinguish an image: on a cityscape full of drones it dropped drone, and on a cluttered living room it dropped bookshelf, archway and bureau.

Quality

Against the fp32 export on 5 images, at the official thresholds: 3 of 5 identical, and across the other two, 4 tags lost and 3 gained — every one of them sitting within ~0.01 of its threshold. Numeric jitter at the decision boundary, not lost capability. drone survives.

This is a 5-image check, not a benchmark. It is enough to say INT8 does not break the model. It is not enough to characterise the tail — rare tags, dark or low-contrast images, unusual subjects. Measure on your own data before trusting it with anything that matters.

Usage

Preprocessing must match ram/transform.py exactly:

Resize((384, 384))          # a squash to square — NOT a crop, NOT a letterbox
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406],
          std =[0.229, 0.224, 0.225])

Preserving aspect ratio here feels like a fix and is a bug — the model was never trained on letterboxed input.

The output is raw logits. Apply sigmoid once, then compare against the per-tag threshold:

tag_fires = sigmoid(logits[i]) > threshold[i]

If every score lands near 0.5–0.73 and almost nothing clears its threshold, the export has been sigmoided twice.

On the thresholds

Two things about ram_tag_list_threshold.txt are worth knowing, because neither is documented upstream:

• 74.5% of tags are already at 0.65 (3414 of 4585). The mean is 0.691. The threshold=0.68 default in the source code is overwritten by this file and never really applies.
• 10 tags are set to 1.0: body, cocktail table, French, group, lie, show, see, set, stop, use. Sigmoid cannot reach 1.0, so this is a kill switch, not a strict cut. The authors switched these off on purpose.

If you need higher recall, cap the thresholds rather than flattening them:

threshold[i] = threshold[i] if threshold[i] >= 1.0 else min(threshold[i], 0.65)

This lowered 1131 tags and roughly 2.5× the tag count on our images, while a flat 0.65 did two things wrong: it revived the ten disabled tags, and it raised the bar for the 30 tags tuned below 0.65, suppressing correct detections such as stool (0.54) and night view (0.51).

Reproducing

from onnxruntime.quantization import quantize_dynamic, QuantType

quantize_dynamic("ram_plus.onnx", "ram_plus_int8.onnx",
                 weight_type=QuantType.QInt8)

Input was ram_plus.onnx from CannotFindObject/RAM_ONNX. Took 47 s; 1859 MB → 873 MB. No calibration data is involved — dynamic quantization computes activation ranges at inference time.

Citation

@article{huang2023open,
  title={Open-Set Image Tagging with Multi-Grained Text Supervision},
  author={Huang, Xinyu and Huang, Yi-Jie and Zhang, Youcai and Tian, Weiwei and
          Feng, Rui and Zhang, Yuejie and Xie, Yanchun and Li, Yaqian and
          Zhang, Lei},
  journal={arXiv e-prints},
  pages={arXiv--2310},
  year={2023}
}