twilson63/humanizer-INT4-ov

Humanizer 12B — OpenVINO INT4

OpenVINO IR conversion of jialinyyzz/humanizer (Gemma4Unified 12B completion model), produced on a DGX Spark (aarch64, 121GB unified RAM).

What's here

file	notes
openvino_model.bin/.xml	language model, int4-sym group-128 (NNCF)
openvino_text_embeddings_model.bin/.xml	token → embeddings (fp16)
openvino_vision_embeddings_*	image path (fp16)
openvino_audio_embeddings_model.bin/.xml	audio path (fp16)
openvino_tokenizer.* / openvino_detokenizer.*	tokenizer pair

How to run (text-only completion)

The upstream model is a completion model (no chat template). Provide its exact byte-format prompt (INSTR + draft + "\n\n### Rewritten:\n\n", see the upstream USAGE.md) and sample with temperature=1.0, top_p=0.95 and nothing else.

import openvino_genai as og

# VLMPipeline handles the tokenizer → text-embeddings → language wiring internally
pipe = og.VLMPipeline("twilson63/humanizer-INT4-ov", "GPU")
out = pipe.generate(prompt, max_new_tokens=128, do_sample=True, temperature=1.0, top_p=0.95)

For the exact INSTR block, grab it byte-for-byte from upstream USAGE.md §The prompt.

Bench (Arc Pro B70 32GB, rig 9)

128-token decode: 56.6 tok/s (3-run mean) with openvino-genai 2026.4.0, VLMPipeline text-only.

Conversion notes

• Exported with transformers 5.10.0 + from-source optimum-intel (Gemma4Unified needs transformers 5.5–5.10)
• Language submodel quantized post-export via NNCF int4_sym, group_size=128, ratio=1.0
• Vision/audio/text-embedding submodels stay fp16 (small, latency-dominated)
• The IR takes inputs_embeds (VLM signature) — VLMPipeline wires the token→embedding submodel internally; a plain LLMPipeline on the language .xml alone will report a 5-input mismatch.

Reuse

If you have ~37GB+ of GPU memory, the Ornith-1.5-35B-A3B INT4 from the same pipeline is also up.