humanizer-12b-8bit

An 8-bit MLX version of jialinyyzz/humanizer, a 12B Gemma 4 fine-tune that rewrites AI-written drafts (English and Chinese) so they read like a person wrote them. It is trained to keep every number, unit, date, name and quotation, and to add nothing.

This pack is made with stock tools and works through the standard chat API: mlx_lm.server, the mlx_lm Python API and mlx-serve all load it as it is.

What this pack is

• Converted with mlx_lm.convert (mlx-lm 0.32.2), nothing custom:

  mlx_lm.convert --hf-path jialinyyzz/humanizer --mlx-path humanizer-12b-8bit -q --q-bits 8 --q-group-size 64
  

  Affine quantization, group size 64, 8.5 bits per weight, 12.7 GB.
• Source: the original model.safetensors (bf16, sha256 f6e8c0d176f9340cc2af82efac085e4ba292781bb0783a08eef07540d6b95548).
• Standard mlx-lm layout (model_type: gemma4_unified, language_model.model.* weights), all in top-level shards.
• Text model only. mlx-lm drops the original's encoder-free vision embedder when converting, so image and audio input are not supported.

Three files differ from what mlx_lm.convert writes:

File	Change
chat_template.jinja	The original safetensors ship no chat template; the authors' GGUF files carry one. This is that template with one change: it emits bos_token first. The GGUF template leaves BOS to llama.cpp, which adds it itself; transformers, mlx-lm and vLLM tokenize chat output without special tokens, so here the template has to carry it. Everything else is the authors' text.
generation_config.json	top_k is 0 (off) instead of 64. The authors recommend temperature 1.0 and top-p 0.95 with every other sampler off.
prompt_format.json	Copied unchanged from the original.

Using it

humanizer is a text-completion model: it was trained on one plain-text prompt and ends the rewrite with EOS. The chat template lets chat clients build that prompt: it takes the last user message as the draft and ignores system prompts and earlier turns, so each message is rewritten on its own.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("ddalcu/humanizer-12b-8bit")
draft = "It is important to note that the deadline has been extended to Friday, June 12."
prompt = tok.apply_chat_template([{"role": "user", "content": draft}], add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

mlx_lm.server --model ddalcu/humanizer-12b-8bit
curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "ddalcu/humanizer-12b-8bit",
  "messages": [{"role": "user", "content": "<your draft>"}],
  "temperature": 1.0, "top_p": 0.95, "max_tokens": 512
}'

• Sampling: temperature 1.0, top-p 0.95, nothing else. mlx-lm takes sampling from the request or the sampler, not from generation_config.json, so set both values yourself.
• Stop on EOS only. Do not add stop strings, especially not ###.
• If an English draft comes back in Chinese, sample again (the original card says the same).
• Context: the original recommends 8192 tokens for instruction, draft and rewrite; split long documents at paragraph breaks.

Building the prompt yourself

prompt = <bos> + INSTR + "\n\n" + draft.strip() + "\n\n### Rewritten:\n\n"

INSTR is in prompt_format.json. If you pass text to mlx-lm's generate, its tokenizer adds BOS itself; do not also write a literal <bos>. As a check, the first 16 hex characters of sha256(INSTR + "\n\n" + "X" + sep) (the text after BOS) are cc51d66b4c593fbe, the same fingerprint the original card gives.

What was checked

• Loads and generates in mlx-lm 0.32.2 (Python API and mlx_lm.server) and in mlx-serve 26.10.2.
• The chat path produces exactly the token ids you get from encoding the prompt directly (114 tokens for the test draft above, BOS first), and all three runtimes give the same greedy output on it.
• Not measured: agreement with bf16 (top-1 or KL) or any fact-judge score for this pack. The numbers on the original card are for the authors' own files, not for this conversion.

Known behavior

• Read the output before you send it, especially numbers, dates, names and quotations. In small informal tests at the recommended sampling, numbers and dates were almost always kept; a direct quotation was paraphrased or dropped in 2 of 12 samples of one test paragraph.
• A one-sentence draft can come back with an extra short line.

Credits and license

All of the model's training is the work of the original authors: jialinyyzz/humanizer, fine-tuned from google/gemma-4-12B. This conversion is not affiliated with the original authors or with Google.

Apache License 2.0, as the original. LICENSE and NOTICE are the original's, unchanged. Changes from the original: 8-bit quantization, the chat template and generation_config.json as listed above.