prose-rewriter-4b-v1.3

A paragraph-level prose rewriter: it takes prose written by a large model and re-renders it to be more human, preserving the semantics it was given.

Qwen/Qwen3-4B-Base with a rank-32 LoRA merged in at strength 1.2.

Successor to prose-rewriter-4b-v1.2, trained on a larger pool with two new data gates. It invents less and copies less than v1.2; see Evaluation. This is why it allows for a merge at higher strength.

Variants

Path	Format	Use with
/	safetensors bf16, qwen3 arch	transformers
GGUF/prose-rewriter-4b-v1.3-Q8_0.gguf	GGUF Q8_0, 4.69 GB	llama.cpp / llama-cpp-python
GGUF/prose-rewriter-4b-v1.3-Q4_K_M.gguf	GGUF Q4_K_M, 2.72 GB	llama.cpp / llama-cpp-python

The quants carry the chat template and stop on <|im_end|>, and the adapted output head is kept separate from the token embeddings in both — Q8_0 stores it at Q8_0, Q4_K_M at Q6_K.

Prompt format

<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite

The chat template in this repo builds exactly that string, byte for byte, from two roles:

messages = [
    {"role": "source", "content": paragraph},
    {"role": "edit",   "content": "match"},
]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It is not a chat model. The template rejects an edit value outside the three modes rather than quietly building a prompt the weights have never seen. Any other role is treated as the source paragraph, so a runtime that probes the template with a user message still gets a valid prompt.

One paragraph per call. Generation stops on <|im_end|>, which is set as an eos_token_id in generation_config.json. Temperature is the only knob.

prompt_contract.json carries the same facts as machine-readable JSON, for code that would rather read it than parse this file.

The edit block is mandatory

edit names which of three length transforms is being asked for. The values are the corruptor's, so they read backwards. They name what was done to build the input, not what the model should do to it:

edit	what it says about the input	what the model does
match	the source is the human's length	rewrite in place
inflate	the source was padded relative to the human original	cut
compress	the source was flattened and shortened	open it back out

match is the setting for "rewrite it, do not trim it". It's strongly recommended you use this mode.

Sending no block is the worst thing you can do to this checkpoint. It was trained with the block, so omitting it collapses the model onto its deletion-heaviest mode.

Serving recipe

Sampled at temperature=0.9, top_p=0.9.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-4b-v1.3"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph, mode="match"):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph},
         {"role": "edit",   "content": mode}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.9, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()

Same thing under llama.cpp. The roles are source and edit, which no chat API models, so build the string yourself; <|im_end|> stops it:

llama-cli -m GGUF/prose-rewriter-4b-v1.3-Q8_0.gguf -no-cnv -n 512 --temp 0.9 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite
'

Input length

The training pool's median input is 50 words and most of it is under 80, so serve it on anything from a full sentence up.

The practical floor is about 15 words. Below it the failure mode is padding and fabrication rather than gibberish: the model stretches the line toward its learned length and adds material the input never supported. Below 80 bytes, pass the text through unchanged.

Evaluation

Final training probe, v1.2 against v1.3:

	v1.2	v1.3
invention rate	0.146	0.104
copy rate	0.062	0.042
distinct-4	0.982	0.990
copy excess vs target	−0.009	+0.049
entropy	0.506	0.505
length ratio	1.004	1.016
train loss	0.9104	0.974
val loss	0.7539	0.8418

Invention drops by 29% and copy rate by 32%, at slightly better diversity.

A known quirk is that human-written texts use less rare vocabulary than LLM texts. This is a trade-off that must be accepted.

Training

Corrupt forward, train backward. The human paragraph is the target; an on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: roughly 57/42 r/WritingPrompts (Mollymo/Human-to-AI-writing) and AO3 (midwestern-simulation-active/ao3_random_subset), with a sliver of fanfiction.net (atom-in-the-universe/fanfics-10k-10k).

The input side was generated by eleven corruptor endpoints — a few of which are two routes onto one set of weights — weighted and share-capped so no single model's tics dominate:

pool axis	composition
rows	67,696 over 54,209 distinct targets
corruptor	artemis-local 15%, ds-flash-nano 15%, ds-flash 13%, gemma-nano 12%, gemma-local 12%, qwen-flash 9%, ox-alpha 8% + 7%, then artemis, muse, ds-pro
corruption band	heavy 39%, medium 38%, light 20%
len_mode	match 64%, inflate 22%, compress 8%, unmarked 5%
kind	prose 96%, dialogue 4%, structural no-ops 1%

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the scene, grammatical correctness on the target side, content recall stratified by target length, and NLI entailment both ways.

Loss on the target paragraph only. Everything before rewrite is masked.

LoRA	r=32, alpha=32, dropout 0.05
target modules	q, k, v, o, gate, up, down, and lm_head
trainable	71,004,160 params (1.73%)
schedule	1 epoch, lr 1e-4 cosine, batch 12 × accum 3, seq 2048
steps	1,864 on one RTX 3090, 2h27
loss	train 0.974, val 0.8418 (600 val rows, document-disjoint)

The merge

Merged at strength 1.2. Rank 32 with alpha 32 is a LoRA scaling of 1.0, so the effective scaling is 1.2: W + (B @ A) * 1.2. Merged in float32, stored bfloat16.

lm_head is adapted, and Qwen3-4B-Base ties lm_head.weight to embed_tokens.weight. This checkpoint is untied: the merged output head is stored separately and the input embeddings are bit-identical to the base model's, which is what training assumed. config.json says tie_word_embeddings: false and it means it. Do not re-tie it, and if you convert to another format, check that the head survived.

Limitations

• Not an instruct model. It has one job and one prompt. There is nothing to ask it.
• Works on fictional prose only. May not work on technical documentation.
• One paragraph per call. Longer input degrades; split it.
• Will not pass AI detectors. Pangram and such will still know because this model preserves word choices and certain sentence structures.
• English only, narrative register (third and first person fiction, dialogue with quoted speech).
• Short input pads and invents. The floor is about 15 words, and below it the failure is fabrication rather than gibberish. See Input length.

License

The weights in this repository are released under the GNU Affero General Public License, version 3. The full text is in LICENSE.

This is a derivative of Qwen/Qwen3-4B-Base, which is licensed under Apache License 2.0. That license is preserved and its terms continue to apply to the base weights this model was built from; the AGPL covers the combined work as distributed here. Apache-2.0 is one-way compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.