prose-rewriter-1.7b-v1.2

A paragraph-level prose rewriter: it takes prose written by a large model and re-renders it to be more human, preserving the semantics it was given.

Qwen/Qwen3-1.7B-Base with a rank-16 LoRA merged in at strength 1.1.

This is the smaller of two checkpoints trained on the same pool and pipeline. A larger version here: 4B

Variants

Path	Format	Use with
/	safetensors bf16, qwen3 arch	transformers
GGUF/prose-rewriter-1.7b-v1.2-Q8_0.gguf	GGUF Q8_0, 2.17 GB	llama.cpp / llama-cpp-python

VRAM

Measured on an RTX 3090 with llama-server -ngl 99 at stock settings (flash attention auto, f16 KV), read as the server process's resident VRAM after load and before the first request.

-c is the TOTAL context and llama.cpp divides it by --parallel. A -c 4096 --parallel 8 server gives each slot 512 tokens and silently truncates long paragraphs. Ask for slots x 1280: 512 source tokens is the documented input ceiling, the generation budget never exceeds 512, and the prompt's three blocks are a dozen more.

slots	-c	Q8_0
1	1280	2,222 MiB
2	2560	2,362 MiB
4	5120	2,642 MiB
8	10240	3,202 MiB

It is linear, so extrapolate freely: the weights are a fixed 2,082 MiB and every 1280-token slot adds 140 MiB of KV cache. Only the total matters -- -c 2048 --parallel 1 and -c 2048 --parallel 2 allocate the same bytes.

Four slots is a sensible default. Eight reserves better than a gigabyte of KV before the first request arrives, which on a small card is the difference between fitting and not.

Prompt format

<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite

The chat template in this repo builds exactly that string, byte for byte, from two roles:

messages = [
    {"role": "source", "content": paragraph},
    {"role": "edit",   "content": "match"},
]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It is not a chat model. The template rejects user / assistant / system, and an edit value outside the three modes, rather than quietly building a prompt the weights have never seen.

One paragraph per call. Generation stops on <|im_end|>, which is set as an eos_token_id in generation_config.json. Temperature is the only knob.

The edit block is mandatory

edit names which of three length transforms is being asked for. The values are the corruptor's, so they read backwards. They name what was done to build the input, not what the model should do to it:

edit	what it says about the input	what the model does
match	the source is the human's length	rewrite in place
inflate	the source was padded relative to the human original	cut
compress	the source was flattened and shortened	open it back out

match is the setting for "rewrite it, do not trim it". It's strongly recommended you use this mode.

Sending no block is the worst thing you can do to this checkpoint. It was trained with the block, so omitting it collapses the model onto its deletion-heaviest mode.

Serving recipe

Sampled at temperature=0.9, top_p=0.9.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-1.7b-v1.2"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph, mode="match"):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph},
         {"role": "edit",   "content": mode}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.9, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()

Same thing under llama.cpp. The roles are source and edit, which no chat API models, so build the string yourself; <|im_end|> stops it:

llama-cli -m GGUF/prose-rewriter-1.7b-v1.2-Q8_0.gguf -st -n 512 --temp 0.9 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite
'

Input length

input words	generations	length ratio	invention	3-gram self-repetition
< 15	288	1.58	72%	0.002
15-24	219	1.00	34%	0.002
25-39	108	0.95	31%	0.003
40-59	60	0.93	28%	0.001
60-79	180	0.89	23%	0.002
80-119	170	0.90	25%	0.004

length ratio is median output words over input words. invention is the share of generations carrying a sentence the input does not entail (DeBERTa-MNLI, min entailment < 0.5). 3-gram self-repetition is the fraction of a generation's 3-grams that repeat inside it -- the degeneracy signal.

Nothing degenerates at any length. Self-repetition sits at noise across the whole range, and not one of these generations failed to stop or came back empty. What actually happens below ~15 words is that the model pads and invents: it stretches the line toward its learned length and roughly 70% of outputs add material the input never supported, against ~25% at paragraph length. Between 15 and 120 words nothing moves much.

So serve it on anything from a full sentence up. Below 80 bytes, pass the text through unchanged.

Evaluation

	1.7B	4B
val loss	0.8214	0.7539
train loss	0.9735	0.9104
invention rate	0.177	0.146
copy rate	0.062	0.062
copy excess vs target	+0.001	−0.009
distinct-4	0.982	0.982
entropy	0.530	0.506
length ratio	1.025	1.004

The 4B invents less and copies no more, at flat diversity, which is the direction scale was expected to move.

A known quirk is that human-written texts use less rare vocabulary than LLM texts. This is a trade-off that must be accepted.

Training

Corrupt forward, train backward. The human paragraph is the target; an on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: roughly 60/40 r/WritingPrompts (Mollymo/Human-to-AI-writing) and AO3 (midwestern-simulation-active/ao3_random_subset), with a sliver of fanfiction.net (atom-in-the-universe/fanfics-10k-10k).

The input side was generated by eight corruptor endpoints, weighted so no single model's tics dominate:

pool axis	composition
rows	60,323 over 48,395 distinct targets
corruptor	ds-flash-nano 21%, artemis 21%, ds-flash 18%, gemma-31b 16%, qwen-flash 12%, then muse-spark, ds-pro
corruption band	medium 38%, heavy 34%, light 25%
len_mode	match 60%, inflate 23%, compress 10%, unmarked 7%
kind	prose 93%, dialogue 7%, structural no-ops 0.7%

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the scene, grammatical correctness on the target side, and NLI entailment both ways. About 44k of 317k raw corruptions survive; the training pool is the seeded subset of those. This is the same pool the 4B was trained on, in the same order.

Loss on the target paragraph only. Everything before rewrite is masked.

LoRA	r=16, alpha=32, dropout 0.05
target modules	q, k, v, o, gate, up, down, and lm_head
trainable	19,896,320 params (1.14%)
schedule	1 epoch, lr 1e-4 cosine, batch 4 × accum 8, seq 2048
steps	1,867 on one RTX 3090, 65 min
loss	train 0.9735, val 0.8214 (601 val rows, document-disjoint)

The merge, and one thing not to do

Merged at strength 1.1. Rank 16 with alpha 32 is a LoRA scaling of 2.0, so the effective scaling is 2.2: W + (B @ A) * 2.2. Merged in float32, stored bfloat16.

lm_head is adapted, and Qwen3-1.7B-Base ties lm_head.weight to embed_tokens.weight. This checkpoint is untied: the merged output head is stored separately and the input embeddings are bit-identical to the base model's, which is what training assumed. config.json says tie_word_embeddings: false and it means it. Do not re-tie it, and if you convert to another format, check that the head survived.

Limitations

• Not an instruct model. It has one job and one prompt. There is nothing to ask it.
• Works on fictional prose only. May not work on technical documentation.
• One paragraph per call. Longer input degrades; split it.
• Will not pass AI detectors. Pangram and such will still know because this model preserves word choices and certain sentence structures.
• English only, narrative register (third and first person fiction, dialogue with quoted speech).
• Short input pads and invents. The floor is about 15 words, and below it the failure is fabrication rather than gibberish. See Input length.

License

The weights in this repository are released under the GNU Affero General Public License, version 3. The full text is in LICENSE.

This is a derivative of Qwen/Qwen3-1.7B-Base, which is licensed under Apache License 2.0. That license is preserved and its terms continue to apply to the base weights this model was built from; the AGPL covers the combined work as distributed here. Apache-2.0 is one-way compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.