vishwr/claim_drafter-merged

Claim Drafter — merged model (Qwen3.5-9B)

The full, standalone merged model for drafting US patent claims from a plain-English invention disclosure. This is Qwen/Qwen3.5-9B with the vishwr/claim_drafter LoRA adapter merged into the base weights, so it can be served directly — no PEFT or adapter loading required.

Architecture	Qwen3_5ForConditionalGeneration (qwen3_5), 32 layers, hidden 4096
Base model	Qwen/Qwen3.5-9B
Precision	bfloat16, 4 safetensors shards (~19.3 GB total)
Adapter (un-merged)	vishwr/claim_drafter (LoRA r=32, α=32)
Training data	vishwr/claim_drafter — SFT + examiner-labelled DPO
Training	SFT → DPO → GRPO, on Tinker

Note on modality. The base checkpoint carries the Qwen3.5 vision/video stack (Qwen3VLProcessor, image/video tokens), so the merged model retains a multimodal architecture. It was fine-tuned on text only (patent-claim drafting); feed it text prompts. The vision path is untouched by fine-tuning.

What makes it different (it's in the data, not the training loop)

• Targets are as-GRANTED claims, fetched per patent number — not the as-filed claims that ship with HUPD. Across 1,596 measured patents, 62% of as-filed claim 1s were substantially amended during prosecution, so training on as-filed teaches the model to draft claims that draw rejections.
• Prompts don't leak the answer. Dropping the patent summary cut measured verbatim leakage from 46% to 4%, so the model must draft rather than reformat.
• Preference pairs come from real examiners. 5,614 applications where the allowed (as-granted) claims are preferred over the refused (as-filed) claims — a real label with no LLM judge.
• The RL reward is a program, not a reward model — patent claims have formal properties (numbering, dependency validity, single-sentence form) checked exactly. Calibrated at 0.991 on real granted claims.

Usage

Serve with vLLM (recommended)

vllm serve vishwr/claim_drafter-merged --max-model-len 16384 --port 8000

Transformers

Requires a recent transformers (the qwen3_5 architecture; ≥ 4.57).

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

repo = "vishwr/claim_drafter-merged"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(
    repo, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "You are an expert US patent attorney. Draft a set of independent and dependent claims based on the invention disclosure."},
    {"role": "user", "content": "<your plain-English invention disclosure here>"},
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Validate before you trust it

The same program that provided the RL reward is the production guardrail (claim_drafter/rewards.py in the code repo):

from claim_drafter.rewards import claim_reward
if claim_reward(generated) < 0.9:
    ...  # numbering or dependency defect: regenerate or route to review

Honest limitations

• Not legal advice. The model drafts claim form. It cannot assess novelty, non-obviousness or patentability. Every output needs attorney review.
• Chemistry is weakest — claim sets referencing drawn structures or sequence listings were dropped; it trains on the method/device claims that survive.
• No design or plant patents (HUPD is utility applications only).
• Evaluation is in-distribution — every training prompt is patent-office prose; real users write rough disclosures. Hand-write a few and read the outputs.
• Corpus is 2014 + Jan-2016 filings, reflecting drafting practice from ~a decade ago.

Provenance & license

Patent text is US government work and not subject to copyright. Applications come from the Harvard USPTO Patent Dataset; granted and as-filed claims were fetched from Google Patents by the pipeline in the code repository. Code and weights are released under the MIT license.

Related repositories

• Code + LoRA adapter — https://huggingface.co/vishwr/claim_drafter
• Datasets (SFT + DPO) — https://huggingface.co/datasets/vishwr/claim_drafter
• GitHub — https://github.com/rahvis/claim_drafter