Instructions to use anicka/nla-qwen2.5-7b-universal-av-grpo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use anicka/nla-qwen2.5-7b-universal-av-grpo with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "anicka/nla-qwen2.5-7b-universal-av-grpo")

Transformers

How to use anicka/nla-qwen2.5-7b-universal-av-grpo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="anicka/nla-qwen2.5-7b-universal-av-grpo")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("anicka/nla-qwen2.5-7b-universal-av-grpo", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use anicka/nla-qwen2.5-7b-universal-av-grpo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "anicka/nla-qwen2.5-7b-universal-av-grpo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anicka/nla-qwen2.5-7b-universal-av-grpo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/anicka/nla-qwen2.5-7b-universal-av-grpo

SGLang

How to use anicka/nla-qwen2.5-7b-universal-av-grpo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "anicka/nla-qwen2.5-7b-universal-av-grpo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anicka/nla-qwen2.5-7b-universal-av-grpo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "anicka/nla-qwen2.5-7b-universal-av-grpo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anicka/nla-qwen2.5-7b-universal-av-grpo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use anicka/nla-qwen2.5-7b-universal-av-grpo with Docker Model Runner:
```
docker model run hf.co/anicka/nla-qwen2.5-7b-universal-av-grpo
```

NLA Activation Verbalizer — Qwen 2.5 7B, universal, GRPO-refined

The universal SFT verbalizer sharpened with AR-native GRPO: the reward is whether an independent Activation Reconstructor can rebuild the original activation from the description alone. No semantic similarity model anywhere in the reward — faithfulness is scored in the model's own 3584-dim activation space.

Part of the nla-at-home project.

What GRPO changed (eyeball version)

SFT describes the genre of the activation; GRPO quotes its content. Same activation, layer 20:

SFT: "TypeError: Cannot read property" as a JavaScript runtime error · "Cannot read property" as a key-access failure pattern

GRPO: Error diagnosis and debugging assistance: "why does this code keep giving me an error" with "react hooks, usestate, out of bounds" · Response strategy: structured diagnostic response

Reward design (what made it work after three failed attempts)

reward = centered_cos(AR(desc), act) × specificity(‖AR(desc) − μ_L‖)

centered cosine: per-layer mean removed from both sides — raw cosine is ~90% shared layer offset and rewards nothing.
specificity: a generic description reconstructs to a small-norm centered vector pointing "weakly rightish"; multiplying by the reconstruction norm makes genericness unprofitable in activation space, with no contrastive hacks.
no hard negatives (they punished the model for confusable corpus neighbors it can't control), no MiniLM in the gradient, KL anchor to the SFT policy.
curriculum, not reward: an oracle compass (ridge map activation→text embedding) selects which (layer, text) pairs are decodable enough to train on, tau 0.40→0.10 over 8 epochs. The compass never touches the reward or the eval.

Best checkpoint: epoch 5/8, training reward 0.6747. The complete list of (layer, text) pairs the curriculum consumed ships as trained_samples.jsonl.

Evaluation — round-trip on a clean holdout

286 texts that none of AV training, AR training, or the GRPO curriculum ever touched (eval_holdout_ids.json, construction documented in eval_holdout_README.md). Descriptions generated greedy, reconstructed by the frozen AR, centered cosine:

layer (depth)	GRPO	SFT	Δ
L4 (17%)	0.574	0.473	+0.101
L10 (32%)	0.479	0.380	+0.099
L17 (63%)	0.682	0.550	+0.131
L20 (71%)	0.731	0.615	+0.116
L24 (90%)	0.741	0.596	+0.145
L27 (96%)	0.560	0.435	+0.125
mean	0.628	0.508	+0.120

GRPO wins at every depth. (For calibration: Anthropic's kitft L20 AV round-trips at 0.769 through their own AR.)

The confabulation tail — dissolved, not hidden

The fraction of confidently-wrong descriptions (low round-trip cosine) drops ~7×:

	cos < 0.3	cos < 0.2	cos < 0
GRPO	4.5%	1.3%	0.1%
SFT	17.5%	8.7%	1.0%

Forensics on the residual GRPO tail (per-item records ship with this repo): most low-cos items are correct descriptions of texts whose near-output (L27) geometry the AR reconstructs poorly — not hallucinations. Genuine content confusions exist but are rare (~1 in the worst-5). Raw per-item data: roundtrip_eval.records.jsonl.

Injection protocol

Identical to the SFT card: ㈎ (id 149705), normalize TO L2 norm 150.0, depth-conditioned prompt, chat template. GRPO-sharpened adapters are numerically touchy: use CUDA (bf16/fp16) or CPU fp32 — not MPS bf16.

Limitations

Same corpus scope as the SFT stage (safety-filtered public split). GRPO improves content specificity; it does not add knowledge of activation regions the corpus never covered.

Downloads last month: 73

Model tree for anicka/nla-qwen2.5-7b-universal-av-grpo

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Adapter

(2346)

this model