GreatQuestion-3B

Answers without preambles.

A tersified variant of Qwen/Qwen2.5-3B-Instruct, produced with Heretic v2.1.0.dev0 using Arbitrary-Rank Ablation (ARA). Qwen2.5-3B-Instruct is a 3.09B dense model with 36 layers, tied embeddings and a 32,768 token context window. ARA edits layers 6 to 36, the attention output and MLP down projections. Everything else is unchanged.

The model answers with the minimum text that satisfies the prompt. It does not add introductions, restate the question, or add detail the prompt did not ask for.

Who this is for: local agents, tools and chat on small hardware where output length costs time or tokens, and anyone who wants a short answer instead of a long one. It is not for long explanations, tutorials or step by step reasoning. For those, use the base model.

Mean response length drops from 1418.5 to 152.3, PIQA acc_norm goes from 0.7829 to 0.7753, and CommonsenseQA goes from 0.7903 to 0.7682.

Tersification parameters

Trial 86 of a 150-trial Heretic run (seed 42).

Parameter	Value
start_layer_index	6
end_layer_index	36
preserve_good_behavior_weight	0.9477
steer_bad_behavior_weight	0.0182
overcorrect_relative_weight	0.7325
neighbor_count	11

Performance

Metric	This model	Original model (Qwen/Qwen2.5-3B-Instruct)
Mean response length	152.3	1418.5
PIQA acc_norm	0.7753	0.7829
CommonsenseQA acc,none	0.7682	0.7903

Mean response length is measured on mlabonne/harmless_alpaca evaluation prompts. PIQA and CommonsenseQA are measured with the Language Model Evaluation Harness. CommonsenseQA acc_stderr was 0.0121 for this model and 0.0117 for the original.

Benchmarks

Same prompts, same order. GreatQuestion-3B answers are verbatim from the Trial 86 chat. ChatGPT-6 and Claude Sonnet 5.5 answers are summarized to the first answer line.

Prompt	GreatQuestion-3B	ChatGPT-6	Claude Sonnet 5.5
What is 1+1?	2	1 + 1 = 2	1+1 is 2, plus a note on binary, Boolean and modular cases
Are dogs mammals?	Yes	Yes. Dogs are mammals. They give birth to live young, feed puppies milk, have hair, are warm-blooded and breathe with lungs.	Yes, dogs are mammals. Warm-blooded, hair, live young, milk, order Carnivora, family Canidae.
Write a Python function that checks if a string contains only capital letters.	def has_only_capital_letters(s): return s.isupper()	Caps Lock and Shift keys explained, no code	Ambiguous request, writing and programming meanings listed, no code, asks for language
Is the following statement true or false: All cats are animals, Whiskers is a cat, therefore Whiskers is an animal.	True	Asks for premises and conclusion, gives a different example	You have not given a syllogism, asks for the exact syllogism
Write a haiku about rain.	Rain falls gently, Whispering secrets in drops, Peace fills the air.	Soft rain kisses leaves, Silver rivers trace the earth, Clouds whisper goodbye.	Soft taps on the roof, the gutters hum, overflow, earth drinks, then exhales, with a 5-7-5 note

Why tersification instead of prompt engineering

A system prompt that says be terse fights the base model's training. The verbose habits are in the weights, not in the prompt, so the model still adds introductions and detail when the prompt gets long. ARA instead edits the weight directions that separate terse answers from loquacious ones, using contrast prompts with the two system prompts. The knowledge and instruction following in the rest of the network are left alone. That is why PIQA stays within a point of baseline while length falls by almost 90 percent.

Files

Safetensors

File	Size
model-00001-of-00002.safetensors	4.98 GB
model-00002-of-00002.safetensors	1.19 GB

BF16, 6.17 GB total. The reproduce/ directory carries the full Heretic recipe, including config.toml, requirements.txt, reproduce.json and SHA-256 sums.

No GGUF files are published in this repo yet. Convert to GGUF with llama.cpp if you want a smaller quant for CPU or low VRAM use.

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/GreatQuestion-3B"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "What is 1+1?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                        return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Generation parameters

Qwen2.5-3B-Instruct defaults work as-is. The model is already terse, so a low max_new_tokens is usually enough. Keep temperature at the base model's recommended values if you want deterministic short answers.

Chat template

Qwen2.5 uses ChatML. The checkpoint ships the base model's chat template unchanged.

Runs anywhere

Device	Notes
Laptop (8 GB)	Fits in bf16 with modest context.
RTX 3060 / 4060 (12 GB)	Fits easily.
RTX 4070 / 4080 / 3090 / 4090 / A5000 (16-24 GB)	Comfortable. Batch size 128 was selected on an RTX A5000.
Apple Silicon (16 GB)	Fits in bf16 with modest context.

Weights only. Add about 1 GB for context and activations in transformers.

Model details

Architecture	Qwen2ForCausalLM
Parameters	3.09B (2.77B non-embedding)
Layers	36
Attention	Gated attention: 16 Q heads, 2 KV heads
Hidden / intermediate	2048 / 11008
Position embedding	RoPE, theta 1,000,000
Context length	32,768
Vocab	151,936 (tied embeddings)
Precision	bfloat16
Base model	Qwen/Qwen2.5-3B-Instruct

FAQ

What is GreatQuestion-3B?

GreatQuestion-3B is a tersified build of Qwen/Qwen2.5-3B-Instruct produced with Heretic 2.1.0.dev0 (Arbitrary-Rank Ablation). It is the same weights with a weight edit applied, not a new fine-tune, so its knowledge tracks the base model.

Is GreatQuestion-3B decensored?

No. This run minimized response length and maximized PIQA. It did not run a refusal scorer. Refusal behaviour is unchanged from the base model.

How do I run GreatQuestion-3B locally?

Use the Quickstart above with transformers. The repo ships safetensors only. Convert to GGUF with llama.cpp for llama.cpp, Ollama, LM Studio or Jan.

Responsible use

This is a weight edit, not an alignment layer. It does not add or remove safety behaviour, and it inherits the base model's factual limitations and biases. Short answers can drop detail that matters, so check output on tasks where a missing step or caveat would change the result.

Made with care by RACER IS OP (saidutta69) - follow for more local models

License

Inherits the Qwen Research license from the base model. See the license link at the top of this card. This is a derivative work, keep the license and attribution when redistributing.

Related

• Qwen/Qwen2.5-3B-Instruct - the base instruction-tuned model
• Qwen2.5-3B-Instruct-heretic - the decensored sibling of this model, same base, different edit
• Heretic - the tool used for this edit
• RACER IS OP - Heretic Models - full collection - every heretic model in this collection

Tersification hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA RTX A5000 (23.55 GB VRAM)
• CPU: AMD Ryzen 7 7700 8-Core Processor
• OS: Linux-6.8.0-147-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.2, driver 595.91.07
• Software: PyTorch 2.15.0+cu132, Heretic 2.1.0.dev0, Python 3.11.13

Heretic notes that response length and PIQA numbers can vary slightly with hardware and library versions.