PrimeMind-9B

A fine-tuned version of Qwen3.5-9B trained on compressed reasoning datasets to produce concise, structured thinking patterns.

What It Does

PrimeMind-9B uses <think> tags to show its reasoning process before providing answers. The reasoning is compact and information-dense -- cutting unnecessary verbosity while retaining accuracy.

Training

• Base model: Qwen/Qwen3.5-9B (multimodal, 5.8B active params)
• Method: LoRA SFT (rank 64, alpha 128)
• Datasets:
  • catsaresupercool/synthetic-caveman-thinking (600 math/reasoning examples)
  • nibauman/objectnav-sft-claude-caveman (600 navigation reasoning examples with images)
• Total samples: 1,200 (600 text + 600 multimodal)
• Training: 300 steps (1 epoch), batch size 1, grad accum 4, learning rate 2e-4, 4-bit quantization
• Loss: 5.59 -> 0.78
• Hardware: RTX 4060 Ti (16GB), ~3 hours 15 minutes

LoRA Config

• Rank: 64
• Alpha: 128
• Dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Usage

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "CrowdMind/PrimeMind-9B",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("CrowdMind/PrimeMind-9B", trust_remote_code=True)

messages = [{"role": "user", "content": "What is 25 + 37?"}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7)

response = processor.tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Example Output

<think>
25 + 37 = 62
</think>

62

Architecture

Qwen3.5-9B uses a hybrid Gated DeltaNet + MoE architecture:

• 32 hidden layers (mix of linear_attention and full_attention)
• 4,096 hidden size
• 262,144 context length
• Vision encoder for multimodal input (27-layer ViT)
• 4-bit quantized (NF4)

Limitations

• Fine-tuned for 1 epoch on 1,200 samples -- more training would improve results
• May occasionally use verbose thinking instead of compressed format

License

Apache 2.0