PrimeMind-4B

A fine-tuned version of Qwen3.5-4B trained on compressed reasoning datasets to produce concise, structured thinking patterns.

What It Does

PrimeMind-4B uses <think> tags to show its reasoning process before providing answers. The reasoning is compact and information-dense — cutting unnecessary verbosity while retaining accuracy.

Training

• Base model: Qwen/Qwen3.5-4B (multimodal, 2.6B active params)
• Method: LoRA SFT (rank 64, alpha 128)
• Datasets:
  • catsaresupercool/synthetic-caveman-thinking (2,326 math/reasoning examples with <think> format)
  • nibauman/objectnav-sft-claude-caveman (638 navigation reasoning examples)
• Training: 300 steps, batch size 2, learning rate 2e-4, 4-bit quantization
• Hardware: RTX 4060 Ti (16GB), ~25 minutes

Usage

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "CrowdMind/PrimeMind-4B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("CrowdMind/PrimeMind-4B", trust_remote_code=True)

messages = [{"role": "user", "content": "What is 25 + 37?"}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor.tokenizer(text, return_tensors="pt", add_special_tokens=False).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7)

response = processor.tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Example Output

<think>
25 + 37 = 62
</think>

62

Architecture

Qwen3.5-4B uses a hybrid Gated DeltaNet + MoE architecture:

• 32 hidden layers (mix of linear_attention and full_attention)
• 2,560 hidden size
• 262,144 context length
• Vision encoder for multimodal input (24-layer ViT)
• 4-bit quantized (NF4)

Limitations

• Fine-tuned for 300 steps on ~3K samples — more training would improve results
• Text-only training on multimodal data (images skipped during training)
• May occasionally use verbose thinking instead of compressed format

License

Apache 2.0