Qwen3-8B-24SST-Sparse-AdamW

Qwen3-8B-24SST-Sparse-AdamW is a 2:4 semi-structured sparse version of Qwen3-8B, retrained with 24SST dynamic sparse training so it keeps high accuracy while retaining only half of the weights.

Model details

• Base model: Qwen/Qwen3-8B
• Architecture: Qwen3ForCausalLM (36 layers, hidden size 4096, 32 attention heads, 8 KV heads)
• Sparsity pattern: 2:4 semi-structured (at most 2 non-zero weights in every group of 4 consecutive weights)
• Training data: OpenWebText (OWT)
• Training method: CHTsnm dynamic sparse training (24SST)
• Optimizer: AdamW

Why sparsity?

2:4 semi-structured sparsity keeps only 2 of every 4 weights, which gives:

• ~2× faster matrix multiplies on hardware with sparse tensor cores (e.g. NVIDIA Ampere and later), which skip the zeroed weights in hardware.
• ~1.8× smaller weight storage — each group of 4 weights is stored as 2 values plus 2 compact 2-bit indices instead of 4 full values.
• Lower energy per token at inference, since fewer multiply-accumulate operations are executed.

Unlike one-shot pruning, retraining the sparse network lets the remaining weights adapt, so the model keeps much more of the dense model's quality.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("SparseMindAI/qwen3-8b-24sst-sparse-adamw", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("SparseMindAI/qwen3-8b-24sst-sparse-adamw")

messages = [{"role": "user", "content": "Hello!"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Reference

Original dense model: Qwen/Qwen3-8B