LLaDA 8B Base pruned models
Collection
SparseGPT, sink-aware, and SliceGPT at 20%, 30%, and 40%. • 9 items • Updated
llada-8b-base-sparsegpt-40
Source: GSAI-ML/LLaDA-8B-Base at 0f2787f2d87eac5eed8a087d5ecd24277e6255b2. Structured pruning target: 40%; pattern: 4:10. Measured values are in pruning_statistics.json. The weights and tokenizer were saved and verified by a forward pass after reload. Benchmark evaluation has not been run.
Load with transformers.AutoModel.from_pretrained(repo_id, trust_remote_code=True).
Pruned checkpoint evaluation
Full completed benchmarks only. Original weight revisions are pinned below; updating this model card does not change evaluated weights. Settings: BF16, seed 42, MC=128, batch size 32. LLaDA generation uses 256 steps and 1024 output tokens; GSM8K uses its built-in 5-shot setting, other tasks are zero-shot.
| Benchmark | Metric | Value | Weights | Raw JSON |
|---|---|---|---|---|
| task:arc_challenge | acc,none | 0.430034 | 4f63215a6558 | JSON |
| task:arc_challenge | acc_norm,none | 0.469283 | 4f63215a6558 | JSON |
| task:winogrande | acc,none | 0.668508 | 4f63215a6558 | JSON |
An absent benchmark is pending, not a zero score. Individual raw files record the complete evaluation configuration and checkpoint provenance.
- Downloads last month
- 57
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for vanshnawander/LLaDA-8B-Base-SparseGPT-40
Base model
GSAI-ML/LLaDA-8B-Base