SSA
Collection
Models and Datasets of paper: [Simplified Sparse Attention via Gist Tokens] • 30 items • Updated
GSA-PT-Qwen2-7B-Instruct-chunk8
This model is continued pretrained from Qwen/Qwen2-7B-Instruct using Gist Sparse Attention (GSA) with chunk size chunk8.
Paper
GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding
Model Details
| Field | Value |
|---|---|
| Base model | Qwen/Qwen2-7B-Instruct |
| Training type | Continued Pretraining |
| Chunk size | chunk8 |
| Architecture | Qwen2-7B |
- Downloads last month
- 9
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support