bcacdwk commited on
Commit
be13d93
·
verified ·
1 Parent(s): e3e27f1

Add README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -0
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - slidesparse
5
+ - sparse
6
+ - quantization
7
+ - int8
8
+ - fp8
9
+ - llama
10
+ - qwen
11
+ ---
12
+
13
+ # SlideSparse Checkpoints
14
+
15
+ Pre-converted sparse model checkpoints using the **SlideSparse** technique.
16
+
17
+ ## Overview
18
+
19
+ This repository contains model weights converted with various sparsity configurations:
20
+ - **2:4** - Standard N:M sparsity (50% sparse)
21
+ - **2:6** - Extended sparsity (67% sparse)
22
+ - **2:8** - Higher sparsity (75% sparse)
23
+ - **2:10** - Maximum sparsity (80% sparse)
24
+
25
+ ## Models Included
26
+
27
+ | Base Model | Quantization | Sparsity Variants |
28
+ |------------|--------------|-------------------|
29
+ | Llama-3.2-1B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
30
+ | Llama-3.2-3B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
31
+ | Qwen2.5-7B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
32
+ | Qwen2.5-14B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
33
+
34
+ ## Source Models
35
+
36
+ These checkpoints are derived from:
37
+ - [RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8)
38
+ - [RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8)
39
+ - [RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8)
40
+ - [RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8)
41
+
42
+ ## License
43
+
44
+ - **Qwen models**: Apache 2.0
45
+ - **Llama models**: Please refer to [Meta's Llama license](https://llama.meta.com/llama3/license/)
46
+
47
+ ## Usage
48
+
49
+ ```bash
50
+ # Download all checkpoints
51
+ huggingface-cli download bcacdwk/slidesparse-checkpoints --local-dir ./checkpoints_slidesparse
52
+
53
+ # Download specific model
54
+ huggingface-cli download bcacdwk/slidesparse-checkpoints Llama3.2-1B-INT8-SlideSparse-2_4 --local-dir ./checkpoints_slidesparse/Llama3.2-1B-INT8-SlideSparse-2_4
55
+ ```
56
+
57
+ ## Citation
58
+
59
+ If you use these checkpoints, please cite the SlideSparse paper (coming soon).