Instructions to use ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,7 @@ EXL3 quants of [baichuan-inc/Baichuan-M2-32B](https://huggingface.co/baichuan-in
|
|
| 19 |
### Quants
|
| 20 |
| Quant(Revision) | Bits per Weight | Head Bits |
|
| 21 |
| -------- | ---------- | --------- |
|
|
|
|
| 22 |
| [3.0_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
|
| 23 |
| [3.5_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
|
| 24 |
| [4.0_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
|
|
|
|
| 19 |
### Quants
|
| 20 |
| Quant(Revision) | Bits per Weight | Head Bits |
|
| 21 |
| -------- | ---------- | --------- |
|
| 22 |
+
| [2.5_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/2.5bpw_H6) | 2.5 | 6 |
|
| 23 |
| [3.0_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
|
| 24 |
| [3.5_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
|
| 25 |
| [4.0_H6](https://huggingface.co/ArtusDev/baichuan-inc_Baichuan-M2-32B-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
|