File size: 1,343 Bytes
ccd394b 00821b2 ccd394b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
pipeline_tag: text-generation
license: apache-2.0
language:
- zh
- en
---
# Model Card for Breeze-7B-Instruct-64k-v0_1 with ExLlamaV2 Quantization
Original model 原始模型: https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0_1
This is a quantizated model from [MediaTek-Research/Breeze-7B-Instruct-64k-v0_1](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0_1) in exl2 format.
You are currently at the [main](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/main) branch, which provides only [measurement.json](measurement.json) used in the ExLlamaV2 quantization. Please take a look of your choices in following table of branches.
這裡是main branch, 只提供EvLlamaV2量化時所用到的[measurement.json](measurement.json)檔案。
[8.0bpw-h8](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/8.0bpw-h8) 8 bits per weight.
[6.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/6.0bpw-h6) 6 bits per weight.
[5.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/5.0bpw-h6) 5 bits per weight.
[4.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/4.0bpw-h6) 4 bits per weight.
[3.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/3.0bpw-h6) 3 bits per weight.
## Citation
```
@article{breeze7b2024,
title={},
author={},
journal={arXiv},
year={2024}
}
```
|