---
license: apache-2.0
tags:
- text2text-generation
pipeline_tag: text2text-generation
language:
- zh
- en
---
# GPTQ-for-Bloom
4 bits quantization of [Bloom](https://arxiv.org/pdf/2211.05100.pdf) using [GPTQ](https://arxiv.org/abs/2210.17323)

GPTQ is SOTA one-shot weight quantization method.

The code of inference can be found in our Github project repository: https://github.com/LianjiaTech/BELLE/gptq.

**This code is based on [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa)**

## Model list

| model name       |  file size | GPU memory |
| -------------------------------------------------- |  ------------------- | ------------------ |
|           bloom7b-2m-8bit-128g.pt                  |          9.7G        |       11G          |
|           bloom7b-2m-4bit-128g.pt                  |          6.9G        |        8G          |
|           bloom7b-2m-3bit-128g.pt                  |          6.2G        |        7.7G        |