File size: 396 Bytes
5a4b70d 8070070 5a4b70d 8070070 5a4b70d 8070070 5a4b70d 8070070 5a4b70d 8070070 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
library_name: transformers
tags:
- internlm
- custom_code
---
# InterLM2-Chat NF4 Quant
## Quantization config
```python
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16,
)
```
## Model Details
- **Repository:** https://huggingface.co/internlm/internlm2-chat-20b |