---
library_name: transformers
tags:
- internlm
- custom_code
---

# InterLM2-Chat NF4 Quant

## Quantization config

```python
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
```

## Model Details

- **Repository:** https://huggingface.co/internlm/internlm2-chat-20b