README.md

metadata

library_name: transformers
tags:
  - internlm
  - custom_code

InterLM2-Chat NF4 Quant

Quantization config

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

Model Details

• Repository: https://huggingface.co/internlm/internlm2-chat-20b