Model Card for Model ID

This is Meta's Llama 2 7B quantized in 2-bit using AutoGPTQ from Hugging Face Transformers.

Model Details

Model Description

Developed by: The Kaitchup
Model type: Causal (Llama 2)
Language(s) (NLP): English
License: Apache 2.0, Llama 2 license agreement

Model Sources

The method and code used to quantize the model are explained here: Quantize and Fine-tune LLMs with GPTQ Using Transformers and TRL

Uses

This model is pre-trained and not fine-tuned. You may fine-tune it with PEFT using adapters. Note that the 2-bit quantization significantly decreases the performance of Llama 2.

Other versions

kaitchup/Llama-2-7b-gptq-4bit
kaitchup/Llama-2-7b-gptq-3bit

Model Card Contact

The Kaitchup