hlhsiao/llama-3.2-3b-instruct-quantized

LLaMA 3.2 3B Instruct - Quantized (w8a8)

This is a 3B parameter LLaMA 3.2 Instruct model, quantized with 8-bit weights and activations (w8a8). Model files include safetensors-formatted weights and generation_config.json.

Files

• model-00001-of-00002.safetensors + index.json: quantized model weights
• config.json + generation_config.json: model and generation configuration
• recipe.yaml: quantization configuration (if applicable)

How to load

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("your-username/llama-3.2-3b-instruct-quantized", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("your-username/llama-3.2-3b-instruct-quantized")