Hugging Face model: fbaldassarri/meta-llama_Llama-3.2-3B-Instruct-auto_gptq-int4-gs64-sym
Author: fbaldassarri
Created: 2026-07-06 21:17:35
Last modified: 2026-07-06 21:22:51
Pipeline: text-generation
Library: transformers
Likes: 0 | Downloads: 9
Tags: transformers, safetensors, llama, text-generation, gptq, auto-gptq, autogptq, pytorch, causal-lm, autoround, auto-round, intel-autoround, intel, woq, weights-only-quantization, meta, llama-3, 4-bit, conversational, en, de, fr, it, pt, hi, es, th, base_model:meta-llama/Llama-3.2-3B-Instruct, base_model:quantized:meta-llama/Llama-3.2-3B-Instruct, license:llama3.2, text-generation-inference, region:us
Base model relation: quantized
Base models: id: meta-llama/Llama-3.2-3B-Instruct
Card metadata:
language: en, de, fr, it, pt, hi, es, th
tags: gptq, auto-gptq, autogptq, pytorch, causal-lm, autoround, auto-round, intel-autoround, intel, woq, weights-only-quantization, meta, llama, llama-3, 4-bit
license: llama3.2
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
library_name: transformers
model_name: Llama 3.2 3B Instruct
base_model: meta-llama/Llama-3.2-3B-Instruct
base_model_relation: quantized
model_type: llama
inference: no
model_creator: fbaldassarri
pipeline_tag: text-generation
quantized_by: fbaldassarri
quantization_config: method: auto_gptq
bits: 4
group_size: 64
sym: yes
auto_round_version: 0.13.1
torch_dtype: torch.bfloat16
device: cpu
nsamples: 128
iters: 200
seqlen: 512
batch_size: 4