Hugging Face model: fbaldassarri/ibm-granite_granite-3.2-2b-instruct-auto_gptq-int8-gs64-sym
Author: fbaldassarri
Created: 2026-07-05 22:00:29
Last modified: 2026-07-05 22:08:57
Pipeline: text-generation
Library: transformers
Likes: 0 | Downloads: 9
Tags: transformers, safetensors, granite, text-generation, gptq, auto-gptq, autogptq, pytorch, causal-lm, autoround, auto-round, intel-autoround, intel, woq, weights-only-quantization, ibm, 8-bit, conversational, en, base_model:ibm-granite/granite-3.2-2b-instruct, base_model:quantized:ibm-granite/granite-3.2-2b-instruct, license:apache-2.0, region:us
Base model relation: quantized
Base models: id: ibm-granite/granite-3.2-2b-instruct
Card metadata:
language: en
tags: gptq, auto-gptq, autogptq, pytorch, causal-lm, autoround, auto-round, intel-autoround, intel, woq, weights-only-quantization, granite, ibm, 8-bit
license: apache-2.0
license_link: https://choosealicense.com/licenses/apache-2.0/
library_name: transformers
model_name: Granite 3.2 2b instruct
base_model: ibm-granite/granite-3.2-2b-instruct
base_model_relation: quantized
model_type: granite
inference: no
model_creator: fbaldassarri
pipeline_tag: text-generation
quantized_by: fbaldassarri
quantization_config: method: auto_gptq
bits: 8
group_size: 64
sym: yes
auto_round_version: 0.13.1
torch_dtype: torch.bfloat16
device: cpu
nsamples: 128
iters: 200
seqlen: 512
batch_size: 4