You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

MH-Q35-9B-v3.2-AWQ

Merged fine-tuned checkpoint, AWQ W4A16 quantized. Gated — access by request.

Identity

Ships with a John Snow Labs identity chat template; the model identifies as a John Snow Labs AI assistant.

Quantization

4-bit AWQ (W4A16_ASYM, group size 128) via llmcompressor, calibrated on 512 samples drawn from the full MedHELM + OpenMed corpora. Vision tower and linear-attention (GDN) layers kept at full precision; only full-attention layer projections quantized.

Usage

from transformers import AutoModelForImageTextToText, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Zaynoid/MH-Q35-9B-v3.2-AWQ")
model = AutoModelForImageTextToText.from_pretrained("Zaynoid/MH-Q35-9B-v3.2-AWQ", torch_dtype="bfloat16", device_map="auto")

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

I32

BF16