mesolitica
/

malaysian-mistral-7b-32k-instructions-AWQ

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

malaysian-mistral-7b-32k-instructions-AWQ / quant_config.json

huseinzol05's picture

Upload quant_config.json with huggingface_hub

b035dcd 12 months ago

72 Bytes

{"zero_point": true, "q_group_size": 128, "w_bit": 4, "version": "GEMM"}