akkky02's picture
Upload folder using huggingface_hub
14b8326 verified
raw
history blame
147 Bytes
{
"load_in_4bit": true,
"bnb_4bit_quant_type" : "fp4",
"bnb_4bit_use_double_quant" : false,
"bnb_4bit_compute_dtype" : "bfloat16"
}