phi-1_5-nf4-fp16compute-doublequant-BNB / generation_config.json
DaertML's picture
ADD doublequant 4 bit quantization of phi 1.5B model
18d7875
{
"_from_model_config": true,
"transformers_version": "4.33.1"
}