Spaces:

BoZhaoHuggingFace
/

Bunny

Running on Zero

BoZhaoHuggingFace commited on 28 days ago

Commit

9f37abb

•

1 Parent(s): 3f18ebc

Update bunny/model/builder.py

Files changed (1) hide show

bunny/model/builder.py CHANGED Viewed

@@ -31,7 +31,7 @@ def load_pretrained_model(model_path, model_base, model_name, model_type, load_8
             bnb_4bit_quant_type='nf4'
         )
     else:
-        kwargs['torch_dtype'] = torch.bfloat16
     # Load Bunny model
     if 'lora' in model_name.lower() and model_base is None:

             bnb_4bit_quant_type='nf4'
         )
     else:
+        kwargs['torch_dtype'] = torch.float16
     # Load Bunny model
     if 'lora' in model_name.lower() and model_base is None: