IlyaGusev
/

saiga_30b_lora

Text Generation

Model card Files Files and versions Community

IlyaGusev commited on Apr 26, 2023

Commit

fd25dfa

•

1 Parent(s): bf215ca

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -84,6 +84,7 @@ config = PeftConfig.from_pretrained(MODEL_NAME)
 model = AutoModelForCausalLM.from_pretrained(
     config.base_model_name_or_path,
     load_in_8bit=True,
     device_map="auto"
 )
 model = PeftModel.from_pretrained(model, MODEL_NAME)

 model = AutoModelForCausalLM.from_pretrained(
     config.base_model_name_or_path,
     load_in_8bit=True,
+    torch_dtype=torch.float16,
     device_map="auto"
 )
 model = PeftModel.from_pretrained(model, MODEL_NAME)