Spaces:

taufiqdp
/

phi-3-mini-128k-instruct

Running on Zero

taufiqdp commited on Apr 24

Commit

c96c1a0

•

1 Parent(s): 161110e

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -16,7 +16,7 @@ model_id = "microsoft/Phi-3-mini-128k-instruct"
 tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
-    dtype=torch.bfloat16,
     device_map="auto",
     trust_remote_code=True,
     attn_implementation="flash_attention_2"

 tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
+    torch_dtype=torch.bfloat16,
     device_map="auto",
     trust_remote_code=True,
     attn_implementation="flash_attention_2"