MediaTek-Research
/

Breeze-7B-Instruct-v0_1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Splend1dchan commited on Jan 17

Commit

55ba48a

•

1 Parent(s): b14be99

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -183,7 +183,7 @@ model = AutoModelForCausalLM.from_pretrained(
     "MediaTek-Research/Breeze-7B-Instruct-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
-    use_flash_attn_2=True # optional
 )
 ```

     "MediaTek-Research/Breeze-7B-Instruct-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
+    attn_implementation="flash_attention_2" # optional
 )
 ```