MediaTek-Research
/

Breeze-7B-Instruct-v0_1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

cllatMTK commited on Jan 12

Commit

942dcea

•

1 Parent(s): 5c3d80a

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -180,10 +180,9 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
 model = AutoModelForCausalLM.from_pretrained(
-    model="MediaTek-Research/Breeze-7B-Instruct-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
-    use_flash_attn_2=True # optional
 )
 ```

 import torch
 model = AutoModelForCausalLM.from_pretrained(
+    "MediaTek-Research/Breeze-7B-Instruct-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
 )
 ```