cloudyu
/

Llama-3-8Bx2-MOE-DPO

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

cloudyu commited on Apr 26

Commit

c4b8a5a

•

1 Parent(s): 25d956c

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -2,7 +2,7 @@
 license: apache-2.0
 ---
-code example
 ```
 import transformers
@@ -14,13 +14,13 @@ pipeline = transformers.pipeline(
     "text-generation",
     model=model_id,
     model_kwargs={"torch_dtype": torch.float16},
-    device_map="auto",
 )
 prompt = "what is biggest animal in earth?"
 while len(prompt)>0:
   messages = [
-    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
     {"role": "user", "content": prompt},
   ]

 license: apache-2.0
 ---
+Apple MPS code example
 ```
 import transformers
     "text-generation",
     model=model_id,
     model_kwargs={"torch_dtype": torch.float16},
+    device_map="mps",
 )
 prompt = "what is biggest animal in earth?"
 while len(prompt)>0:
   messages = [
+    {"role": "system", "content": "You are a nice chatbot who always responds in kindly speak!"},
     {"role": "user", "content": prompt},
   ]