Text Generation
Transformers
Safetensors
mixtral
reasoning
preference_learning
nca
conversational
text-generation-inference
Inference Endpoints
ganqu commited on
Commit
0169f18
1 Parent(s): 4b0e984

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -147,6 +147,33 @@ We conducted overall coding, math, reasoning, knowledge, instruction-following a
147
 
148
  ## Usage
149
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
150
  We apply tailored prompts for coding and math, consistent with UltraInteract data formats:
151
 
152
  **Coding**
 
147
 
148
  ## Usage
149
 
150
+ ```python
151
+ # pip install 'transformers>=4.39.3'
152
+ # pip install accelerate
153
+
154
+ import torch
155
+ from transformers import pipeline
156
+
157
+ pipe = pipeline(
158
+ "text-generation",
159
+ model="openbmb/Eurux-8x22b-nca",
160
+ device_map="auto",
161
+ torch_dtype=torch.bfloat16,
162
+ )
163
+ messages = [
164
+ {"role": "user", "content": "What does Eurus mean?"},
165
+ ]
166
+ outputs = pipe(
167
+ messages,
168
+ max_new_tokens=512,
169
+ do_sample=True,
170
+ temperature=0.7,
171
+ top_k=50,
172
+ top_p=0.95,
173
+ )
174
+ print(outputs[0]["generated_text"][-1]["content"])
175
+ ```
176
+
177
  We apply tailored prompts for coding and math, consistent with UltraInteract data formats:
178
 
179
  **Coding**