Text Generation
Transformers
PyTorch
English
llama
text-generation-inference
Inference Endpoints
bleysg commited on
Commit
ed00e88
1 Parent(s): 7f556f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -2
README.md CHANGED
@@ -74,7 +74,8 @@ Suffix (space after colon):
74
  ```
75
  <|end_of_turn|>\nAssistant:
76
  ```
77
- ### Oobabooga's webui instructions can be found further down the page
 
78
 
79
 
80
  # Evaluation
@@ -154,6 +155,8 @@ Pre-quantized models are now available courtesy of our friend TheBloke:
154
  * **GGML**: https://huggingface.co/TheBloke/OpenOrcaxOpenChat-Preview2-13B-GGML
155
  * **GPTQ**: https://huggingface.co/TheBloke/OpenOrcaxOpenChat-Preview2-13B-GPTQ
156
 
 
 
157
 
158
  ## Serving with OpenChat
159
 
@@ -200,7 +203,9 @@ For "`Bot string`" ...
200
  ```
201
  Assistant:
202
  ```
203
- For "`Context`", it is not necessary but we have found good results with ...
 
 
204
  ```
205
  You are a helpful assistant. Please answer truthfully and write out your thinking step by step to be sure you get the right answer. If you make a mistake or encounter an error in your thinking, say so out loud and attempt to correct it. If you don't know or aren't sure about something, say so clearly. You will act as a professional logician, mathematician, and physicist. You will also act as the most appropriate type of expert to answer any particular question or solve the relevant problem; state which expert type your are, if so. Also think of any particular named expert that would be ideal to answer the relevant question or solve the relevant problem; name and act as them, if appropriate.
206
  ```
 
74
  ```
75
  <|end_of_turn|>\nAssistant:
76
  ```
77
+
78
+ **Oobabooga's text-generation-webui instructions can be found [further down the page](https://huggingface.co/Open-Orca/OpenOrcaxOpenChat-Preview2-13B#serving-with-oobabooga--text-generation-webui).**
79
 
80
 
81
  # Evaluation
 
155
  * **GGML**: https://huggingface.co/TheBloke/OpenOrcaxOpenChat-Preview2-13B-GGML
156
  * **GPTQ**: https://huggingface.co/TheBloke/OpenOrcaxOpenChat-Preview2-13B-GPTQ
157
 
158
+ The serving instructions below only apply to the unquantized model being presented in the repository you are viewing here.
159
+ There are some notes, such as on use of the prompt format, that will still apply to the quantized models though.
160
 
161
  ## Serving with OpenChat
162
 
 
203
  ```
204
  Assistant:
205
  ```
206
+ For "`Context`", this is analogous to system prompt.
207
+ It is not necessary, but we have found good results with the below example.
208
+ System prompts used in the Orca training also work well. ...
209
  ```
210
  You are a helpful assistant. Please answer truthfully and write out your thinking step by step to be sure you get the right answer. If you make a mistake or encounter an error in your thinking, say so out loud and attempt to correct it. If you don't know or aren't sure about something, say so clearly. You will act as a professional logician, mathematician, and physicist. You will also act as the most appropriate type of expert to answer any particular question or solve the relevant problem; state which expert type your are, if so. Also think of any particular named expert that would be ideal to answer the relevant question or solve the relevant problem; name and act as them, if appropriate.
211
  ```