Text Generation
Transformers
English
llama
custom_code
Inference Endpoints
text-generation-inference
chargoddard commited on
Commit
c0cd0b1
1 Parent(s): e4a68c6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -13,4 +13,11 @@ tags:
13
  Trained on a flavorful melange of the WizardLM, Airoboros, and Wizard Vicuna datasets.
14
  This model was trained using both linear and NTK-aware RoPE scaling in tandem. When loading, ensure that `compress_pos_emb` (or `scale`) is set to 2, and `alpha_value` is set to 4. *Both* values must be set.
15
 
16
- Expect context length of up to 8192 to work for sure. It will probably maintain coherence into the ~12k range, but I have not tested that.
 
 
 
 
 
 
 
 
13
  Trained on a flavorful melange of the WizardLM, Airoboros, and Wizard Vicuna datasets.
14
  This model was trained using both linear and NTK-aware RoPE scaling in tandem. When loading, ensure that `compress_pos_emb` (or `scale`) is set to 2, and `alpha_value` is set to 4. *Both* values must be set.
15
 
16
+ Expect context length of up to 8192 to work for sure. It will probably maintain coherence into the ~12k range, but I have not tested that.
17
+
18
+ Prompt format is vicuna 1.1:
19
+ ```
20
+ <whatever nonsense system prompt you want>
21
+ USER: ...
22
+ ASSISTANT: ...
23
+ ```