English
TheBloke commited on
Commit
ceb337d
1 Parent(s): ebcc201

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -11
README.md CHANGED
@@ -42,26 +42,21 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
42
 
43
  ## Prompt template
44
 
45
- According to the original model's README, the following template should be used:
46
 
47
  ```
48
  <|user|>
49
  prompt goes here
50
  <|assistant|>
51
- ```
52
-
53
- However in my own testing, this seems to return no response at all. But I do get good responses using:
54
 
55
  ```
56
- ### Instruction: prompt goes here
57
- ### Response:
58
- ```
59
 
60
- and
 
 
61
 
62
  ```
63
- USER: prompt goes here
64
- ASSISTANT:
65
  ```
66
 
67
  <!-- compatibility_ggml start -->
@@ -118,7 +113,7 @@ Refer to the Provided Files table below to see what files use which methods, and
118
  I use the following command line; adjust for your tastes and needs:
119
 
120
  ```
121
- ./main -t 10 -ngl 32 -m tulu-7b.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
122
  ```
123
  Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
124
 
 
42
 
43
  ## Prompt template
44
 
45
+ The following template should be used:
46
 
47
  ```
48
  <|user|>
49
  prompt goes here
50
  <|assistant|>
 
 
 
51
 
52
  ```
 
 
 
53
 
54
+ **Note**: There should be a newline after `<|assistant|>`. This appears to be very important for getting this model to respond correctly.
55
+
56
+ In other words, the prompt is:
57
 
58
  ```
59
+ <|user|>\nprompt goes here\n<|assistant|>\n
 
60
  ```
61
 
62
  <!-- compatibility_ggml start -->
 
113
  I use the following command line; adjust for your tastes and needs:
114
 
115
  ```
116
+ ./main -t 10 -ngl 32 -m tulu-7b.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "<|user|>\nprompt goes here\n<|assistant|>\n"
117
  ```
118
  Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
119