Updated llama.cpp example
Browse filesReference: https://github.com/ggerganov/llama.cpp/pull/2304
:)
README.md
CHANGED
@@ -107,7 +107,7 @@ Refer to the Provided Files table below to see what files use which methods, and
|
|
107 |
I use the following command line; adjust for your tastes and needs:
|
108 |
|
109 |
```
|
110 |
-
./main -t 10 -ngl 32 -m llama-2-13b-chat.ggmlv3.q4_0.bin --color -c
|
111 |
```
|
112 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
113 |
|
|
|
107 |
I use the following command line; adjust for your tastes and needs:
|
108 |
|
109 |
```
|
110 |
+
./main -t 10 -ngl 32 -m llama-2-13b-chat.ggmlv3.q4_0.bin --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 --in-prefix-bos --in-prefix ' [INST] ' --in-suffix ' [/INST]' -i -p "[INST] <<SYS>> You are a helpful, respectful and honest assistant. <</SYS>> Write a story about llamas. [/INST]"
|
111 |
```
|
112 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
113 |
|