MaziyarPanahi commited on
Commit
b375889
1 Parent(s): 274c92c

Update README.md (#3)

Browse files

- Update README.md (c564564b4351ea1d73d08147874f5d426fc0f2b7)

Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -27,6 +27,19 @@ quantized_by: MaziyarPanahi
27
 
28
  ## How to use
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ### About GGUF
31
 
32
  GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
 
27
 
28
  ## How to use
29
 
30
+ ## Load GGUF models
31
+
32
+ You `MUST` follow the prompt template provided by Llama-3:
33
+
34
+
35
+ ```sh
36
+ ./llama.cpp/main -m Llama-3-Smaug-8B.Q2_K.gguf -r '<|eot_id|>' --in-prefix "\n<|start_header_id|>user<|end_header_id|>\n\n" --in-suffix "<|eot_id|><|start_header_id|>assistant<|end_header_id|>
37
+ \n\n" -p "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.<|eot_id|>\n<|start_header_id|>user<|end_header_id|>\n\nHi!<|eot_id|>\n<|start_header_id|>assistant<|end_header_id|>\n\n" -n 1024
38
+ ```
39
+
40
+
41
+
42
+
43
  ### About GGUF
44
 
45
  GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.