alexcovo commited on
Commit
0edd524
1 Parent(s): 86a51ec

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +64 -0
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - Llama-3
6
+ - instruct
7
+ - finetune
8
+ - chatml
9
+ - DPO
10
+ - RLHF
11
+ - gpt4
12
+ - synthetic data
13
+ - distillation
14
+ - function calling
15
+ - json mode
16
+ - axolotl
17
+ - llama-cpp
18
+ - gguf-my-repo
19
+ base_model: NousResearch/Meta-Llama-3-8B
20
+ datasets:
21
+ - teknium/OpenHermes-2.5
22
+ widget:
23
+ - example_title: Hermes 2 Pro
24
+ messages:
25
+ - role: system
26
+ content: You are a sentient, superintelligent artificial general intelligence,
27
+ here to teach and assist me.
28
+ - role: user
29
+ content: Write a short story about Goku discovering kirby has teamed up with Majin
30
+ Buu to destroy the world.
31
+ model-index:
32
+ - name: Hermes-2-Pro-Llama-3-8B
33
+ results: []
34
+ ---
35
+
36
+ # alexcovo/Hermes-2-Pro-Llama-3-8B-Q4_0-GGUF
37
+ This model was converted to GGUF format from [`NousResearch/Hermes-2-Pro-Llama-3-8B`](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
38
+ Refer to the [original model card](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B) for more details on the model.
39
+ ## Use with llama.cpp
40
+
41
+ Install llama.cpp through brew.
42
+
43
+ ```bash
44
+ brew install ggerganov/ggerganov/llama.cpp
45
+ ```
46
+ Invoke the llama.cpp server or the CLI.
47
+
48
+ CLI:
49
+
50
+ ```bash
51
+ llama-cli --hf-repo alexcovo/Hermes-2-Pro-Llama-3-8B-Q4_0-GGUF --model hermes-2-pro-llama-3-8b.Q4_0.gguf -p "The meaning to life and the universe is"
52
+ ```
53
+
54
+ Server:
55
+
56
+ ```bash
57
+ llama-server --hf-repo alexcovo/Hermes-2-Pro-Llama-3-8B-Q4_0-GGUF --model hermes-2-pro-llama-3-8b.Q4_0.gguf -c 2048
58
+ ```
59
+
60
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
61
+
62
+ ```
63
+ git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m hermes-2-pro-llama-3-8b.Q4_0.gguf -n 128
64
+ ```