quanthunter commited on
Commit
2fcf7df
1 Parent(s): 77e81c5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +64 -0
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - Mistral
7
+ - instruct
8
+ - finetune
9
+ - chatml
10
+ - DPO
11
+ - RLHF
12
+ - gpt4
13
+ - synthetic data
14
+ - distillation
15
+ - function calling
16
+ - json mode
17
+ - llama-cpp
18
+ - gguf-my-repo
19
+ base_model: mistralai/Mistral-7B-v0.1
20
+ datasets:
21
+ - teknium/OpenHermes-2.5
22
+ widget:
23
+ - example_title: Hermes 2 Pro
24
+ messages:
25
+ - role: system
26
+ content: You are a sentient, superintelligent artificial general intelligence,
27
+ here to teach and assist me.
28
+ - role: user
29
+ content: Write a short story about Goku discovering kirby has teamed up with Majin
30
+ Buu to destroy the world.
31
+ model-index:
32
+ - name: Hermes-2-Pro-Mistral-7B
33
+ results: []
34
+ ---
35
+
36
+ # quanthunter/Hermes-2-Pro-Mistral-7B-Q5_K_M-GGUF
37
+ This model was converted to GGUF format from [`NousResearch/Hermes-2-Pro-Mistral-7B`](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
38
+ Refer to the [original model card](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) for more details on the model.
39
+ ## Use with llama.cpp
40
+
41
+ Install llama.cpp through brew.
42
+
43
+ ```bash
44
+ brew install ggerganov/ggerganov/llama.cpp
45
+ ```
46
+ Invoke the llama.cpp server or the CLI.
47
+
48
+ CLI:
49
+
50
+ ```bash
51
+ llama-cli --hf-repo quanthunter/Hermes-2-Pro-Mistral-7B-Q5_K_M-GGUF --model hermes-2-pro-mistral-7b.Q5_K_M.gguf -p "The meaning to life and the universe is"
52
+ ```
53
+
54
+ Server:
55
+
56
+ ```bash
57
+ llama-server --hf-repo quanthunter/Hermes-2-Pro-Mistral-7B-Q5_K_M-GGUF --model hermes-2-pro-mistral-7b.Q5_K_M.gguf -c 2048
58
+ ```
59
+
60
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
61
+
62
+ ```
63
+ git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m hermes-2-pro-mistral-7b.Q5_K_M.gguf -n 128
64
+ ```