Commit
•
bf93d08
1
Parent(s):
1d1c4b2
Update README.md
Browse files
README.md
CHANGED
@@ -1,49 +1,43 @@
|
|
1 |
-
---
|
2 |
-
language:
|
3 |
-
- en
|
4 |
-
license: mit
|
5 |
-
tags:
|
6 |
-
-
|
7 |
-
-
|
8 |
-
-
|
9 |
-
-
|
10 |
-
license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
|
11 |
-
pipeline_tag: text-generation
|
12 |
-
inference:
|
13 |
-
parameters:
|
14 |
-
temperature: 0.7
|
15 |
-
widget:
|
16 |
-
- messages:
|
17 |
-
- role: user
|
18 |
-
content: Can you provide ways to eat combinations of bananas and dragonfruits?
|
19 |
-
---
|
20 |
|
21 |
# tinybiggames/Phi-3-mini-4k-instruct-Q4_K_M-GGUF
|
22 |
This model was converted to GGUF format from [`microsoft/Phi-3-mini-4k-instruct`](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
23 |
Refer to the [original model card](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) for more details on the model.
|
24 |
-
## Use with
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
```
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
```
|
44 |
-
|
45 |
-
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
|
46 |
-
|
47 |
-
```
|
48 |
-
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m phi-3-mini-4k-instruct.Q4_K_M.gguf -n 128
|
49 |
-
```
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
license: mit
|
5 |
+
tags:
|
6 |
+
- llama-cpp
|
7 |
+
- gguf-my-repo
|
8 |
+
- Infero
|
9 |
+
- Dllama
|
10 |
+
license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
|
11 |
+
pipeline_tag: text-generation
|
12 |
+
inference:
|
13 |
+
parameters:
|
14 |
+
temperature: 0.7
|
15 |
+
widget:
|
16 |
+
- messages:
|
17 |
+
- role: user
|
18 |
+
content: Can you provide ways to eat combinations of bananas and dragonfruits?
|
19 |
+
---
|
20 |
|
21 |
# tinybiggames/Phi-3-mini-4k-instruct-Q4_K_M-GGUF
|
22 |
This model was converted to GGUF format from [`microsoft/Phi-3-mini-4k-instruct`](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
23 |
Refer to the [original model card](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) for more details on the model.
|
24 |
+
## Use with tinyBigGAMES's Local LLM Inference Libraries
|
25 |
+
|
26 |
+
Add to **config.json**
|
27 |
+
|
28 |
+
```Json
|
29 |
+
{
|
30 |
+
"filename": "phi-3-mini-4k-instruct.Q4_K_M.gguf",
|
31 |
+
"name": "phi3:4K:Q4_K_M",
|
32 |
+
"max_context": 4000,
|
33 |
+
"template": "<|%s|>%s<|im_end|>",
|
34 |
+
"template_end": "<|assistant|>",
|
35 |
+
"stop": [
|
36 |
+
"<|user|>",
|
37 |
+
"<|assistant|>",
|
38 |
+
"<|system|>",
|
39 |
+
"<|end|>",
|
40 |
+
"<|endoftext|>"
|
41 |
+
]
|
42 |
+
}
|
43 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|