Update README.md
Browse files
README.md
CHANGED
@@ -4,13 +4,17 @@ language:
|
|
4 |
- ko
|
5 |
---
|
6 |
# Llama-2-ko-7b-ggml
|
|
|
|
|
7 |
Llama-2-ko-7b-ggml ์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์ **GGML** ํฌ๋งท ๋ชจ๋ธ์
๋๋ค.
|
8 |
|
9 |
- Llama2 tokenizer ์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์์ ์ฌ์ฉ๋ ํ๊ตญ์ด Additaional Token ์ ๋ฐ์ํ์ฌ ์์ฑํ์ต๋๋ค.
|
10 |
- **GGML** ํฌ๋งท ๋ชจ๋ธ์ [llama.cpp](https://github.com/ggerganov/llama.cpp) ๋ฅผ ์ฌ์ฉํ์ฌ C/C++ ๊ธฐ๋ฐ์ผ๋ก Inference ํฉ๋๋ค.
|
|
|
11 |
- [llama.cpp](https://github.com/ggerganov/llama.cpp) ์ Python Binding ํจํค์ง์ธ [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) ์ ์ฌ์ฉํ๋ฉด python ํ๊ฒฝ์์๋ Inference ๊ฐ๋ฅํฉ๋๋ค.
|
|
|
12 |
|
13 |
-
์ฐธ๊ณ ๋ก, [Llama-2-ko-7b-chat-ggml](https://huggingface.co/StarFox7/Llama-2-ko-7B-ggml) ์์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์ [nlpai-lab/kullm-v2](https://huggingface.co/datasets/nlpai-lab/kullm-v2) ์ ์ถ๊ฐ ํ์ตํ [Llama-2-ko-7b-chat](https://huggingface.co/kfkas/Llama-2-ko-7b-Chat) ์ **GGML** ํฌ๋งท ๋ชจ๋ธ์ ์ฐพ์ ์ ์์ต๋๋ค.
|
14 |
|
15 |
---
|
16 |
# ์์ํ (Quantization)
|
@@ -24,10 +28,13 @@ Llama-2-ko-7b-ggml ์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-
|
|
24 |
# Inference Code Example (Python)
|
25 |
๋ค์์ Inference ๋ฅผ ์ํ ๊ฐ๋จํ Example Code ์
๋๋ค. [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) ๊ทธ๋ฆฌ๊ณ ์ด Repository ์ Llama-2-ko-7b-ggml-q4_0.bin ๊ฐ ํ์ํฉ๋๋ค.
|
26 |
```python
|
|
|
27 |
from llama_cpp import Llama
|
28 |
|
29 |
llm = Llama(model_path = 'Llama-2-ko-7b-ggml-q4_0.bin',
|
30 |
-
n_ctx=1024
|
|
|
|
|
31 |
|
32 |
output = llm("Q: ์ธ์์ ๋ํด์ ์ค๋ช
ํ์์ค. A: ", max_tokens=1024, stop=["Q:", "\n"], echo=True)
|
33 |
|
|
|
4 |
- ko
|
5 |
---
|
6 |
# Llama-2-ko-7b-ggml
|
7 |
+
<img src=https://huggingface.co/StarFox7/Llama-2-ko-7B-ggml/resolve/main/cute.png style="max-width: 200px; width: 100%" />
|
8 |
+
|
9 |
Llama-2-ko-7b-ggml ์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์ **GGML** ํฌ๋งท ๋ชจ๋ธ์
๋๋ค.
|
10 |
|
11 |
- Llama2 tokenizer ์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์์ ์ฌ์ฉ๋ ํ๊ตญ์ด Additaional Token ์ ๋ฐ์ํ์ฌ ์์ฑํ์ต๋๋ค.
|
12 |
- **GGML** ํฌ๋งท ๋ชจ๋ธ์ [llama.cpp](https://github.com/ggerganov/llama.cpp) ๋ฅผ ์ฌ์ฉํ์ฌ C/C++ ๊ธฐ๋ฐ์ผ๋ก Inference ํฉ๋๋ค.
|
13 |
+
- **GGML** ํฌ๋งท ๋ชจ๋ธ์ ๋น๊ต์ ๋ฎ์ ์ฌ์์ ์ปดํจํ
์์์์๋ Inference ๊ฐ๋ฅํฉ๋๋ค. ( ์: 4๋นํธ ์์ํ ๋ชจ๋ธ (q4) ์ CPU,7-8GB RAM ํ๊ฒฝ์์ Inference ๊ฐ๋ฅ )
|
14 |
- [llama.cpp](https://github.com/ggerganov/llama.cpp) ์ Python Binding ํจํค์ง์ธ [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) ์ ์ฌ์ฉํ๋ฉด python ํ๊ฒฝ์์๋ Inference ๊ฐ๋ฅํฉ๋๋ค.
|
15 |
+
|
16 |
|
17 |
+
์ฐธ๊ณ ๋ก, [Llama-2-ko-7b-chat-ggml](https://huggingface.co/StarFox7/Llama-2-ko-7B-chat-ggml) ์์ [beomi/llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b) ์ [nlpai-lab/kullm-v2](https://huggingface.co/datasets/nlpai-lab/kullm-v2) ์ ์ถ๊ฐ ํ์ตํ [kfkas/Llama-2-ko-7b-chat](https://huggingface.co/kfkas/Llama-2-ko-7b-Chat) ์ **GGML** ํฌ๋งท ๋ชจ๋ธ์ ์ฐพ์ ์ ์์ต๋๋ค.
|
18 |
|
19 |
---
|
20 |
# ์์ํ (Quantization)
|
|
|
28 |
# Inference Code Example (Python)
|
29 |
๋ค์์ Inference ๋ฅผ ์ํ ๊ฐ๋จํ Example Code ์
๋๋ค. [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) ๊ทธ๋ฆฌ๊ณ ์ด Repository ์ Llama-2-ko-7b-ggml-q4_0.bin ๊ฐ ํ์ํฉ๋๋ค.
|
30 |
```python
|
31 |
+
# !pip install llama-cpp-python # llama-cpp-python ์ด ์ค์น๋์ด ์์ง ์๋ค๋ฉด ์ฃผ์์ ํด์ ํ์ฌ ์ค์นํฉ๋๋ค.
|
32 |
from llama_cpp import Llama
|
33 |
|
34 |
llm = Llama(model_path = 'Llama-2-ko-7b-ggml-q4_0.bin',
|
35 |
+
n_ctx=1024,
|
36 |
+
# n_gpu_layers=1 #gpu ๊ฐ์์ ์ํ๋ ๊ฒฝ์ฐ ์ฃผ์์ ํด์ ํ๊ณ Metal(Apple M1) ์ 1, Cuda(Nvidia) ๋ Video RAM Size ๋ฅผ ๊ณ ๋ คํ์ฌ ์ ์ ํ ์์น๋ฅผ ์
๋ ฅํฉ๋๋ค.)
|
37 |
+
)
|
38 |
|
39 |
output = llm("Q: ์ธ์์ ๋ํด์ ์ค๋ช
ํ์์ค. A: ", max_tokens=1024, stop=["Q:", "\n"], echo=True)
|
40 |
|